Posts tagged aws

In the AWS Ruby blog, Trevor Rowe covers Logging HTTP Wire Traces with the AWS SDK for Ruby (aws-sdk gem) and overriding the default Logger class.

By default, aws-sdk logs to $stdout for Ruby apps or Rails.logger for Rails apps. Trevor’s example passes in an alternative Logger instance that is backed by $stderr:

AWS.config(:logger => Logger.new($stderr))

Let’s send these logs to a syslog destination like Papertrail, using Papertrail’s open-source remote_syslog_logger gem.

remote_syslog_logger

remote_syslog_logger provides a Logger-compatible class that’s backed by a UDP remote syslog destination (rather than an instance of IO, such as $stderr). To use it, add remote_syslog_logger to your Gemfile (or require it), then instantiate RemoteSyslogLogger with the syslog destination hostname and optionally, port. For example:

AWS.config(:logger => RemoteSyslogLogger.new('syslog.domain.com', 1234)

See Papertrail’s Add Systems page for these settings.

This works without any other system-wide config changes. A RemoteSyslogLogger instance can be passed to any Ruby class which expects a Logger instance.

Customize

By default, RemoteSyslogLogger will use its running program as the program name in the syslog message. Often this is just rails. Let’s set the program to something more descriptive, like myapp-development. Let’s also enable the AWS Ruby SDK http_wire_trace option to output HTTP headers (which one would typically only do in development). Here it is:

AWS.config(:logger => RemoteSyslogLogger.new('syslog.domain.com', 1234, :program => "myapp-#{RAILS_ENV}"), :http_wire_trace => true)

The README has more.

Extend

To use two Loggers, one could wrap RemoteSyslogLogger within a container class which simply invoked each of them.

Other ways to accomplish a similar result include writing logs to local syslog and configuring it to transmit them, or write to a text log file and use a daemon to watch and send it (see rsyslog, syslog-ng, or remote_syslog).

Happy debugging!

We formally announced public availability of Papertrail hosted log management yesterday. This is as formal as we get!

Read the announcement below (or on Business Wire).

Cloud-Based Logging Service Papertrail Launches

Hosted log management service helps engineers detect, resolve, and avoid infrastructure problems

SEATTLE—(BUSINESS WIRE)—Cloud-based log management service Papertrail™ announced public availability today. Papertrail delivers effortless log aggregation and management as a hosted service (SaaS).

In 30 seconds, an engineer or startup entrepreneur can obtain a searchable, scalable, always-on repository for their server and application log messages, over the Internet. With a simple one-line change, logs are sent to Papertrail as they happen.

Bringing all log messages together is the gateway to powerful operations visibility, monitoring, and analysis with Papertrail. For example:

• systems administrators can watch infrastructure events in realtime — from the Web, command-line, or REST API.
• developers diagnosing app problems can search for anything they know (like an IP address, error message, or customer name), or seek right to a specific time or server.
• support staff can view events from only the product that they support, without access to the underlying systems.
• architects can setup alerts to notify monitoring systems about important events (web hooks), and analyze months of logs in Hadoop to identify trends.

“Engineers often discover Papertrail as the easiest way to aggregate server log files, then they try it and realize that log collection is just the beginning,” said Troy Davis, co-founder and CEO of Papertrail’s operating company, Seven Scale. “Seeing operating system logs, app server requests, database queries, and router logs in one live interface is incredibly useful.”

A single Web request might generate a firewall log message, a Redis lookup, a dozen MySQL queries, a Ruby on Rails request log, a handful of static file logs from Apache, and syslog from a background task. Because Papertrail works with all common log methods, engineers can see these related events together as an end-to-end operational narrative.

Pre-launch customers already use Papertrail to manage many terabytes every month. It seamlessly scales with each customer’s needs, whether a few messages per hour or thousands per second. Systems and apps can reside on physical servers, hosting services like Amazon Elastic Compute Cloud (EC2), and app clouds like Heroku.

“DevOps engineers want productivity. It’s not just about more data or more features, but the right info at the right time,” Davis said. “Papertrail provides that, and in less than a minute.”

Demonstrations will be available at the O’Reilly Velocity Web Performance & Operations Conference.

Seven Scale operates the Papertrail hosted log management service, which brings smiles to the faces of systems engineers, developers, and entrepreneurs.

Everyone deserves great logging. Sign up and log to the cloud™ at http://papertrailapp.com/