Today we announced an integration between SolarWinds® AppOptics™ and SolarWinds Papertrail™ to allow you to quickly move from service-level metrics, down to a trace, and then down to the logs specific to that trace.
The integration between AppOptics and Papertrail provides the ability to group the log messages from a traced transaction and add trace context to your logs in Papertrail. Connecting the dots between the distributed trace and the related logs makes your life easier when troubleshooting application issues—especially in complex environments.
So How Does it Work?
With both Papertrail and AppOptics installed, you can skip the manual steps usually required for inserting context in your logs. AppOptics libraries use auto-instrumentation and context propagation through HTTP headers. This means if your application is in Go, Java, Node.js, PHP, or Ruby, there is no instrumentation needed. You can leverage context tracking out of the box. From a trace view in AppOptics that allows you to explore latency and span counts, you can click on the “Search Logs” button to quickly transition down to the logs for that specific request in Papertrail.
The “Search Logs” button launches Papertrail with the trace ID automatically inserted into the search bar. You will see all the logs with the matching trace ID in the event viewer. As you can see in the screenshot below, AppOptics is inserting an “ao.traceid=” tag into the logs which allows you to isolate the logs that are specific to each request. You can use the Papertrail filter options to select specific systems or further refine your search.
Why is This a Big Deal?
Since requests can travel across many different systems throughout their life, and there is no built-in connection between logs, it can be extremely challenging to track a request. A common way application engineers approach this problem is to look at the surrounding logs.
Logs are typically in chronological order. When you see an exception, you typically look at the log lines from just before the exception occurred. The challenge in complex cloud application is you might be looking at completely unrelated logs. In a busy application with thousands of requests per minute, it’s difficult to understand which logs belong to a specific request, and which logs relate to the exception you are trying to understand.
Teams have tried to solve this problem by logging session IDs, API tokens, and other types of unique identifiers. The approach addresses part of the problem but requires upfront manual instrumentation work, and breaks down once multiple systems and services are involved.
How to Get Started
Distributed tracing on its own is amazing, but adding a simple, single click to drill down directly from that trace into the appropriate logs grouped together by trace ID is game-changing. By using distributed tracing to generate and propagate context (i.e., trace IDs) throughout the lifetime of requests, then having this context annotated on associated log lines, you now have access to logs with distributed context. These context-rich logs allow you to isolate individual requests across all systems for better diagnostics.
Want to add context to your logs? Take AppOptics out for a spin by trying it for free for 14 days and see what the combination of Papertrail and AppOptics can do for yourself.