Papertrail

The revolution will be verbosely {,b}logged

Hello cron job monitoring & alerts, goodbye silent failures

Posted by @coryduncan on

Papertrail has had the ability to alert on searches that match events for years, but what about when they don’t? When a cron job, backup, or other recurring job doesn’t run, it’s not easy to notice the absence of an expected message. But now, Papertrail can do the noticing for you. Today we’re excited to release inactivity alerts, offering the ability to alert when searches don’t match events.

Set up an inactivity alert

From the create/edit alert form, choose “Trigger when no new events match”

Inactivity alerts

Once saved, the alert will send notifications when there are no matching events within the chosen time period. Use this for:

  • cron jobs
  • background jobs which should run nearly all the time, like system monitors/pollers and database or offsite backups
  • lower-frequency scheduled jobs, like nightly billing

Try it

If you have cron jobs, backup jobs, or other recurring or scheduled jobs, they almost certainly already generate logs. Here’s how to have Papertrail tell you when they don’t run or run but don’t complete successfully:

  1. Search for the message emitted when a cron job finishes successfully (example: cron)
  2. Click “Save Search”
  3. Attach an alert, like to notify a Slack channel or send an email.

No logs? No problem.

Very rarely, a recurring job doesn’t generate log messages on its own. For those, use the shell && operator and logger to generate a post-success log message. For example, ./run-billing-job && logger "billing succeeded" will send the message billing succeeded to syslog if and only if run-billing-job finishes with a successful exit code. Use "billing succeeded" as the Papertrail search.

What do you think?

Give inactivity alerts a try and if you have questions or feedback, let us know.