What?

Open source, fast, non-distributed stream processing for monitoring.

Why?

Thanks to such large companies as Google, Facebook, Linkedin and Netflix, the culture and practice of modern infrastructure monitoring has vastly improved and many good and free tools have been released publicly. Those tools understandably focus on large distributed infrastructure.

Ramen is designed for the scale most mortals are dealing with.

Ramen has been designed to process network monitoring data in embedded appliances but can be useful in other contexts. If you need an all-purpose stream processor to turn inputs time series into dashboards or alerts but do not want to deploy Kubernetes in your three racks of hardware or have only a couple of GiB left of RAM for monitoring, then you might want to consider using Ramen.

How?

The key design principles are:

Ramen is not based on a persistent message queue; Instead messages are sent directly from producers to consumers, using memory-mapped ring-buffers (or, as a possible optimisation, direct function calls).
Operations are defined using a strongly typed SQL-like language that's compiled down to machine code for efficiency. The type system knows about NULLs, integers up to 128 bits, network addresses, compound types such as lists and structures, etc... Available functions include the usual online algorithms and then some.

This is how it looks like:

DEFINE memory_alert AS
  SELECT time, host, free + used + cached AS total
  FROM memory
  GROUP BY host
  NOTIFY "Low RAM" WHEN free / total < 0.5;

An HTTP daemon mimics the Graphite API so Ramen works with Grafana out of the box.
Past data can be archived so that Ramen can also answer queries about the past and be useful not only for real-time dashboarding and alerting but also for troubleshooting/capacity planning/...

What?

Why?

How?

More information