Ramen Documentation
Ramen consists of a single executable that can act as the supervisor daemon, the compiler, a data extractor, and so on.
It is self-documented so only a very rough summary of the main commands is given hereafter. You will find an actual copy of the embedded documentation at the end.
Programs are stored on mere text files which extension should be ".ramen". To compile one, use the ramen compile command. This will parse, type-check and then generate the executable binary (which extension will be ".x").
For type-checking it is likely that the compiler will need to know the output types of referenced functions from other programs. It will then look for those programs in the path provided by wither the RAMEN_ROOT+ environment variable or the --root command line option. Those referenced programs must have been compiled already, which forces a given order when compiling multiple programs.
As an example, suppose this program:
SELECT hostname, last uptime FROM monitoring/hosts GROUP BY hostname COMMIT WHEN in.time > group.first.time + 30
to know the types of hostname, uptime and time ramen must look at the monitoring/hosts operation output type. It will then look for the executable file $RAMEN_ROOT/monitoring/hosts.x which will then provide this type (as each executable worker can display its types and other information about itself). Therefore this program must have been compiled previously.
See ramen compile --help for details.
Contrary to normal executables, Ramen programs must be started by the supervisor, which will make sure all workers are connected properly.
ramen supervisor will start a daemon that will essentially check that the workers specified in the running-configuration are indeed working. When starting new worker it will first check that types are compatible with the other workers (or report an error), create the ring buffers and "connect" the parents to their children.
See ramen supervisor --help for more.
To stop the supervisor, merely sends it a TERM or INT signal. It will then stops all workers and exit. Workers themselves will save their state so that little is lost by stopping and restarting Ramen.
To start a program use the ramen run command. This command does not in itself start anything, though. Instead, it merely adds that program to the list of programs that the supervisor must keep running. Therefore the actual workers won't start until the process supervisor itself is running.
See ramen run --help for details.
Similarly, ramen kill will remove a program from the running configuration of the supervisor.
See ramen kill --help for more.
ramen ps will list the running programs and display some runtime statistics which makes it a `ps` as well as a `top` equivalent.
See ramen ps --help for more.
ramen tail followed by the fully qualified name of an operation will display the last output tuples by that operation.
If this is the first time in a while that this worker has been asked for its output then it may take some time to start receiving data as old tuples may not be archived (so ramen tail will wait for new tuples instead).
Refer to ramen tail --help for more information.
Similarly, ramen timeseries will extract a single field from an operation, indexed by time. Time points will be evenly spaced and events will be "bucketed" into the requested time scale.
See ramen timeseries --help for more details.
In addition to tuples, workers can emit notifications. Those are received by a dedicated daemon: ramen alerter that will route them according to its configuration file.
See ramen alerter --help for more details.
Other daemons are required in production, to clean the archived output, compute various statistics, and so on.
See ramen gc --help and ramen archivist --help for more details.
Some commands are also available to help with diagnosing issues. See ramen --help for an exhaustive list of commands.
As Ramen is designed to connect to alerting systems, reliability has been an important design consideration. The main source of errors in production systems being configuration changes, it is therefore important to test any change in the configuration.
The ramen test command takes text files each describing a test and, independently of any already running instance of ramen, will run all specified programs, provide them with the test input, and check the output matches the ones described in the test file.
See for instance this test from ramen own test suite for an example of a test specification.
Refer to ramen test --help for details.