What?
Junkie is an open source, fast, extensible, deep packet inspection tool (aka DPI).
Other distinguishing features include:
- Can take advantage of many threads while still handling packet reassembly and connection tracking;
- Can handle out-of-order traffic, missing and even truncated packets (as in
tcpdump -s) and keep on parsing, skipping the gaps;
- Can see through TLS if provided with private keys;
- Can be controlled via a scripting language although Junkie itself is written in C;
- Does nothing but parsing by itself but comes with many plugins to perform various tasks.
Examples of what's doable with Junkie default plugins from the command line:
- Dumping live traffic à la tshark;
- Protocol aware net-top;
- Monitor SSL certificate expiry dates;
- Dump response times for all known protocols;
- ...
See plugins for a complete list and reference.
Why?
A DPI was needed to analyse mirrored traffic for application monitoring.
At the time (around 2010) there were simply no open source tools that could do deep packet inspection in any capacity on live traffic.
Today, other open source DPI have emerged. From a quick survey of those Junkie still looks very relevant, but a more thorough comparison has to be carried out and published.
How?
The key design principles are:
- Be prepared to receive packets out-of-order in a way they could not possibly have been emitted. This is particularly common when the sniffer is sent mirrored traffic instead of standing across the traffic path, even doubly so when the traffic is mirrored from several locations. Temporary buffering is the norm rather than the exception.
- Be prepared for a thread decisions to depend on data managed by another thread. There is no kernel trick clever enough to assist the sniffer in splitting the data amongst several threads in a way that connection tracking does not require interlocks. Better use many threads, many fine-grained locks and a deadlock detector than rely on tricks that prevent the sniffer to access data when it needs it.
- Assume every byte of traffic can be missing and be prepared to skip over gaps to keep parsing whenever possible (which is often, as the interesting bits are often located in the very beginning of the packets/frames/messages/etc).
- Avoid copying packets around when reassembling; rather, use some abstract pointers pretending data chunks are sequential when they are not. Also helps with skipping over missing parts: as long as the pointer is not dereferenced, who care if the data is missing?
- Do not assume any preconceived protocol stack. In particular, be prepared that the same protocol can appear several times in the stack (tunnelling...) When writing a parser lower and upper layers must be abstracted away.
- For maximum versatility, do not design Junkie to do anything beyond parsing as fast and deep as doable in live traffic, reporting findings to user defined callbacks at specific locations (hooks).
- Junkie being designed to monitor traffic without interruption, make it observable and extensible live via an extension language. Using this language, one can connect to Junkie and change its settings, load/unload plugins, etc... Junkie uses Guile as it's the only extension language able to share memory and Posix threads with C.
More information