Building low-overhead metrics collection for high-performance systems

Design decisions that allowed us to capture metrics in Wallaroo with a low-overhead.

Keep reading

Latency Histograms and Percentile Distributions In Wallaroo Performance Metrics

How we implemented Wallaroo’s low overhead performance counters and the philosophy behind the choices we made.

Keep reading

Why we wrote our Kafka Client in Pony

Why we wrote our Kafka Client in Pony instead of relying on the existing C Kafka Client and creating bindings for it. We talk about the pros/cons of our decision and how far we’ve come including some preliminary performance numbers.

Keep reading

You might have a streaming data problem if...

When processing data, we often categorize a job as either a batch or streaming job. However, this is a bit of a false dichotomy. In this post, I’ll explore how we ordinarily define batch and stream processing, and show how some tasks that we think of as batch jobs can be seen as a subset of stream processing.

Keep reading

Serverless, Scale-Independent Processing, and Wallaroo

Our open-source framework, Wallaroo, shares many high-level goals as the existing serverless frameworks, but is also different in key areas and thus better suited for many use-cases.

Keep reading