Reasons to Scale Horizontally

An introduction to horizontal scaling: what is it and why you might want to do it. The alternate title was going to be ‘Would you rather fight 1 big cow or 15 small cows?’

Keep reading

The Treacherous Tangle of Redundant Data: Resilience for Wallaroo

Introduction: we need data redundancy, but how, exactly? You now have your distributed system in production, congratulations! Your cluster is starting at six machines, but it is expected to grow quickly as it is assigned more work. The cluster’s main application is stateful, and that’s a problem. What if you lose a local disk drive? Or a sysadmin runs rm -rf on the wrong directory? Or else the entire machine cannot reboot, due to a power failure or administrator error that destroys an entire virtual machine?…

Keep reading

Checkpointing and Consistent Recovery Lines: How We Handle Failure in Wallaroo

In which we show you some of the key issues we considered when choosing how to handle failure in our system, and, in the process, introduce you to some concepts and resources that will help you in thinking about how to build resilient distributed systems of your own.

Keep reading

Real-time Streaming Pattern: Analyzing Trends

This week, we continue to look at data processing patterns used to build event driven stream processing applications with Wallaroo. One of our most frequent use-cases is analyzing trends, either from third-party external or your internal data.

Keep reading

Utilizing Elixir as a lightweight tool to store real-time metrics data

How we use Elixir to store and aggregate Wallaroo’s metrics for end-user consumption.

Keep reading