Over the past year at Pusher, we have been using Haskell to write a new message bus for our core pubsub system.
Customers currently send more than 50,000 messages per second through our existing system and we aim to keep latency below 50ms. This means we have hard requirements on the performance of the new system. On top of this we want to provide durability guarantees in the face of failovers and client disconnections.
In this talk I will describe our experience with using Haskell for this problem.
On the one hand we found Haskell's type system and testing capabilities excellent for writing reliable production software. As an example of this you can read about how we used QuickCheck to test our raft implementation here.
On the other hand we encountered challenges when attempting to optimise for high throughput and low latency. The most fundamental of these problems was the length of GC pauses when there was a large working set in memory. You can read more about this in James Fisher's blog post here.