Pipelines help split your architecture into distinct components. They're also a fantastic way to design software. By designing your architecture as a pipeline, you decouple all of your components and make sure that each part has a very specific responsibility.
This talk will look at how to start putting together a pipeline architecture and how to test them, as well as looking at a few case studies of how existing systems can be refactored to behave as a pipeline.
Michael's a fixer at DataSift. He works with various high volume data sources (including the Facebook firehose) in real time. Day to day, you can usually find him working with PHP or Go, with a bit of NodeJS or Python thrown in to keep things interesting.