An introduction into PgQ and SkyTools - efficient queue implementation on top of PostgreSQL and the tools to manage it. Description of PgQ, the different kinds of consumers, queue applications and their usage scenarios and how to tie it all together with SkyTools.
SkyTools is a batch processing framework, initially developed at Skype to simplify the management of distributed databases. It's most prominent features are PgQ queuing and londiste replication system. SkyTools has been released to open source since 2007, and is now approaching release 3.0.
This presentation makes an introduction to PgQ - producing and consuming events, different consumer types, batch processing. Solving common problems with pre-packaged applications - replication, archiving and data loading. Writing custom queue applications with SkyTools framework. Other programming interfaces - SQL, Java, PHP.
We will also discuss upcoming features of SkyTools 3: distributed queues, cooperative consuming and cascaded consumers.