Over the past year, OpenTable have been re-architecting their system from a monolithic architecture to move more towards microservices and small applications. As the infrastructure has changed, so to the logging infrastructure has had to change. Originally we had a logging solution where all logs where based in SQL Server. We then adopted the ELK stack. This allowed us to be able to scale more.
As the company moved into the cloud, we had to be able to scale even more. We decided that we would move to Apache Kafka.
This talk is all about the steps we went through and how we approached the work as well as the lessons learned. We are now in a position where we are able to elastically scale our logging infrastructure easily.
At the end of this talk, Paul will have demonstrated why Apache Kafka is a perfect addition to the ELK stack and how Apache Kafka allows us to add more resiliency and redundancy into our logging infrastructure