Two recent papers have given more detail on Twitter’s distributed stream processing system Heron (the successor to Storm). The first paper, Twitter Heron: Towards Extensible Streaming Engines , details how Heron has evolved into a modular architecture that makes it extremely flexible. Adrian Colyer gives a good summary of the paper on his blog The Morning Paper. The main takeaway for me is the ease with which Heron can be augmented with new features.
This month I officially take the reins of the Python North East user group. Myself and Scott Walton are taking over from Rowan Hargreaves and Kieran Darcy’s fine stewardship. I’m looking forward to the opportunity to meet many Python people and encourage new members to try the best programming language there is! We are in the process of refreshing the website and have set up a new chat channel on Gitter.
Yesterday I took part in a hack-a-thon hosted by the Digital Catapult and the Royal Society. It was based on using data analysis and technology to try and measure and improve happiness. It was a great event, with lots of teams from all different disciplines and backgrounds. I was part of a team from Newcastle University, as well as members from Brighton and Central St. Martins. Our idea was to create an app that could find the happiest route between two points, avoiding high pollution areas and preferring green spaces.
I had an amazing time at DEBS 2016! It was great to meet other researchers in my field, discuss my ideas (and have them dissected) and make some new friends. I was extremely excited to meet Karthik Ramasamy who is the head of real time analytics at Twitter. He gave a very interesting talk about the evolution of Twitter’s new distributed stream processing engine (Heron) and how they use it in production.
I am going to my first academic conference! Distributed Event Based Systems (DEBS) in Irvine, California which takes place at the end of June this year. I will be presenting as part of their Doctoral Symposium, where I will explain the basis, current progress and future plan for my PhD research and get feedback from experts in the field! DEBS is one of the main sources of literature for my PhD research and I am incredibly excited to be able to attend.