Introduction
In this post, I’ll present a hackathon we did in our team which revolved around our Retrospective meetings. A hackathon provides an opportunity for the team to dive into a certain subject. We allocate a short period of time (a day or two usually) and use all the tools we can think of to solve a problem. Sort of an accelerated PoC. We’ve been keeping our Retrospective data for a while, we identified the opportunity to get creative around it and learn a thing or two regarding the way we operate as a team.
What is a Retrospective?
Like many other teams in the tech industry, our team also practices the Agile methodology. We have been doing so practically since the team was formed. One of the Agile meetings we hold is the Retrospective (more formally known as the Sprint Retrospective). The main idea is to learn from past experiences so that we can do better next time. In the meeting, we talk about the past two weeks, and team members raise cases worth discussing. There can be times when something didn’t work well, like the way one (or more) of the team members handled a task, some need that they are missing in their work (e.g. a comparison tool between successive runs), or a lack of communication with a colleague from another team in a joint task. It can also be an example of things that actually did work well, where we can learn from that good experience, or just enjoy patting our backs. We have been holding the Retro every two weeks, since we started, around two years ago.
What can we do with all the Retro data?
We are an NLP team, which implies a lot of things. Amongst other things, we like data, especially in the format of free text. We like to draw conclusions from that data, and we enjoy applying a diverse set of tools to draw them. We have been saving these Retro tickets for almost two years, without any actual purpose, wondering if we’re ever going to look at them.
And then, it clicked
What we have here at hand, is a nice, not-so-little, tagged data set, which we can play with. For every Retro ticket that we raised in the past two years, we saved the following information: title, description, when was it raised, who wanted to talk about it (we don’t get to talk about all the issues we raised), and was it a “to improve” or “to preserve” kind of issue. Those of you readers that are NLP enthusiasts can already start rolling up the potential of possible work using these data. We decided to hold an NLP Retro Hackathon! We pitched a few ideas of what we can do with the data, paired up within the team (randomly), and cleared one day to go crazy on Retro data. The primary idea was of course to have fun, doing what we enjoy doing, but on an entirely different data set, that is very much connected to our work, but from a different perspective. If we can learn something about our work process along the way — even better.
Some of the ideas we thought of
Predict the person assigned to the ticket
Find topics in retro tickets
Find trends in topics
See if topics resurface
Classify tickets into the “to improve” or “to preserve” buckets
Automatically generate retro tickets
The Retro Hackathon
We arrived in the early morning of the Hackathon day and started working. Each team took a different direction, and on the following day, we presented our results. Some interesting conclusions emerged. We discovered that when the team first started, we had a lot of debugging needs related-tickets, and that most of them were resolved. This means the Retros were actually helpful! We saw that a lot of the action items were on the team leader (me) and that most of them started with a verb: take (with another colleague), solve, fix, set (a meeting). We saw that there’s a relatively high number of tickets in the “to improve” bucket that include interaction with another team — a known issue, especially in the interface between a Data Science team and an Engineering team. We managed to build an accurate classifier between the “to improve” and “to preserve” items. We showed a promising direction in being able to summarize long tickets into a shorter version using the title and the description. We discovered that certain people tend to vote together on tickets (they want to talk about the same issues).
Some next-level tasks we tried, revolve around text generation. We tried to generate the ticket description using the title (and vice versa). This is a very interesting task but definitely needs more than one day to get solid results. It did, however, produce some funny lengthy descriptions. And of course, we tried to build a retro-ticket-generator, that could do the work for us. We used quite a few tools in the hackathon — LDA for topic extraction, different transformer-based models (BERT and the likes and T5) for the classification and summarization task, and more.
When the hackathon day was over (and after each of us consumed a considerable amount of pastries along the day), each pair summed up their results in a presentable manner. The format was free — some of us used slides, others used their Jupyter notebooks and interesting csvs were also a valid option. The following day we presented our work to the entire company. We enjoyed presenting it, people were keen on hearing about it, and the overall process generated an interesting discussion around Retro, hackathon, and fun activities we can hold within the teams. And of course — it made everyone want to do NLP!
Summary
What we did is take one of our routine activities in the team, made something fun out of it, kept it attached to our work using the data and the very nature of that activity, and also tried to learn something along the way. Retrospectives have served us well over the course of time, and it was great seeing action items come to life following them. The insights that emerged during the Retro hackathon reassured us that we’re on a good path. Everyone who worked on it had a lot of fun, and the rest of our colleagues enjoyed our presentation.