This morning I had the opportunity to chat with software engineers and data scientists at the AI Dev World Conference on a topic I just happen to be v...
Engineering managers have the responsibility to set their engineers up for success. This success includes helping engineers be productive, efficient, effective, work as a team, and grow their careers. Managers often directly influence how successful individual developers are — because managers are responsible for assigning developers to projects that fit their strengths and interests, while ensuring that team dynamics work well for everyone.
When I think about setting my team up for success, here are the areas I focus on:
Access to the right metrics makes my job easier. What I mean by “right” metrics is meaningful metrics: outliers, hotspots, and metrics related to the aforementioned areas . Knowing there are 1,000 issues in the backlog isn’t interesting nor that helpful, but being able to zero in on the 10 out of 1000 issues that are causing some churn or could affect some mission-critical items, is helpful.
Before I talk about a few engineering metrics I use to help engineering teams, I do want to make one important point: Using any of these metrics in isolation, especially for performance comparisons, is a fool’s errand. To quote Goodhart’s Law:
When a measure becomes a target, it ceases to be a good measure.
and more descriptive still:
Any observed statistical regularity will tend to collapse once pressure is placed upon it for control purposes.
That’s why I emphasize: These metrics are to help engineering teams according to the areas I outlined above. There’s a big difference between using metrics to compare performance and using metrics to help engineers get better.
So with that as our foundation, here are some useful metrics I look at to ensure we are on pace to achieve our goals:
Engineering managers want to know that work is moving forward. At first glance, cycle time is about how long it takes us to get things done, but there’s more to cycle time than that. It also helps us understand if stories and issues are being scoped correctly and if teams are able to understand the requirements for feature requests.
Trends in cycle time can also point to larger issues that might indicate some tooling or architectural problems because of debt or some work process that is off. Cycle time is often the canary in the coal mine, providing the first hint that something in the organization isn’t working the way that it should or there is some outside roadblock that needs to be addressed.
High rework rates are often the cause of cycle time issues, so being able to quickly see which issues have a high rework rate can help keep your cycle time down. Often, the rework rate is a sign that issues aren’t being properly understood, requirements aren’t clear, or downstream steps like QA or creative review have different expectations. Rework outliers help identify where some of the bottlenecks reside or where we need more communication, teamwork, and mentoring.
These two metrics are interesting, especially together. PR review rate can be an indicator of several things: how much knowledge is being spread among team members, how stable your code might be, who’s a cowboy on the team and ignores any feedback from peers, etc. Merge rate is similar and is also an indicator of the speed your code is shipped to your customers. Again, by themselves, the metrics aren’t as helpful, but when you can use rework rate and cycle time together, it’s much more useful.
At a previous company, there was a situation where my team’s productivity seemed to be tanking — and it turned out it was. It was tanking because they were being increasingly distracted and pulled into meetings with other teams. Understanding how much time teams and individuals are spending on which projects or in meetings is key to being a good steward of engineers.
Workload balance metrics help me see if the team is spending their time on meaningful work. If not, these metrics help me advocate on behalf of my team — even if that means talking to other engineering managers and finding a better way to utilize their time while still helping everyone get work done.
This is also important to look at for individuals, too. When a top performer seems to be struggling with declining productivity, it’s easy to jump to conclusions and think that this person might be burning out. But it’s just as likely that word got around that he or she is great, and now other teams are constantly asking for their input. and for them to attend their meetings.
At Pinpoint, we believe metrics are important, but we also want to encourage individuals and managers to consider metrics as the beginning, not the end, of the conversation. They provide a shortcut or an early warning that allows us to fix problems earlier or find interesting outliers sooner.
Director of Engineering
Automating data science is hard, and we do a lot of it.
As part of our latest release, our Agent underwent a complete transformation in order to simplify the installation of in...