Plea from HR: how can we help engineers shine?

A couple of weeks ago, at the invitation of the Gartner Research Board, I spoke to a group of human resources executives from some of the biggest companies in America. The topic, in essence: For software engineers, how do we know whether the money we spend on hiring, training and mentoring is helping?

It’s strange this question is so hard to answer. What, exactly, constitutes good engineering performance, and whether it can be measured at all—these are vague, sometimes contentious topics for people who’ve grown up in engineering. Imagine what it’s like for people who haven’t. It makes software engineering an odd outlier in the business: an incredibly important function whose performance is largely a mystery.

In a report on performance measurement, McKinsey & Company had this to say:

"In the best performance-management systems, the entire organization operates from a single, verified version of the truth, and all employees understand both the organization’s overall performance and how they contributed to it."

A single, verified version of the truth. All employees understand both the organization’s overall performance and how they contributed to it. Yes, and yes. But what we’ve had in software engineering is almost the anti-McKinsey: a “system” based on intuition, manually compiled and/or spot data, with little to connect a developer’s personal performance to the business impact.

Not great.

Hard questions, meet hard data

In talking with the HR leaders, this is where we started: What are the engineering performance questions we’d like to be able to answer? Never mind what we think is or isn’t measurable. Assume an ideal world—what would we like to know?

Here’s what we came up with:

Who are our top performers?
Who might be in need of mentoring or training?
What percentage of our engineering team is at risk to leave?
How well are we using our current engineering resources?
What are our performance trends over time? Are we getting better?
Which of our delivery locations returns the most for money spent?
How long does it take for a new hire to become productive?

A pretty good list. As it happens, the science exists to answer these questions. This is because engineering actually has a rich set of hard data, just begging to be used.

Take the question of predicting what percent of staff might be at risk to leave. One obvious and powerful signal for understanding developer engagement is their code commit activity:

Developer Code Commit Activity

In this anonymized example, we see Natalie’s commit activity fell off a cliff in November, and the trend only got worse in December. Even accounting for a slowdown through the holidays (which we’ve quantified), something is clearly off.

Now, imagine we compare this signal trend to what we know about the risk factors for churn—something we’ve uncovered by using data science to find the correlative signal activity of employees who’ve already left. We’re now in a position to understand the probability of Natalie leaving. More importantly, we’re positioned to know how to help.

For example, maybe the data show that Natalie’s been working on a team with below average Workload Balance, a signal we derive to understand how evenly work is distributed across people. (This also goes straight to one of the other questions above: How well are we using our resources?) Maybe this is a case of overwork: Natalie’s tired of being the anchor leg for her team.

Team Workload Balance

How we help Natalie may depend on what the data says about her probability of leaving. If it’s high, some kind of work or team rotation might be in order. A new team, new people. If it’s moderate, there may be time yet to work with her and her manager on a plan for better balance. (A conversation that needs to happen with her manager in any case.)

Data activates the discussion

To state the obvious: this focus on data shouldn’t be read as some sort of computerized replacement for actually talking to people about how they feel. Just the opposite. In this example, the data exist to signal that some special attention is in order—not only that managers need to engage, but how we can best engage.

Compared to most other parts of the business, engineering may’ve lagged where performance measurement is concerned. But we have an in-built advantage to change that: most of what we do generates a data trail. Just as top athletes have done with advanced analytics, we can use that data to better understand where we’re great, where we’re falling short, and what actions will turn those shortcomings into strengths.

Pinpoint Engineering

The TL;DR from my session on AI and EngOps at AIDe...