This morning I had the opportunity to chat with software engineers and data scientists at the AI Dev World Conference on a topic I just happen to be very passionate about — Engineering Operations. More specifically, on how we can use machine learning and AI with data we already from all the systems used to build software to eliminate the manual data gathering, meeting cycles, and ad hoc reporting that weighs down engineering teams’ ability to just build great products. 

Here are the highlights from my talk to save you 25 minutes 😏 

  • We’ve seen an incredible amount of digital transformation in just two months, but the way we build software hasn’t changed much in 20 years. 
  • Every other team in an organization has an operations function but not engineering teams. 
  • The business wants to understand how we progress towards our goals, and engineering just wants to build great stuff. An EngOps function gives both the business and engineering what they need. 
  • Engineering is ripe for applying AI to their workflow. We already have the data from the systems and processes used to build software — using AI to improve and automate is a no-brainer. 

I also introduced an Engineering Operations Solution Architecture to take us from:

Disorganized Engineering Workflow

to this:

Engineering Operations Solution Architecture

Here are two examples of how we are tackling this along with the libraries, data points, and predictions of the data science models for each:

  • Issue Forecasting - Predicting how long an issue will take to complete using scikit learn, NLTK, pandas
  • Pull Request Impact - Predicting the impact on other repos when a pull request is opened using PyTorch and spaCy.

You can download the slides from the presentation here.

Lastly, please join the conversation around engineering operations, data science, and data engineering in our Slack community

- Jeff

Get the data science behind high-performance teams.