How to Empower Your Data Science Team to Drive a Successful Enterprise MLOps Capability
Everyone wants to pursue ML and AI. It’s one of the most popular domains within data and IT, and yet few know how to make it successful.
Contino’s research report, Data Maturity in the Public Cloud, reveals that “63% of organisations have one or more AI/ML projects in production on the cloud. However, while some organisations claim to have delivered dozens or even hundreds of machine learning products to production, most (over half) have delivered fewer than ten.” To boot, “76% [of C-suite executives] report they struggle with how to scale [beyond a pilot],” according to Accenture.
The sentiment that ML is important but leveraging it is tricky is widely expressed within our industry. And there are plenty of red flags to indicate when things aren’t going well.
In this blog, I’m going to focus on how to drive an MLOps capability through a cross-functional data science team. This requires consideration for the cultural divide between ‘science’ and ‘engineering’. I’ll reflect on the ML lifecycle, which influences the structure of effective data science teams. My ultimate goal in this post is to articulate a better way to design, deliver and deploy Artificial Intelligence (AI) and Machine Learning (ML) through the right team structure.
What Is MLOps?
Before diving in, if you need a quick intro to MLOps, there are many resources out there. May I recommend a previous blog I wrote about the state of The MLOps Community in 2020 as well as another article my colleague wrote Best Practices for MLOps and the Machine Learning Lifecycle.
Also, the diagram below is useful in setting the context of MLOps. It breaks down what is involved (code, data and models), the challenges it addresses and how it compares to DataOps and DevOps.
3 Pitfalls to Avoid When Building Data Science Teams
Ever heard someone say, “We have a team of 20 data scientists... No, no! We have 200 data scientists!” It’s obviously not true.
The ‘we got a big team’ mentality is usually a sign of false promise—throwing people at a problem instead of actually forming high-performance, long-lasting team structures. This is one of many potential pitfalls when developing not just a data science team but particularly one that embraces MLOps.
In our experience, the three most common pitfalls when forming data science teams include:
- The perceived need for a large team (i.e. “Neigh, I boast we have 2,000 data scientists!”)
- Only 1 out of 10 models makes it to production (i.e. “What design? Just throw that salami slice against the wall. It’ll stick!”)
- Persistence of the “that’s not my job” mentality (i.e.That thing you call “modularity” and “observability” - us Data Scientists will chuck it over to the Data Engineers and SREs. Which one does which? I don’t know - it’s not my job!” )
Pardon my sarcasm, but those bullet-pointed sentiments do really exist—more frequently than you think. Those sentiments make it all the more important to ask questions like:
- How does the team structure overcome the inherent divide?
- What is done to minimise wasted effort and forge tractable and measurable paths?
- Are skills and knowledge silos effectively being minimised?
If the right questions aren’t asked, and pitfalls are not addressed, time and money are wasted. You certainly can’t expect to form a good culture and practice, to generate products instead of pockets of results, to create a united effort to tackle a common problem.
These pitfalls are particularly damaging in a domain like data science that is known for its breadth and depth of complexity, spanning a number of different roles.
ML Journey Matrix: How to Unlock MLOps Capabilities
From Contino’s point of view, scaling ML beyond a pilot is not purely about tactically delivering technology but rather supporting transformational change through technology along with people and process. In this instance, we are talking about the transformation of data science teams to become cross-functional and unlock MLOps capabilities.
If you’re wondering, “Where does data science end and MLOps begin?”, it might be helpful to bear in mind that data science is a broad discipline. Moreover, there are tools and techniques that support MLOps, which should fundamentally be grounded in a practice and culture that is foundational to team formation.
To make that answer clearer, I like to talk about it in terms of the ML Journey Matrix.
In the diagram below, we can see data science teams do not exist without identifying and driving value (Path A), which MLOps techniques and tooling maturity (Path B) are dependent on. Path B is also dependent on the people and processes that support the continued maintenance and future development of the practice and culture of MLOps (Path C).
From this perspective, we can begin to intuitively understand the complexity of the ML journey and therefore the importance of a cross-functional data science team.
The Divide Between Data Scientists and Engineers
That which divides us , makes us grow… in complexity.
Data scientists and engineers are famously on the same team yet don’t play well together. While ML pipelines are the unifying asset, each archetypal role on the team brings its own values and vocabulary to the table.
ML pipelines consist of code, data and the ML model—all three elements require version control and continuous integration (CI), which is not performed in the same way for each. In short, there is a wall these role types must mutually surmount in order to function as a collective.
Yes, data scientists and engineers are crucial to the development of ML pipelines, and that is why we saw the advent of the ML Engineer over the last two to four years. That title’s rise to prominence, arguably more so than that of ‘data scientist’, shows just how important bridging this divide between scientist and engineer has become.
So, how do we steer the ship?
Data Maturity in the Public Cloud: Research Report 2021
We asked 272 IT decision-makers from around the globe about the state of data maturity in the public cloud in their organisation.
Download this report to get useful insights and the benchmark metrics you need to help drive a successful cloud data implementation.
The ML Lifecycle and How It Shapes the Team
ML pipelines do not drive value. Well designed products do. ML pipelines are only one part of the end result. That means the divide is actually much larger—less Lego set and more Legoland.
There needs to be emphasis on a broader coalition of roles and their association with the various aspects of the ML lifecycle. So, let’s consider the ML lifecycle (a version of a product lifecycle) and how the team is involved at each phase of that lifecycle.
I’m assuming you can appreciate this lifecycle without the granular details. What’s most important is alignment to these phases, so each one is serviced by the individuals with the right skills and knowledge at the right time.
This requires collaboration across the entirety of the lifecycle. To miss a phase, or have it partially serviced, can be problematic.
Example: Internet Bank Fails to Include the Right People When Building an ML Model
For example, a neo-bank was developing an ML model to support a Know Your Customer (KYC) automation feature. They went through use case selection as well as plan and build.
However, they did not include the KYC business owners—a small mistake with massive consequences.
In fact, it derailed their release and generated more work for the team as a whole. Had those domain experts been involved earlier, this pitfall could have been avoided by prioritising a different use case or addressing concerns before they morph into stubborn refusal.
Mind you, the solution isn’t to throw loads of people at the challenge. That generates competition, complacency and chaos. Adding as many people as possible to a team does not mean more problems solved—even at a lower cost per engineer that can sometimes be used as the justification to pile more individuals into a team.
In the medium- to long-term, more people are harder to manage. Even if the total cost of ownership (TCO) is the same, the likelihood of the cross-functional team’s ROI being greater is higher.
Why You Need a Cross-Functional Data Science Team
Success spans a great divide that should be filled with a diverse, cross-functional team. Each member of such a team is representative of a block of knowledge, which needs to be broadly shared and advocated for as well as expertly implemented. This maximises the team’s effort.
This combination of advocacy and expertise requires highly skilled individuals who own the challenges they are faced with, yet are flexible enough to understand the broader impact of their roles and mitigate that impact.
Example: FinTech Struggles to See MLOps Success Due to Divided Team
The inverse of this was painfully on display at a FinTech, which had a model that was causing a CPU-bound workload to fail at scaling up. Its latency was dreadful. However, the bigger problem was the ‘quants’ (aka data scientists) and the data engineers were both tossing work between each other—neither really spending the time to understand the other’s work.
This meant the quants didn’t understand their options to scale up and the engineers didn’t understand the limitation of the statistical model so they couldn’t really recommend the right option. The “that is not my job” fixation freezes us to the point where we cannot see solutions.
So, what are the roles in a cross-functional data science team?
7 Roles You Need For a Successful Cross-Functional Data Science Team
- Data engineers will have the most knowledge about databases, ETL processes, steaming pipelines and data manipulation in general, including how to build and deploy data tools.
- Data analysts (or junior data scientists) will have a strong knowledge of descriptive analytics and machine learning. They are adept at extracting insights.
- ML engineers (or senior data scientists with data engineering chops) understand the gap between data engineering and science, which allows them to create ML pipelines that integrate into CI/CD processes.
- Data science leaders are the decision-makers in their squad. They embody the best of each role above, likely serve as scrum master and know how to bridge the above and below roles.
- Statisticians help decision-makers come to conclusions safely beyond the data. They are the ones that serve as a safety valve for the squad.
- Domain experts can be wide-ranging from e-commerce marketing to senior wind turbine engineers, etc. They are the connection to “the business” that will benefit from the model.
- Software engineers (cloud, site reliability and security engineers included) are the “proverbial rug” that ties the room together and most needed after discovery and model development, when building and integrating models into broader applications.
These roles serve as archetypal roles. There are individuals who can wear multiple ‘hats’. However, it is imperative that no one wears too many hats. It’s not scalable; together we go further, unicorns do not exist (at least not without scotch tape and a toilet paper roll).
Perhaps, most importantly, because they operate in a single team, limits are placed on the notion of ‘us and them’; instead it is all ‘we’ with shared objective, shared history, shared workspace, shared culture and shared agile structure.
Ultimately this shared ownership structure dissolves barriers and empower individuals to act in the interests of the group.
The Data Science Leader
The data science leader role demands special attention as they serve as a conduit between three points:
- the core data capabilities of the team
- the core application capabilities of the team
- the business.
Data science leaders are the decision-makers in their squad. They embody the best of the data analyst/scientist, data engineer and ML engineer. They likely serve as scrum master and know how to bridge the core data roles with domain experts, statisticians, and software engineers.
Anecdotally, a data science leader runs this cross-functional data science squad of around six people. They solve challenging problems by leveraging design thinking to lay the foundation of valuable products and gain consensus. They use cutting-edge technology like transformers from HuggingFace and Google Cloud’s AutoML to prototype at speed. They work with key engineers to navigate technology choices like MLflow vs Kubeflow. They see their team’s ML models from design all the way through to production where they can be observed and where value creation can be truly validated.
Importantly, their team is a critical part of a high-performing data-driven culture, which continuously improves based on insights after each iteration and integrates advanced analytics at all levels—even at the core of the business.
This leadership role is critical and should be the direct beneficiaries of the C-suite, business champion or product owner’s vocal support for digital efforts, consistent feedback, continual empowerment and direction.
The data science leader is the single role that serves as the transformational linchpin between becoming a laggard or leader in ML and AI. Research, as illustrated below, highlights the broad actions the C-Suite must do to empower and enable the data science leader to forge a way forward and avoid stagnation.
In Summary
So, how do we steer the MLOps ship? We implement a cross-functional data science team with a Data Science Leader at the helm. We make sure that the team is focused on ownership and sharing. The right way to steer it is to hire a lean and diverse combination of skills and empower who’s at the helm. We focus on MLOps as a culture and practice—a bullseye encircled by a union of technology, people and process.
Where is your team on its journey to empowering your cross-functional data science team? Not so sure? Connect with us.
Like what I write? You can find more on Medium or follow me on LinkedIn.
