Operationalizing Analytics with DataOps and ModelOps


Click to learn more about co-author Prashanth Southekal.

Click to learn more about co-author Harsh Vardhan.

If you
look to separate the hype in the market from the reality faced by organizations
in operationalizing analytics — you’ll notice a gap. In 2019, Gartner reported
that over 80 percent of analytics initiatives did not deliver business value, and,
according to McKinsey & Company, less than 20 percent of the companies had
achieved analytics at scale. [1]

In this article, we want to highlight how an organization can avoid missteps in their analytics journey and ensure their analytics initiatives deliver business value. How can they leverage analytics at scale to drive outcomes and avoid becoming part of the 72 percent whose AI initiatives are in the lab and not in production? [2]

In our opinion, to scale AI and analytics initiatives and embed these technologies within operational processes, organizations need to look at the entire analytics lifecycle and identify opportunities to automate aspects of their modeling process. Enter the practices of DevOps and its application within analytical modeling (DataOps, ModelOps, and DecisionOps).

Figure 1: Analytical Lifecycle
Image Source: SAS Institute Inc.

To level-set,
DevOps brings together development (Dev) and IT operations (Ops) into a set of organizational
practices intended to shorten the application development lifecycle while
resulting in improved quality, reduced risk/downtime, and increased feature set.
Similarly, DataOps, ModelOps, and DecisionOps focus on practices intended to
get the data ready, expedite model development from lab to production, and
deploy decision frameworks leveraging the models underneath. The goal is to
reduce development, prototyping, testing, and deployment cycles while ensuring
quality results and outcomes can be achieved in a timely manner.

Application of these practices within the
analytical lifecycle can benefit in the following three ways:

Break Organizational Silos:

These practices focus on collaboration across business stakeholders, data
engineers, data scientists, IT operations, and application development teams. The
continuous feedback loop helps to ensure that business outcomes are kept front
and center during the design and development process and that all stakeholders
are working towards the same goals, thereby improving the likelihood of success.

Strategically Sourced Data:

Over 80 percent of the work in analytics is getting the data ready for
analytical processing. DataOps reduces this effort with an automated,
process-oriented methodology
that spans the entire data lifecycle, ensuring that you can provide timely
access to high-quality data from diverse sources while maintaining stewardship
and governance requirements.

3. Scaled Analytical Responsiveness:
The automation built-in across DataOps, ModelOps, and DecisionOps enables the
organization to quickly respond to a decay in model performance, allowing
analytical insights to be embedded in more processes, thereby scaling solutions
and democratizing the analytical capabilities.

So, how
can organizations shorten the curve and realize business value associated with
these practices? In our experience, there are three keys to adopting DevOps
practices in your analytical lifecycle:

  • Establish
    a CI/CD (Continuous Improvement/Continuous Delivery) pipeline that automates
    the model versioning, scoring, challenger/champion tournaments, deployment, and
    testing. This ensures that changes to the model logic can be tested quickly,
    deployed easily, and reverted if needed without significant overhead.
  • Establish
    a process to monitor models in production, ensuring that a data pipeline
    automatically feeds both the model training and validation processes. Along
    with appropriate alerts, this can allow for automatic switching of models in
    production or trigger a human intervention if model performance falls below
    acceptable business thresholds.
  • Use
    A/B testing (or Canary deployments) to test alternate what-if scenarios to
    ensure that the business assumptions behind automated decisions are still

For analytics
initiatives to be successful, organizations need to transform themselves by
looking holistically at the business case, culture, processes, data, and
technologies that enable them to efficiently develop and deploy more integrated
advanced-analytics solutions more frequently. A well-defined ModelOps-DataOps
approach will enable an organization to have an iterative,
fail-fast, learn-fast, agile process that provides timely access to insights, resulting in
better, more informed decisions.

For additional information, you can read about how to get the most of your AI investment by operationalizing analytics.


[1] Southekal, Prashanth, Analytics Best Practices, Technics, 2020
[2] Leone, Mike, ”ESG Brief: Artificial Intelligence and Analytics Predictions for 2020,” 2019st

Credit: Source link