Most organizations still find data to be the major source of frustration for their digital transformation journeys.
The Gist
- Growing up, digitally. Business leaders will be digitally transforming their companies throughout their careers, and digital and AI transformation relies on data, which is constantly changing.
- Oh, data. Data remains a major source of frustration for most organizations on their digital transformation journey, with 70% of AI-based solution development involving data wrangling and harmonization.
- Where's my data? 80% of business users face difficulties accessing the data they need, indicating a lack of a modern data stack, data integration challenges, and data quality and security concerns that need to be addressed to achieve successful digital transformation.
In my recent review of Rewired by McKinsey consultants Eric Lamarre, Kate Smaje and Rodney Zemmel, the authors suggested that business leaders will be digitally transforming their companies for the rest of their careers. Even more significant, the authors claimed digital and AI transformation — which is built on data — is not a one and done program because digital capabilities are constantly changing.
Data Is a Major Source of Frustration
Without question, the vanguard — the digital leaders — will outperform others within their marketplaces. This will involve continually improving marketing capabilities around personalization analytics, digital marketing campaigns and omnichannel experience.
Yet the authors of Rewired say most organizations still find data to be the major source of frustration for their digital transformation journeys. Even more startling, their research says that 70% of the development effort for AI-based solutions involves wrangling and harmonizing data.
So where are organizations really at today in delivering the fuel of digital transformation —data? And how far is the journey forward for most organizations?
Related Article: Balancing Customer Data Privacy and Usefulness
What Is the State of Data Analytics and Digital Transformation Today?
A new study sponsored by Hitachi Vantara provides some interesting insights into the state of data. Honestly, the results should be of concern to CIOs, CMOs and CEOs.
Just Accessing Data Is a Problem
The study found that 80% of business users simply can’t access the data they need to make a decision, create a report or build a transformative data model. Interestingly, 25% of IT leaders say their users don’t know the data they need already exists.
The Hitachi study goes on to suggest this is because data is hidden away in organizations behind layers of technology or siloed systems or buried under mountains of useless data. The latter should represent the business case for not simply re-hosting, the so called "lift and shift," data into the cloud.
Yet something needs to be done, too, about legacy environments as well, which the Hitachi Vantara study suggests are often a “Frankenstein monster of cobbled together systems.”
The Need for a Modern Data Stack
However, I would contend that Bob Muglia, a former Microsoft executive, former CEO of Snowflake and a venture investor is correct when he suggested a "modern data stack" that helps organizations manage and analyze their growing data. In contrast to past waves of technology, the modern data stack is not provided by one vendor; it is instead an ecosystem of technologies provided by many companies.
A key element, found in the seminal article on this topic by Matt Bornstein, Jennifer Li and Martin Casado at Andreessen Horowitz, is data discovery via a data catalog.
There must be still room for growth in this market category or for fixing the process of granting data access if 80% of users can’t access the data they need. Supportive of this idea is RightData’s announcement on the July 20 that it is entering the congested data catalog market.
Challenges for Integrating Data Analytics
A problem for those that can discover data is that 34% say they need a third party help to integrate their data together. This is scary because a key element of an AI strategy is putting together diverse datasets. It is also scary that these companies lack an effective "digital bench."
Lamarre, Smaje and Zemmel believe that going digital starts by putting together an internal talent bench. They are clear being digital can’t be outsourced. Success comes through an organization’s own talent bench.
Lack of Trust in Data Analytics Quality
And for those that get through this gate, 23% say users, according to the Hitachi Vantara study, don’t trust their data quality. This includes issues such as data lineage, knowing where the data comes from. This is yet again another issue considered by the Andreessen Horowitz white paper.
In a modern data stack, data quality is solved by a mix of data discovery plus lineage, data governance and data observability.
Data Analytics Security Is A Concern
A final component of the study deals with the defensive side of data trust. The report finds that 41% say data security is a concern and 30% admit their regulatory audits could not be substantiated. For organizations that are in industries of compliance, the financial impact of this could be significant.
This is especially the case for organizations that need to worry about the consequences of the failure to comply. In the modern data stack, this is addressed via sensitive data discovery, data governance, and entitlements and security.
Clearly, it is still early, and most organizations are relatively early in the process of putting in place a modern data stack. And for organizations that are well along the journey, it can still take time to see the results from a data maturation program.
Related Article: 10 Potential Data Privacy Pitfalls for Marketers
Data Maturity Research From MIT-CISR
A few years ago, Stephanie Woerner and Jeanne Ross at MIT-CISR investigated the process of data maturity. They found that only 28% of organizations have their data acts together. Their research showed that 51% of organizations have their data in silos and 21% have their data managed with bailing wire and duct tape. This is like the "Frankenstein monster" suggested in the Hitachi Vantara study. In fact, in the book “Enterprise Architecture as Strategy,” Ross and her co-authors say, “company data, one of its most important assets, is patchy, error-prone, and not up to date.”
The Problem of Immature Data Analytics Processes
The results of the Hitachi Vantara study, which to be honest covers much more ground, confirm that many organizations are still living with immature data processes. But there are growing consequences today for data immaturity. The research of MIT-CISR laid out in the book “Future Ready” finds that firms that get to the other side — industrialize their data and fix their customer experience — have “revenue growth of 17.3 percentage points and a net margin of 14 percentage points about their industry average.”
Parting Words on Data Analytics and Digital Transformation
Put simply, organizations that want to be digital leaders need to start fixing their data immaturity so data analytics and digital transformation are not only possible but prioritized! This means that it is time for CMOs to stand up and be counted. CIOs and CDOs need your guidance, counsel and support in putting in place transformative customer experience and the data that enables it.
According to former CIO and active board member Wayne Sadin, “Data silos slow down decision making at best ('which report is right?') and lead to wrong decisions at worst ('I never saw the data about slowing sales when I was placing big component orders.').” It is time to fix data and win!
