Despite its immense potential and widespread hype, machine learning and artificial intelligence analytics do not always live up to expectations when deployed in business. This is often due to factors such as irrelevant or misaligned use cases, poor data governance and a lack of synergy between business and data science teams, according to the Knowledge Integration Dynamics (KID) data science experts.

KID data scientist Janco van Niekerk said: “It is a common misconception that you can simply hire a data scientist armed with ML tools and expect them to deliver value with no realistic objectives and additional guidance. This happens often in less mature data organisations.”

A solid use case for machine learning

“For an ML project to deliver the expected value, it is critical to have input from business from the start. During the ML use case ideation process, it is preferable to identify the problem to be solved instead of proposing a solution in the initial stages. This allows the data scientist to consider alternatives and leverage different algorithms, approaches and objective functions to reach the goal,” Van Niekerk says.

DataRobot KID partnership manager Markus Top outlined questions organisations should consider when assessing what makes a good ML use case. “Organisations need to consider what they want to predict, and whether they have any historical data to work from. They need to understand what changes would be made by having a prediction, whether any action could be taken based on the prediction, and what impact this would have.”

KID data scientist Daniel Charters said: “Although often overlooked, good data governance is the foundation of an ML project, so ensuring consistent data governance is essential to the integrity of ML projects. Inconsistent or poor-quality data will not generate the desired value.”

Bridging the business – data science gap

Identifying the problem is just the first step, Van Niekerk said. “It is also important that stakeholders know and understand what type of problem can be solved with ML. It will not be fruitful to approach machine learning specialists with a problem which requires only a simple/standard software process to be built, such as a data Extract, Transform and Load (ETL) pipeline.”

ML application has a different objective to that of pipelining and executes complex learning logic models not only to support analytics, but also AI.

“Therefore, it is important for business to have knowledge of basic ML concepts such as the difference between supervised and unsupervised learning, regression, classification, model training, model evaluation and the difference between target and predictor variables.”

Both sides need to have an understanding of the business domain and data science, Van Niekerk said. “ML projects are not one size fits all undertakings and require many different skill sets.”

Top said: “Effective communication is essential. Organisations must ensure that both teams understand each other’s goals, processes and terminologies. They should use regular meetings, workshops and documentation to facilitate this understanding. Defining key performance indicators (KPIs) helps measure the success of collaborative projects and provides a clear way to assess the impact of data science on business outcomes.”

“Joint training sessions can be organised where business experts provide insights into domain-specific knowledge, and data scientists explain the technical aspects,” Charters said.

DataRobot

Top added that ML projects are supported by fostering a data-driven and data-fluent culture within the organisation.

DataRobot, a leading AI platform, helps bridge this gap through a few of its capabilities.

“The ease of use means you do not have to have a deep technical knowledge of the different models built. The models and results derived are easily explained and second to none in DataRobot, with full documentation for every aspect, so a non-technical person can have peace of mind and a full understanding of the models and whole process.

“And lastly, the new generative AI capabilities of the platform adds a new dimension to model interpretation, as well as code generation for further model customisation,” Charters said.

Moreover, the KID team observes that DataRobot’s rapid and consistent platform enhancements solidifies their status as an industry leader.

Building trust and achieving value

Charters says that while complex models may offer better accuracy, they can be more difficult for the business side to understand and trust.

“Choosing simpler, more interpretable models can sometimes be more effective in driving actionable insights. Interpretability tools can be used to provide insights into what a complex model is doing, thus bridging the gap between accuracy and interpretability.”

The KID team agreed that defining and sharing responsibilities helps ensure maximum ROI. Top said: “Teams should align their objectives with the overall business goals. Data science projects should directly contribute to solving business problems and achieving strategic outcomes.”

And Charters added: “Both teams should know who’s doing what for data collection, cleaning, model development, implementation and feedback gathering to avoid redundancy and to streamline the processes. Constant alignment is needed between technical and business teams to make sure everyone’s on the same page.”

Data science projects should directly contribute to solving business problems and achieving strategic outcomes

To ensure that the end-users have trust in the models, Top recommends that organisations create mechanisms for continuous feedback. “Business teams should provide feedback on the usefulness of data science insights, and data scientists should iterate on their models based on this feedback,” he said. “Organisations should select and implement the right tools and technology to facilitate collaboration. Consider platforms that do everything you need to build, deploy, manage and govern generative and predictive AI models.”

KID’s standards of ensuring insight and prediction accuracy by actively managing data drift when data is prepared and curated – and using truth verification techniques – eg triangulation, corroboration or substantiation, is crucial for enabling trust.

According to Charters: “End user confidence can also be built by ensuring that models are robust and tested against a variety of scenarios, and regularly updating the models based on new data and feedback. Here, again, DataRobot has an edge over traditional ML as it offers full documentation for every step of the process; there are no ‘black boxes’ in the process.”