Currently, Data Science, including this year of 2022, constitutes indispensable perspectives in doing business for its stakeholders. As center stage of discourse, this article emphasizes that statistical performance metrics aren’t enough to pick the right models to bring to market.

, President and Founder at Analytic Strategies & Consulting, LLC on April 19, 2022 in Data Science elaborates further the aformentioned indispensable perspectives.

Source and adapted from https://www.kdnuggets.com/2022/04/prioritizing-data-science-models-production.html

Very few businesses have unconstrained budgets for data science. The dollars for people, technology, analytic environments, and platforms are often much smaller than the corporate appetite for knowledge. In one company where I worked, the wish list for new analytic models would have required doubling our research staff, something that was clearly impossible.

Therese Gorski, the leader of several prioritization efforts, told me that the prioritization requirement is common in many product management organizations. It is often part of a larger process of vetting many ideas for offerings to bring to market. She said it is therefore useful to manage the prioritization process like any other investment in product development.

In this post I propose several criteria for companies to consider as they allocate scarce resources for data science work. My advice is to discuss these with people in every department involved in producing, marketing, selling, financing, supporting, or maintaining corporate offerings. Insights from company staff should come from all levels, not just leadership. This should be supplemented by insights from external stakeholders who would be affected by each model. Broad input will help maximize client utility, increasing sales and profits for them and your firm. Broad input will also increase the social value of your offerings.

A Dozen Criteria for Consideration

The criteria included in the following table have been inspired by Therese and colleagues at several companies, and by recent literature about attributes that make models useful and which pitfalls to avoid. Some of these criteria can be weighed prior to production; others address models that have already been produced.

Model attributes should be weighed before and after production because it may not become clear until late in the training process, or even afterward, whether any model is likely to be or remain useful. Moreover, new models typically compete for limited funding with models that have been sold for a long time, but which require periodic yet significant investments to maintain or update. Thus, it is important to consider how much every model is likely to contribute to business success. This requires weighing a variety of criteria and perspectives when ranking the utility of investing or continuing to invest in data science models.