Artificial Intelligence in IT: best practices

Arguably the most important part in any AI-project is to understand its corresponding business case. Particularly, before the development and deployment of an AI-solution, one should consider how an AI-model can be applied to create business value for an organization. For instance, think about how the output of a machine learning model can be used for further steps in solving the business case, or consider whether the deployment of an AI solution outweighs the cost of maintaining it. The answers to these kinds of questions form the main motivation behind an AI-project. The more clear these answers are, the easier it becomes to filter some models and algorithms from your eventual AI-solution.

To give the aforementioned considerations more structure, you can gather requirements from the business case. These business requirements could describe specific objects needed for the AI-solution, such as datasets, models and evaluation metrics. On the other hand, the business requirements can formulate some more general characteristics of the AI-solution. For example, you may want to emphasize the accuracy of the model, rather than its efficiency. These business requirements partly define the necessary steps in an AI-project that are necessary to solve the business case. Furthermore, the requirements help in keeping the goal of an AI project in sight, as they can be used in re-evaluating the AI solution during development time.

Now that there is a clear motivation for the AI project, it is time to start developing the AI solution. Before developing the AI model itself, the dataset relevant to the business case needs to be prepared. Data preparation entails cleaning the dataset from errors and biases that might negatively affect a model’s performance. It also means that the dataset is transformed such that it is more suitable to train the AI-model. In general, the goal of preparing the data is to ensure its quality. The necessary steps to prepare the dataset usually depend on the underlying patterns and relationships that are already present. Therefore, it is crucial to perform a data analysis to thoroughly understand the dataset. A data analysis can be simple by calculating some basic statistics and providing a quick overview of the distribution of the values of your dataset. On the other hand, data analysis can be more complex by making visualizations and highlighting certain patterns and relationships from the dataset. All of these insights not only determine the data preparation steps but also additional datasets that may be needed to properly solve the business case. This makes it a cyclic process between data preparation and data analysis, as more data means more patterns and relationships to uncover. This is the stage in any AI project that requires the most effort, as the higher the quality of the data, the better the performance of the AI model.

Once the dataset is of sufficient quality, it can then be used to train the AI model. By now, it should be possible to determine the type of task that the AI model needs to perform (e.g. regression, classification, clustering) and, by extension, the output that it needs to predict. Once these are known, the choice of the eventual AI model has become much more limited. With this limited set of models, it is recommended to start with the simplest possible machine learning model, and then gradually increase its complexity if necessary. There are two reasons for this. First, simple models usually contain only a few parameters, meaning that they can be swiftly trained on new or updated datasets. This makes simple models considerably easy to maintain. Second, simple machine learning models are easier to formulate mathematically, making them easier to explain to others. Together with the aforementioned insights from your data analysis, simple machine learning models allow you to explain more easily how the AI model has generated its output from the data. The amount in which the output of a model can be explained from its input (i.e. the dataset) is called the model’s transparency. This is important to consider, as a lack of transparency is one of the main reasons why AI solutions are currently not so prevalent in IT. The topic of transparency is interesting in itself, especially with the rise of deep learning models. However, it warrants a separate blog and, as such, is beyond the scope of this blog.

Now that we have fitted an AI-model to our dataset, the next step is to evaluate its performance, both in terms of accuracy and in its behavior. Particularly, it is essential to examine whether the relationships estimated by the AI-model between the input and the output are correct and consistent with respect to the business case. To achieve this, it is highly recommended to evaluate the AI-model both quantitatively and qualitatively. On the one hand, a quantitative approach gives you a quick overview of the model’s accuracy by calculating metric values, measuring the error between the predicted output and the actual target values. It is recommended to calculate at least three different metric values, as each metric measures the performance of the model differently (e.g. some metrics penalize large errors more harshly than smaller errors). On the other hand, a qualitative approach provides insights into the model’s relationships and overall behaviour. This could be done by visualizing the model’s results to get a general overview of its relationships. Another approach is to use the AI-model on individual samples and to inspect the consistency of the results.

After assessing that the AI-model has sufficiently solved the business case, it is time to deploy the AI-solution. Since the world around us changes, so does our dataset. This means that over time the performance of the deployed AI-model will decrease, requiring the model to be rebuilt. As such, it is important to regularly monitor the results of the AI-solution over time and retrain the underlying AI-model should it be necessary. You do not need to go through all the steps described above since this is simply fine-tuning your model to the current situation instead of building a model from scratch.

The general tips described in this blog only form the tip of the iceberg when realizing an AI-solution. However, it shows us that each stage in the development and maintenance of AI-solutions depend on one another. As such, smooth communication between people (i.e. data engineers, data scientists, machine learning engineers) at each level of the development and maintenance phase is essential in order to successfully release an AI-solution in the market. That way, AI and machine learning becomes a smart way to optimize business processes, increasing the productivity of an organization while reducing costs at the same time. In other words, AI and machine learning become a smart set of tools that allows an organization to be efficient and agile, instead of a replacement of redundant employees. At Expertum, we realize this fact and are eager to help you to make this scenario a reality in your organization. Enrich your enterprise with the power of AI!