Machine Learning Uncertainty: What, Why and How

An introduction to the concept and importance of uncertainty in machine learning

Table of Contents

1. What is Uncertainty in Machine Learning?

Uncertainty is a fundamental aspect of machine learning. It refers to the degree of confidence or doubt that a machine learning model has about its predictions, outputs, or parameters. Uncertainty can arise from various sources, such as:

Data uncertainty: This is the uncertainty that comes from the data that is used to train and test the model. Data uncertainty can be caused by noise, outliers, missing values, measurement errors, or insufficient data.
Model uncertainty: This is the uncertainty that comes from the model itself. Model uncertainty can be caused by the choice of model architecture, hyperparameters, optimization algorithm, regularization, or approximation methods.
Task uncertainty: This is the uncertainty that comes from the task that the model is trying to solve. Task uncertainty can be caused by the complexity, ambiguity, or novelty of the task, or by the presence of multiple possible solutions or interpretations.

Why is uncertainty important for machine learning? How can we quantify and estimate uncertainty in machine learning? How can we communicate and visualize uncertainty in machine learning? These are some of the questions that we will explore in this blog. But first, let’s see some examples of uncertainty in machine learning in action.

2. Why is Uncertainty Important for Machine Learning?

Uncertainty is important for machine learning because it affects the performance, reliability, and usability of machine learning models. By understanding and quantifying uncertainty, we can:

Improve model evaluation: Uncertainty can help us measure how well a model fits the data, how confident it is about its predictions, and how generalizable it is to new data. Uncertainty can also help us compare different models and select the best one for a given task.
Enhance decision making: Uncertainty can help us make better decisions based on the model’s predictions, by taking into account the risks and trade-offs involved. Uncertainty can also help us handle situations where the model is uncertain, such as asking for human feedback, exploring new data, or updating the model.
Increase human-AI trust: Uncertainty can help us communicate and visualize the model’s predictions and outputs to the users, by showing them how certain or uncertain the model is, and why. Uncertainty can also help us elicit user preferences, expectations, and feedback, and adjust the model accordingly.

In the next sections, we will see how to quantify and estimate uncertainty in machine learning, and how to communicate and visualize uncertainty in machine learning. But first, let’s see some examples of how uncertainty can affect machine learning models and applications.

2.1. Uncertainty for Model Evaluation

One of the main applications of uncertainty in machine learning is to evaluate the performance of a model. By measuring the uncertainty of a model, we can assess how well it fits the data, how confident it is about its predictions, and how generalizable it is to new data. There are two types of uncertainty that are relevant for model evaluation: aleatoric uncertainty and epistemic uncertainty.

Aleatoric uncertainty is the uncertainty that comes from the inherent randomness or variability of the data. For example, if we are trying to predict the weather, there is always some uncertainty due to the chaotic nature of the atmospheric system. Aleatoric uncertainty cannot be reduced by collecting more data or changing the model, as it is a property of the data itself. Aleatoric uncertainty can be further divided into homoscedastic uncertainty, which is constant for all data points, and heteroscedastic uncertainty, which varies depending on the input.

Epistemic uncertainty is the uncertainty that comes from the lack of knowledge or information about the data or the model. For example, if we are trying to predict the outcome of a coin toss, there is some uncertainty due to the fact that we do not know the exact bias of the coin. Epistemic uncertainty can be reduced by collecting more data or changing the model, as it is a property of the model itself. Epistemic uncertainty can be further divided into model uncertainty, which is the uncertainty about the model parameters or architecture, and data uncertainty, which is the uncertainty about the data distribution or quality.

How can we measure and estimate aleatoric and epistemic uncertainty in machine learning? There are different methods and techniques that can be used, depending on the type of model and the type of task. In the next section, we will see some examples of how to quantify and estimate uncertainty in supervised learning, unsupervised learning, and reinforcement learning.

2.2. Uncertainty for Decision Making

Another important application of uncertainty in machine learning is to enhance decision making based on the model’s predictions. By taking into account the uncertainty of a model, we can make better decisions that reflect the risks and trade-offs involved. We can also handle situations where the model is uncertain, such as asking for human feedback, exploring new data, or updating the model. There are two types of uncertainty that are relevant for decision making: predictive uncertainty and active uncertainty.

Predictive uncertainty is the uncertainty that comes from the model’s predictions or outputs. For example, if we are trying to classify an image, there is some uncertainty about the probability of each class label. Predictive uncertainty can help us make decisions that are optimal, robust, or cautious, depending on the goal and the context. Predictive uncertainty can also help us identify outliers, anomalies, or adversarial examples, that might indicate errors or attacks.

Active uncertainty is the uncertainty that comes from the model’s actions or inputs. For example, if we are trying to learn from data, there is some uncertainty about which data points are the most informative or valuable. Active uncertainty can help us make decisions that are efficient, adaptive, or exploratory, depending on the goal and the context. Active uncertainty can also help us acquire new data, solicit human feedback, or update the model, to improve the model’s performance or reliability.

How can we measure and estimate predictive and active uncertainty in machine learning? There are different methods and techniques that can be used, depending on the type of model and the type of task. In the next section, we will see some examples of how to quantify and estimate uncertainty in supervised learning, unsupervised learning, and reinforcement learning.

3. How to Quantify and Estimate Uncertainty in Machine Learning?

In this section, we will see how to quantify and estimate uncertainty in machine learning, using different methods and techniques. We will focus on three types of machine learning tasks: supervised learning, unsupervised learning, and reinforcement learning. For each task, we will see how to measure and estimate both aleatoric and epistemic uncertainty, as well as predictive and active uncertainty.

Supervised learning is the task of learning a function that maps inputs to outputs, given a set of labeled data. For example, if we are trying to classify images of cats and dogs, we have a set of images as inputs and a set of labels as outputs. In supervised learning, we can quantify and estimate uncertainty in the following ways:

Aleatoric uncertainty: This is the uncertainty that comes from the inherent randomness or variability of the data. We can measure aleatoric uncertainty by using the entropy of the output distribution. Entropy is a measure of how uncertain or unpredictable a random variable is. The higher the entropy, the higher the aleatoric uncertainty. We can estimate aleatoric uncertainty by using probabilistic models that output a distribution over the possible outputs, rather than a single point estimate. For example, we can use a logistic regression model that outputs the probability of each class label, rather than a binary classification.
Epistemic uncertainty: This is the uncertainty that comes from the lack of knowledge or information about the data or the model. We can measure epistemic uncertainty by using the variance of the output distribution. Variance is a measure of how much the output changes when the input or the model changes. The higher the variance, the higher the epistemic uncertainty. We can estimate epistemic uncertainty by using Bayesian models that output a distribution over the possible model parameters, rather than a single point estimate. For example, we can use a Bayesian neural network that outputs the posterior distribution over the weights and biases, rather than a deterministic neural network.
Predictive uncertainty: This is the uncertainty that comes from the model’s predictions or outputs. We can measure predictive uncertainty by using the confidence interval of the output distribution. Confidence interval is a range of values that contains the true output with a certain probability. The wider the confidence interval, the higher the predictive uncertainty. We can estimate predictive uncertainty by using bootstrap methods that output a set of predictions based on resampled data or model parameters, rather than a single prediction. For example, we can use a bootstrap aggregating or bagging method that outputs the average and the standard deviation of the predictions from multiple models trained on different subsets of the data, rather than a single model trained on the whole data.
Active uncertainty: This is the uncertainty that comes from the model’s actions or inputs. We can measure active uncertainty by using the information gain of the input distribution. Information gain is a measure of how much the input reduces the uncertainty of the output. The higher the information gain, the higher the active uncertainty. We can estimate active uncertainty by using active learning methods that output a set of inputs that are the most informative or valuable for the model, rather than a random or fixed set of inputs. For example, we can use an uncertainty sampling method that outputs the inputs that have the highest entropy, variance, or confidence interval, rather than the inputs that have the lowest or the highest probability.

In the next subsection, we will see how to quantify and estimate uncertainty in unsupervised learning.

3.1. Uncertainty in Supervised Learning

In this subsection, we will see how to quantify and estimate uncertainty in supervised learning, using different methods and techniques. Supervised learning is the task of learning a function that maps inputs to outputs, given a set of labeled data. For example, if we are trying to classify images of cats and dogs, we have a set of images as inputs and a set of labels as outputs. In supervised learning, we can quantify and estimate uncertainty in the following ways:

Aleatoric uncertainty: This is the uncertainty that comes from the inherent randomness or variability of the data. We can measure aleatoric uncertainty by using the entropy of the output distribution. Entropy is a measure of how uncertain or unpredictable a random variable is. The higher the entropy, the higher the aleatoric uncertainty. We can estimate aleatoric uncertainty by using probabilistic models that output a distribution over the possible outputs, rather than a single point estimate. For example, we can use a logistic regression model that outputs the probability of each class label, rather than a binary classification.
Epistemic uncertainty: This is the uncertainty that comes from the lack of knowledge or information about the data or the model. We can measure epistemic uncertainty by using the variance of the output distribution. Variance is a measure of how much the output changes when the input or the model changes. The higher the variance, the higher the epistemic uncertainty. We can estimate epistemic uncertainty by using Bayesian models that output a distribution over the possible model parameters, rather than a single point estimate. For example, we can use a Bayesian neural network that outputs the posterior distribution over the weights and biases, rather than a deterministic neural network.
Predictive uncertainty: This is the uncertainty that comes from the model’s predictions or outputs. We can measure predictive uncertainty by using the confidence interval of the output distribution. Confidence interval is a range of values that contains the true output with a certain probability. The wider the confidence interval, the higher the predictive uncertainty. We can estimate predictive uncertainty by using bootstrap methods that output a set of predictions based on resampled data or model parameters, rather than a single prediction. For example, we can use a bootstrap aggregating or bagging method that outputs the average and the standard deviation of the predictions from multiple models trained on different subsets of the data, rather than a single model trained on the whole data.
Active uncertainty: This is the uncertainty that comes from the model’s actions or inputs. We can measure active uncertainty by using the information gain of the input distribution. Information gain is a measure of how much the input reduces the uncertainty of the output. The higher the information gain, the higher the active uncertainty. We can estimate active uncertainty by using active learning methods that output a set of inputs that are the most informative or valuable for the model, rather than a random or fixed set of inputs. For example, we can use an uncertainty sampling method that outputs the inputs that have the highest entropy, variance, or confidence interval, rather than the inputs that have the lowest or the highest probability.

In the next subsection, we will see how to quantify and estimate uncertainty in unsupervised learning.

3.2. Uncertainty in Unsupervised Learning

In this subsection, we will see how to quantify and estimate uncertainty in unsupervised learning, using different methods and techniques. Unsupervised learning is the task of learning a function that maps inputs to outputs, without a set of labeled data. For example, if we are trying to cluster images of cats and dogs, we have a set of images as inputs and no labels as outputs. In unsupervised learning, we can quantify and estimate uncertainty in the following ways:

Aleatoric uncertainty: This is the uncertainty that comes from the inherent randomness or variability of the data. We can measure aleatoric uncertainty by using the entropy of the output distribution. Entropy is a measure of how uncertain or unpredictable a random variable is. The higher the entropy, the higher the aleatoric uncertainty. We can estimate aleatoric uncertainty by using probabilistic models that output a distribution over the possible outputs, rather than a single point estimate. For example, we can use a Gaussian mixture model that outputs the probability of each cluster, rather than a hard clustering.
Epistemic uncertainty: This is the uncertainty that comes from the lack of knowledge or information about the data or the model. We can measure epistemic uncertainty by using the variance of the output distribution. Variance is a measure of how much the output changes when the input or the model changes. The higher the variance, the higher the epistemic uncertainty. We can estimate epistemic uncertainty by using Bayesian models that output a distribution over the possible model parameters, rather than a single point estimate. For example, we can use a Bayesian Gaussian mixture model that outputs the posterior distribution over the cluster means and covariances, rather than a fixed Gaussian mixture model.
Predictive uncertainty: This is the uncertainty that comes from the model’s predictions or outputs. We can measure predictive uncertainty by using the confidence interval of the output distribution. Confidence interval is a range of values that contains the true output with a certain probability. The wider the confidence interval, the higher the predictive uncertainty. We can estimate predictive uncertainty by using bootstrap methods that output a set of predictions based on resampled data or model parameters, rather than a single prediction. For example, we can use a bootstrap clustering method that outputs the average and the standard deviation of the cluster assignments from multiple models trained on different subsets of the data, rather than a single model trained on the whole data.
Active uncertainty: This is the uncertainty that comes from the model’s actions or inputs. We can measure active uncertainty by using the information gain of the input distribution. Information gain is a measure of how much the input reduces the uncertainty of the output. The higher the information gain, the higher the active uncertainty. We can estimate active uncertainty by using active learning methods that output a set of inputs that are the most informative or valuable for the model, rather than a random or fixed set of inputs. For example, we can use an uncertainty sampling method that outputs the inputs that have the highest entropy, variance, or confidence interval, rather than the inputs that have the lowest or the highest probability.

In the next subsection, we will see how to quantify and estimate uncertainty in reinforcement learning.

3.3. Uncertainty in Reinforcement Learning

In this subsection, we will see how to quantify and estimate uncertainty in reinforcement learning, using different methods and techniques. Reinforcement learning is the task of learning a function that maps states to actions, given a reward signal that indicates the quality of the actions. For example, if we are trying to play a video game, we have a set of states as inputs and a set of actions as outputs, and we receive a reward for each action. In reinforcement learning, we can quantify and estimate uncertainty in the following ways:

Aleatoric uncertainty: This is the uncertainty that comes from the inherent randomness or variability of the data. We can measure aleatoric uncertainty by using the entropy of the output distribution. Entropy is a measure of how uncertain or unpredictable a random variable is. The higher the entropy, the higher the aleatoric uncertainty. We can estimate aleatoric uncertainty by using probabilistic models that output a distribution over the possible actions, rather than a single point estimate. For example, we can use a stochastic policy that outputs the probability of each action, rather than a deterministic policy.
Epistemic uncertainty: This is the uncertainty that comes from the lack of knowledge or information about the data or the model. We can measure epistemic uncertainty by using the variance of the output distribution. Variance is a measure of how much the output changes when the input or the model changes. The higher the variance, the higher the epistemic uncertainty. We can estimate epistemic uncertainty by using Bayesian models that output a distribution over the possible model parameters, rather than a single point estimate. For example, we can use a Bayesian Q-network that outputs the posterior distribution over the Q-values, rather than a fixed Q-network.
Predictive uncertainty: This is the uncertainty that comes from the model’s predictions or outputs. We can measure predictive uncertainty by using the confidence interval of the output distribution. Confidence interval is a range of values that contains the true output with a certain probability. The wider the confidence interval, the higher the predictive uncertainty. We can estimate predictive uncertainty by using bootstrap methods that output a set of predictions based on resampled data or model parameters, rather than a single prediction. For example, we can use a bootstrap DQN method that outputs the average and the standard deviation of the Q-values from multiple Q-networks trained on different subsets of the data, rather than a single Q-network trained on the whole data.
Active uncertainty: This is the uncertainty that comes from the model’s actions or inputs. We can measure active uncertainty by using the information gain of the input distribution. Information gain is a measure of how much the input reduces the uncertainty of the output. The higher the information gain, the higher the active uncertainty. We can estimate active uncertainty by using active learning methods that output a set of inputs that are the most informative or valuable for the model, rather than a random or fixed set of inputs. For example, we can use an uncertainty-based exploration method that outputs the actions that have the highest entropy, variance, or confidence interval, rather than the actions that have the highest or the lowest Q-value.

In the next section, we will see how to communicate and visualize uncertainty in machine learning.

4. How to Communicate and Visualize Uncertainty in Machine Learning?

In this section, we will see how to communicate and visualize uncertainty in machine learning, using different methods and techniques. Communicating and visualizing uncertainty is important for machine learning because it affects the trust, understanding, and usability of machine learning models. By communicating and visualizing uncertainty, we can:

Inform the users: Communicating and visualizing uncertainty can help us inform the users about how certain or uncertain the model is, and why. This can help the users interpret the model’s predictions and outputs, and understand the sources and types of uncertainty involved.
Engage the users: Communicating and visualizing uncertainty can help us engage the users in the machine learning process, and elicit their preferences, expectations, and feedback. This can help the users interact with the model, and provide guidance, correction, or validation to the model.
Empower the users: Communicating and visualizing uncertainty can help us empower the users to make better decisions based on the model’s predictions and outputs, by taking into account the risks and trade-offs involved. This can help the users act on the model’s recommendations, and handle situations where the model is uncertain.

How can we communicate and visualize uncertainty in machine learning? There are different methods and techniques that can be used, depending on the type of model, the type of task, and the type of user. In the next subsections, we will see some examples of how to communicate and visualize uncertainty for human-AI interaction and for data visualization.

4.1. Uncertainty for Human-AI Interaction

In this subsection, we will see how to communicate and visualize uncertainty for human-AI interaction, using different methods and techniques. Human-AI interaction is the process of communication and collaboration between humans and AI systems, such as chatbots, virtual assistants, or recommender systems. For human-AI interaction, we can communicate and visualize uncertainty in the following ways:

Verbal communication: This is the communication of uncertainty using natural language, such as words, phrases, or sentences. Verbal communication can help us convey the level, type, and source of uncertainty, as well as the implications and actions for the user. For example, we can use phrases like “I am not sure”, “There is a high chance”, or “It depends on” to express uncertainty, and phrases like “You may want to”, “I suggest you”, or “Please confirm” to suggest actions. Verbal communication can be done through text or speech, depending on the mode of interaction.
Numerical communication: This is the communication of uncertainty using numbers, such as probabilities, percentages, or scores. Numerical communication can help us quantify the degree of uncertainty, and provide a precise and objective measure of confidence. For example, we can use numbers like “0.8”, “80%”, or “8 out of 10” to indicate the probability of an outcome, and numbers like “+/- 0.1”, “10% error”, or “95% confidence interval” to indicate the range of uncertainty. Numerical communication can be done through text or graphics, depending on the mode of presentation.
Graphical communication: This is the communication of uncertainty using graphics, such as icons, colors, or shapes. Graphical communication can help us visualize the distribution of uncertainty, and provide a intuitive and appealing representation of uncertainty. For example, we can use icons like “⚠️”, “❓”, or “🔮” to indicate the presence of uncertainty, and colors like “red”, “yellow”, or “green” to indicate the level of uncertainty. Graphical communication can be done through images or animations, depending on the mode of display.

How can we choose the best method and technique to communicate and visualize uncertainty for human-AI interaction? There are different factors that can influence the choice, such as the type of model, the type of task, the type of user, and the type of context. In the next subsection, we will see some examples of how to communicate and visualize uncertainty for data visualization.

4.2. Uncertainty for Data Visualization

In this subsection, we will see how to communicate and visualize uncertainty for data visualization, using different methods and techniques. Data visualization is the process of presenting and exploring data using graphical elements, such as charts, graphs, or maps. For data visualization, we can communicate and visualize uncertainty in the following ways:

Verbal communication: This is the communication of uncertainty using natural language, such as words, phrases, or sentences. Verbal communication can help us convey the level, type, and source of uncertainty, as well as the implications and actions for the user. For example, we can use phrases like “The data is incomplete”, “The results are uncertain”, or “The trend is unclear” to express uncertainty, and phrases like “You should be cautious”, “You should explore more”, or “You should update the data” to suggest actions. Verbal communication can be done through text or speech, depending on the mode of interaction.
Numerical communication: This is the communication of uncertainty using numbers, such as probabilities, percentages, or scores. Numerical communication can help us quantify the degree of uncertainty, and provide a precise and objective measure of confidence. For example, we can use numbers like “0.8”, “80%”, or “8 out of 10” to indicate the probability of an outcome, and numbers like “+/- 0.1”, “10% error”, or “95% confidence interval” to indicate the range of uncertainty. Numerical communication can be done through text or graphics, depending on the mode of presentation.
Graphical communication: This is the communication of uncertainty using graphics, such as icons, colors, or shapes. Graphical communication can help us visualize the distribution of uncertainty, and provide a intuitive and appealing representation of uncertainty. For example, we can use icons like “⚠️”, “❓”, or “🔮” to indicate the presence of uncertainty, and colors like “red”, “yellow”, or “green” to indicate the level of uncertainty. Graphical communication can be done through images or animations, depending on the mode of display.
Visual encoding: This is the communication of uncertainty using visual properties, such as size, position, or transparency. Visual encoding can help us integrate the uncertainty information with the data information, and provide a coherent and consistent view of uncertainty. For example, we can use size to indicate the magnitude of uncertainty, position to indicate the direction of uncertainty, or transparency to indicate the confidence of uncertainty. Visual encoding can be done through charts, graphs, or maps, depending on the type of data.

How can we choose the best method and technique to communicate and visualize uncertainty for data visualization? There are different factors that can influence the choice, such as the type of data, the type of task, the type of user, and the type of context. In the next section, we will conclude this blog and provide some future directions for uncertainty in machine learning.

5. Conclusion and Future Directions

In this blog, we have introduced the concept and importance of uncertainty in machine learning, and how to quantify, estimate, communicate, and visualize it. We have seen some examples of how uncertainty can affect machine learning models and applications, and how different methods and techniques can be used to measure and handle uncertainty. We have also seen some examples of how to communicate and visualize uncertainty for human-AI interaction and for data visualization.

Uncertainty in machine learning is a challenging and fascinating topic, with many open questions and opportunities for future research and development. Some of the possible future directions are:

Developing new methods and techniques for uncertainty quantification and estimation: There is a need for more efficient and accurate methods and techniques for quantifying and estimating uncertainty in machine learning, especially for complex and large-scale models and tasks. Some of the possible approaches are using deep probabilistic models, variational inference, Monte Carlo methods, or adversarial learning.
Improving the communication and visualization of uncertainty for different users and contexts: There is a need for more effective and user-friendly methods and techniques for communicating and visualizing uncertainty in machine learning, especially for different types of users and contexts. Some of the possible approaches are using natural language generation, multimodal interaction, interactive visualization, or explainable AI.
Integrating uncertainty into the machine learning pipeline and lifecycle: There is a need for more systematic and holistic methods and techniques for integrating uncertainty into the machine learning pipeline and lifecycle, from data collection and preprocessing, to model training and testing, to model deployment and maintenance. Some of the possible approaches are using uncertainty-aware data quality assessment, model selection and validation, model monitoring and updating, or model governance and ethics.

We hope that this blog has given you a comprehensive and accessible introduction to uncertainty in machine learning, and has inspired you to explore more about this topic. If you have any questions, comments, or feedback, please feel free to contact us. Thank you for reading!