AI Principles: Representing Uncertainty with Probabilities

Artificial Intelligence systems often deal with incomplete or uncertain information. Probabilities provide a mathematical framework to represent and reason about this uncertainty, enabling more robust decision-making.

Core Principles

Uncertainty is inherent in many real-world AI problems.
Probability theory offers a principled way to model and manage uncertainty.
Key concepts include possible worlds, probability distributions, conditional probability, and independence.
Bayes' Rule is fundamental for updating beliefs based on new evidence.
Bayesian Networks provide a graphical model for representing dependencies between variables.
Hidden Markov Models (HMMs) are used for systems with unobserved states and observed events over time.
Inference algorithms (e.g., enumeration, sampling) allow AI systems to draw conclusions from probabilistic models.

Action Steps

Identify uncertain variables in the AI problem.
Define the sample space (possible worlds) for each variable.
Assign probabilities to events or states.
Determine conditional probabilities based on dependencies.
Apply Bayes' Rule to update probabilities with new evidence.
Construct Bayesian Networks to visualize relationships.
Use inference algorithms to answer queries about the system.

Formulas

$P(\omega) \text{ where } 0 \le P(\omega) \le 1$
$\sum_{\omega \in \Omega} P(\omega) = 1$
$P(a|b) = \frac{P(a \land b)}{P(b)}$
$P(a \land b) = P(b)P(a|b) = P(a)P(b|a)$
$P(\neg a) = 1 - P(a)$
$P(a \lor b) = P(a) + P(b) - P(a \land b)$
$P(a) = P(a, b) + P(a, \neg b)$
$P(X=x_i) = \sum_{j} P(X=x_i, Y=y_j)$
$P(a) = P(a|b)P(b) + P(a|¬b)P(¬b)$
$P(X|e) = \alpha \sum_{y} P(X, e, y)$

Key Terms

Uncertainty: The lack of complete knowledge or certainty about a situation or outcome.
Probability: A measure of the likelihood that an event will occur, ranging from 0 (impossible) to 1 (certain).
Possible Worlds (ω): The set of all possible outcomes or states of a system.
Probability Distribution: A function that assigns probabilities to each possible outcome in a sample space.
Unconditional Probability: The probability of an event occurring without considering any other evidence.
Conditional Probability: The probability of an event occurring given that another event has already occurred.
Independence: Two events are independent if the occurrence of one does not affect the probability of the other.
Random Variable: A variable whose value is a numerical outcome of a random phenomenon.
Bayesian Network: A directed acyclic graph representing probabilistic relationships among a set of variables.
Markov Assumption: The assumption that the future state depends only on the current state, not on past states.
Markov Chain: A sequence of random variables where each variable's distribution depends only on the previous variable.
Hidden Markov Model (HMM): A statistical model where the system being modeled is assumed to be a Markov process with unobserved (hidden) states.
Inference: The process of deriving conclusions or making predictions based on available evidence and a probabilistic model.