Probability Theory

see also:

• Marginalization (Probability)
• Error Propagation
• Markov random fields
• Bayesian Networks
• Weibull Distribution

Probability theory is a branch of mathematics concerned with analyzing random phenomena. The fundamental object of study in probability theory is the probability measure, a function that assigns a probability to events in a certain space, satisfying specific axioms. Probability theory enables the quantification of uncertainty, the modeling of stochastic processes, and the derivation of outcomes’ likelihoods based on the mathematical properties of those processes. It lays the foundation for statistical inference, where data are analyzed to make decisions or predictions about a population.

Axiomatic Foundation

The modern axiomatic foundation of probability theory was established by Andrey Kolmogorov in the 1930s, which is based on three axioms:

1. Non-negativity: For any event $A$, the probability of $A$, denoted as $P(A)$, is non-negative: $P(A)\ge 0$.
2. Normalization: The probability of the sample space $S$, which represents the set of all possible outcomes, is 1: $P(S)=1$.
3. Additivity: For any two mutually exclusive events $A$ and $B$ (i.e., $A$ and $B$ cannot both occur), the probability of their union is the sum of their probabilities: $P(A\cup B)=P(A)+P(B)$.

From these axioms, many other properties and rules of probability can be derived, including the law of total probability, Bayes’ theorem, and the concept of conditional probability.

Key Concepts

Random Variables

A random variable is a function that assigns a real number to each outcome in a sample space. Random variables can be discrete, taking on a countable number of distinct values, or continuous, taking on any value in an interval of numbers.

Probability Distributions

The probability distribution of a random variable describes how probabilities are distributed over the possible values of the random variable. For discrete variables, this is often given by a probability mass function (PMF), and for continuous variables, by a probability density function (PDF).

Expected Value and Variance

• Expected Value: The expected value (or mean) of a random variable is a measure of its central tendency and is calculated as the weighted average of all possible values, weighted by their probabilities.
• Variance: The variance of a random variable measures the spread of its possible values around the mean, indicating how much the values of the random variable differ from the expected value.

Applications

Probability theory is fundamental to many fields and applications, including:

• Statistics: Provides the theoretical basis for estimating population parameters, testing hypotheses, and making predictions based on sample data.
• Finance: Used to model financial markets, assess risks, and price derivatives.
• Computer Science: Underpins algorithms in machine learning, data mining, and artificial intelligence.
• Engineering: Applied in reliability engineering, signal processing, and control systems.
• Natural Sciences: Models randomness and uncertainty in physical, biological, and social systems.

Stochastic Processes

A stochastic process is a collection of random variables indexed by time or space, representing the evolution of some random value or system over time or space. Examples include random walks, Markov chains, and Poisson processes, each of which has specific properties and applications in modeling various phenomena in science and engineering.

Probability theory, with its rigorous mathematical framework, provides a powerful language for describing and understanding the randomness and complexity inherent in the world around us.