import numpy as np
import pandas as pd
from scipy import stats

%matplotlib inline

import matplotlib.pyplot as plt
import seaborn as sns
sns.set()

Graphical Models, Probability Distributions, and Independence#

Graphical Models#

Coming soon

Relating graphical models to probability distributions#

Coming soon

Independence and Conditional Independence#

Review: independence and conditional independence#

We say that two random variables \(w\) and \(v\) are independent if knowing the value of one tells us nothing about the distribution of the other. Notationally, we write \(w \perp\!\!\!\perp v\). The following statements are all true for independent random variables \(w\) and \(v\):

  • If \(w\) and \(v\) are independent (\(w \perp\!\!\!\perp v\)), then the joint distribution \(p(w, v)\) can be written as the product of the marginal distributions: \(p(w, v) = p(w)p(v)\).

  • If \(w\) and \(v\) are independent (\(w \perp\!\!\!\perp v\)), then the conditional distributions are equal to the marginal distributions: \(p(w|v) = p(w)\) and \(p(v|w) = p(v)\). Exercise: using the definition of conditional distributions, show that this condition and the previous condition are mathematically equivalent.

We say that two random variables \(w\) and \(v\) are conditionally independent given a third random variable \(u\) if, when we condition on \(u\), knowing the value of one of \(v\) or \(w\) tells us nothing about the distribution of the other. Notationally, we write \(w \perp\!\!\!\perp v \mid u\).

For example, suppose \(x_1\) and \(x_2\) are the heights of two people randomly sampled from a very specific population with some average height \(\mu\). If we know the value of \(\mu\), then \(x_1\) and \(x_2\) are conditionally independent, because they’re random samples. Suppose instead that we don’t know the value of \(\mu\), but we find out that \(x_1 = 7' 1''\). In this case, we might guess that the ‘specific population’ is likely a very tall group, such as NBA players. This will affect our belief about the distribution of \(x_2\) (i.e., we should expect the second person to be tall too). So, in this case:

  • \(x_1\) and \(x_2\) are conditionally independent given \(\mu\): \(x_1 \perp\!\!\!\perp x_2 \mid \mu\).

  • \(x_1\) and \(x_2\) are not unconditionally independent: it is not true that \(x_1 \perp\!\!\!\perp x_2\).

Independence and conditional independence in graphical models#

In progress