KLD / JSD / Wasserstein
Overview
You have two distributions P and Q. These metrics measure: "How different are P and Q?"
Metric Comparison
Loading diagram...
Summary
KLD: Selfish — direction matters. NOT symmetric.
JSD: Fair version of KLD. Symmetric.
Wasserstein: Effort to reshape P into Q. Symmetric.
KLD — Gaussian
P = N(μ₁, σ₁²) Q = N(μ₂, σ₂²) KLD(P||Q) = log(σ₂/σ₁) + [σ₁² + (μ₁-μ₂)²]/(2σ₂²) - ½ To get KLD(Q||P): swap ALL 1s and 2s
Wasserstein Distance
W₂ = √[(μ₁-μ₂)² + (σ₁-σ₂)²] = √[center difference² + spread difference²]
JSD Shortcut
For Gaussians: "M = ½(P+Q) is a mixture. Closed form not possible. JSD(P||Q) = JSD(Q||P) by symmetry." — Full marks, no calculation needed.