|
KL-distance from p1 to p2
- E.g. Consider these two distributions over {A, C, G, T}:
|
A | C | G | T |
|
p1 |
1/4 | 1/4 | 1/4 | 1/4 |
pr |
2 | 2 | 2 | 2 |
-log pr, bits |
p2 |
1/2 | 1/4 | 1/8 | 1/8 |
pr |
1 | 2 | 3 | 3 |
-log pr, bits |
1 1 1 1 1
KL(p1, p2) = - -1 + -0 + -1 + -1 = - bits
4 4 4 4 4
1 1 1 1 1
KL(p2, p1) = -1 + -0 - -1 - -1 = - bits
2 4 8 8 4
- So KL(p1, p2) = KL(p2, p1)
in this case -- but not in general.
-
-
- Now you calculate the two distances for these two distributions:
| A | C | G | T |
p1 | 1/2 | 1/4 | 1/8 | 1/8 |
p2 | 1/8 | 1/2 | 1/4 | 1/8 |
KL(p1, p2) = ...
KL(p2, p1) = ...
|
|