## Multistate, Multinomial

 home1 home2  Bib  Algorithms  Bioinfo  FP  Logic  MML  Prog.Lang and the  Book

MML
KL
M-state
Normal
M-state

Also see:  M-state

## KL-distance from p1 to p2

E.g. Consider these two distributions over {A, C, G, T}:
A C G T
p1 1/4 1/4 1/4 1/4 pr
2 2 2 2 -log pr, bits
p2 1/2 1/4 1/8 1/8 pr
1 2 3 3 -log pr, bits
```               1    1    1    1    1
KL(p1, p2) = - -1 + -0 + -1 + -1 = - bits
4    4    4    4    4
```
```               1    1    1    1    1
KL(p2, p1) =   -1 + -0 - -1 - -1 = - bits
2    4    8    8    4
```
So KL(p1, p2) = KL(p2, p1) in this case -- but not in general.

Now you calculate the two distances for these two distributions:
ACGT
p1 1/21/41/81/8
p2 1/81/21/41/8
```KL(p1, p2) = ...
```
```KL(p2, p1) = ...
```
 Coding Ockham's Razor, L. Allison, Springer A Practical Introduction to Denotational Semantics, L. Allison, CUP

 Linux  Ubuntu free op. sys. OpenOffice free office suite The GIMP ~ free photoshop Firefox web browser

 © L. Allison   http://www.allisons.org/ll/   (or as otherwise indicated), Faculty of Information Technology (Clayton), Monash University, Australia 3800 (6/'05 was School of Computer Science and Software Engineering, Fac. Info. Tech., Monash University, was Department of Computer Science, Fac. Comp. & Info. Tech., '89 was Department of Computer Science, Fac. Sci., '68-'71 was Department of Information Science, Fac. Sci.) Created with "vi (Linux + Solaris)",  charset=iso-8859-1,  fetched Tuesday, 16-Jul-2024 14:06:22 AEST.