Multistate, Multinomial

home1 home2
 Bib
 Algorithms
 Bioinfo
 FP
 Logic
 MML
 Prog.Lang
and the
 Book

MML
 KL
  M-state
  Normal
  M-state

Also see:  M-state

KL-distance from p1 to p2

E.g. Consider these two distributions over {A, C, G, T}:
  A C G T  
p1 1/4 1/4 1/4 1/4 pr
2 2 2 2 -log pr, bits
p2 1/2 1/4 1/8 1/8 pr
1 2 3 3 -log pr, bits
               1    1    1    1    1
KL(p1, p2) = - -1 + -0 + -1 + -1 = - bits
               4    4    4    4    4
               1    1    1    1    1
KL(p2, p1) =   -1 + -0 - -1 - -1 = - bits
               2    4    8    8    4
So KL(p1, p2) = KL(p2, p1) in this case -- but not in general.
 
 
Now you calculate the two distances for these two distributions:
  ACGT
p1 1/21/41/81/8
p2 1/81/21/41/8
KL(p1, p2) = ...
KL(p2, p1) = ...
Coding Ockham's Razor, L. Allison, Springer

A Practical Introduction to Denotational Semantics, L. Allison, CUP

Linux
 Ubuntu
free op. sys.
OpenOffice
free office suite
The GIMP
~ free photoshop
Firefox
web browser

© L. Allison   http://www.allisons.org/ll/   (or as otherwise indicated),
Faculty of Information Technology (Clayton), Monash University, Australia 3800 (6/'05 was School of Computer Science and Software Engineering, Fac. Info. Tech., Monash University,
was Department of Computer Science, Fac. Comp. & Info. Tech., '89 was Department of Computer Science, Fac. Sci., '68-'71 was Department of Information Science, Fac. Sci.)
Created with "vi (Linux + Solaris)",  charset=iso-8859-1,  fetched Tuesday, 16-Jul-2024 14:06:22 AEST.