^MML^ ^Information^

## Examples K-L Distance

### Code lengths for {A, C, G, T}

Jumping ahead a little, we would get these average code lengths,   SUMi{ pi.log(qi) },   for {A,C,G,T} for the following true and assumed probabilities of the bases:

assumed probabilities q = (1/4, 1/4, 1/4, 1/4) {A, C, G, T} 2 = 1/4*2+1/4*2+1/4*2+1/4*2 2 1/4 = 1/4*1+1/4*2+1/4*3+1/4*3 2 = 1/2*2+1/4*2+1/8*2+1/8*2 1 3/4 = 1/2*1+1/4*2+1/8*3+1/8*3

### K-L Distance

It just happens that in the above example the KL distances between the two distributions, KL(fair->biased) = KL(biased->fair) = 1/4, are equal - in this case.

In general, and in the following example, the K-L distance is not symmetric:

assumed probabilities q = (1/2, 1/4, 1/8, 1/8) {A, C, G, T} 1 3/4 = 1/2*1+1/4*2+1/8*3+1/8*3 2 1/4 = 1/2*2+1/4*3+1/8*3+1/8*1 2 3/8 = 1/4*1+1/8*2+1/8*3+1/2*3 1 3/4 = 1/4*2+1/8*3+1/8*3+1/2*1

KL(p->p') = 1/2,  but  KL(p'->p) = 5/8.

© L. Allison 2000, School of Computer Science and Software Engineering, Monash University, Australia.
Created with "vi (Linux + IRIX)",   charset=iso-8859-1