Elementary Probability

home1 home2
 Bib
 Algorithms
 Bioinfo
 FP
 Logic
 MML
 Prog.Lang
and the
 Book

MML
 Tables
 Probability
 >Information>
  • A sample-space (e.g. S={a,c,g,t}) is the set of possible outcomes of some experiment.
  • Events A, B, C, ..., H, ... An event is a subset (possibly a singleton) of the sample space, e.g. Purine={a,g}.
  • Events have probabilities P(A), P(B), etc.
  • Random variables X, Y, Z, ... A random variable X takes values, with certain probabilities, from the sample space.
  • We may write P(X=a), P(a) or P({a}) for the probability that X=a.

Thomas Bayes (1702-1761)

Thomas Bayes made an early study of probability and games of chance.

Bayes' Theorem

If B1, B2, ..., Bk is a partition of a set B (of causes) then

P(Bi|A) = P(A|Bi) P(Bi) / ∑j=1..k P(A|Bj) P(Bj)
i = 1, 2, ..., k
One and only one of the Bi must occur because they are a partition of B.

Inference

Bayes theorem is relevant to inference because we may be entertaining a number of exclusive and exhaustive hypotheses H1, H2, ..., Hk, and wish to know which is the best explanation of some observed data D. In that case P(Hi|D) is called the posterior probability of Hi, "posterior" because it is the probability after the data has been observed.

j=1..k P(D|Hj) P(Hj) = P(D)
 
P(Hi|D) = P(D|Hi) P(Hi) / P(D)   --posterior
Note that the Hi can even be an infinite enumerable set.

P(Hi) is called the prior probability of Hi, "prior" because it is the probability before D is known.

Notes

  • T. Bayes. An essay towards solving a problem in the doctrine of chance. Phil. Trans. of the Royal Soc. of London, 53, pp.370-418, 1763.
    Reprinted in Biometrika, 45, pp.296-315, 1958.

Conditional Probability

The probability of B given A is written P(B|A). It is the probability of B provided that A is true; we do not care, either way, if A is false. Conditional probability is defined by:

P(A&B) = P(A).P(B|A) = P(B).P(A|B)
 
P(A|B) = P(A&B) / P(B)
P(B|A) = P(A&B) / P(A)
These rules are a special case of Bayes' theorem for k=2.

There are four combinations for two Boolean variables:

  A not A margin
B A & B not A & B (A or not A)& B = B
not B A & not B not A & not B (A or not A)& not B = not B
margin A = A&(B or not B) not A = not A &(B or not B) LA 1999
We can still ask what is the probability of A, say, alone
P(A) = P(A & B) + P(A & not B)
P(B) = P(A & B) + P(not A & B)

Independence

A and B are said to be independent if the probability of A does not depend on B and v.v.. In that case P(A|B)=P(A) and P(B|A)=P(B) so

P(A&B) = P(A).P(B)
P(A & not B) = P(A).P(not B)
P(not A & B) = P(not A).P(B)
P(not A & not B) = P(not A).P(not B)

A Puzzle

I have a dice (made it myself, so it might be "tricky") which has 1, 2, 3, 4, 5 & 6 on different faces. Opposite faces sum to 7. The results of rolling the dice 100 times (good vigorous rolls on carpet) were:

 1- 20:  3 1 1 3 3 5 1 4 4 2    3 4 3 1 2 4 6 6 6 6
21- 40:  3 3 5 1 3 1 5 3 6 5    1 6 2 4 1 2 2 4 5 5
41- 60:  1 1 1 1 6 6 5 5 3 5    4 3 3 3 4 3 2 2 2 3
61- 80:  5 1 3 3 2 2 2 2 1 2    4 4 1 4 1 5 4 1 4 2
81-100:  5 5 6 4 4 6 6 4 6 6    6 3 1 1 1 6 6 2 4 5
Can you learn anything about the dice from these results? What would you predict might come up at the next roll? How certain are you of your prediction?

-- LA 1999
Coding Ockham's Razor, L. Allison, Springer

A Practical Introduction to Denotational Semantics, L. Allison, CUP

Linux
 Ubuntu
free op. sys.
OpenOffice
free office suite
The GIMP
~ free photoshop
Firefox
web browser

© L. Allison   http://www.allisons.org/ll/   (or as otherwise indicated),
Faculty of Information Technology (Clayton), Monash University, Australia 3800 (6/'05 was School of Computer Science and Software Engineering, Fac. Info. Tech., Monash University,
was Department of Computer Science, Fac. Comp. & Info. Tech., '89 was Department of Computer Science, Fac. Sci., '68-'71 was Department of Information Science, Fac. Sci.)
Created with "vi (Linux + Solaris)",  charset=iso-8859-1,  fetched Friday, 29-Mar-2024 18:17:21 AEDT.