(See the page re-formatted and perhaps out of date here.)

Papers on theory behind Snob and papers on applications of Snob.

Mixture modelling (or mixture modeling) concerns modelling a statistical distribution by a mixture (or weighted sum) of other distributions. Mixture modelling is also known as

Name of program:

As per sec. 0.2.4, p535, footnote 113 in D. L. Dowe (2008), "Foreword re C. S. Wallace", Computer Journal, Vol. 51, No. 5 (Sept. 2008) [Christopher Stewart WALLACE (1933-2004) memorial special issue], pp523-560 (and here), the program was given the name ``

Minimum Message Length (MML) is a method of machine learning, statistical inference, inductive learning, "knowledge discovery" and "data mining" very much in line with the notions of Kolmogorov complexity and algorithmic information theory pioneered by R. J. Solomonoff, A. N. Kolmogorov and Greg Chaitin. See also Wallace & Dowe (1999a), "Minimum Message Length and Kolmogorov complexity", Comp. J., Vol 42, No. 4, 270-283 [which is the Computer Journal's most downloaded ``full text as .pdf'' article - see, e.g., here].

[Possibly see also Ray Solomonoff (1926-2009) 85th memorial conference (Wedn 30 Nov - Fri 2 Dec 2011), 1st Call for Papers and conference proceedings.]

The original Snob paper was: Wallace, C. S. and Boulton, D.M. (1968), `An Information Measure for Classification', Computer Journal, Vol. 11, No. 2, 1968, pp. 185-194. This is the same paper in which MML was developed. (See also more recent Snob theory and application papers.)

See also D. L. Dowe (2008), "Foreword re C. S. Wallace", Computer Journal, Vol. 51, No. 5 (Sept. 2008) [Christopher Stewart WALLACE (1933-2004) memorial special issue], pp523-560 (and here) for a survey of all of Wallace's works - including all his MML work and his

Snob currently deals with finite mixture models (or a finite mixture model) of

Chris Wallace extended Snob in 1998 to deal with spatial correlation (and Markov fields), as occurs in images.

Gerhard Visser and David Dowe (2007, 2009) endeavoured to extend Wallace (1998, above) with "G. Visser and D. Dowe" (2007) "Minimum Message Length Clustering Of Spatially-Correlated Data with Varying Inter-Class Penalties" and with G. Visser, D. L. Dowe and J. P. Uotila (2009) "Enhancing MML Clustering using Context Data with Climate Applications".

Russell Edwards and David Dowe created a version of Snob (also in 1998) which deals with single Gaussian factor analysis in sequentially and spatially uncorrelated data. It uses total assignment. (See publications.) Yudi Agusta and David Dowe have also developed MML mixture modelling software for other correlated Gaussians, t distributions (2002) and Gamma distributions (2003, .pdf); and Jon Oliver and David Dowe published a note (1996) on MML mixture modelling of von Mises-Fisher spherical distributions.

A useful set of links on Snob is given in the next few lines immediately below:

Snob ReadMe, documentation and (data) sd1.raw files, .ps of Wallace and Dowe (1997) and .pdf of more recent, 2000, paper:

Wallace, C.S. and D. L. Dowe (2000). MML clustering of multi-state, Poisson, von Mises circular

and Gaussian distributions, Statistics and Computing, Vol. 10, No. 1, Jan. 2000, pp73-83.

p73, p74, p75, p76, p77, p78, p79, p80, p81, p82, p83

http://www.wkap.nl/issuetoc.htm/0960-3174+10+1+2000

http://www.wkap.nl/sampletoc.htm?0960-3174+10+1+2000

See also Wallace and Dowe (1994b) Wallace, C.S. and D.L. Dowe (1994b), Intrinsic classification by MML - the Snob program. Proc. 7th Australian Joint Conf. on Artificial Intelligence, UNE, Armidale, Australia, November 1994, pp37-44.

http://www.csse.monash.edu.au/~dld/Publications/1994/Wallace+Dowe1994.ref http://www.csse.monash.edu.au/~dld/Publications/1994/Wallace+Dowe1994_IntrinsicClassification_MML_AI94.Snob.pdf http://www.csse.monash.edu.au/~dld/Publications/1994/Wallace+Dowe1994_IntrinsicClassification_MML_AI94.Snob.ps (reference here) in C.S. Wallace publications and/or D.L. Dowe publications.

The Snob software is available subject to conditions.

Snob

Papers on theory behind Snob and papers on applications of Snob.

Chris Wallace MML publications, 1968-1991 and Chris Wallace MML publications, 1990- .

Chris Wallace MML applications, 1968-1996 and Chris Wallace MML applications, 1990- .

If you would like to download a Linux

A C version is currently under construction.

(Pseudo-)Random number generation software in Fortran :

uniform (for multinomial), Gaussian (Normal), von Mises (circular) and Poisson.

http://www.csse.monash.edu.au/research/mdmc/software/random.

Link to Lloyd Allison's Short note on Snob.

Link to K D Mine's S*i*ftware Snob notes, based on material supplied by D. Dowe and L. Allison.

Minimum Description Length (MDL) and comparisons with MML (on pp270-283 and elsewhere) in Comp. J., Vol 42, No. 4, 1999.

Bayesian networks using MML,

clustering, mixture modelling and finite mixture models,

comparisons between MML and the subsequent Minimum Description Length principle,

data repositories,

decision trees and decision graphs using MML,

medical research,

"Minimum Message Length, MDL and Generalised Bayesian Networks with Asymmetric Languages", by J. W. Comley and D.L. Dowe; Chapter 11 (pp265-294) in P. Grunwald, M. A. Pitt and I. J. Myung (eds.), Advances in Minimum Description Length: Theory and Applications, M.I.T. Press, April 2005, ISBN 0-262-07262-9. {This is about Generalised Bayesian nets, generalising MML Bayesian nets or MML Bayesian networks or MML Bayes nets (or Generalised directed graphical models, generalising MML directed graphical models); and it deals with a mix of both continuous and discrete variables. (See also Comley and Dowe (2003), .pdf.)}

Minimum Message Length (MML),

Occam's razor (Ockham's razor),

a probabilistic sports prediction competition (and further reading on probabilistic scoring),

Snob theory publications and applications publications,

(econometric) time series using MML,

Chris Wallace (1933-2004) (developer of MML in 1968) and his (2005) [posthumous]

chess and game theory research,

Feeding the world (TheHungerSite), TheRainforestSite, and "do-goody"/"do-goody stuff, improving the world and saving the planet".

Dr David Dowe, Dept. of Computer Science, Monash University, Clayton, Vic. 3168, Austra lia

(and was started on Sat 8th Mar. 1997) and was last updated no earlier than Mon 3rd Mar. 1998.

Copyright David L. Dowe, Monash University, Australia, 8 Mar 1997, 3 Mar 1998, 7 May 1998, etc.

Copying is not permitted without expressed permission from David L. Dowe.