Welcome to David Dowe's minimum encoding length inference page. This page discusses primarily the information-theoretic inference methods of:

Minimum Message Length (MML) is an information-theoretic Bayesian principle of inductive inference. It was first published in C. S. Wallace and D.M. Boulton, Comp. J., 1968.

The first MML papers were C. S. Wallace and D.M. Boulton, Comp. J., 1968; D.M. Boulton and C. S. Wallace, 1969; D.M. Boulton and C. S. Wallace, 1970; D.M. Boulton and C. S. Wallace, 1973; D.M. Boulton and C. S. Wallace, 1975; C. S. Wallace and D.M. Boulton, 1975; and David M. Boulton's 1975 Ph.D. thesis.

[See also Ray Solomonoff (1926-2009) 85th memorial conference (Wedn 30 Nov - Fri 2 Dec 2011), 3rd Call for Papers, 2nd Call for Papers, 1st Call for Papers, Invited speakers, accepted papers.]

Pieces of literature I particularly recommend include:

Below are some frequently asked questions (FAQs):

What is the difference between MML and MDL?

What are the differences between MML and MDL?

What is the similarity between MML and MDL?

What are the similarities between MML and MDL?

Can you compare MML and MDL, and give a comparison between MML and MDL?

``Answers'': Have a look at \cite[sec. 10.2]{Wallace2005} and

\cite[sec. 11.4.3, pp272-273]{ComleyDowe2005} and

\cite[sec. 6.7]{Dowe2010} and the references within these, and

also have a look at Comp. J., Vol 42, No. 4, Wallace and Dowe (1999a), and anything else you can find.

There are also links below to related topics and to people doing related research. Please e-mail me if you'd like to be included.

Computer Journal special issue on Kolmogorov complexity and algorithmic information theory, Volume 42, Issue 4: 1999. This issue features contributions by many authors - including articles by Rissanen, by Solomonoff, by Wallace and Dowe, and by others; and responses and rejoinders on MDL and MML by Rissanen and by Wallace and Dowe respectively.

Calendar of Machine Learning, RUUG, CSSE, Monash Univ. MML talks and

CSE455 Minimum Message Length (formerly Learning and Prediction II: MML Data Mining [formerly before that CSC423 Learning and Prediction course]), CSSE, Monash Univ.

Information, Statistics and Induction in Science (ISIS) conference, Aug. 1996.

"Minimum Message Length and Kolmogorov complexity", Comp. J., Vol 42, No. 4 (1999a), pp270-283, by C. Wallace and D. Dowe [which is the Computer Journal's most downloaded ``full text as .pdf'' article - see, e.g., here].

"Minimum Message Length, MDL and Generalised Bayesian Networks with Asymmetric Languages", by J. W. Comley and D.L. Dowe; Chapter 11 (pp265-294) in P. Grunwald, M. A. Pitt and I. J. Myung (eds.), Advances in Minimum Description Length: Theory and Applications, M.I.T. Press, April 2005, ISBN 0-262-07262-9. {This is about Generalised Bayesian nets (or even the special case of hybrid Bayesian nets), generalising MML Bayesian nets or MML Bayesian networks or MML Bayes nets; and it deals with a mix of both continuous and discrete variables. (See also Comley and Dowe (2003), .pdf.)}

Artificial Intelligence (AI) on the Web: Machine Learning and Artificial Intelligence Resources.

Bayesian Net Repository - see also MML Bayes Nets: Comley & Dowe (2005) and (2003, .pdf).

Bayesian Statisticians worldwide (was here and was previously here), a link re Rev. Thomas Bayes, a link to a small drawing of Rev. Thomas Bayes and (his posthumously published) "An Essay towards solving a Problem in the Doctrine of Chances". MML uses Bayesian information theory.

Bayesianiam, compression and intelligence [Dowe and Hajek (1997, 1998)].

Boosting Research Site: boosting.org.

Clustering, mixture modelling (or finite mixture modeling) and/or unsupervised learning using MML: Snob - see, e.g., Wallace and Dowe (2000) (was here).

Compression and intelligence [Dowe and Hajek (1997, 1998)], and other compression pointers.

Computational complexity links.

Computational Learning Theory (CoLT) page: theorists and resources, and Thomas Zeugmann's COLTBIB.

CSE455, 4th Year Hons course entitled

Data collections and Vlad's KD and data mining page.

Kernel machines WWW site: www.kernel-machines.org; an Alex Smola SVM talk; and Support Vector Machine mailing list.

Machine Learning and CBR people and Mach. Learning Resources (maintained by David Aha). Online Mach. Learn. Resources.

Occam's razor links: Minimum Encoding Length Inference is an operational form of Ockham's razor.

Probabilistic prediction, probabilistic prediction competition and Gaussian prediction competition for Australian football, and history.

Statistics: University of Florida Department of Statistics's Statistics page; and AskDrMath's Prob/Stat and Statistics.

Support Vector Machine mailing list; and Kernel machines WWW site, www.kernel-machines.org. David Albrecht's SVM site, another, and another; and Tan and Dowe (2004, .pdf) on MML SVMs.

[A, B, C, D, E, F, G, H, I, J, K, L, M, N, O, P, Q, R, S, T, U, V, W, X, Y, Z]

Yudi Agusta.

Akaike Information Criterion (AIC): To compare MML and AIC, see, e.g., here.

David Albrecht (and his SVM site).

Lloyd Allison's MDL (stochastic complexity) and MML pages.

Christoph Arndt's information theory notes and "Information measures" (in English) book.

Azat Arslanov

Andrew Barron

Rohan Baxter

Rev. Thomas Bayes (dec. 1761), a small drawing of Rev. Thomas Bayes, a German article on him; and Bayesian Statisticians worldwide. MML uses Bayesian information theory.

Bayesian Nets using Minimum message length (MML).

Adrian Bickerstaffe.

Olivier Bousquet's Kolmogorov complexity resources, Kolmogorov complexity mail list and list archive.

David M. Boulton

Matthew Brand

Peter G. Bryant

Cristian S. Calude

CDMS.

Greg Chaitin

Bertrand Clarke

John G. Cleary

Clustering, mixture modelling (or finite mixture modeling) and/or unsupervised learning using MML: Snob - see, e.g., Wallace and Dowe (2000).

Josh Comley and (with D. L. Dowe) "Minimum Message Length, MDL and Generalised Bayesian Networks with Asymmetric Languages", Chapter 11 (pp265-294) in P. Grunwald, M. A. Pitt and I. J. Myung (eds.), Advances in Minimum Description Length: Theory and Applications, M.I.T. Press, April 2005, ISBN 0-262-07262-9. {This is about Generalised Bayesian nets, generalising MML Bayesian nets or MML Bayesian networks or MML Bayes nets (or generalised directed graphical models, or generalised MML directed graphical models); and it deals with a mix of both continuous and discrete variables. (See also Comley and Dowe (2003), .pdf.)}

Thomas M. Cover

Ian Davidson.

A. Phil Dawid

Ivanoe De Falco (and colleagues)'s papers and Genetic Programming Approach to Kolmogorov Complexity page.

Trevor Dix

Byron Dom's publications.

David Dowe and (with C. S. Wallace): "Minimum Message Length and Kolmogorov complexity" Comp. J., Vol 42, No. 4 (1999a), pp270-283 [this article is the Computer Journal's most downloaded ``full text as .pdf'' - see, e.g., here]; and (with J. W. Comley) "Minimum Message Length, MDL and Generalised Bayesian Networks with Asymmetric Languages", Chapter 11 (pp265-294) in P. Grunwald, M. A. Pitt and I. J. Myung (eds.), Advances in Minimum Description Length: Theory and Applications, M.I.T. Press, April 2005, ISBN 0-262-07262-9; and D. Dowe publications (or other D. Dowe publications).

Mike Dowman.

Bruce Edmonds's publications and Complexity Related Links.

Russell Edwards, Astronomical Institute "Anton Pannekoek", Faculty of Mathematics, Computer Science, Physics and Astronomy (WINS), University of Amsterdam.

Mark Ellison (was here).

Vladimir Estivill-Castro

Graham Farr's publications.

Ronald Aylmer Fisher (1890-1962): While R. A. Fisher is generally considered

Leigh Fitzgibbon's publications.

Simon Fox

Roberto Fraile (and here).

Pasi Fra:nti.

Norio Fukuda

Peter Gacs

Alex Gammerman

Andres Garcia

Russell Greiner

Peter Grünwald (Peter Grunwald, Peter Gru:nwald) and here; and (MDL source book) ``Advances in Minimum Description Length'', April 2005, MIT Press.

Alan Hajek

Brian Hanlon

Hongxing He

José Hernàndez i Orallo (and in English, Jose Hernandez-Orallo).

Phil Hingston

Marcus Hutter

Laurence Irlicht

Bruno Janvier.

Tao Jiang

Murray Jorgensen

Paul Kabaila

Kenichi Kanatani's recent publications.

Aram Karalic

Gary Warren King

Andrei N. Kolmogorov (1903-1987) sites: http://www.Kolmogorov.com/Kolmogorov.html, a c.v. (in Russian), http://www.pms.ru/kolmogorov (in Russian), exploratorium.edu, dcs.st-and.ac.uk and (by Paul Vitanyi) a biography.

Petri Kontkanen

Aaron Lanterman (was here).

Thomas Lee's publications and technical reports.

Shane Legg

Leonid Levin

Mengxiang Li

Ming Li

Xiaodong Li

Dean McKenzie's publications.

Enes Makalic.

Steve Maybank (and here).

www.MDL-research.org, people, reading, demonstrations and related topics; and (MDL source book) ``Advances in Minimum Description Length'', April 2005, MIT Press.

Paul Ming's Virtual Turing Machine (VTM) and Virtual Turing Machine 2 (VTM2).

Minimum Description Length (MDL) and comparisons with MML (on pp270-283 and elsewhere) in Comp. J., Vol 42, No. 4, 1999.

Mixture modelling (or finite mixture modeling), clustering and/or unsupervised learning using MML: Snob - see, e.g., Wallace and Dowe (2000).

MML and the "Turing test".

Suzie Molloy.

Robert J. Munro

Petri Myllymäki

Julian R. Neil

Doug Newlands

Ockham, William of (circa. 1280 or 1285 till circa. 1347 or 1349, apparently 10th April 1349), Ockham's razor and town of Ockham.

R. D. Ogden, Computer Science Department, Southwest Texas State University, San Marcos, TX 78666, USA; ro01@swt.edu.

Jon Oliver

Graham Oppy

Jon Patrick.

Philosophy, Stanford Encyclopaedia of.

Guoqi Qian

Anand Venkataraman (was Anand (Venkt) Raman).

Fengrong Ren

Jorma Rissanen (and publications, although links seems dead) and some sample publications.

Ricardo Rocha

Teemu Roos

Juho Rousu.

Pritika Sanghi.

J. G. Sanjayan

Ju:rgen Schmidhuber

Stanley L. Sclove's journal publications and working papers.

Claude Shannon ("father of information theory")'s obituary (and another obituary), 1916-2001.

Alexander Shen

Tomi Silander

Jatinder Singh.

Snob (software): mixture modelling (to infer a finite mixture model) using MML.

Ray Solomonoff's biography, publications and obituary (New York Times, Wedn 13/Jan/2010): online and scanned - and Solomonoff 85th memorial conference (Wedn 30 Nov - Fri 2 Dec 2011) : 3rd Call for Papers, 2nd Call for Papers, 1st Call for Papers, Invited speakers, accepted papers.

David Suter.

Peter Jing Tan; and P. J. Tan and D. L. Dowe (2003), "MML Inference of Decision Graphs with Multi-Way Joins and Dynamic Attributes", Proc. 16th Australian Joint Conference on Artificial Intelligence (AI'03), Perth, Australia, 3-5 Dec. 2003, Published in Lecture Notes in Artificial Intelligence (LNAI) 2903, Springer-Verlag, pp269-281.

Hiroshi Tanaka

Kai Ming Ting

Henry Tirri

Peter Tischer

Phil H. S. Torr (was here).

Alan Turing (1912-1954), developer of (Universal) Turing Machines, among many other things. Sites maintained by Andrew Hodges and dcs.st-and.ac.uk; The Turing archive for the history of computing (maintained by Jack Copeland and Gordon Aston); and image of Alan Turing.

A.M. Turing's (1950) "Computing Machinery and Intelligence", Mind, 59, pp433-460 - which is the paper which introduced the imitation game or "Turing Test".

Turing Machine simulator: http://wap03.informatik.fh-wiesbaden.de/weber1/turing/tm.html and documentation.

Virtual Turing Machine (VTM) and Virtual Turing Machine 2 (VTM2), by Paul Ming.

The "Turing test" and MML.

Charles Twardy and Bayesian Models for Search & Rescue.

Unsupervised learning, mixture modelling (or finite mixture modeling) and/or clustering using MML: Snob - see, e.g., Wallace and Dowe (2000).

Aleksey M. Urmanov.

William Uther (was here).

Farshid Vahid

Manuela Veloso.

Brani Vidakovic.

Murli Viswananthan (and here)'s publications.

Paul Vitanyi's Publications & Areas of Interest, Kolmogorov complexity publications, selected recent papers and publications.

Volodya Vovk

V. V. V'yugin

Chris Wallace's publications, including "Minimum Message Length and Kolmogorov complexity" (with D. L. Dowe), Comp. J., Vol 42, No. 4 (1999a), pp270-283 [this article is the Computer Journal's most downloaded ``full text as .pdf'' - see, e.g., here];

Chris Wallace MML publications, 1990-

Ian H. Witten

Kenji Yamanishi

Bin Yu's publications.

Alexander K. Zvonkin

Chris Wallace (1933-2004) (developer of MML in 1968) and the Computer J's Christopher Stewart WALLACE (1933-2004) memorial special issue [Vol. 51, No. 5 (Sept. 2008)],

Bayesian Nets using Minimum message length (MML),

clustering and mixture modelling,

data repositories,

decision trees using Minimum message length (MML),

Occam's razor (Ockham's razor),

Snob (program for MML clustering and mixture modelling) to infer finite mixture models,

(econometric) time series using MML,

medical research,

a probabilistic sports prediction competition (and further reading on probabilistic scoring),

chess and game theory research;

Feeding the world (TheHungerSite), TheRainforestSite, "do-goody"/"do-goody stuff, improving the world and saving the planet".

Copyright David L. Dowe, Monash University, Australia, 31 Jan. 2000, etc.

Copying is not permitted without expressed permission from David L. Dowe.