Solomonoff's theory of inductive inference

Solomonoff's theory of inductive inference is a mathematical theory of induction introduced by Ray Solomonoff, based on probability theory and theoretical computer science.^[1]^[2]^[3] In essence, Solomonoff's induction derives the posterior probability of any computable theory, given a sequence of observed data. This posterior probability is derived from Bayes' rule and some universal prior, that is, a prior that assigns a positive probability to any computable theory.

Solomonoff proved that this induction is incomputable, but noted that "this incomputability is of a very benign kind", and that it "in no way inhibits its use for practical prediction".^[2]

Solomonoff's induction naturally formalizes Occam's razor^[4]^[5]^[6]^[7]^[8] by assigning larger prior credences to theories that require a shorter algorithmic description.

^ Zenil, Hector (2011-02-11). Randomness Through Computation: Some Answers, More Questions. World Scientific. ISBN 978-981-4462-63-1.
^ ^a ^b Solomonoff, Ray J. (2009), Emmert-Streib, Frank; Dehmer, Matthias (eds.), "Algorithmic Probability: Theory and Applications", Information Theory and Statistical Learning, Boston, MA: Springer US, pp. 1–23, doi:10.1007/978-0-387-84816-7_1, ISBN 978-0-387-84816-7, retrieved 2020-07-21
^ Hoang, Lê Nguyên (2020). The equation of knowledge : from Bayes' rule to a unified philosophy of science (First ed.). Boca Raton, FL. ISBN 978-0-367-85530-7. OCLC 1162366056.{{cite book}}: CS1 maint: location missing publisher (link)
^ JJ McCall. Induction: From Kolmogorov and Solomonoff to De Finetti and Back to Kolmogorov – Metroeconomica, 2004 – Wiley Online Library.
^ D Stork. Foundations of Occam's razor and parsimony in learning from ricoh.com – NIPS 2001 Workshop, 2001
^ A.N. Soklakov. Occam's razor as a formal basis for a physical theory from arxiv.org – Foundations of Physics Letters, 2002 – Springer
^ Jose Hernandez-Orallo (1999). "Beyond the Turing Test" (PDF). Journal of Logic, Language and Information. 9.
^ M Hutter. On the existence and convergence of computable universal priors arxiv.org – Algorithmic Learning Theory, 2003 – Springer

[1] Zenil, Hector (2011-02-11). Randomness Through Computation: Some Answers, More Questions. World Scientific. ISBN 978-981-4462-63-1.

[Theory_and_Applications-2] Solomonoff, Ray J. (2009), Emmert-Streib, Frank; Dehmer, Matthias (eds.), "Algorithmic Probability: Theory and Applications", Information Theory and Statistical Learning, Boston, MA: Springer US, pp. 1–23, doi:10.1007/978-0-387-84816-7_1, ISBN 978-0-387-84816-7, retrieved 2020-07-21

[:0-3] Hoang, Lê Nguyên (2020). The equation of knowledge : from Bayes' rule to a unified philosophy of science (First ed.). Boca Raton, FL. ISBN 978-0-367-85530-7. OCLC 1162366056.{{cite book}}: CS1 maint: location missing publisher (link)

[ReferenceA-4] JJ McCall. Induction: From Kolmogorov and Solomonoff to De Finetti and Back to Kolmogorov – Metroeconomica, 2004 – Wiley Online Library.

[ReferenceB-5] D Stork. Foundations of Occam's razor and parsimony in learning from ricoh.com – NIPS 2001 Workshop, 2001

[ReferenceC-6] A.N. Soklakov. Occam's razor as a formal basis for a physical theory from arxiv.org – Foundations of Physics Letters, 2002 – Springer

[Hernandez.1999-7] Jose Hernandez-Orallo (1999). "Beyond the Turing Test" (PDF). Journal of Logic, Language and Information. 9.

[Hutter.2003-8] M Hutter. On the existence and convergence of computable universal priors arxiv.org – Algorithmic Learning Theory, 2003 – Springer

[1]

[2]

[3]

[4]

[5]

[6]

[7]

[8]