Home

Occam’s Razor/Ockham’s Razor

Occam’s razor (also spelled Ockham’s razor) is a law of parsimony: the principle gives precedence to simplicity; of two competing theories, the simplest explanation of an entity is to be preferred. The principle is also expressed as ‘Entities are not to be multiplied beyond necessity’.

There is no free lunch (Wolpert 1996) for Occam’s razor, so it can not be proved from first principles. However, if when choosing priors, one applies the ‘principle of insufficient reason’ to models (such as ax + b), rather than functions (such as 3x + 4), then Bayesian inference inherently prefers simpler models because the likelihood, P(data|hypothesis), of more complex models is more spread out. For example, suppose that I throw n dice and tell you that the spots sum to 3. How many dice did I throw? P(3 spots|1 dice) = 1/6 = 0.1667, P(3 spots|2 dice) = 2×(1/6)×(1/6) = 0.0556, P(3 spots|3 dice) = (1/6)×(1/6)×(1/6) = 0.0046. Plugging this into Bayes formula shows that, even with equal priors, the simpler model (one die) is to be preferred.

Top 10 Papers

  1. BLUMER, A., et al., 1987. Occam's Razor. Information Processing Letters. [Cited by 465] (21.71/year)
  2. JEFFERYS, W. and J. BERGER, 1992. Ockham's razor and Bayesian analysis(statistical theory for systems evaluation). American Scientist. [Cited by 134] (8.16/year)
  3. FORD, A.P., et al., 1994. Alpha 1-adrenoceptor classification: sharpening Occam's razor.. Trends Pharmacol Sci. [Cited by 157] (10.89/year)
  4. GLIICK, R. and A.V. KLIMOV, Static Analysis. Proceedings. Occam's razor in metacomputation: the notion of a perfect process tree. [Cited by 91] (?/year)
  5. DOMINGOS, P., 1999. The Role of Occam's Razor in Knowledge Discovery. Data Mining and Knowledge Discovery. [Cited by 128] (13.59/year)
  6. MURPHY, P.M. and M.J. PAZZANI, 1997. Exploring the Decision Forest: An Empirical Investigation of Occam's Razor in Decision Tree …. Computational Learning Theory and Natural Learning Systems. [Cited by 102] (8.93/year)
  7. WEBB, G.I., 1996. Further Experimental Evidence against the Utility of Occam's Razor. Arxiv preprint cs.AI/9605101. [Cited by 81] (6.52/year)
  8. BALASUBRAMANIAN, V., 1997. Statistical Inference, Occam's Razor, and Statistical Mechanics on the Space of Probability …. Neural Computation. [Cited by 88] (7.71/year)
  9. ZHANG, B.T. and H. MUEHLENBEIN, 1993. Evolving Optimal Neural Networks Using Genetic Algorithms with Occam's Razor. COMPLEX SYSTEMS-CHAMPAIGN-. [Cited by 64] (4.15/year)
  10. HOLLAND, M.A., 1997. Occam's Razor Applied to Hormonology (Are Cytokinins Produced by Plants?). Plant Physiology. [Cited by 61] (5.34/year)

Links

Bibliography

Webmaster: Martin Sewell