You are previewing Bayesian Probability Theory.
O'Reilly logo
Bayesian Probability Theory

Book Description

From the basics to the forefront of modern research, this book presents all aspects of probability theory, statistics and data analysis from a Bayesian perspective for physicists and engineers. The book presents the roots, applications and numerical implementation of probability theory, and covers advanced topics such as maximum entropy distributions, stochastic processes, parameter estimation, model selection, hypothesis testing and experimental design. In addition, it explores state-of-the art numerical techniques required to solve demanding real-world problems. The book is ideal for students and researchers in physical sciences and engineering.

Table of Contents

  1. Cover Page
  2. Half Title Page
  3. Title Page
  4. Copyright Page
  5. Contents
  6. Preface
  7. PART I INTRODUCTION
    1. 1 The meaning of ‘probability’
      1. 1.1 Classical definition of ‘probability’
      2. 1.2 Statistical definition of ‘probability’
      3. 1.3 Bayesian understanding of ‘probability’
    2. 2 Basic definitions for frequentist statistics and Bayesian inference
      1. 2.1 Definition of mean, moments and marginal distribution
      2. 2.2 Worked example: The three-urn problem
      3. 2.3 Frequentist statistics versus Bayesian inference
    3. 3 Bayesian inference
      1. 3.1 Propositions
      2. 3.2 Selected examples
      3. 3.3 Ockham’s razor
    4. 4 Combinatorics
      1. 4.1 Preliminaries
      2. 4.2 Partitions, binomial and multinomial distributions
      3. 4.3 Occupation number problems
      4. 4.4 Geometric and hypergeometric distributions
      5. 4.5 The negative binomial distribution
    5. 5 Random walks
      1. 5.1 First return
      2. 5.2 First lead
      3. 5.3 Random walk with absorbing wall
    6. 6 Limit theorems
      1. 6.1 Stirling’s formula
      2. 6.2 de Moivre–Laplace theorem/local limit theorem
      3. 6.3 Bernoulli’s law of large numbers
      4. 6.4 Poisson’s law
    7. 7 Continuous distributions
      1. 7.1 Continuous propositions
      2. 7.2 Distribution function and probability density functions
      3. 7.3 Application in statistical physics
      4. 7.4 Definitions for continuous distributions
      5. 7.5 Common probability distributions
      6. 7.6 Order statistic
      7. 7.7 Transformation of random variables
      8. 7.8 Characteristic function
      9. 7.9 Error propagation
      10. 7.10 Helmert transformation
    8. 8 The central limit theorem
      1. 8.1 The theorem
      2. 8.2 Stable distributions
      3. 8.3 Proof of the central limit theorem
      4. 8.4 Markov chain Monte Carlo (MCMC)
      5. 8.5 The multivariate case
    9. 9 Poisson processes and waiting times
      1. 9.1 Stochastic processes
      2. 9.2 Three ways to generate Poisson points
      3. 9.3 Waiting time paradox
      4. 9.4 Order statistic of Poisson processes
      5. 9.5 Various examples
  8. PART II ASSIGNING PROBABILITIES
    1. 10 Prior probabilities by transformation invariance
      1. 10.1 Bertrand’s paradox revisited
      2. 10.2 Prior for scale variables
      3. 10.3 The prior for a location variable
      4. 10.4 Hyperplane priors
      5. 10.5 The invariant Riemann measure (Jeffreys’ prior)
    2. 11 Testable information and maximum entropy
      1. 11.1 Discrete case
      2. 11.2 Properties of the Shannon entropy
      3. 11.3 Maximum entropy for continuous distributions
    3. 12 Quantified maximum entropy
      1. 12.1 The entropic prior
      2. 12.2 Derivation of the entropic prior
      3. 12.3 Saddle-point approximation for the normalization
      4. 12.4 Posterior probability density
      5. 12.5 Regularization and good data
      6. 12.6 A technical trick
      7. 12.7 Application to ill-posed inversion problems
    4. 13 Global smoothness
      1. 13.1 A primer on cubic splines
      2. 13.2 Second derivative prior
      3. 13.3 First derivative prior
      4. 13.4 Fisher information prior
  9. PART III PARAMETER ESTIMATION
    1. 14 Bayesian parameter estimation
      1. 14.1 The estimation problem
      2. 14.2 Loss and risk function
      3. 14.3 Confidence intervals
      4. 14.4 Examples
    2. 15 Frequentist parameter estimation
      1. 15.1 Unbiased estimators
      2. 15.2 The maximum likelihood estimator
      3. 15.3 Examples
      4. 15.4 Stopping criteria for experiments
      5. 15.5 Is unbiasedness desirable at all?
      6. 15.6 Least-squares fitting
    3. 16 The Cramer–Rao inequality
      1. 16.1 Lower bound on the variance
      2. 16.2 Examples
      3. 16.3 Admissibility of the Cramer–Rao limit
  10. PART IV TESTING HYPOTHESES
    1. 17 The Bayesian way
      1. 17.1 Some illustrative examples
      2. 17.2 Independent measurements with Gaussian noise
    2. 18 The frequentist approach
      1. 18.1 Introduction
      2. 18.2 Neyman–Pearson lemma
    3. 19 Sampling distributions
      1. 19.1 Mean and median of i.i.d. random variables
      2. 19.2 Mean and variance of Gaussian samples
      3. 19.3 z-Statistic
      4. 19.4 Student’s t-statistic
      5. 19.5 Fisher–Snedecor F -statistic
      6. 19.6 Chi-squared in case of missing parameters
      7. 19.7 Common hypothesis tests
    4. 20 Comparison of Bayesian and frequentist hypothesis tests
      1. 20.1 Prior knowledge is prior data
      2. 20.2 Dependence on the stopping criterion
  11. PART V REAL-WORLD APPLICATIONS
    1. 21 Regression
      1. 21.1 Linear regression
      2. 21.2 Models with nonlinear parameter dependence
      3. 21.3 Errors in all variables
    2. 22 Consistent inference on inconsistent data
      1. 22.1 Erroneously measured uncertainties
      2. 22.2 Combining incompatible measurements
    3. 23 Unrecognized signal contributions
      1. 23.1 The nuclear fission cross-section[sup(239)] Pu(n, f)
      2. 23.2 Electron temperature in a tokamak edge plasma
      3. 23.3 Signal–background separation
    4. 24 Change point problems
      1. 24.1 The Bayesian change point problem
      2. 24.2 Change points in a binary image
      3. 24.3 Neural network modelling
      4. 24.4 Thin film growth detected by Auger analysis
    5. 25 Function estimation
      1. 25.1 Deriving trends from observations
      2. 25.2 Density estimation
    6. 26 Integral equations
      1. 26.1 Abel’s integral equation
      2. 26.2 The Laplace transform
      3. 26.3 The Kramers–Kronig relations
      4. 26.4 Noisy kernels
      5. 26.5 Deconvolution
    7. 27 Model selection
      1. 27.1 Inelastic electron scattering
      2. 27.2 Signal–background separation
      3. 27.3 Spectral line broadening
      4. 27.4 Adaptive choice of pivots
      5. 27.5 Mass spectrometry
    8. 28 Bayesian experimental design
      1. 28.1 Overview of the Bayesian approach
      2. 28.2 Optimality criteria and utility functions
      3. 28.3 Examples
      4. 28.4 N-step-ahead designs
      5. 28.5 Experimental design: Perspective
  12. PART VI PROBABILISTIC NUMERICAL TECHNIQUES
    1. 29 Numerical integration
      1. 29.1 The deterministic approach
      2. 29.2 Monte Carlo integration
      3. 29.3 Beyond the Gaussian approximation
    2. 30 Monte Carlo methods
      1. 30.1 Simple sampling
      2. 30.2 Variance reduction
      3. 30.3 Markov chain Monte Carlo
      4. 30.4 Expectation value of the sample mean
      5. 30.5 Equilibration
      6. 30.6 Variance of the sample mean
      7. 30.7 Taming rugged PDFs by tempering
      8. 30.8 Evidence integral and partition function
    3. 31 Nested sampling
      1. 31.1 Motivation
      2. 31.2 The theory behind nested sampling
      3. 31.3 Application to the classical ideal gas
      4. 31.4 Statistical uncertainty
      5. 31.5 Concluding remarks
  13. Appendix A Mathematical compendium
  14. Appendix B Selected proofs and derivations
  15. Appendix C Symbols and notation
  16. References
  17. Index