You are previewing Mathematical Foundations of Infinite-Dimensional Statistical Models.
O'Reilly logo
Mathematical Foundations of Infinite-Dimensional Statistical Models

Book Description

In nonparametric and high-dimensional statistical models, the classical Gauss-Fisher-Le Cam theory of the optimality of maximum likelihood estimators and Bayesian posterior inference does not apply, and new foundations and ideas have been developed in the past several decades. This book gives a coherent account of the statistical theory in infinite-dimensional parameter spaces. The mathematical foundations include self-contained 'mini-courses' on the theory of Gaussian and empirical processes, on approximation and wavelet theory, and on the basic theory of function spaces. The theory of statistical inference in such models - hypothesis testing, estimation and confidence sets - is then presented within the minimax paradigm of decision theory. This includes the basic theory of convolution kernel and projection estimation, but also Bayesian nonparametrics and nonparametric maximum likelihood estimation. In the final chapter, the theory of adaptive inference in nonparametric models is developed, including Lepski's method, wavelet thresholding, and adaptive inference for self-similar functions.

Table of Contents

  1. Cover
  2. Half-title page
  3. Series page
  4. Title
  5. Copyright
  6. Dedication
  7. Contents
  8. Preface
  9. 1 Nonparametric Statistical Models
    1. 1.1 Statistical Sampling Models
      1. 1.1.1 Nonparametric Models for Probability Measures
      2. 1.1.2 Indirect Observations
    2. 1.2 Gaussian Models
      1. 1.2.1 Basic Ideas of Regression
      2. 1.2.2 Some Nonparametric Gaussian Models
      3. 1.2.3 Equivalence of Statistical Experiments
    3. 1.3 Notes
  10. 2 Gaussian Processes
    1. 2.1 Definitions, Separability, 0-1 Law, Concentration
      1. 2.1.1 Stochastic Processes: Preliminaries and Definitions
      2. 2.1.2 Gaussian Processes: Introduction and First Properties
    2. 2.2 Isoperimetric Inequalities with Applications to Concentration
      1. 2.2.1 The Isoperimetric Inequality on the Sphere
      2. 2.2.2 The Gaussian Isoperimetric Inequality for the Standard Gaussian Measure on RN
      3. 2.2.3 Application to Gaussian Concentration
    3. 2.3 The Metric Entropy Bound for Suprema of Sub-Gaussian Processes
    4. 2.4 Anderson’s Lemma, Comparison and Sudakov’s Lower Bound
      1. 2.4.1 Anderson’s Lemma
      2. 2.4.2 Slepian’s Lemma and Sudakov’s Minorisation
    5. 2.5 The Log-Sobolev Inequality and Further Concentration
      1. 2.5.1 Some Properties of Entropy: Variational Definition and Tensorisation
      2. 2.5.2 A First Instance of the Herbst (or Entropy) Method: Concentration of the Norm of a Gaussian Variable about Its Expectation
    6. 2.6 Reproducing Kernel Hilbert Spaces
      1. 2.6.1 Definition and Basic Properties
      2. 2.6.2 Some Applications of RKHS: Isoperimetric Inequality, Equivalence and Singularity, Small Ball Estimates
      3. 2.6.3 An Example: RKHS and Lower Bounds for Small Ball Probabilities of Integrated Brownian Motion
    7. 2.7 Asymptotics for Extremes of Stationary Gaussian Processes
    8. 2.8 Notes
  11. 3 Empirical Processes
    1. 3.1 Definitions, Overview and Some Background Inequalities
      1. 3.1.1 Definitions and Overview
      2. 3.1.2 Exponential and Maximal Inequalities for Sums of Independent Centred and Bounded Real Random Variables
      3. 3.1.3 The Lévy and Hoffmann-Jørgensen Inequalities
      4. 3.1.4 Symmetrisation, Randomisation, Contraction
    2. 3.2 Rademacher Processes
      1. 3.2.1 A Comparison Principle for Rademacher Processes
      2. 3.2.2 Convex Distance Concentration and Rademacher Processes
      3. 3.2.3 A Lower Bound for the Expected Supremum of a Rademacher Process
    3. 3.3 The Entropy Method and Talagrand’s Inequality
      1. 3.3.1 The Subadditivity Property of the Empirical Process
      2. 3.3.2 Differential Inequalities and Bounds for Laplace Transforms of Subadditive Functions and Centred Empirical Processes, λ ≥ 0
      3. 3.3.3 Differential Inequalities and Bounds for Laplace Transforms of Centred Empirical Processes, λ < 0
      4. 3.3.4 The Entropy Method for Random Variables with Bounded Differences and for Self-Bounding Random Variables
      5. 3.3.5 The Upper Tail in Talagrand’s Inequality for Nonidentically Distributed Random Variables*
    4. 3.4 First Applications of Talagrand’s Inequality
      1. 3.4.1 Moment Inequalities
      2. 3.4.2 Data-Driven Inequalities: Rademacher Complexities
      3. 3.4.3 A Bernstein-Type Inequality for Canonical U-statistics of Order 2
    5. 3.5 Metric Entropy Bounds for Suprema of Empirical Processes
      1. 3.5.1 Random Entropy Bounds via Randomisation
      2. 3.5.2 Bracketing I: An Expectation Bound
      3. 3.5.3 Bracketing II: An Exponential Bound for Empirical Processes over Not Necessarily Bounded Classes of Functions
    6. 3.6 Vapnik-Červonenkis Classes of Sets and Functions
      1. 3.6.1 Vapnik-Červonenkis Classes of Sets
      2. 3.6.2 VC Subgraph Classes of Functions
      3. 3.6.3 VC Hull and VC Major Classes of Functions
    7. 3.7 Limit Theorems for Empirical Processes
      1. 3.7.1 Some Measurability
      2. 3.7.2 Uniform Laws of Large Numbers (Glivenko-Cantelli Theorems)
      3. 3.7.3 Convergence in Law of Bounded Processes
      4. 3.7.4 Central Limit Theorems for Empirical Processes I: Definition and Some Properties of Donsker Classes of Functions
      5. 3.7.5 Central Limit Theorems for Empirical Processes II: Metric and Bracketing Entropy Sufficient Conditions for the Donsker Property
      6. 3.7.6 Central Limit Theorems for Empirical Processes III: Limit Theorems Uniform in P and Limit Theorems for P-Pre-Gaussian Classes
    8. 3.8 Notes
  12. 4 Function Spaces and Approximation Theory
    1. 4.1 Definitions and Basic Approximation Theory
      1. 4.1.1 Notation and Preliminaries
      2. 4.1.2 Approximate Identities
      3. 4.1.3 Approximation in Sobolev Spaces by General Integral Operators
      4. 4.1.4 Littlewood-Paley Decomposition
    2. 4.2 Ortho-Normal Wavelet Bases
      1. 4.2.1 Multiresolution Analysis of L2
      2. 4.2.2 Approximation with Periodic Kernels
      3. 4.2.3 Construction of Scaling Functions
    3. 4.3 Besov Spaces
      1. 4.3.1 Definitions and Characterisations
      2. 4.3.2 Basic Theory of the Spaces
      3. 4.3.3 Relationships to Classical Function Spaces
      4. 4.3.4 Periodic Besov Spaces on [0,1]
      5. 4.3.5 Boundary-Corrected Wavelet Bases*
      6. 4.3.6 Besov Spaces on Subsets of Rd
      7. 4.3.7 Metric Entropy Estimates
    4. 4.4 Gaussian and Empirical Processes in Besov Spaces
      1. 4.4.1 Random Gaussian Wavelet Series in Besov Spaces
      2. 4.4.2 Donsker Properties of Balls in Besov Spaces
    5. 4.5 Notes
  13. 5 Linear Nonparametric Estimators
    1. 5.1 Kernel and Projection-Type Estimators
      1. 5.1.1 Moment Bounds
      2. 5.1.2 Exponential Inequalities, Higher Moments and Almost-Sure Limit Theorems
      3. 5.1.3 A Distributional Limit Theorem for Uniform Deviations*
    2. 5.2 Weak and Multiscale Metrics
      1. 5.2.1 Smoothed Empirical Processes
      2. 5.2.2 Multiscale Spaces
    3. 5.3 Some Further Topics
      1. 5.3.1 Estimation of Functionals
      2. 5.3.2 Deconvolution
    4. 5.4 Notes
  14. 6 The Minimax Paradigm
    1. 6.1 Likelihoods and Information
      1. 6.1.1 Infinite-Dimensional Gaussian Likelihoods
      2. 6.1.2 Basic Information Theory
    2. 6.2 Testing Nonparametric Hypotheses
      1. 6.2.1 Construction of Tests for Simple Hypotheses
      2. 6.2.2 Minimax Testing of Uniformity on [0,1]
      3. 6.2.3 Minimax Signal-Detection Problems in Gaussian White Noise
      4. 6.2.4 Composite Testing Problems
    3. 6.3 Nonparametric Estimation
      1. 6.3.1 Minimax Lower Bounds via Multiple Hypothesis Testing
      2. 6.3.2 Function Estimation in L∞ Loss
      3. 6.3.3 Function Estimation in Lp -Loss
    4. 6.4 Nonparametric Confidence Sets
      1. 6.4.1 Honest Minimax Confidence Sets
      2. 6.4.2 Confidence Sets for Nonparametric Estimators
    5. 6.5 Notes
  15. 7 Likelihood-Based Procedures
    1. 7.1 Nonparametric Testing in Hellinger Distance
    2. 7.2 Nonparametric Maximum Likelihood Estimators
      1. 7.2.1 Rates of Convergence in Hellinger Distance
      2. 7.2.2 The Information Geometry of the Likelihood Function
      3. 7.2.3 The Maximum Likelihood Estimator over a Sobolev Ball
      4. 7.2.4 The Maximum Likelihood Estimator of a Monotone Density
    3. 7.3 Nonparametric Bayes Procedures
      1. 7.3.1 General Contraction Results for Posterior Distributions
      2. 7.3.2 Contraction Results with Gaussian Priors
      3. 7.3.3 Product Priors in Gaussian Regression
      4. 7.3.4 Nonparametric Bernstein–von Mises Theorems
    4. 7.4 Notes
  16. 8 Adaptive Inference
    1. 8.1 Adaptive Multiple-Testing Problems
      1. 8.1.1 Adaptive Testing with L2-Alternatives
      2. 8.1.2 Adaptive Plug-in Tests for L∞ -Alternatives
    2. 8.2 Adaptive Estimation
      1. 8.2.1 Adaptive Estimation in L2
      2. 8.2.2 Adaptive Estimation in L∞
    3. 8.3 Adaptive Confidence Sets
      1. 8.3.1 Confidence Sets in Two-Class Adaptation Problems
      2. 8.3.2 Confidence Sets for Adaptive Estimators I
      3. 8.3.3 Confidence Sets for Adaptive Estimators II: Self-Similar Functions
      4. 8.3.4 Some Theory for Self-Similar Functions
    4. 8.4 Notes
  17. References
  18. Author Index
  19. Index