You are previewing Theory and Methods of Statistics.
O'Reilly logo
Theory and Methods of Statistics

Book Description

Theory and Methods of Statistics covers essential topics for advanced graduate students and professional research statisticians. This comprehensive resource covers many important areas in one manageable volume, including core subjects such as probability theory, mathematical statistics, and linear models, and various special topics, including nonparametrics, curve estimation, multivariate analysis, time series, and resampling. The book presents subjects such as "maximum likelihood and sufficiency," and is written with an intuitive, heuristic approach to build reader comprehension. It also includes many probability inequalities that are not only useful in the context of this text, but also as a resource for investigating convergence of statistical procedures.



  • Codifies foundational information in many core areas of statistics into a comprehensive and definitive resource
  • Serves as an excellent text for select master’s and PhD programs, as well as a professional reference
  • Integrates numerous examples to illustrate advanced concepts
  • Includes many probability inequalities useful for investigating convergence of statistical procedures

Table of Contents

  1. Cover image
  2. Title page
  3. Table of Contents
  4. Copyright
  5. Dedication
  6. Preface
  7. 1: Probability Theory
    1. Abstract
    2. 1.1 Random Experiments and Their Outcomes
    3. 1.2 Set Theory
    4. 1.3 Axiomatic Definition of Probability
    5. 1.4 Some Simple Propositions
    6. 1.5 Equally Likely Outcomes in Finite Sample Space
    7. 1.6 Conditional Probability and Independence
    8. 1.7 Random Variables and Their Distributions
    9. 1.8 Expected Value, Variance, Covariance, and Correlation Coefficient
    10. 1.9 Moments and the Moment Generating Function
    11. 1.10 Independent Random Variables and Conditioning When There Is Dependence
    12. 1.11 Transforms of Random Variables and Their Distributions
    13. Exercises
  8. 2: Some Common Probability Distributions
    1. Abstract
    2. 2.1 Discrete Distributions
    3. 2.2 Continuous Distributions
    4. Exercises
  9. 3: Infinite Sequences of Random Variables and Their Convergence Properties
    1. Abstract
    2. 3.1 Introduction
    3. 3.2 Modes of Convergence
    4. 3.3 Probability Inequalities
    5. 3.4 Asymptotic Normality: The Central Limit Theorem and Its Generalizations
    6. Exercises
  10. 4: Basic Concepts of Statistical Inference
    1. Abstract
    2. 4.1 Population and Random Samples
    3. 4.2 Parametric and Nonparametric Models
    4. 4.3 Problems of Statistical Inference
    5. 4.4 Statistical Decision Functions
    6. 4.5 Sufficient Statistics
    7. 4.6 Optimal Decision Rules
    8. Exercises
  11. 5: Point Estimation in Parametric Models
    1. Abstract
    2. 5.1 Optimality Under Unbiasedness, Squared-Error Loss, UMVUE
    3. 5.2 Lower Bound for the Variance of an Unbiased Estimator
    4. 5.3 Equivariance
    5. 5.4 Bayesian Estimation Using Conjugate Priors
    6. 5.5 Methods of Estimation
    7. Exercises
  12. 6: Hypothesis Testing
    1. Abstract
    2. 6.1 Early History
    3. 6.2 Basic Concepts
    4. 6.3 Simple Null Hypothesis vs Simple Alternative: Neyman-Pearson Lemma
    5. 6.4 UMP Tests for One-Sided Hypotheses Against One-Sided Alternatives in Monotone Likelihood Ratio Families
    6. 6.5 Unbiased Tests
    7. 6.6 Generalized Neyman-Pearson Lemma
    8. 6.7 UMP Unbiased Tests for Two-Sided Problems
    9. 6.8 Locally Best Tests
    10. 6.9 UMP Unbiased Tests in the Presence of Nuisance Parameters: Similarity and Completeness
    11. 6.10 The <span xmlns="http://www.w3.org/1999/xhtml" xmlns:epub="http://www.idpf.org/2007/ops" class="italic">p</span>-Value: Another Way to Report the Result of a Test-Value: Another Way to Report the Result of a Test
    12. 6.11 Sequential Probability Ratio Test
    13. 6.12 Confidence Sets
    14. Exercises
  13. 7: Methods Based on Likelihood and Their Asymptotic properties
    1. Abstract
    2. 7.1 Asymptotic Properties of the MLEs: Consistency and Asymptotic Normality
    3. 7.2 Likelihood Ratio Test
    4. 7.3 Asymptotic Properties of MLE and LRT Based on Independent Nonidentically Distributed Data
    5. 7.4 Frequency X<sup xmlns="http://www.w3.org/1999/xhtml" xmlns:epub="http://www.idpf.org/2007/ops">2</sup>
    6. Exercises
  14. 8: Distribution-Free Tests for Hypothesis Testing in Nonparametric Families
    1. Abstract
    2. 8.1 Ranks and Order Statistics
    3. 8.2 Locally Most Powerful Rank Tests
    4. 8.3 Tests Based on Empirical Distribution Function
    5. Exercises
  15. 9: Curve Estimation
    1. Abstract
    2. 9.1 Introduction
    3. 9.2 Density Estimation
    4. 9.3 Regression Estimation
    5. 9.4 Nearest Neighbor Approach
    6. 9.5 Curve Estimation in Higher Dimension
    7. 9.6 Curve Estimation Using Local Polynomials
    8. 9.7 Estimation of Survival Function and Hazard Rates Under Random Right-Censoring
    9. Exercises
  16. 10: Statistical Functionals and Their Use in Robust Estimation
    1. Abstract
    2. 10.1 Introduction
    3. 10.2 Functional Delta Method
    4. 10.3 The <span xmlns="http://www.w3.org/1999/xhtml" xmlns:epub="http://www.idpf.org/2007/ops" class="italic">L</span>-Estimators-Estimators
    5. 10.4 The <span xmlns="http://www.w3.org/1999/xhtml" xmlns:epub="http://www.idpf.org/2007/ops" class="italic">M</span>-Estimators-Estimators
    6. 10.5 A Relation Between <span xmlns="http://www.w3.org/1999/xhtml" xmlns:epub="http://www.idpf.org/2007/ops" class="italic">L</span>-Estimators and -Estimators and <span xmlns="http://www.w3.org/1999/xhtml" xmlns:epub="http://www.idpf.org/2007/ops" class="italic">M</span>-Estimators-Estimators
    7. 10.6 The Remainder Term <span xmlns="http://www.w3.org/1999/xhtml" xmlns:epub="http://www.idpf.org/2007/ops" class="italic">R</span><sub xmlns="http://www.w3.org/1999/xhtml" xmlns:epub="http://www.idpf.org/2007/ops"><span class="italic">n</span></sub>
    8. 10.7 The Jackknife and the Bootstrap
    9. Exercises
  17. 11: Linear Models
    1. Abstract
    2. 11.1 Introduction
    3. 11.2 Examples of Gauss-Markov Models
    4. 11.3 Gauss-Markov Models: Estimation
    5. 11.4 Decomposition of Total Sum of Squares
    6. 11.5 Estimation Under Linear Restrictions on β
    7. 11.6 Gauss-Markov Models: Inference
    8. 11.7 Analysis of Covariance
    9. 11.8 Model Selection
    10. 11.9 Some Alternate Methods for Regression
    11. 11.10 Random- and Mixed-Effects Models
    12. 11.11 Inference: Examples From Mixed Models
    13. Exercises
  18. 12: Multivariate Analysis
    1. Abstract
    2. 12.1 Introduction
    3. 12.2 Wishart Distribution
    4. 12.3 The Role of Multivariate Normal Distribution
    5. 12.4 One-Sample Inference
    6. 12.5 Two-Sample Problem
    7. 12.6 One-Factor MANOVA
    8. 12.7 Two-Factor MANOVA
    9. 12.8 Multivariate Linear Model
    10. 12.9 Principal Components Analysis
    11. 12.10 Factor Analysis
    12. 12.11 Classification and Discrimination
    13. 12.12 Canonical Correlation Analysis
    14. Exercises
  19. 13: Time Series
    1. Abstract
    2. 13.1 Introduction
    3. 13.2 Concept of Stationarity
    4. 13.3 Estimation of the Mean and the Autocorrelation Function
    5. 13.4 Partial Autocorrelation Function (PACF)
    6. 13.5 Causality and Invertibility
    7. 13.6 Forecasting
    8. 13.7 ARIMA Models and Forecasting
    9. 13.8 Parameter Estimation
    10. 13.9 Selection of an Appropriate ARMA model
    11. 13.10 Spectral Analysis
    12. Exercises
  20. Appendix A: Results From Analysis and Probability
    1. A.1 Some Important Results in Integration Theory
    2. A.2 Convex Functions
    3. A.3 Stieltjes Integral
    4. A.4 Characteristic Function, Weak Law of Large Number, and Central Limit Theorem
    5. A.5 Weak Convergence of Probabilities on C[0,1]
  21. Appendix B: Basic Results From Matrix Algebra
    1. B.1 Some Elementary Facts
    2. B.2 Eigenvalues and Eigenvectors
    3. B.3 Functions of Symmetric Matrices
    4. B.4 Generalized Eigenvalues
    5. B.5 Matrix Derivatives
    6. B.6 Orthogonal Projection
    7. B.7 Distribution of Quadratic Forms
  22. Bibliography
  23. Index