You are previewing Hilbert Space Methods in Signal Processing.
O'Reilly logo
Hilbert Space Methods in Signal Processing

Book Description

This lively and accessible book describes the theory and applications of Hilbert spaces and also presents the history of the subject to reveal the ideas behind theorems and the human struggle that led to them. The authors begin by establishing the concept of 'countably infinite', which is central to the proper understanding of separable Hilbert spaces. Fundamental ideas such as convergence, completeness and dense sets are first demonstrated through simple familiar examples and then formalised. Having addressed fundamental topics in Hilbert spaces, the authors then go on to cover the theory of bounded, compact and integral operators at an advanced but accessible level. Finally, the theory is put into action, considering signal processing on the unit sphere, as well as reproducing kernel Hilbert spaces. The text is interspersed with historical comments about central figures in the development of the theory, which helps bring the subject to life.

Table of Contents

  1. Cover
  2. Title Page
  3. Copyright
  4. Dedication
  5. Contents
  6. Preface
  7. I: Hilbert Spaces
    1. 1: Introduction
      1. 1.1 Introduction to Hilbert spaces
        1. 1.1.1 The basic idea
        2. 1.1.2 Application domains
        3. 1.1.3 Broadbrush structure
        4. 1.1.4 Historical comments
      2. 1.2 Infinite dimensions
        1. 1.2.1 Why understand and study infinity?
        2. 1.2.2 Primer in transfinite cardinals
        3. 1.2.3 Uncountably infinite sets
        4. 1.2.4 Continuum as a power set
        5. 1.2.5 Countable sets and integration
    2. 2: Spaces
      1. 2.1 Space hierarchy: algebraic, metric, geometric
      2. 2.2 Complex vector space
      3. 2.3 Normed spaces and Banach spaces
        1. 2.3.1 Norm and normed space
        2. 2.3.2 Convergence concepts in normed spaces
        3. 2.3.3 Denseness and separability
        4. 2.3.4 Completeness of the real numbers
        5. 2.3.5 Completeness in normed spaces
        6. 2.3.6 Completion of spaces
        7. 2.3.7 Complete normed spaces — Banach spaces
      4. 2.4 Inner product spaces and Hilbert spaces
        1. 2.4.1 Inner product
        2. 2.4.2 Inner product spaces
        3. 2.4.3 When is a normed space an inner product space?
        4. 2.4.4 Orthonormal sets and sequences
        5. 2.4.5 The space l2
        6. 2.4.6 The space L2(Ω)
        7. 2.4.7 Inner product and orthogonality with weighting in L2(Ω)
        8. 2.4.8 Complete inner product spaces — Hilbert spaces
      5. 2.5 Orthonormal polynomials and functions
        1. 2.5.1 Legendre polynomials
        2. 2.5.2 Hermite polynomials
        3. 2.5.3 Complex exponential functions
        4. 2.5.4 Associated Legendre functions
      6. 2.6 Subspaces
        1. 2.6.1 Preamble
        2. 2.6.2 Subsets, manifolds and subspaces
        3. 2.6.3 Vector sums, orthogonal subspaces and projections
        4. 2.6.4 Projection
        5. 2.6.5 Completeness of subspace sequences
      7. 2.7 Complete orthonormal sequences
        1. 2.7.1 Definitions
        2. 2.7.2 Fourier coefficients and Bessel’s inequality
      8. 2.8 On convergence
        1. 2.8.1 Strong convergence
        2. 2.8.2 Weak convergence
        3. 2.8.3 Pointwise convergence
        4. 2.8.4 Uniform convergence
      9. 2.9 Examples of complete orthonormal sequences
        1. 2.9.1 Legendre polynomials
        2. 2.9.2 Bessel functions
        3. 2.9.3 Complex exponential functions
        4. 2.9.4 Spherical harmonic functions
      10. 2.10 Gram-Schmidt orthogonalization
        1. 2.10.1 Legendre polynomial construction
        2. 2.10.2 Orthogonalization procedure
      11. 2.11 Completeness relation
        1. 2.11.1 Completeness relation with weighting
      12. 2.12 Taxonomy of Hilbert spaces
        1. 2.12.1 Non-separable Hilbert spaces
        2. 2.12.2 Separable Hilbert spaces
        3. 2.12.3 The big (enough) picture
  8. II: Operators
    1. 3: Introduction to operators
      1. 3.1 Preamble
        1. 3.1.1 A note on notation
      2. 3.2 Basic presentation and properties of operators
      3. 3.3 Classification of linear operators
    2. 4: Bounded operators
      1. 4.1 Definitions
      2. 4.2 Invertibility
      3. 4.3 Boundedness and continuity
      4. 4.4 Convergence of a sequence of bounded operators
      5. 4.5 Bounded operators as matrices
      6. 4.6 Completing the picture: bounded operator identities
      7. 4.7 Archetype case: Hilbert-Schmidt integral operator
        1. 4.7.1 Some history and context
        2. 4.7.2 Hilbert-Schmidt integral operator definition
        3. 4.7.3 Matrix presentation and relations
      8. 4.8 Road map
      9. 4.9 Adjoint operators
        1. 4.9.1 Special forms of adjoint operators
      10. 4.10 Projection operators
        1. 4.10.1 Finite rank projection operators
        2. 4.10.2 Further properties of projection operators
      11. 4.11 Eigenvalues, eigenvectors and more
    3. 5: Compact operators
      1. 5.1 Definition
      2. 5.2 Compact operator: some explanation
      3. 5.3 Compact or not compact regions
      4. 5.4 Examples of compact and not compact operators
      5. 5.5 Limit of finite rank operators
      6. 5.6 Weak and strong convergent sequences
      7. 5.7 Operator compositions involving compact operators
      8. 5.8 Spectral theory of compact operators
      9. 5.9 Spectral theory of compact self-adjoint operators
    4. 6: Integral operators and their kernels
      1. 6.1 Kernel Fourier expansion and operator matrix representation
      2. 6.2 Compactness
        1. 6.2.1 Approximating kernels with kernels of finite rank
      3. 6.3 Self-adjoint integral operators
        1. 6.3.1 Establishing self-adjointness
        2. 6.3.2 Spectral theory
        3. 6.3.3 Operator decomposition
        4. 6.3.4 Diagonal representation
        5. 6.3.5 Spectral analysis of self-adjoint integral operators
        6. 6.3.6 Synopsis
  9. III: Applications
    1. 7: Signals and systems on 2-sphere
      1. 7.1 Introduction
      2. 7.2 Preliminaries
        1. 7.2.1 2-sphere and spherical coordinates
        2. 7.2.2 Regions on 2-sphere
        3. 7.2.3 Understanding rotation
        4. 7.2.4 Single matrix representation of rotation
        5. 7.2.5 Single rotation along the x, y or z axes
        6. 7.2.6 Intrinsic and extrinsic successive rotations
        7. 7.2.7 Rotation convention used in this book
        8. 7.2.8 Three rotation angles from a single rotation matrix
      3. 7.3 Hilbert space L2(S2)
        1. 7.3.1 Definition of Hilbert space L2(S2)
        2. 7.3.2 Signals on 2-sphere
        3. 7.3.3 Definition of spherical harmonics
        4. 7.3.4 Orthonormality and other properties
        5. 7.3.5 Spherical harmonic coefficients
        6. 7.3.6 Shorthand notation
        7. 7.3.7 Enumeration
        8. 7.3.8 Spherical harmonic Parseval relation
        9. 7.3.9 Dirac delta function on 2-sphere
        10. 7.3.10 Energy per degree
        11. 7.3.11 Vector spectral representation
      4. 7.4 More on spherical harmonics
        1. 7.4.1 Alternative definition of complex spherical harmonics
        2. 7.4.2 Complex spherical harmonics: a synopsis
        3. 7.4.3 Real spherical harmonics
        4. 7.4.4 Unnormalized real spherical harmonics
        5. 7.4.5 Complex Hilbert space with real spherical harmonics …
        6. 7.4.6 Real spherical harmonics: a synopsis
        7. 7.4.7 Visualization of spherical harmonics
        8. 7.4.8 Visual catalog of spherical harmonics
      5. 7.5 Useful subspaces of L2(S2)
        1. 7.5.1 Subspace of bandlimited signals
        2. 7.5.2 Subspace of spacelimited signals
        3. 7.5.3 Subspace of azimuthally symmetric signals
      6. 7.6 Sampling on 2-sphere
        1. 7.6.1 Sampling distribution
        2. 7.6.2 Sampling theorem on 2-sphere
      7. 7.7 Bounded linear operators on 2-sphere
        1. 7.7.1 Systems on 2-sphere
        2. 7.7.2 Matrix representation
        3. 7.7.3 Kernel representation
        4. 7.7.4 Obtaining matrix elements from kernel
      8. 7.8 Spectral truncation operator
      9. 7.9 Spatial truncation operator
      10. 7.10 Spatial masking of signals with a window
        1. 7.10.1 Different operator representations
        2. 7.10.2 Wigner 3j symbols
        3. 7.10.3 Spherical harmonic coefficients of Bhf
      11. 7.11 Rotation operator
        1. 7.11.1 Rotation operator matrix
        2. 7.11.2 Rotation operator kernel
        3. 7.11.3 Important relations pertaining to rotation operation …
        4. 7.11.4 Wigner d-matrix and Wigner D-matrix properties
        5. 7.11.5 Wigner d-matrix symmetry relations
        6. 7.11.6 Fourier series representation of Wigner D-matrix
        7. 7.11.7 Spherical harmonics revisited
        8. 7.11.8 Fast computation
      12. 7.12 Projection into H0(S2)
      13. 7.13 Rotation of azimuthally symmetric signals
      14. 7.14 Operator classification based on operator matrix
      15. 7.15 Quadratic functionals on 2-sphere
      16. 7.16 Classification of quadratic functionals
    2. 8: Advanced topics on 2-sphere
      1. 8.1 Introduction
      2. 8.2 Time-frequency concentration reviewed
        1. 8.2.1 Preliminaries
        2. 8.2.2 Problem statement
        3. 8.2.3 Answer to time concentration question Q1
        4. 8.2.4 Answer to frequency concentration question Q2
        5. 8.2.5 Answer to minimum angle question Q3
        6. 8.2.6 Answer to time-frequency concentration question Q4 …
      3. 8.3 Introduction to concentration problem on 2-sphere
      4. 8.4 Optimal spatial concentration of bandlimited signals
        1. 8.4.1 Orthogonality relations
        2. 8.4.2 Eigenfunction kernel representation
      5. 8.5 Optimal spectral concentration of spacelimited signals
      6. 8.6 Area-bandwidth product or spherical Shannon number
      7. 8.7 Operator formulation
      8. 8.8 Special case: azimuthally symmetric polar cap region
      9. 8.9 Azimuthally symmetric concentrated signals in polar cap
      10. 8.10 Uncertainty principle for azimuthally symmetric functions
      11. 8.11 Comparison with time-frequency concentration problem
      12. 8.12 Franks generalized variational framework on 2-sphere
        1. 8.12.1 Variational problem formulation
        2. 8.12.2 Stationary points of Lagrange functional G(f)
        3. 8.12.3 Elaborating on the solution
      13. 8.13 Spatio-spectral analysis on 2-sphere
        1. 8.13.1 Introduction and motivation
        2. 8.13.2 Procedure and SLSHT definition
        3. 8.13.3 SLSHT expansion
        4. 8.13.4 SLSHT distribution and matrix representation
        5. 8.13.5 Signal inversion
      14. 8.14 Optimal spatio-spectral concentration of window function
        1. 8.14.1 SLSHT on Mars topographic data
    3. 9: Convolution on 2-sphere
      1. 9.1 Introduction
      2. 9.2 Convolution on real line revisited
      3. 9.3 Spherical convolution of type 1
        1. 9.3.1 Type 1 convolution operator matrix and kernel
      4. 9.4 Spherical convolution of type 2
        1. 9.4.1 Characterization of type 2 convolution
        2. 9.4.2 Equivalence between type 1 and 2 convolutions
      5. 9.5 Spherical convolution of type 3
        1. 9.5.1 Alternative characterization of type 3 convolution
      6. 9.6 Commutative anisotropic convolution
        1. 9.6.1 Requirements for convolution on 2-sphere
        2. 9.6.2 A starting point
        3. 9.6.3 Establishing commutativity
        4. 9.6.4 Graphical depiction
        5. 9.6.5 Spectral analysis
        6. 9.6.6 Operator matrix elements
        7. 9.6.7 Special case — one function is azimuthally symmetric
      7. 9.7 Alt-azimuth anisotropic convolution on 2-sphere
        1. 9.7.1 Background
    4. 10: Reproducing kernel Hilbert spaces
      1. 10.1 Background to RKHS
        1. 10.1.1 Functions as sticky-note labels
        2. 10.1.2 What is wrong with L2(Ω)?
      2. 10.2 Constructing Hilbert spaces from continuous functions
        1. 10.2.1 Completing continuous functions
      3. 10.3 Fourier weighted Hilbert spaces
        1. 10.3.1 Pass the scalpel, nurse
        2. 10.3.2 Forming a new inner product
        3. 10.3.3 Inner product conditions
        4. 10.3.4 Finite norm condition
        5. 10.3.5 Setting up an isomorphism
        6. 10.3.6 Function test condition
        7. 10.3.7 Fundamental operator
        8. 10.3.8 Weighting sequence considerations
        9. 10.3.9 Orthonormal sequences
        10. 10.3.10 Isomorphism equations
      4. 10.4 O Kernel, Kernel, wherefore art thou Kernel?
        1. 10.4.1 Kernel of an integral operator
        2. 10.4.2 Mercer’s theorem
        3. 10.4.3 Square summable weighting
      5. 10.5 Reproducing kernel Hilbert spaces
        1. 10.5.1 Complete orthonormal functions
        2. 10.5.2 Completeness relation and Dirac delta functions
        3. 10.5.3 Reproducing kernel property
        4. 10.5.4 Feature map and kernel trick
      6. 10.6 RKHS on 2-sphere
        1. 10.6.1 RKHS construction
        2. 10.6.2 Isomorphism
        3. 10.6.3 Two representation of vectors
        4. 10.6.4 Closed-form isotropic reproducing kernels
      7. 10.7 RKHS synopsis
  10. Answers to problems in Part I
    1. Answers to problems in Chapter 1
    2. Answers to problems in Chapter 2
  11. Answers to problems in Part II
    1. Answers to problems in Chapter 3
    2. Answers to problems in Chapter 4
    3. Answers to problems in Chapter 5
    4. Answers to problems in Chapter 6
  12. Answers to problems in Part III
    1. Answers to problems in Chapter 7
    2. Answers to problems in Chapter 8
    3. Answers to problems in Chapter 9
    4. Answers to problems in Chapter 10
  13. Bibliography
  14. Notation
  15. Index