Machine Learning

Problems

17.1 Let

$\mu :=\mathbb{E}[f(\mathbf{x})]=\int f(\mathit{x})p(\mathit{x})\text{d}\mathit{x}$

and q(x) be the proposal distribution. Show that if

$w(\mathit{x}):=\frac{p(\mathit{x})}{q(\mathit{x})},$

and

$\widehat{\mu }=\frac{1}{N}\sum _{i=1}^{N}w({\mathit{x}}_i)f({\mathit{x}}_i),$

then the variance

${\sigma }_f^2=\mathbb{E}\left[{\left(\widehat{\mathrm{\mu }}-\mathbb{E}\left[\widehat{\mathrm{\mu }}\right]\right)}^2\right]=\frac{1}{N}\left(\int \frac{f^2(\mathit{x})p^2(\mathit{x})}{q(\mathit{x})}d\mathit{x}-{\mu }^2\right).$

Observe that if f²(x)p²(x) goes to zero slower than q(x), then for fixed N, ${\sigma }_f^2\to \infty$.

17.2 In importance sampling, with weights defined as

$w(\mathit{x})=\frac{\varphi (}{}$