Variational Approximation

変分近似

概要

濾波推定値 \(\hat{\mathbf w}_{t/t}\) \[\hat{\mathbf w}_{t/t}=\mathbf P_{t/t}\left(\mathbf P_{t/t-1}^{-1}\hat{\mathbf w}_{t/t-1}+(y_t-1/2)\mathbf x_t\right)\]

推定誤差共分散行列 \(\mathbf P_{t/t}\)

\[\mathbf P_{t/t}=\mathbf P_{t/t-1}-\frac{2\lambda(\xi_t)}{1+2\lambda(\xi_t)\mathbf x_t^T\mathbf P_{t/t-1}\mathbf x_t}\left(\mathbf P_{t/t-1}\mathbf x_t\right)\left(\mathbf P_{t/t-1}\mathbf x_t\right)^T \]

\[\mathbf P_{t/t}^{-1}=\mathbf P_{t/t-1}^{-1}+2\lambda(\xi_t)\mathbf x_t\mathbf x_t^T\]

一段予測推定値 \(\hat{\mathbf w}_{t/t-1}\) を使う変分パラメータ (pre) \[\xi_t=\sqrt{\mathbf x_t^T\left(\mathbf P_{t/t-1}+\hat{\mathbf w}_{t/t-1}\hat{\mathbf w}_{t/t-1}^T\right)\mathbf x_t}\]

濾波推定値 \(\hat{\mathbf w}_{t/t}\) を使う変分パラメータ (EM) \[\xi_t=\sqrt{\mathbf x_t^T\left(\mathbf P_{t/t}+\hat{\mathbf w}_{t/t}\hat{\mathbf w}_{t/t}^T\right)\mathbf x_t}\]

各変数の情報の利用

\(\!\)	\(\mathbf P_{t/t}\)	\(\hat{\mathbf w}_{t/t}\)	\(\mathbf P_{t/t-1}\)	\(\hat{\mathbf w}_{t/t-1}\)	\(\mathbf x_t\)	\(y_t\)	\(\xi_t\)
\(\mathbf P_{t/t}\)	\(\!\)	\(\!\)	\(\bigcirc\)	\(\!\)	\(\bigcirc\)	\(\!\)	\(\bigcirc\)
\(\hat{\mathbf w}_{t/t}\)	\(\bigcirc\)	\(\!\)	\(\bigcirc\)	\(\bigcirc\)	\(\bigcirc\)	\(\bigcirc\)	\(\bigcirc\)
\(\xi_t\) (pre)	\(\!\)	\(\!\)	\(\bigcirc\)	\(\bigcirc\)	\(\bigcirc\)	\(\!\)	\(\!\)
\(\xi_t\) (EM)	\(\bigcirc\)	\(\bigcirc\)	\(\!\)	\(\!\)	\(\bigcirc\)	\(\!\)	\(\!\)

関数等

---

source

lam

def lam(
    x:Float[Array, ''], # $x$
)->Float[Array, '']: # $\lambda(x)$

*\(\!\)** Lambda 関数 \[\lambda(x)=\frac{1}{2x}\left[\sigma(x)-\frac{1}{2}\right]\] *\(\!\)

---

source

Ptt

def Ptt(
    Ptm:Float[Array, 'N N'], # $\mathbf P_{t/t-1}$
    x:Float[Array, 'N'], # $\mathbf x_t$
    xi:Float[Array, ''], # $\xi_t$
)->Float[Array, 'N N']: # $\mathbf P_{t/t}$

*\(\!\)** 推定誤差共分散行列 \(\mathbf P_{t/t}\)

\[\mathbf P_{t/t}=\mathbf P_{t/t-1}-\frac{2\lambda(\xi_t)}{1+2\lambda(\xi_t)\mathbf x_t^T\mathbf P_{t/t-1}\mathbf x_t}\left(\mathbf P_{t/t-1}\mathbf x_t\right)\left(\mathbf P_{t/t-1}\mathbf x_t\right)^T \]

\[\mathbf P_{t/t}^{-1}=\mathbf P_{t/t-1}^{-1}+2\lambda(\xi_t)\mathbf x_t\mathbf x_t^T\] *\(\!\)

---

source

wtt

def wtt(
    Ptm:Float[Array, 'N N'], # $\mathbf P_{t/t-1}$
    Ptt_:Float[Array, 'N N'], # $\mathbf P_{t/t}$
    w:Float[Array, 'N'], # $\hat{\mathbf w}_{t/t-1}$
    x:Float[Array, 'N'], # $\mathbf x_t$
    y:Float[Array, ''], # $y_t$
)->Float[Array, 'N']: # $\hat{\mathbf w}_{t/t}$

*\(\!\)** 濾波推定値 \(\hat{\mathbf w}_{t/t}\) \[\hat{\mathbf w}_{t/t}=\mathbf P_{t/t}\left(\mathbf P_{t/t-1}^{-1}\hat{\mathbf w}_{t/t-1}+(y_t-1/2)\mathbf x_t\right)\] *\(\!\)

---

source

xit

def xit(
    Cov:Float[Array, 'N N'], # $\boldsymbol\Sigma$
    w:Float[Array, 'N'], # $\hat{\mathbf w}$
    x:Float[Array, 'N'], # $\mathbf x_t$
)->Float[Array, '']: # $\xi_t$

*\(\!\)** 変分パラメータ \(\xi_t\) \[\xi_t=\sqrt{\mathbf x_t^T\mathbb E[\mathbf w\mathbf w^T]\mathbf x_t}=\sqrt{\mathbf x_t^T\left(\boldsymbol \Sigma+\hat{\mathbf w}\hat{\mathbf w}^T\right)\mathbf x_t}\] *\(\!\)

---

source

VApre_out

def VApre_out(
    args:VAR_POSITIONAL, kwargs:VAR_KEYWORD
):

*\(\!\)** VApre 関数の返り値

\(\!\)	Type	Details
W	Float[Array, ‘T N’]	\(\{\hat{\mathbf w}_{t/t}\}_{t=0,\ldots,T-1}\)
P	Float[Array, ‘T N N’]	\(\{\mathbf P_{t/t}\}_{t=0,\ldots,T-1}\)
Xi	Float[Array, ‘T’]	\(\{\xi_t\}_{t=0,\ldots,T-1}\)

*\(\!\)

---

source

VApre

def VApre(
    N:int, # $N$
    T:int, # $T$
    x:Float[Array, '{T} {N}'], # $\{ \mathbf x_t \}_{t=0,\ldots,T-1}$
    y:Float[Array, '{T}'], # $\{ y_t \}_{t=0,\ldots,T-1}$
    G:Float[Array, '{N} {N}'], # $\boldsymbol\Gamma$
    w0:Float[Array, '{N}'], # $\hat{\mathbf w}_{0/-1}$
    P0:Float[Array, '{N} {N}'], # $\mathbf P_{0/-1}$
)->VApre_out:

*\(\!\)** 一段予測推定値 \(\hat{\mathbf w}_{t/t-1}\) を使う変分近似法 \[\xi_t=\sqrt{\mathbf x_t^T\left(\mathbf P_{t/t-1}+\hat{\mathbf w}_{t/t-1}\hat{\mathbf w}_{t/t-1}^T\right)\mathbf x_t}\] *\(\!\)

---

source

VAEM_out

def VAEM_out(
    args:VAR_POSITIONAL, kwargs:VAR_KEYWORD
):

*\(\!\)** VAEM 関数の返り値

\(\!\)	Type	Details
W	Float[Array, ‘T N’]	\(\{\hat{\mathbf w}_{t/t}\}_{t=0,\ldots,T-1}\)
P	Float[Array, ‘T N N’]	\(\{\mathbf P_{t/t}\}_{t=0,\ldots,T-1}\)
Xi	Float[Array, ‘T’]	\(\{\xi_t\}_{t=0,\ldots,T-1}\)
Iters	Int[Array, ‘T’]	\(\{\mathrm{Iter}_t\}_{t=0,\ldots,T-1}\)

*\(\!\)

---

source

VAEM

def VAEM(
    N:int, # $N$
    T:int, # $T$
    x:Float[Array, '{T} {N}'], # $\{ \mathbf x_t \}_{t=0,\ldots,T-1}$
    y:Float[Array, '{T}'], # $\{ y_t \}_{t=0,\ldots,T-1}$
    G:Float[Array, '{N} {N}'], # $\boldsymbol\Gamma$
    w0:Float[Array, '{N}'], # $\hat{\mathbf w}_{0/-1}$
    P0:Float[Array, '{N} {N}'], # $\mathbf P_{0/-1}$
    epsilon:Float[Array, ''], # $\epsilon\ge |\xi\\^{\text{new}}_t-\xi\\^{\text{old}}_t|$
    max_iter:int=100, # 繰り返し回数の上限
)->VAEM_out:

*\(\!\)** 濾波推定値 \(\hat{\mathbf w}_{t/t}\) を使う変分近似法。EMアルゴリズムを使う。 \[\xi_t=\sqrt{\mathbf x_t^T\left(\mathbf P_{t/t}+\hat{\mathbf w}_{t/t}\hat{\mathbf w}_{t/t}^T\right)\mathbf x_t}\] *\(\!\)