본문 바로가기

Study/Lecture - Basic

 (38)
W9.L5. Hidden Markov Model - Baum-Welch Algorithm | Learning Parameters with Only $X$ X 만 주어져 있을 때 모든 Parameters 알아내보자! 이전에는 $\pi, a, b$ 안다는 가정: 이 말은 즉슨 $X$ 와 $Z$ 를 모두 안다는 가정 ... 현실에서는 어려움 EM Algorithm Iteratively optimizing $\hat{\pi}, \hat{a}, \hat{b}$ and $Z$ 1. Finding the optimized $\hat{\pi}, \hat{a}, \hat{b}$ with $X$ 2. Finding the most probable $Z$ with $X, \hat{\pi}, \hat{a}, \hat{b}$ | EM Algorithm Initialize $\theta^{0}$ to an arbit..
W9.L4. Hidden Markov Model - Viterbi Decoding | Viterbi Decoding The most probable assignment to a single latent variable $z_t$, given the whole observed sequence X 특정 시간 t 의 Latent factor 가 특정 k cluster 에 속할 확률 Recursive formation $$ \begin{align*} P(z_{t}^{k}=1, X) &= \alpha_{t}^{k} \cdot \beta_{t}^{k} \\ &= (b_{k, x_t} \cdot \sum_{i} \alpha_{t-1}^{i} \cdot a_{i, k}) \cdot (\sum_{i} \alpha_{k, i} \cdot b_{i, x_{t}} \cdot \beta_{t+1}^{i}) ..
W9.L3. Hidden Markov Model - Forward-Backward Probability | Forward Probability Calculation Forward probability : $\alpha_{t}^{k}$ Dimension: Time x States Initialize $ \alpha_{1}^{k}=b_{k, x_1}\pi_{k} $ Iterate until time T $ \alpha_{t}^{k}=b_{k, x_t} \cdot \sum_{i}\alpha_{t-1}^{i} \cdot a_{i, k} $ Proof of correctness (Answer of evaluation question without Z) $\sum_{i} \alpha_{T}^{i} = \sum_{i}P(x_1, ..., x_T, z_{T}^{i}=1) = P(x_1, ..., x_t)$ 특정 시간 T..
W9.L2. Hidden Markov Model - Joint, Marginal Probability | Main Questions on HMM Given the topology of the M Evaluation question - Given $\pi,\ a,\ b,\ X$ - Find $P(X\mid M, \pi, a, b)$ - How much is $X$ likely to be observed in the trained model? Decoding question - Given $\pi,\ a,\ b,\ X$ - Find $argmax_{z}\ P(Z\mid X, M, \pi, a, b)$ - What would be the most probable sequences of latent states? - 보이는 데이터들을 가장 잘 설명할 수 있는 Latent 들의 sequence 를 알아내봐라! L..
W9.L1. Hidden Markov Model - Concept | INTRO 앞서 Clustering ... 2D Space, 3D Space ... 예제들을 살펴봤었는데 공간과 더불어 시간이라는 개념이 추가되었을 때를 한 번 살펴보겠다. | Time Series Data for GMM N 개의 Data Point X N 개의 Data Point 들마다 Z ... 어떤 Cluster 에 Assign 되었는지를 나타내는 Latent Factor 이제 새롭게 Temporal relation / Graph notation 으로 나타내보면 다음 Time 의 Latent 에 영향 주는 형태 Observation $X$ - $X_{1},\ X_{2},\ ... \ , X_{T}$ : observation from time $1$ to time $T$ - $x_{i} \in \..
W8.L8-9. Gaussian Mixture Model - EM Algorithm | Inference with Latent Variables Difference between classification and clustering Complete set of variables: { $X, Z$ } $X$ : observed variables $Z$ : hidden (latent) variables $\theta$ : parameters for distributions Classification (supervised learning) $ P(X \mid \theta) $ → $ \textup{ln}\ P(X \mid \theta) $ Clustering (unsupervised learning) 모르는 녀석 $Z$ 를 어떻게 하면 좋을까 라는 고민이 함께 필요 $ P(X \mid \th..
W8.L5-7. Gaussian Mixture Model - G.M.M | Gaussian Mixture Model Let's assume that the data points are drawn from a mixture distribution of multiple multivariate Gaussian distributions $$ \begin{align*} P(x) &= \sum_{k=1}^{K} P(z_k) \cdot P(x \mid z ) \\ &= \sum_{k=1}^{K} \pi_{k}\cdot N (x \mid \mu_{k},\ \sigma_{k} ) \\ \end{align*} $$ 어떤 특정 $x_n$ 이 주어졌을 때, $k$ 번째 Cluster 에 Assign 될 확률은 무엇일까? $x_n$ 이 Given 일 때, $z_k = 1$ 일 확률은 무엇일까? $..
W8.L3-4. Gaussian Mixture Model - Multinomial Distribution | Multinomial Distribution Binary variable - Selecting 0 or 1 → binomial distribution - Selecting 0, 1, 2, ... → multinomial distribution How about K options? Multinomial distribution (A generalization of binomial distribution) 주사위 한 번 던지는 경우 One observation: $X_1=(0, 0, 1, 0, 0, 0)$ $P(X \mid \mu) = \prod_{k=1}^{K} \mu_{k}^{x_k}$ such that $\mu_{k} \geqslant 0,\ \sum_{k}\mu_{k}=1$ $\sum_{k} x_{..

티스토리툴바