\(\)where \(\theta\)m(j-1) and \(\theta\)k(j-1) denote parameters of components m and k, in the iteration j−1, respectively.
M-step: Given the expected values of hm(j)(xi) obtained in the E-step the set of parameters Θ(j) is calculated as:
\(\)\(\)\(\)
\(\)\(\alpha_{m}^{(j)}={1\over N}\sum\limits_{i=1}^{N}h_{m}^{(j)}({\bf x}^{i})\)
\(\)
\(\mu_{m}^{(j)}={\sum\nolimits_{i=1}^{N}h_{m}^{(j)}({\bf x}^{i}){\ast}{\bf x}^{i} \over \sum\nolimits_{i=1}^{N}h_{m}^{(j)}({\bf x}^{i})}\)
\(\)
\(\Sigma_{m}^{(j)}={\sum\nolimits_{i=1}^{N}h_{m}^{(j)}({\bf x}^{i})({\bf x}^{i}-\mu_{m}^{(j)})({\bf x}^{i}-\mu_{m}^{(j)})^{T} \over \sum\nolimits_{i=1}^{N}h_{m}^{(j)}({\bf x}^{i})}\)
Convergence check: The loglikelihood logp(X|Θ(j)) is computed according to likelihood function. The algorithm is terminated if the following convergence criterion is met.
\({\log p(X\vert \Theta^{(j)})-\log p(X\vert \Theta^{(j-1)}) \over \log p(X\vert \Theta(j))}<\epsilon,\)
where ϵ≪1 is a user defined termination threshold. If the convergence criterion is not met algorithm proceeds to Step 1.
For our question we have value of k=2 i.e. we have two components of gaussian mixture.The above generalized method for K components of gaussian mixture can be used to derive the formula for two component model.
DATA ANALYSIS
In data analysis, i will analyze the results on the basis of the values of the parameter space i.e. Θ={μ1,Σ1,μ2,Σ2,\(\pi\)} which i have calculated using the EM algorithm.(The code has been attached in the appendix for the reference of algorithm).
First I initialized the means and the covariance of the components using k-means and giving input of initial values.Then i have calculated the expected value using the E-step formula and then using the M-step formula, i have calculated the value of Θ={μ1,Σ1,μ2,Σ2,\(\pi\)} for the next stage.
Then giving the convergence condition, i have formulated the while loop which will rum till the convergence of these values does not take place.Once the convergence take place, then we will get the final values of Θ={μ1,Σ1,μ2,Σ2,\(\pi\)} using EM-Algorithm.
INITIALISATION:
My initial values that are used by me for the initialisation step are as follows:
pi= [ 0.5 , 0.5 ]
mu1= [ 214 , 130 , 130.5 , 8.1 , 9.7 , 140.5 ]
mu2= [ 214 , 130 , 130.5 , 8.1 , 9.7 , 140.5 ]
sigma1= [ 1, 1, 1, 1, 1, 1 ; 1, 1, 1, 1, 1, 1 ; 1, 1, 1, 1, 1, 1 ; 1, 1, 1, 1, 1, 1 ; 1, 1, 1, 1, 1, 1 ; 1, 1, 1, 1, 1, 1] (6 by 6 matrix)
sigma2= [ 1, 1, 1, 1, 1, 1 ; 1, 1, 1, 1, 1, 1 ; 1, 1, 1, 1, 1, 1 ; 1, 1, 1, 1, 1, 1 ; 1, 1, 1, 1, 1, 1 ; 1, 1, 1, 1, 1, 1] (6 by 6 matrix)
CLASSIFICATION METHOD and RESULTS
In the classification step, based on the final values of parameter space Θ={μ1,Σ1,μ2,Σ2,\(\pi\)}, I have calculated the expected values of all the 200 data points and classified them as fake currency (if the value is close to 1) and original currency (if the value is close to 0).Also by comparing this data with the original data we can calculate the number of data points which show a difference in their original status of currency and see how precise is the EM-Algorithm.
RUNNING E-STEP AND M-STEP ALGORITHM:
After running the EM-Algorithm code, the final values of parameter space that i got are as follows: