Weighted GMM by Gebru et al. (2016)

When each observation $x_i$ is associated with a weight $w_i > 0$, modifying the GMM formulation is required. Gebru et al. (2016) proposed a method to use scaled covariance based on an observation that $$\mathcal{N}\left(x\vert \mu, \Sigma\right)^w \propto \mathcal{N}\left(x\vert \mu, \frac{\Sigma}{w}\right)$$ by considering the positive weight as a role of precision. Currently, we provide a method with fixed weight case only while the paper also considers a Bayesian formalism on the weight using Gamma distribution.

gmm16G(data, k = 2, weight = NULL, ...)

Arguments

data	an $(n\times p)$ matrix of row-stacked observations.
k	the number of clusters (default: 2).
weight	a positive weight vector of length $n$. If `NULL` (default), uniform weight is set.
...	extra parameters including maxiter the maximum number of iterations (default: 10). usediag a logical; covariances are diagonal if `TRUE`, or full covariances are returned for `FALSE` (default: `FALSE`).

Value

a named list of S3 class T4cluster containing

cluster: a length-$n$ vector of class labels (from $1:k$).
mean: a $(k\times p)$ matrix where each row is a class mean.
variance: a $(p\times p\times k)$ array where each slice is a class covariance.
weight: a length-$k$ vector of class weights that sum to 1.
loglkd: log-likelihood of the data for the fitted model.
algorithm: name of the algorithm.

References

Gebru ID, Alameda-Pineda X, Forbes F, Horaud R (2016). “EM Algorithms for Weighted-Data Clustering with Application to Audio-Visual Scene Analysis.” IEEE Transactions on Pattern Analysis and Machine Intelligence, 38(12), 2402--2415. ISSN 0162-8828, 2160-9292.

Examples

# -------------------------------------------------------------
#            clustering with 'iris' dataset
# -------------------------------------------------------------
## PREPARE
data(iris)
X   = as.matrix(iris[,1:4])
lab = as.integer(as.factor(iris[,5]))

## EMBEDDING WITH PCA
X2d = Rdimtools::do.pca(X, ndim=2)$Y  

## CLUSTERING WITH DIFFERENT K VALUES
cl2 = gmm16G(X, k=2)$cluster
cl3 = gmm16G(X, k=3)$cluster
cl4 = gmm16G(X, k=4)$cluster

## VISUALIZATION
opar <- par(no.readonly=TRUE)
par(mfrow=c(1,4), pty="s")
plot(X2d, col=lab, pch=19, main="true label")
plot(X2d, col=cl2, pch=19, main="gmm16G: k=2")
plot(X2d, col=cl3, pch=19, main="gmm16G: k=3")
plot(X2d, col=cl4, pch=19, main="gmm16G: k=4")
par(opar)

data	an \((n\times p)\) matrix of row-stacked observations.
k	the number of clusters (default: 2).
weight	a positive weight vector of length \(n\). If `NULL` (default), uniform weight is set.
...	extra parameters including maxiter the maximum number of iterations (default: 10). usediag a logical; covariances are diagonal if `TRUE`, or full covariances are returned for `FALSE` (default: `FALSE`).