Author: Anders Ellern Bilgrau
Date: 2019-08-28

# Unsupervised clustering with general GMCMs

#### Anders Ellern Bilgrau

#### 2019-08-28

This is a quick tutorial for using the general GMCM for unsupervised clustering.

## Initialization

The GMCM1 package is loaded.

    #install.packages("GMCM")  # Uncomment to install the GMCM package
    library("GMCM")

If GMCM is not installed, please uncomment the above line and rerun the script to install from CRAN. The development version can be installed from GitHub following the instructions there.

## Simulation of data

First, we simulate some toy data. We wish to simulate, say, 1000 observations of 2-dimensions each of which stems from one of 3 components. In order to do so, we construct a parameter object `theta` and simulate from the GMCM.

    true_theta <- rtheta(m = 3, d = 2)
    plot(true_theta)

    ds <- SimulateGMCMData(n = 1000, theta = true_theta)
    str(ds)

    ## List of 4
    ##  $ u    : num [1:1000, 1:2] 0.0452 0.5879 0.8876 0.0706 0.4786 ...
    ##  $ z    : num [1:1000, 1:2] -2.71 5.19 7.37 -1.77 4.46 ...
    ##  $ K    : int [1:1000] 3 3 1 3 1 1 3 1 1 1 ...
    ##  $ theta:List of 5
    ##   ..$ m    : num 3
    ##   ..$ d    : num 2
    ##   ..$ pie  : Named num [1:3] 0.5084 0.0989 0.3928
    ##   .. ..- attr(*, "names")= chr [1:3] "pie1" "pie2" "pie3"
    ##   ..$ mu   :List of 3
    ##   .. ..$ comp1: num [1:2] 5.99 -8.08
    ##   .. ..$ comp2: num [1:2] 3.97 -3.4
    ##   .. ..$ comp3: num [1:2] 1.31 4.03
    ##   ..$ sigma:List of 3
    ##   .. ..$ comp1: num [1:2, 1:2] 2.55 -0.44 -0.44 3.77
    ##   .. ..$ comp2: num [1:2, 1:2] 1.06 -1.6 -1.6 3.19
    ##   .. ..$ comp3: num [1:2, 1:2] 11.2561 -0.0595 -0.0595 10.2816
    ##   ..- attr(*, "class")= chr "theta"

As can be seen, the `SimulateGMCMData` function returns a `list` containing the copula observations `u`, the unobserved process `z`, the latent groups `K`, and the `theta` used for simulation.

Plotting the true copula realizations and coloring by the true classes shows what we intend to estimate and recover:

    plot(ds$u, col = ds$K)

A ranking of `u` (or `z`) corresponds to what would be the data in a real application.

    uhat <- Uhat(ds$u)
    plot(uhat)

## Initial parameters

To fit a general GMCM, we must choose some initial parameters. The `choose.theta` is a (sometimes) helpful default we here invoke explicitly:

    init_theta <- choose.theta(uhat, m = 3)
    print(init_theta)

    ## theta object with d = 2 dimensions and m = 3 components:
    ##
    ## $pie
    ##  pie1  pie2  pie3
    ## 0.300 0.345 0.355
    ##
    ## $mu
    ## $mu$comp1
    ## [1] 0 0
    ##
    ## $mu$comp2
    ## [1] -2.097679  1.557202
    ##
    ## $mu$comp3
    ## [1]  1.039218 -1.846066
    ##
    ##
    ## $sigma
    ## $sigma$comp1
    ##      [,1] [,2]
    ## [1,]    1    0
    ## [2,]    0    1
    ##
    ## $sigma$comp2
    ##           [,1]     [,2]
    ## [1,] 0.4001925 0.000000
    ## [2,] 0.0000000 0.489384
    ##
    ## $sigma$comp3
    ##           [,1]      [,2]
    ## [1,] 0.8003757 0.0000000
    ## [2,] 0.0000000 0.5226159

The function needs to know how many components we want to estimate though this number may very well be unknown in practice.

## Model fitting

With the data loaded and defined initial parameters, the model is now fitted.

    est_theta <- fit.full.GMCM(u = uhat,  # Ranking function is applied automatically
                               theta = init_theta,
                               method = "NM",
                               max.ite = 5000,
                               verbose = FALSE)
    print(est_theta)

    ## theta object with d = 2 dimensions and m = 3 components:
    ##
    ## $pie
    ##      pie1      pie2      pie3
    ## 0.2085000 0.2745018 0.5169982
    ##
    ## $mu
    ## $mu$comp1
    ## [1] 0 0
    ##
    ## $mu$comp2
    ## [1] -3.1515792  0.8279297
    ##
    ## $mu$comp3
    ## [1]  0.8631953 -1.5608324
    ##
    ##
    ## $sigma
    ## $sigma$comp1
    ##           [,1]      [,2]
    ## [1,] 1.0000000 0.3978458
    ## [2,] 0.3978458 1.0000000
    ##
    ## $sigma$comp2
    ##             [,1]        [,2]
    ## [1,]  1.49969539 -0.01741992
    ## [2,] -0.01741992  0.43467752
    ##
    ## $sigma$comp3
    ##            [,1]       [,2]
    ## [1,]  0.2798609 -0.0292142
    ## [2,] -0.0292142  0.1083622

The fitting method is set to `"NM"` with a maximum number of iterations of `100`.

## Unsupervised clustering

The estimated parameters are used to calculated posterior component probabilities on which the classification is based:

    membership <- classify(uhat, est_theta)
    str(membership)

    ##  int [1:1000] 2 1 3 2 3 3 2 3 3 1 ...

If it is of interest, the posterior probability can be computed directly using

    post_prob <- get.prob(uhat, est_theta)  # Compute component probabilities

## Results

The number of observations in each class can be e.g. counted by

    table(membership)

    ## membership
    ##   1   2   3
    ## 200 272 528

The results are also displayed by plotting

    par(mfrow = c(1,2))
    plot(uhat, col = membership, asp = 1) # Plot of estimated copula values
    z <- GMCM:::qgmm.marginal(uhat, theta = est_theta) # Estimate latent process
    plot(z, col = membership, asp = 1) # Plot of estimated latent process

The fitted `theta` object can also be plotted directly:

    plot(est_theta)

### Session information

This report was generated using rmarkdown2 and knitr3 under the session given below.

    sessionInfo()

    ## R version 3.6.1 (2019-07-05)
    ## Platform: x86_64-w64-mingw32/x64 (64-bit)
    ## Running under: Windows 10 x64 (build 18362)
    ##
    ## Matrix products: default
    ##
    ## locale:
    ## [1] LC_COLLATE=C                     LC_CTYPE=English_Denmark.1252
    ## [3] LC_MONETARY=English_Denmark.1252 LC_NUMERIC=C
    ## [5] LC_TIME=English_Denmark.1252
    ##
    ## attached base packages:
    ## [1] stats     graphics  grDevices utils     datasets  methods   base
    ##
    ## other attached packages:
    ## [1] GMCM_1.4
    ##
    ## loaded via a namespace (and not attached):
    ##  [1] compiler_3.6.1  magrittr_1.5    htmltools_0.3.6 tools_3.6.1
    ##  [5] yaml_2.2.0      Rcpp_1.0.1      ellipse_0.4.1   stringi_1.4.3
    ##  [9] rmarkdown_1.13  knitr_1.23      stringr_1.4.0   digest_0.6.19
    ## [13] xfun_0.7        evaluate_0.14

### References

Please cite the GMCM paper1 if you use the package or shiny app.

1. Bilgrau AE, Eriksen PS, Rasmussen JG, Johnsen HE, Dybkaer K, Boegsted M (2016). “GMCM: Unsupervised Clustering and Meta-Analysis Using Gaussian Mixture Copula Models.” Journal of Statistical Software, 70(2), 1-23. doi: 10.18637/jss.v070.i02 (URL: https://doi.org/10.18637/jss.v070.i02).
2. Xie Y (2019). knitr: A General-Purpose Package for Dynamic Report Generation in R. R package version 1.23, <URL: https://yihui.name/knitr/>.
3. Xie Y (2015). Dynamic Documents with R and knitr, 2nd edition. Chapman and Hall/CRC, Boca Raton, Florida. ISBN 978-1498716963, <URL: https://yihui.name/knitr/>.
4. Xie Y (2014). “knitr: A Comprehensive Tool for Reproducible Research in R.” In Stodden V, Leisch F, Peng RD (eds.), Implementing Reproducible Computational Research. Chapman and Hall/CRC. ISBN 978-1466561595, <URL: http://www.crcpress.com/product/isbn/9781466561595>.
5. Allaire J, Xie Y, McPherson J, Luraschi J, Ushey K, Atkins A, Wickham H, Cheng J, Chang W, Iannone R (2019). rmarkdown: Dynamic Documents for R. R package version 1.13, <URL: https://rmarkdown.rstudio.com>.
6. Xie Y, Allaire J, Grolemund G (2018). R Markdown: The Definitive Guide. Chapman and Hall/CRC, Boca Raton, Florida. ISBN 9781138359338, <URL: https://bookdown.org/yihui/rmarkdown>.