Author: Andrea Martino
Date: 2018-04-06

# gmfd

#### Andrea Martino

#### 2018-04-06

# Introduction

Package gmfd (Generalized Mahalanobis Functional Distance) is an R package which gathers statistical methods for both inference and clustering of functional data based on a generalization of Mahalanobis distance. The package supports both univariate and multivariate functional data.

# Simulation of functional data

Let us consider a process \(X(t) \in L^2(I)\), with \(I \in \mathbb{R}\), with mean function \(m(t) = \mathbb{E}[X(t)]\) and covariance operator \(V\), i.e. V is a linear compact integral operator from \(L^2(I)\) to \(L^2(I)\) such that \((Va)(s) = \int_I v(s,t) a(t) dt \quad \forall a \in L^2(I)\), where \(v\) is the covariance function defined as \(v = \mathbb{E}[(X(t) - m(t))(X(s)-m(s))]\). Then, denote with \(\{\rho_k, k \ge 1\}\) and \(\{\theta_k, k \ge 1\}\) respectively the sequences of eigenvalues and eigenfunctions of \(v\). The Karhunen-Loève expansion decomposes the process \(X(t)\) in a sum of its mean \(m(t)\) and the series of orthonormal functions \(\theta_k(t)\), each one multiplied by zero-mean uncorrelated random variables \(\sqrt{\rho_k} Z_k\), (\(\rho_k>0, \mathbb{V}ar(Z_k) = 1)\). Then we can write \[\begin{equation*} X(t) = m(t) + \sum_{k=1}^\infty Z_{k} \sqrt{\rho_k} \cdot \theta_k(t) \end{equation*}\]

The function `gmfd_simulate( size, mean, covariance, rho, theta )` can simulate an univariate sample of functional data where `size` represents the number of elements to be generated while `mean` is the center of the distribution. The user can choose to use the argument `covariance` for the covariance of the data or alternatively the sequences of eigenvalues and eigenfunctions of the covariance matrix.

    library( gmfd )

    # Define the parameters
    n <- 50
    P <- 100
    K <- 150

    # Grid of the functional dataset
    t <- seq( 0, 1, length.out = P )

    # Define the means and the parameters to use in the simulation
    m1 <- t^2 * ( 1 - t )

    rho <- rep( 0, K )
    theta <- matrix( 0, K, P )
    for ( k in 1:K ) {
      rho[k] <- 1 / ( k + 1 )^2
      if ( k%%2 == 0 )
        theta[k, ] <- sqrt( 2 ) * sin( k * pi * t )
      else if ( k%%2 != 0 && k != 1 )
        theta[k, ] <- sqrt( 2 ) * cos( ( k - 1 ) * pi * t )
      else
        theta[k, ] <- rep( 1, P )
    }

    # Simulate the functional data
    X <- gmfd_simulate( size = n, mean = m1, rho = rho, theta = theta )

# `S3` class `funData` for functional data

For ease of manipulation and visualization, a `S3` class for both univariate and multivariate functional data has been created. A `funData` object represents a functional data and it is defined by the function `funData( grid, F )` where `grid` is the grid of evenly spaced points over which the functional data is defined and `F` is a matrix (if it is a univariate functional data) or a list (if it is a multivariate functional data). A functional data as it has been just described can be then represented by using the function `plot.funData` which takes as argument all the usual customisable graphical parameters.

    # Create a functional data object
    FD1 <- funData( t, X )

    # Plot the funData object
    plot( FD1, col = "blue" , xlab = "grid", ylab = "data")

# Generalized Mahalanobis Distance

Let us consider a \(J\)-dimensional sample of \(n\) realizations \(\mathbf{X}_1(t), ...,\mathbf{X}_n(t)\) of a stochastic process in \((L^2(I))^J\), with \(J\geq 1\). Let \(\bar{\mathbf{X}}_n(t) = n^{-1}(\mathbf{X}_1(t) + \ldots + \mathbf{X}_n(t))\) be the empirical mean. The estimated covariance function is defined as follows: \[\begin{equation} \label{hat_v} \hat{v}(s,t) := \frac{1}{n-1} \sum_{i=1}^n \big(\mathbf{X}_i(s) - \bar{\mathbf{X}}_n(s)\big)\big(\mathbf{X}_i(t) - \bar{\mathbf{X}}_n(t)\big)^\top, \end{equation}\] from which we can compute the sequences of its eigenfunctions \(\{\hat{\boldsymbol{\varphi}}_{k}=(\hat{\varphi}_k^{(1)},..., \hat{\varphi}_k^{(J)})^\top,\, k\geq 1\}\) and the associated eigenvalues \(\{\hat{\lambda}_k;\, k \geq 1\}\). Since in this case the covariance function is computed using \(n\) curves, we have \(\hat{\lambda}_k = 0\) for all \(k\geq n\), and hence the functions \(\{\hat{\boldsymbol{\varphi}}_k;\, k \geq n\}\) can be arbitrary chosen such that \(\{\hat{\boldsymbol{\varphi}}_k;\, k \geq 1\}\) is an orthonormal basis of \((L^2(I))^J\). The empirical version of the generalized Mahalanobis distance based on the covariance estimator \(\hat{v}\) can be written as follows: \[\begin{equation}\begin{aligned} \label{def:hat_dp} \hat{d}^2_p(\textbf{X}_i(t),\textbf{X}_j(t)) &= \sum_{k=1}^{\text{min}\{n-1,P\}} \hat{d}^2_{M,k}(\textbf{X}_i(t), \textbf{X}_j(t)) \hat{h}_k(p) \\ & + \sum_{k=\text{min}\{n-1,P\}+1}^{P} p \Big( \langle \textbf{X}_i(t) - \textbf{X}_j(t), \hat{\boldsymbol{\varphi}}_k \rangle \Big)^2, \end{aligned} \end{equation}\]

where \(P\) is the length of the independent variable grid, while \(\hat{d}^2_{M,k}(\cdot,\cdot)\) and \(\hat{h}(p)\) represent the estimates of the square of the contribution to this distance along the \(k\)-th component and the regularizing function, respectively. The function `funDist( FD1, FD2, metric, p, lambda, phi )` computes the distance between two functional data `FD1` and `FD2` by using the chosen `metric`. The last three parameters are used only for the generalized Mahalanobis distance.

    # We estimate the eigenvalues and eigenfunctions of the covariance matrix of data
    lambda <- eigen( cov( FD1$data[[1]] ) )$values
    phi <- eigen( cov( FD1$data[[1]] ) )$vectors

    # Extract two curves from the samples to compute the distance between them
    x <- funData( t, FD1$data[[1]][1, ] )
    y <- funData( t, FD1$data[[1]][2, ] )

    distance <- funDist( x, y, metric = "mahalanobis", p = 10^5, lambda, phi )
    distance

    ## [1] 2.306482

It is also possible to compute the dissimilarity matrix of a given sample by using the function `gmfd_diss( FD, metric, p )`.

# Inference on the means of functional data: two sample hypotesis tests

We want now to compare the means of two functional data samples. We simulate two samples \(\mathbf{X}_1(t), ...,\mathbf{X}_{n_1}(t)\) and \(\mathbf{Y}_{n_2}(t), ...,\mathbf{Y}_{n_2}(t)\) and consider the following asymptotic hypotesis test: \[\begin{equation} H_0 : m_{1} = m_{2} \qquad \text{vs} \qquad H_1: m_{1} \neq m_{2}. \end{equation}\]

where \(m_1\) and \(m_2\) are the real means of the two simulated samples. We can infer on the means of the two samples by using the function `gmfd_test(FD1, FD2, conf.level, stat_test, p)` where we have the two samples `FD1` and `FD2`, the confidence level for the test `conf.level`, a string to choose the test statistic to use `stat_test` and the parameter of the regularizing function `p`. The function then returns the value of the test statistics, the value of the quantile and the p-value for the test.

    # Simulate another functional sample
    s <- 0
    for ( k in 4:K ) {
      s <- s + sqrt( rho[k] ) * theta[k, ]
    }

    m2 <- m1 + s
    Y <- gmfd_simulate( n, m2, rho = rho, theta = theta )
    FD2 <- funData( t, Y )

    test_output <- gmfd_test(FD1, FD2, conf.level = 0.95, stat_test = "mahalanobis", p = 10^5)
    test_output

    ## $T0
    ## [1] 74.00768
    ##
    ## $quantile
    ## [1] 12.52993
    ##
    ## $pval
    ## [1] 0

# Clustering: the k-means algorithm

The functional \(k\)-means clustering algorithm is an iterative procedure, alternating a step of cluster assignment, where all the curves are assigned to a cluster, and a step of centroid calculation, where a relevant functional representative (the centroid) for each cluster is identified. More precisely, the algorithm is initialized by fixing the number \(k\) of clusters and by randomly selecting a set of \(k\) initial centroids \(\{\boldsymbol{\chi}_1^{(0)}(t), \ldots , \boldsymbol{\chi}_k^{(0)}(t)\}\) among the curves of the dataset. Given this initial choice, the algorithm iteratively repeats the two basic steps mentioned above. Formally, at the \(m^{th}\) iteration of the algorithm, \(m\geq 1\), the two following steps are performed:

- Step 1 (cluster assignment step): each curve is assigned to the cluster with the nearest centroid at the \((m-1)^{th}\) iteration, according to the distance \(\hat{d}_p\). Formally, the \(m^{th}\) cluster assignment \(C_i^{(m)}\) of the \(i^{th}\) statistical unit, for \(i=1,\ldots,n\), can be written as follows: \[\begin{equation} C_i^{(m)} := \underset{l=1,\ldots,k}{\operatorname{argmin}}\:\hat{d}_p(\textbf{X}_i(t), \boldsymbol{\chi}_l^{(m-1)}(t)); \end{equation}\]
- Step 2 (centroid calculation step): the computation of the centroids at the \(m^{th}\) iteration is performed by solving the optimization problems: for any \(l=1,\ldots,k\), \[\begin{equation} \boldsymbol{\chi}_l^{(m)}(t) := \underset{\boldsymbol{\chi} \in (L^2(I))^J}{\operatorname{argmin}} \sum_{i:C_i^{(m)} = l} \hat{d}_p(\textbf{X}_i(t),\boldsymbol{\chi}(t))^2, \end{equation}\] where \(C_i^{(m)}\) is the cluster assignment of the \(i^{th}\) statistical unit at the \(m^{th}\) iteration. The algorithm stops when the same cluster assignments are obtained at two subsequent iterations, i.e. the set of cluster assignments \(\{C_1^{(\bar{m})},\ldots,C_n^{(\bar{m})}\}\) and the set of centroids \(\{\boldsymbol{\chi}_1^{(\bar{m})}(t),\ldots, \boldsymbol{\chi}_k^{(\bar{m})}(t)\}\) are considered final solutions of the algorithm if \(\bar{m}\) is the minimum integer such that \(C_i^{(\bar{m}+1)} \equiv C_i^{(\bar{m})}\) for all \(i=1,\ldots,n\).

We apply the procedure by merging the two samples \(\mathbf{X}_1(t), ...,\mathbf{X}_{n_1}(t)\) and \(\mathbf{Y}_{n_2}(t), ...,\mathbf{Y}_{n_2}(t)\) previously simulated using the function `gmfd_kmeans( FD, n.cl , metric, p )` where `n.cl` is the number of cluster. It returns a vector of the clusters and a vector or a list of vectors of the centers, other than a plot of the clusters along with their empirical means.

    # We estimate the eigenvalues and eigenfunctions of the covariance matrix of all merged data
    lambda <- eigen( cov( rbind( FD1$data[[1]], FD2$data[[1]] ) ) )$values
    phi <- eigen( cov ( rbind( FD1$data[[1]], FD2$data[[1]] ) ) )$vectors

    # Functional data sample containing the merged samples
    FD <- funData( t, rbind( X, Y ) )

    kmeans_output <- gmfd_kmeans( FD, n.cl = 2, metric = "mahalanobis", p = 10^5 )