Cross-Validation Function for Aggregated Frobenius Loss

cvFrobeniusLoss() evaluates the aggregated Frobenius loss over a fold object (from 'origami' (Coyle and Hejazi 2018) ).

cvFrobeniusLoss(fold, dat, estimator_funs, estimator_params = NULL)

Arguments

fold

A fold object (from make_folds()) over which the estimation procedure is to be performed.

dat

A data.frame containing the full (non-sample-split) data, on which the cross-validated procedure is performed.

estimator_funs

An expression corresponding to a vector of covariance matrix estimator functions to be applied to the training data.

estimator_params

A named list of arguments corresponding to the hyperparameters of covariance matrix estimators, estimator_funs. The name of each list element should be the name of an estimator passed to estimator_funs. Each element of the estimator_params is itself a named list, with names corresponding to an estimators' hyperparameter(s). These hyperparameters may be in the form of a single numeric or a numeric vector. If no hyperparameter is needed for a given estimator, then the estimator need not be listed.

Value

A tibble providing information on estimators, their hyperparameters (if any), and their scaled Frobenius loss evaluated on a given fold.

References

Coyle J, Hejazi N (2018). “origami: A Generalized Framework for Cross-Validation in R.” Journal of Open Source Software, 3(21), 512. doi: 10.21105/joss.00512 .

Examples

library(MASS)
#> Warning: package ‘MASS’ was built under R version 4.1.2
library(origami)
#> origami v1.0.7: Generalized Framework for Cross-Validation
library(rlang)
#> Warning: package ‘rlang’ was built under R version 4.1.2

# generate 10x10 covariance matrix with unit variances and off-diagonal
# elements equal to 0.5
Sigma <- matrix(0.5, nrow = 10, ncol = 10) + diag(0.5, nrow = 10)

# sample 50 observations from multivariate normal with mean = 0, var = Sigma
dat <- mvrnorm(n = 50, mu = rep(0, 10), Sigma = Sigma)

# generate a single fold using MC-cv
resub <- make_folds(dat,
  fold_fun = folds_vfold,
  V = 2
)[[1]]
cvFrobeniusLoss(
  fold = resub,
  dat = dat,
  estimator_funs = rlang::quo(c(
    linearShrinkEst, thresholdingEst, sampleCovEst
  )),
  estimator_params = list(
    linearShrinkEst = list(alpha = c(0, 1)),
    thresholdingEst = list(gamma = c(0, 1))
  )
)
#> [[1]]
#> # A tibble: 5 × 4
#>   estimator       hyperparameters       loss  fold
#>   <chr>           <chr>                <dbl> <int>
#> 1 linearShrinkEst alpha = 0            106.      1
#> 2 linearShrinkEst alpha = 1             94.3     1
#> 3 thresholdingEst gamma = 0             94.3     1
#> 4 thresholdingEst gamma = 1            114.      1
#> 5 sampleCovEst    hyperparameters = NA  94.3     1
#>