Skip to main content Skip to navigation

Department of Statistics

Forum

Forum Examples of shrinkage estimators

2 posts 2 followers RSS feed

You need to be logged in to post in this topic.
  1. Shahin Tavakoli Staff Statistics
    #1 14:53, Thu 14 Dec 2017
     
  2. Sigurd Assing Staff Statistics
    #2 11:32, Fri 15 Dec 2017

    shalin, thanks for these links---great stuff. I screened through the papers, and below is a short sketch of how this connects to what was discussed yesterday. but first, a quick reminder of the linear shrinkage ansatz I stressed yesterday several times: the shrunken estimator of the functional relation between X and Y is (1-t_0)\hat{f}+t_0c where t_0 is chosen in a way minimizing some distance to the true function f. for me \hat{f}(\cdot)=\sum_{j=1}^N N_j(\cdot)\hat{\theta}_j where the N_j might be eigenvectors with respect to some PDE problem, or cubic splines, or whatever.

    COVARIANCE SHRINKAGE: this is about a shrinkage method for sample eigenvalues \hat{\lambda}_i\,,\,i=1,\dots,p when estimating a covariance. therefore, the corresponding eigenvectors don't play the same role as the N_j in my structure for \hat{f}. the shrinkage is nonlinear because the shrunken estimator is obtained by \hat{\lambda}_i/[1-c-c\hat{\lambda}_i\check{m}_F(\hat{\lambda}_i)]^2. here \check{m}_F(\hat{\lambda}_i) plays the role of t_0 but it is not found by simply minimizing some distance, and hence further thinking is required when applying this in other contexts. Furthermore, the reference point c is not a vector in R^p but a constant being the same for all principal directions, though the shrinkage is different in each direction. Applying this to my smoothing spline example would mean to shrink each \hat{\theta}_j separately by a similar method, which is different to shrinking things x by x as discussed yesterday.

    SPECTRAL DENSITY MATRIX SHRINKAGE: this is about a p-dimensional weakly stationary discrete time series, and one is after the spectral density matrix denoted by f. so, this f is a matrix-valued function depending on frequencies \omega, and the shrunken estimate for f(\omega) is (1-W_T(\omega))\hat{f}(\omega)+W_T(\omega)\tilde{V}(\omega). this is linear shrinkage as discussed yesterday. First, the shrinkage is worked out \omega by \omega, and \omega plays the role of x, of course. Second, W_T(\omega) plays the role of t_0 with T being the time horizon of the time series (this time dependence should change when the series is NOT stationary). Third, \tilde{V}(\omega) and \hat{f}(\omega) are a parametric and a nonparametric estimate of f\,, respectively, with \tilde{V}(\omega) playing the role of c. So, this is indeed the same I wanted to do, because I wanted to take a linear regression for c and do the whole thing x by x.

    ALL IN ALL, what was suggested yesterday for smoothing splines is applied in practice, and one could even go beyond the linear shrinkage ansatz though this might be less inuitive. the next step to be discussed in january would be to bring in bootstrapping and to understand how this can be applied repeatedly. for christmas, everybody is welcome to think about what shrinks to what and how much when bootstrapping repeatedly.