Maximum Penalized Likelihood Estimation

Volume II: Regression
1. Auflage 2009
ISBN: 978-0-387-68902-9
Verlag: Springer-Verlag
Format: PDF
Kopierschutz: Adobe DRM (»Systemvoraussetzungen)

E-Book, Englisch, 572 Seiten

This is the second volume of a text on the theory and practice of maximum penalized likelihood estimation. It is intended for graduate students in s- tistics, operationsresearch, andappliedmathematics, aswellasresearchers and practitioners in the ?eld. The present volume was supposed to have a short chapter on nonparametric regression but was intended to deal mainly with inverse problems. However, the chapter on nonparametric regression kept growing to the point where it is now the only topic covered. Perhaps there will be a Volume III. It might even deal with inverse problems. But for now we are happy to have ?nished Volume II. The emphasis in this volume is on smoothing splines of arbitrary order, but other estimators (kernels, local and global polynomials) pass review as well. We study smoothing splines and local polynomials in the context of reproducing kernel Hilbert spaces. The connection between smoothing splines and reproducing kernels is of course well-known. The new twist is thatlettingtheinnerproductdependonthesmoothingparameteropensup new possibilities: It leads to asymptotically equivalent reproducing kernel estimators (without quali?cations) and thence, via uniform error bounds for kernel estimators, to uniform error bounds for smoothing splines and, via strong approximations, to con?dence bands for the unknown regression function. ItcameassomewhatofasurprisethatreproducingkernelHilbert space ideas also proved useful in the study of local polynomial estimators.

Autoren/Hrsg.

Inhaltsverzeichnis

1;Preface;6
2;Contents;8
3;Contents of Volume I;12
4;Notations, Acronyms and Conventions;16
5;Nonparametric Regression;20
5.1;1. What and why?;20
5.2;2. Maximum penalized likelihood estimation;26
5.3;3. Measuring the accuracy and convergence rates;35
5.4;4. Smoothing splines and reproducing kernels;39
5.5;5. The local error in local polynomial estimation;45
5.6;6. Computation and the Bayesian view of splines;47
5.7;7. Smoothing parameter selection;54
5.8;8. Strong approximation and confidence bands;62
5.9;9. Additional notes and comments;67
6;Smoothing Splines;68
6.1;1. Introduction;68
6.2;2. Reproducing kernel Hilbert spaces;71
6.3;3. Existence and uniqueness of the smoothing spline;78
6.4;4. Mean integrated squared error;83
6.5;5. Boundary corrections;87
6.6;6. Relaxed boundary splines;91
6.7;7. Existence, uniqueness, and rates;102
6.8;8. Partially linear models;106
6.9;9. Estimating derivatives;114
6.10;10. Additional notes and comments;115
7;Kernel Estimators;117
7.1;1. Introduction;117
7.2;2. Mean integrated squared error;119
7.3;3. Boundary kernels;123
7.4;4. Asymptotic boundary behavior;128
7.5;5. Uniform error bounds for kernel estimators;132
7.6;6. Random designs and smoothing parameters;144
7.7;7. Uniform error bounds for smoothing splines;150
7.8;8. Additional notes and comments;161
8;Sieves;162
8.1;1. Introduction;162
8.2;2. Polynomials;165
8.3;3. Estimating derivatives;170
8.4;4. Trigonometric polynomials;172
8.5;5. Natural splines;178
8.6;6. Piecewise polynomials and locally adaptive designs;180
8.7;7. Additional notes and comments;184
9;Local Polynomial Estimators;185
9.1;1. Introduction;185
9.2;2. Pointwise versus local error;189
9.3;3. Decoupling the two sources of randomness;192
9.4;4. The local bias and variance after decoupling;197
9.5;5. Expected pointwise and global error bounds;199
9.6;6. The asymptotic behavior of the error;200
9.7;7. Refined asymptotic behavior of the bias;206
9.8;8. Uniform error bounds for local polynomials;211
9.9;9. Estimating derivatives;213
9.10;10. Nadaraya-Watson estimators;214
9.11;11. Additional notes and comments;218
10;Other Nonparametric Regression Problems;220
10.1;1. Introduction;220
10.2;2. Functions of bounded variation;223
10.3;3. Total-variation roughness penalization;231
10.4;4. Least-absolute-deviations splines: Generalities;236
10.5;5. Least-absolute-deviations splines: Error bounds;242
10.6;6. Reproducing kernel Hilbert space tricks;246
10.7;7. Heteroscedastic errors and binary regression;247
10.8;8. Additional notes and comments;251
11;Smoothing Parameter Selection;254
11.1;1. Notions of optimality;254
11.2;2. Mallows’ estimator and zero-trace estimators;259
11.3;3. Leave-one-out estimators and cross-validation;263
11.4;4. Coordinate-free cross-validation (GCV);266
11.5;5. Derivatives and smooth estimation;271
11.6;6. Akaike’s optimality criterion;275
11.7;7. Heterogeneity;280
11.8;8. Local polynomials;285
11.9;9. Pointwise versus local error, again;290
11.10;10. Additional notes and comments;295
12;Computing Nonparametric Estimators;299
12.1;1. Introduction;299
12.2;2. Cubic splines;299
12.3;3. Cubic smoothing splines;305
12.4;4. Relaxed boundary cubic splines;308
12.5;5. Higher-order smoothing splines;312
12.6;6. Other spline estimators;320
12.7;7. Active constraint set methods;327
12.8;8. Polynomials and local polynomials;333
12.9;9. Additional notes and comments;337
13;Kalman Filtering for Spline Smoothing;339
13.1;1. And now, something completely different;339
13.2;2. A simple example;347
13.3;3. Stochastic processes and reproducing kernels;352
13.4;4. Autoregressive models;364
13.5;5. State-space models;366
13.6;6. Kalman filtering for state-space models;369
13.7;7. Cholesky factorization via the Kalman filter;373
13.8;8. Diffuse initial states;377
13.9;9. Spline smoothing with the Kalman filter;380
13.10;10. Notes and comments;384
14;Equivalent Kernels for Smoothing Splines;387
14.1;1. Random designs;387
14.2;2. The reproducing kernels;394
14.3;3. Reproducing kernel density estimation;398
14.4;4. L2 error bounds;400
14.5;5. Equivalent kernels and uniform error bounds;402
14.6;6. The reproducing kernels are convolution-like;407
14.7;7. Convolution-like operators on Lp spaces;415
14.8;8. Boundary behavior and interior equivalence;423
14.9;9. The equivalent Nadaraya-Watson estimator;428
14.10;10. Additional notes and comments;435
15;Strong Approximation and Confidence Bands;439
15.1;1. Introduction;439
15.2;2. Normal approximation of iid noise;443
15.3;3. Confidence bands for smoothing splines;448
15.4;4. Normal approximation in the general case;451
15.5;5. Asymptotic distribution theory for uniform designs;460
15.6;6. Proofs of the various steps;466
15.7;7. Asymptotic 100% confidence bands;478
15.8;8. Additional notes and comments;482
16;Nonparametric Regression in Action;484
16.1;1. Introduction;484
16.2;2. Smoothing splines;488
16.3;3. Local polynomials;498
16.4;4. Smoothing splines versus local polynomials;508
16.5;5. Confidence bands;512
16.6;6. The Wood Thrush Data Set;523
16.7;7. The Wastewater Data Set;531
16.8;8. Additional notes and comments;540
17;Bernstein’s Inequality;541
18;The TVDUAL inplementation;544
19;Solutions to Some Critical Exercises;549
19.1;1. Solutions to Chapter 13: Smoothing Splines;549
19.2;2. Solutions to Chapter 14: Kernel Estimators;550
19.3;3. Solutions to Chapter 17: Other Estimators;551
19.4;4. Solutions to Chapter 18: Smoothing Parameters;552
19.5;5. Solutions to Chapter 19: Computing;552
19.6;6. Solutions to Chapter 20: Kalman Filtering;553
19.7;7. Solutions to Chapter 21: Equivalent Kernels;556
20;References;558
21;Author Index;572
22;Subject Index;578