Theoretical Statistics and Regression

Most of the topics of this exam are included in the courses Applied Regression, ESMA 6205, and Theoretical Statistics, ESMA 6661.

Topics in Applied Regression:

• The simple linear regression model, Inference in simple linear regression, residual analysis.
• Multiple linear regression: model estimation and inference.
• Anomalies in linear regression and remedial measures.
• Regression diagnostics for outlier detection as well points of high leverages, residual plots.
• Transformations in regression: Transformations for normality and constant variance. Weighted least squares.
• Regression with qualitative variables, regression with categorical predictors, logistic regression.
• Feature selection in regression: stepwise methods, best subsets methods, criteria for choosing best subsets of predictors.
• Detecting multicollinearity and remedial measures: Diagnostics for multicollinearity.
• Ridge regression and principal component analysis to deal with multicollinearity.

Topics in Theoretical Statistics:

• Random samples: Properties, Distribution, Distribution of the Sum.
• Moment Generating Functions.
• Sampling from the Normal Distribution, Order Statistics.
• Exponential Families, Location-Scale Families.
• Data Reduction: Sufficiency, Minimal Sufficiency, Ancillary and Complete Statistics.
• Point Estimation: Method of Moments, MLE, Bayes Estimators, MSE, Best Unbiased Estimators.
• Interval Estimation: Inverting a Test, Pivotal Quantities, Size and Coverage Probability.
• Hypothesis Testing: LRT, Bayesian Tests, U-I and I-U Tests, Power Function, Most Powerfull Test, p-Values.
• Contingency Tables and Goodness-of-Fit.

Bibliography (Applied Regression Part):

• Notas en Regresión, E. Acuña, (2006). Disponible en math.uprm.edu/~edgar
• Regression Diagnostics, D. Belsley, R. Kuh and R. Welsh, (1980) John Wiley, NY.
• Applied Regression Analysis, N. Draper and H. Smith, (1998) 3rd Ed. John Wiley, NY.
• Applied Logistic Regression, D. Hosner and S. Lemeshow, (2000) 2nd Ed. John Wiley, NY.
• Classical and modern regression with applications, R. Myers, (1990) Duxbury Press, Belmont, CA.
• Introduction to Linear Regression Analysis, D.C. Montgomery and E.A. Peck, (2004) John Wiley, NY.
• Applied Linear Statistical Models, J. Neter, W. Wasserman, M.H. Kutner and C. Nachtsheim, (1996), McGraw-Hill, Boston.
• Linear Statistical Inference and its applications, C.R. Rao, (1973) John Wiley, NY.
• Applied Regression Analysis: A Research Tool, J.O. Rawlings, G.P. Sastry and D.A. Dickey, (1998), Springer-Verlag, NY
• Robust Regression and outlier detection P. Rousseau and A. Leroy, (1987) John Wiley, NY.
• Modern Regression Methods, T. Ryan, (1996) John Wiley, NY.
• Linear Regression Analysis, G.A.F. Seber and A. Lee, (2003) 2nd Ed. John Wiley, NY.
• Applied Linear Regression, S. Weisberg, (2005), 3rd Ed. John Wiley, NY.

Bibliography (Theoretical Statistics Part):

• Statistical Inference, G. Casella and R.L. Berger, (2002). Duxbury Press, Belmont, CA.
• Mathematical Statistics, J. Shao, (1999) Springer.
• All of Statistics: A concise course in Statistical Inference, L. Wasserman, (2004) Springer.
• An introduction to Mathematical Statistics, R.J. Larsen and M.L. Marx, (2006) 4th Ed. Pearson Prentice Hall.
• An Introduction to Probability and Mathematical Statistics, L.J. Bain and M. Engelhard, (1987), Duxbury Press, Belmont, CA.
Previous exams:

Spring-2005