Theoretical Statistics and Regression

Most of the topics of this exam are included in the courses Applied Regression, ESMA 6205, and Theoretical Statistics, ESMA 6661.

Topics in Applied Regression:

  • The simple linear regression model, Inference in simple linear regression, residual analysis.
  • Multiple linear regression: model estimation and inference.
  • Anomalies in linear regression and remedial measures.
  • Regression diagnostics for outlier detection as well points of high leverages, residual plots.
  • Transformations in regression: Transformations for normality and constant variance. Weighted least squares.
  • Regression with qualitative variables, regression with categorical predictors, logistic regression.
  • Feature selection in regression: stepwise methods, best subsets methods, criteria for choosing best subsets of predictors.
  • Detecting multicollinearity and remedial measures: Diagnostics for multicollinearity.
  • Ridge regression and principal component analysis to deal with multicollinearity.

Topics in Theoretical Statistics:

  • Random samples: Properties, Distribution, Distribution of the Sum.
  • Moment Generating Functions.
  • Sampling from the Normal Distribution, Order Statistics.
  • Exponential Families, Location-Scale Families.
  • Data Reduction: Sufficiency, Minimal Sufficiency, Ancillary and Complete Statistics.
  • Point Estimation: Method of Moments, MLE, Bayes Estimators, MSE, Best Unbiased Estimators.
  • Interval Estimation: Inverting a Test, Pivotal Quantities, Size and Coverage Probability.
  • Hypothesis Testing: LRT, Bayesian Tests, U-I and I-U Tests, Power Function, Most Powerfull Test, p-Values.
  • Contingency Tables and Goodness-of-Fit.

Bibliography (Applied Regression Part):

  • Notas en Regresión, E. Acuña, (2006). Disponible en
  • Regression Diagnostics, D. Belsley, R. Kuh and R. Welsh, (1980) John Wiley, NY.
  • Applied Regression Analysis, N. Draper and H. Smith, (1998) 3rd Ed. John Wiley, NY.
  • Applied Logistic Regression, D. Hosner and S. Lemeshow, (2000) 2nd Ed. John Wiley, NY.
  • Classical and modern regression with applications, R. Myers, (1990) Duxbury Press, Belmont, CA.
  • Introduction to Linear Regression Analysis, D.C. Montgomery and E.A. Peck, (2004) John Wiley, NY.
  • Applied Linear Statistical Models, J. Neter, W. Wasserman, M.H. Kutner and C. Nachtsheim, (1996), McGraw-Hill, Boston.
  • Linear Statistical Inference and its applications, C.R. Rao, (1973) John Wiley, NY.
  • Applied Regression Analysis: A Research Tool, J.O. Rawlings, G.P. Sastry and D.A. Dickey, (1998), Springer-Verlag, NY
  • Robust Regression and outlier detection P. Rousseau and A. Leroy, (1987) John Wiley, NY.
  • Modern Regression Methods, T. Ryan, (1996) John Wiley, NY.
  • Linear Regression Analysis, G.A.F. Seber and A. Lee, (2003) 2nd Ed. John Wiley, NY.
  • Applied Linear Regression, S. Weisberg, (2005), 3rd Ed. John Wiley, NY.

Bibliography (Theoretical Statistics Part):

  • Statistical Inference, G. Casella and R.L. Berger, (2002). Duxbury Press, Belmont, CA.
  • Mathematical Statistics, J. Shao, (1999) Springer.
  • All of Statistics: A concise course in Statistical Inference, L. Wasserman, (2004) Springer.
  • An introduction to Mathematical Statistics, R.J. Larsen and M.L. Marx, (2006) 4th Ed. Pearson Prentice Hall.
  • An Introduction to Probability and Mathematical Statistics, L.J. Bain and M. Engelhard, (1987), Duxbury Press, Belmont, CA.
Previous exams: