Math 3330 3.0 BF: Regression Analysis

• Introduction to S-Plus and R.
• Review of basic inference: two-sample confidence intervals and tests, Anova for 3 or more groups.
• The many roles of statistics: scientific research to business decisions.
• Sources of data and the purpose of inference:
• Observation vs Experiment
• Causal inference vs Prediction
• Sampling and Randomization
• Unconditional vs Conditional association: Simpson's Paradox
• The role of regression analysis. How well can statistical control compensate for the lack of experimental control?

• Part1: Complete  Lesson 1 and Lesson 2 of Getting Started with S-Plus. Hand in the exercise at the end of Lesson 2.
• Part2: Hand in your answer to the following question (from p. 20 of FPP)

In 1964, the Public Health Service of the United States studied the effects of smoking on health in a sample of 42,00 households.  For men and for women in each age group, they found that those who had never smoked were on average somewhat healthier than the current smokers, but the current smokers were on average much healthier than the former smokers.
• Why did they study men and women and the different age groups separately?
• The lesson seems to be that you shouldn't start smoking, but once you've started, don't stop. Comment.

• Your textbook for introductory statistics.
• ARA: Chapter 1.
• FPP: Chapters 1 and 2.

• Regression analysis = study of conditional distribution of Y given X
• Main characteristics of conditional distributions
• Patterns of dependency on X
• Basic ideas of non-parametric regression
• Examining data: histograms, density estimation, quantiles, quantile comparison plots, boxplots.
• Bivariate data: scatterplots, side-by-side boxplots.
• Multivariate data: scatterplot matrix, 3-d plots, coplots. [Link with Simpson's Paradox]

• ARA, Chap. 2,  Chap. 3.
• SGS, Chap. 7.11 pp. 158 ff

• S-Plus script files: Right-click to download
•  Week2a_Review.SSC 
•  Week2b_Reg_Intro.SSC 
•  Week2c_Reg_prestige.SSC 
•  Week2d_Examining_Data.SSC 
• Data set: Right-click to download and import to S-Plus with File|Import
• "Swan":  swan.xls 

• See S-Plus script files above.
• Continue working with the S-Plus tutorial  http://www.utstat.toronto.edu/splus/contents.html.  Try to complete it by early October.

Quiz 1
Assignment 1 due

• Transforming distributions: Tukey's ladder of powers and skewness
• Transforming relationships: Tukey's ladder of powers and non-linearity
• Transforming heteroskedasticity
• Putting all the transformations together
• The logit transformation for proportions (not covered)
• Simple linear least-squares, correlation, basic formulas. (assigned for review) 

Links:

• Week 3 Transformations.SSC 
• Assignment 2 

• Simple linear least-squares, correlation, basic formulas. 
• Visualizing the data ellipse, the least-squares line, correlation, R squared.

• ARA, Chap. 5, 97-111. Chap. 9, 204-211, Chap. 6, 112-119.

• Week4a_LinReg.SSC
• Week4b_VarCov.SSC 

Quiz 2
Assignment 2 due

• The Regression Paradox: Galton and Pearson: heights of fathers and sons. The concept of correlation. "Regression to Mediocrity." 
• Multiple regression. Normal equations. Matrix formulation. Multiple correlation. Meaning of coefficients. 
• Simple regression versus multiple regression. Marginal versus conditional linear association.

Links:

• Exercise using Heart, Coffee and Stress data set
• Some useful functions for the course (this will be updated regularly so you might like to download and run it regularly): Fun.SSC
• Coffee data set: coffee.dat
• Assignment 3
• Week5a_Regression.SSC
• Week5b_Coffee_Part1.SSC
• Week5c_MultReg.SSC

Term Test

• Review

• Relationship between simple regression and multiple regression: the chain rule.
• Statistical inference in simple and multiple regression

Part 2 of the script begun in Week 5 on the Heart Damage / Coffee / Stressdata is 

Week7b_Coffee_Part2.SSC 

The data set used in class to illustrate the principle of Extra Sums of Squares and the regression with a dummy varible is available as an Excel file:  blf.xls which you can download (by right-clicking) and import to S-Plus. The script used with this dataset is:  Week7a_BLF.SSC .

Chap. 6, 112-119.

• Relationship between unconditional and conditional effects.
• Regression with a categorical predictor and a continuous predictor: (Using dummy variables)
• Dichotomous and polytomous with parallel lines
• Using interactions to model non-parallel lines.
• Principle of marginality, hypothesis tests for main effects and interactions
• Polytomous predictors and one-way anova.

Script begun in Week 5 on the Heart Damage / Coffee / Stressdata is 

Week7b_Coffee_Part2.SSC 

The data set used in class to illustrate the principle of Extra Sums of Squares and the regression with a dummy varible is available as an Excel file:  blf.xls which you can download (by right-clicking) and import to S-Plus. The script used with this dataset is:  Week7a_BLF.SSC .

An example with a dichotomous variable is in Week7a_BLF.SSC,

and, with a polytomous  variable, in  Week8_prestige.SSC

ARA, Chap. 7, 135-154 

Quiz 4
Assignment 4 due

• Empirical vs structural relations 
• Effect of Measurement error in predictor variable
• Regression with a categorical predictors and a continuous predictor: (Using dummy variables)
• Dichotomous and polytomous with parallel lines
• Using interactions to model non-parallel lines.
• Principle of marginality, hypothesis tests for main effects and interactions
• Polytomous predictors and one-way anova.
• Extra sums of squares, balanced vs unbalanced designs..
• Theory of linear models 

Quiz 4:
Assignment 4 due.

• Properties of LS estimation: BLUE and UMVUE
• The general linear hypothesis.
• Regression diagnostics, outliers and influential observations

• General Linear Hypothesis:  
• Script: Week10_GLH.SSC and 
• Data:  dose.xls 
• Diagnostics:  Week10_Diags.SSC 
• Download latest  Fun.SSC to get an up-to-date set of functions.

Last class
Quiz 5
Reports due

Regression diagnostics: nonlinearity, heteroschedasticity, non-normality

Collinearity

Polynomial regression