In This Chapter

^ When and how to follow up ANOVA with multiple comparisons ^ Comparing two well-known multiple comparison procedures

Ou’re comparing the means of not two, but K Independent populations, and you find out (using ANOVA — see Chapter 9) that you reject Ho: All the population means are equal, and you conclude Ha: At least two of the population means are different. Now you gotta know — which of those populations are different? Answering this question requires a follow-up procedure to ANOVA called Multiple comparisons, Which makes sense because you want to compare the multiple means you have and see which ones are different.

In this chapter, you figure out when you need to use a multiple comparison procedure. You see two of the most well-known multiple comparison procedures: Fisher’s LSD (least significant difference) and Tukey’s test. They can help you answer that burning question: So some of the means are different, but which ones are different?

Following Up after ANOVA

This section runs through the ANOVA procedure in the case where Ho is rejected and leads you to the next step: multiple comparisons.

Suppose you want to compare the average number of cell-phone minutes used per month for children and young adults, where the age groups are the following:

Y

V Group 1: 19 years old and under Group 2: 20-39 years old

Group 3: Adult males 40-59 years old

Group 4: Adult females 60 years old and over

You collect data on a random sample of 10 people from each group (where no one knows anyone else to keep independence), and you record the number of minutes each person used their cell phone in one month. The first ten lines of a hypothetical data set are shown in Table 10-1.

Table 10-1 One Month’s Cell Phone Minutes for Four Age Groups

The means and standard deviations of the sample data are shown in Figure 10-1, as well as confidence intervals for each of the population means separately (see Chapter 3 for info on confidence intervals). Looking at Figure 10-1, it appears that all four means are different, with 19 and under heading the pack, with 40- to 59-year-olds not far behind, and with 20- to 39-year-olds and those over 60 bringing up the rear (in that order).

Knowing that man can’t live by sample results alone, you decide that ANOVA is needed to see whether any differences that appear in the samples can be extended to the population (see Chapter 9). By using the ANOVA procedure, you test whether the average cell minutes used is the same across all groups. The results of the ANOVA, using the data from Table 10-1, are shown in Figure 10-2.

Figure 10-1:

Basic statistics and

Confidence intervals for the cellphone data.

Individual 95% CIs For Mean Based on Pooled StDev

——-1———–1———–1———–1—-

(-*-)

(-*-)

(-*-)

(-*-)

——-1———–1———–1———–1—-

200 400 600 800

Looking at Figure 10-2, the F-test for equality of all four population means has a p-value of 0.000, meaning it is less then 0.001. That says at least two of these groups have a significant difference in their cell-phone use (see Chapter 9 for info on the F-test and its results).

Figure 10-2:

ANOVA results for comparing cell-phone use for four age groups.

Okay, so what’s your next question? You just found out that the average number of cell-phone minutes per month isn’t the same across these four groups. Remember, this doesn’t mean all four groups are different (see Chapter 9). However, it does mean that at least two groups are significantly different in their cell-phone use. So your questions are: Which groups are different, and how are they different?

Determining which populations have differing means after ANOVA has been rejected involves a new data-analysis technique called Multiple comparisons. While many different multiple comparison procedures are out there, statisticians have their favorites, which I present in the next section.

Don’t attempt to explore the data with a multiple comparison procedure if the test for equality of the populations isn’t rejected. In this case, you must conclude that you don’t have enough evidence to say the population means aren’t equal, so you must stop there. Always look at the P-value of the F-test on the ANOVA output before moving on to conduct any multiple comparisons.

PInPoIntinG DiffeRinG Means With Fisher and Tukey

You’ve conducted ANOVA to see whether a group of K Populations have the same mean, and you rejected Ho. You conclude that at least two of those populations have different means. But you don’t have to stop there; you can go on to find out how many and which means are different by conducting multiple comparison tests.

In this section, you see two of the most well-known multiple comparison procedures: Fisher’s paired differences (also known as Fisher’s test Or Fisher’s LSD) And Tukey’s simultaneous confidence intervals (also known as Tukey’s test).

Although I only discuss two procedures in this section, tons of other multiple comparison procedures are out there. Although the other procedures’ methods differ a great deal, their overall goal is the same: to figure out which population means differ by comparing their sample means.

Fishing for differences With Fisher’s LSD

In this section, I outline Fisher’s LSD and apply it to the cell-phone example.

Examining Fisher’s LSD procedure

Suppose you’re comparing K Population means. Fisher’s LSD (short for Least significant difference) Conducts a t-test on each of the —^—h pairs of populations in the study, each one at level a = 0.05. For example, if you have four

Populations labeled A, B, C, D, you would have 4 ^40—— = 6 t-tests to perform: A versus B; A versus C; A versus D; B versus C; B versus D; and C versus D.

The number of tests is calculated by knowing that you have K Possible means for the first one in the pair, then K - 1 left for the second one in the pair. Because the order of the means doesn’t matter, you can divide by 2 to avoid overcounting.

Fisher’s LSD is very straightforward, easy to conduct, and easy to understand. However, Fisher’s LSD has some issues. Because each T-test is conducted at a level 0.05, each test done has a 5 percent chance of making a Type I error (rejecting Ho when you shouldn’t have — see Chapter 3). Although a 5-percent error rate for each test doesn’t seem too bad, the errors have a multiplicative effect as the number of tests increases. For example, the chance of making at least one Type I error with six T-tests, each at level a = 0.05, is 26.50 percent, which would be your Overall error rate For the procedure.

You could help lower the error rate for Fisher’s test if you lower the value of A For each test from 0.05 to, say, 0.01. However, doing so makes it harder to reject Ho for each pair of means. A lower value of A Also doesn’t solve the error-rate problem; it just slows it down for a bit, until the number of tests gets larger, and the error rate goes back up again.

If you want or need to know how I arrived at the number 26.50 percent as the overall error rate in that last example, here it goes: The probability of making a Type I error for each test is 0.05. The chance of making at least one error in six tests equals one minus the probability of making no errors in six tests. The chance of not making an error in one test is 1 – A = 0.95. The chance of no error in six tests is this quantity times itself six times, or (0.95)6, which equals 0.735. Now take one minus this quantity to get 1 – 0.735 = 0.2650 or 26.50 percent.

To conduct Fisher’s LSD, go to Stat>ANOVA>One-way or One-way unstacked. (If your data appear in two columns with Column 1 representing the population number and Column 2 representing the response, just click One-way because your data is stacked. If your data is shown in K Columns, one for each of the K Populations, click One-way unstacked.) In either case, the next step is to highlight the data for the groups you’re comparing and click Select. Then click on Comparisons. Click on Fisher’s. The individual error rate is listed at 5 (percent), which is typical. If you want to change it, type in the desired error rate (between 0.5 and 0.001) and click OK. You may type in your error rate as a decimal, 0.05, or as a number greater than one, such as 5. Numbers greater than one are interpreted as a percentage.

Applying Fisher’s LSD to cell phones

An ANOVA procedure was done on the cell-phone data presented in Table 10-1 to compare the mean number of minutes used for four age groups. Looking at Figure 10-2, you see Ho (all the populations means are equal) was rejected. The next step is to conduct multiple comparisons by using Fisher’s LSD to see which population means differ. Figure 10-3 shows the Minitab output.

The first block of results shows "Group 1 subtracted from" where Group 1 = age 19 and under. Each line after that represents the other age groups (Group 2 = 20- to 39-year-olds, Group 3 = 40- to 59-year-olds, and Group 4 = 60 and over). Each line shows the results of comparing the mean for the other group minus the mean for Group 1. For example, the first line shows Group 2 being compared with Group 1.

Moving to the right in that same row, you see the confidence interval for the difference in these two means, which turns out to be -436.97 to -323.03. Because 0 isn’t contained in this interval, you conclude that these two means are different in the populations also. You can also say, because this difference U,2 - U4 Is negative, that U,2 Is less than U4. Or, a better way to think of it may be that U1 is greater than |U2. That is, Group 1′s mean is greater than Group 2′s

Mean.

Figure 10-3:

Output showing Fisher’s LSD applied to the cellphone data.

Each subsequent row in the "Group 1 subtracted from" section of Figure 10-3 shows similar results. None of the confidence intervals contain 0, so you conclude that the mean cell-phone use for Group 1 isn’t equal to the mean cellphone use for any other group. Moreover, because all confidence intervals are in negative territory, you can conclude that the mean cell-phone use time for those 19 and under is greater than all the others. This process continues as you move down through the output until all six pairs of means are compared. Then you put them all together into one conclusion.

For example, in the second portion of the output, Group 2 is subtracted from Groups 3 and 4. You see the confidence interval for the "Group 3" line is 232.03, 345.97; this gives possible values for Group 3′s mean minus Group 2′s mean. The interval is entirely positive, so conclude that Group 3′s mean is greater than Group 2′s mean (according to this data). On the next line, the interval for Group 4 minus Group 2 is -299.97 to -186.03. All these numbers are negative, so conclude Group 4′s mean is less than Group 2′s. Combine conclusions to say that Group 3′s mean is greater than Group 2′s, which is greater than Group 4′s.

In the cell-phone example, none of the means are equal to each other, and based on the signs of confidence intervals and the results of all the individual pairwise comparisons, the following order of cell-phone mean usage prevails: U4 > U,3 > U,2> u,4. (Hypothetical data aside, it might be the case that 40- to 59-year-olds use a lot of cell phone time because of their jobs.)

Notice near the top of Figure 10-3 that you see "simultaneous confidence level = 80.32 percent." That means the overall error rate for this procedure is 1 – 0.8032 = 0.1968, which is close to 20 percent.

Separating the turkeys with TUkey’s test

This section dives into Tukey’s test and applies it to the cell-phone example.

Setting up Tukey’s test

The basic idea behind Tukey’s test is to provide a series of simultaneous confidence intervals for the differences in the means. It still examines all possible pairs of means and keeps the overall error rate (also known as the Familywise error rate) At a (like Fishers LSD), but it also keeps the individual Type I error rate for each pair of means at a as well. This difference takes care of a lot of issues raised with Fisher’s LSD procedure (refer to the preceding section).

Although the details of the formulas used for Tukey’s test are beyond the scope of this book, they’re not based on the t-test, but rather something called a Stu-dentized range statistic, Which is based on the highest and lowest means in the group, and their difference. The individual error rates are held at 0.05 because Tukey developed a cutoff value for his test statistic, which is based on all pair-wise comparisons (no matter how many means are in each group).

If you calculate the results by hand, you can look at tables to make your conclusions. However, all applications I have ever seen both in the classroom and outside of it use a computer for these calculations. (For sanity’s sake, I suggest you do the same.)

To conduct Tukey’s test, go to Stat>ANOVA>One-way or One-way unstacked. (If your data appears in two columns with Column 1 representing the population number and Column 2 representing the response, just click One-way because your data is stacked. If your data is shown in K Columns, one for each of the K Populations, click One-way unstacked.) The next step is to highlight the data for the groups you’re comparing and click Select. Then click on Comparisons. Click on Tukey’s. The familywise (overall) error rate is listed at 5 (percent), which is typical. If you want to change it, type in the desired error rate (between 0.5 and 0.001) and click OK. You may type in your error rate as a decimal, such as 0.05, or as a number greater than one, such as 5. Numbers greater than one are interpreted as a percentage.

Doing Tukey’s test on the cell phone data

The Minitab output for comparing the groups regarding cell-phone use by using Tukey’s test appears in Figure 10-4. Looking at Figure 10-4, you see that its results can be interpreted in the same was as for Figure 10-3. Some of the numbers in the confidence intervals are different, but in this case, the main conclusions are the same: Those 19 and under use their cell phones most, followed by 40- to 59-year-olds, then 20- to 39-year-olds, and finally those 60 and over.

The results of Fisher and Tukey don’t always agree, usually because the overall error rate of Fisher’s procedure is larger than Tukey’s (except when only two means are involved). Most statisticians I know prefer Tukey’s procedure over Fisher’s. That doesn’t mean they don’t have other procedures they like even better than Tukey’s, but Tukey’s is the most common procedure, and many people like to use it.

Figure 10-4:

Output for Tukey’s test used to compare cell-phone usage.

Another multiple comparison procedure is listed on Minitab’s repertoire after you ask it to do multiple comparisons. This procedure is called Dunnett’s test. Dunnett’s test is a special multiple comparison procedure used in a designed experiment that contains a control group. The test compares each treatment group to the control group and determines which treatments do better than others that way. Dunnett’s test is better able to find real differences in this situation than other multiple comparison procedures, because it focuses only on the differences between each treatment and the control — not the differences between every single pair of treatments in the entire study.

In This Chapter

^ Building and carrying out ANOVA with two factors

^ Getting familiar with (and looking for) interaction effects and main effects

^ Putting the terms to the test

^ Demystifying the two-way ANOVA table

J\ Nalysis of variance (ANOVA) is often used in experiments to see whether T \ Different levels of an explanatory variable (x) Get different results on some quantitative variable Y. (See Chapter 9.) The X Variable in this case is called a Factor, And it has certain levels to it, depending on how the experiment is set up. For example, say you want to compare the average reduction in blood pressure on certain dosages of a drug. The factor is drug dosage. Suppose it has three levels: 10mg per day, 20mg per day, or 30mg per day. Suppose someone else studies the response to that same drug and examines whether the times taken per day (one time or two times) has any effect on blood pressure. In this case, the factor is number of times per day, and it has two levels: once and twice.

Suppose you want to study the effects of dosage And Number of times taken together, because you believe both may have an affect on the response. So what you have is called a Two-way ANOVA, Using two factors together to compare the average response. So it’s an extension of one-way ANOVA (refer to Chapter 9) with a twist, because the two factors you use may operate on the response differently together than they would separately.

In this chapter, you examine two-way ANOVA — setting up the model, making your way through the ANOVA table, taking the F-tests, and drawing the appropriate conclusions.

Setting Up the Two-Way ANOVA Model

The two-way ANOVA model extends the ideas of the one-way ANOVA model and adds an interaction term to examine how various combinations of the two factors affect the response. In this section, you see the building blocks of a two-way ANOVA: the treatments, main effects, the interaction term, and the sums of squares equation that puts everything together.

Determining the treatments

The two-way ANOVA model contains two factors, A And B, And each factor has a certain number of levels (say I Levels of factor A and J Levels of factor B). In the drug study example from the chapter intro, you have A = drug dosage with I = 1, 2, or 3 and B = number of times taken per day with J = 1 or 2. Each person involved in the study is subject to one of the three different drug dosages and will take the drug in one of the two methods given. That means you have 3 * 2 = 6 different combinations of factors A and B that you can apply to the subjects, and you can study it in the two-way ANOVA model.

Each different combination of levels of factors A and B is called a Treatment In the model. Table 11-1 shows the six treatments in the drug study. For example, Treatment 4 is the combination of 20mg of the drug taken in two doses of 10mg each per day.

If factor A has I Levels and factor B has JLevels, you have I * jDifferent combinations of treatments in your two-way ANOVA model.

Stepping through the sums of squares

The two-way ANOVA model contains three terms:

The main effect A: A term for the effect of factor A on the response

The main effect B: A term for the effect of factor B on the response

The interaction of A and B: The effect of the combination of factors A and B (denoted AB)

The sums of squares equation for the one-way ANOVA (see Chapter 9) is SSTO = SST + SSE, where SSTO is the total variability in the response variable, y; SST is the variability explained by the treatment variable (call it factor A); and SSE is the variability left over as error. The purpose of this model is to test to see whether the different levels of factor A produce different responses in the Y Variable. The way you do it is by using Ho: U4 = U,2 = . . . = U,,-, Where I Is the number of levels of factor A (the treatment variable). If you reject Ho, then factor A (which separates the data into the groups being compared) is significant. If you can’t reject Ho, you can’t conclude that factor A is significant.

In the two-way ANOVA, you add another factor to the mix (B) plus an interaction term (AB). The sums of squares equation for the two-way ANOVA model is SSTO = SSA + SSB + SSAB + SSE. Here SSTO is the total variability in the Y-values; SSA is the sums of squares due to factor A (representing the variability in the y-values explained by factor A.); and similarly for SSB and factor B. SSAB is the sums of squares due to the interaction of factors A and B, and SSE is the amount of variability left unexplained, and deemed error. (While the mathematical details of all the formulas for these terms are unwieldy and beyond the focus of this book, they just extend the formulas for one-way ANOVA found in Chapter 9. ANOVA handles the calculations for you, so you don’t have to worry about that part.)

To carry out a two-way ANOVA in Minitab, enter your data in three columns. Column 1 contains the responses (the actual data). Column 2 represents the level of factor A (Minitab calls it the row factor). Column 3 represents the level of factor B (Minitab calls it the Column factor). Go to Stat>Anova>Two-way. Click on Column 1 in the left-hand box and it appears in the Response box on the right-hand side. Click on Column 2 and it appears in the row factor box; click on Column 3 and it appears in the column factor box. Click OK.

For example, suppose you have six data values in Column 1: 11, 21, 38, 14, 15, 62. Suppose Column 2 contains 1, 1, 1, 2, 2, 2, and Column 3 contains 1, 2, 3, 1, 2, 3. This means that factor A has two levels (1, 2), and factor B has three levels (1, 2, 3). The number 11 was the response when Level 1 of factor A and Level 1 of factor B were applied. The second data value, 21, came from Level 1 of A and Level 2 of B. The third value, 38, came from Level 1 of A and Level 3 of B. The fourth number, 14, came from Level 2 of A and Level 1 of B. The number 15 is the response from Level 2 of A and Level 2 of B, and finally, the number 62 corresponds to the result of Level 2 of A and Level 3 of B.

Suppose factor A has I Levels and factor B has J Levels, with a sample of size M Collected on each combination of A and B. The degrees of freedom for factor A, factor B, and the interaction term AB are (i - 1), (j - 1), and (i - 1) * (j - 1) respectively. This formula is just an extension of the degrees of freedom for the one-way model for factors A and B. The degrees of freedom for SSTO is I * J * M - 1, and the degrees of freedom for SSE is I * J * (m - 1).

Understanding Interaction Effects

The interaction effect is the heart of the two-way ANOVA model. Knowing that the two factors may act together in a different way than they would separately is important and must be taken into account. In this section, you see the many ways in which the interaction term AB and the main effects of factors A and B affect the response variable in a two-way ANOVA model.

What is interaction anyway?

Interaction is when two factors meet, or interact with each other, on the response in a way that’s different from how each factor affects the response separately. For example, before you can test to see whether dosage of medicine (factor A) or number of times taken (factor B) are important in explaining changes in blood pressure, you have to look at how they operate together to affect blood pressure. That is, you have to examine the interaction term.

Suppose you’re taking one type of medicine for cholesterol and one medicine for a heart problem. Suppose researchers only looked at the effects of each drug alone, saying each one was good for managing the problem for which it was designed, with little to no side effects. Now you come along and mix the two drugs in your system. As far as the individual study results are concerned, all bets are off. With only those separate studies to go on, they will have no idea how the drugs will interact with each other, and you can be in a great deal of trouble very quickly. Fortunately, drug companies and medical researchers do a great deal of work studying drug interactions, and your pharmacist knows which drugs interact as well. You can bet a statistician was involved in this work from day one!

Baking is another good example of how interaction works. Slurp down one raw egg, drink a cup of milk, and eat a cup of sugar, a cup of flour, and a stick of margarine. Then eat a cup of chocolate chips. Each one of these items has a certain taste, texture, and affect on your taste buds that, in most cases, won’t be all that great. But mix them all together in a bowl and voila! You have a batch of chocolate chip cookie dough, thanks to the magic effects of interaction.

Interacting with interaction plots

In the two-way ANOVA model, you have two factors and their interaction. A number of results could come out of this model in terms of significance of the individual terms, as you can see in the following:

Factors A and B are both significant. Factor A is significant but not factor B. Factor B is significant but not factor A. Neither factors A nor B are significant. The interaction term AB is significant.

Figure 11-1 depicts each of these five situations, respectively, in terms of a diagram, using the drug-study example. Plots that show how factors A and B react separately and together on the response variable Y Are called Interaction plots. In the following sections, I describe each of these five situations in detail in terms of what the plots are telling you and what the results mean in the context of an example.

Factors A and B are significant

Figure 11-1a shows the situation when both A and B are significant in the model (no interaction present). The lines represent the levels of the times-per-day factor (B); the X-axis represents the levels of the dosage factor (A); and the Y-axis represents the average value of the response variable Y, Change in blood pressure, at each combination of treatments.

The top line moving across Figure 11-1a shows that when the drug is taken two times per day, the change in blood pressure increases with dosage level. The bottom line shows the same thing happens when the drug is taken once per day, except that the effects on blood pressure are lower overall than the effects of taking the drug twice a day. That means factor A is significant because blood pressure changes across dosage levels, and factor B is significant because blood pressure is different from one line to another. (Assume the difference is large enough to be significant.) Here the different combinations of factors A and B don’t affect the overall trends, so there’s no interaction effect.

Two parallel lines in an interaction plot means a lack of an interaction effect. In the drug-study example, the levels of A don’t change blood pressure differently for different levels of B.

Factor A is significant but not factor B

Figure 11-1b shows that blood pressure changes across dosage levels for taking the drug once or twice a day. However, the two lines are so close

Together that whether you take the drug once or twice a day has no effect. So factor A (dosage) is significant, and factor B (times per day) isn’t. Parallel lines again means no interaction effect.

2 times/day 1 time/day

10mg 20mg 30mg (dosage)

2 times/day 1 time/day

10mg 20mg 30mg (dosage)

-• 2 times/day -• 1 time/day

10mg 20mg 30mg (dosage)

Figure 11-1:

Five

Examples of the results from a two-way ANOVA with interaction.

2 times/day ~* 1 time/day

10mg 20mg 30mg (dosage)

2 times/day

1 time/day

10mg 20mg 30mg

A.

B.

C.

D.

E.

Factor B is significant but not factor A

Figure 11-1c shows where factor B is significant but A isn’t. The lines are flat across dosage levels indicating that dosage has no effect on blood pressure. However, the two lines for times per day are spread apart, so their effect on blood pressure is significant. Parallel lines mean no interaction effect.

Neither factor is significant

Figure 11-1d shows two flat lines that are very close to each other. By the previous discussions about Figures 11-1b and 11-1c, you can guess that this figure represents the case where neither factor A nor factor B are significant, and you don’t have an interaction effect because the lines are parallel.

Interaction term is significant

Finally you get to Figure 11-1e, the most interesting interaction plot of all. The big picture is that because the two lines cross, then factors A and B interact with each other in the way that they operate on the response. If they didn’t interact, then the lines would be parallel.

Start with the top line of Figure 11-1e. When you take the drug two times per day at the low dose, you get a low change in blood pressure; as you increase dosage, blood pressure increases also. But when you take the drug once per day, the opposite result happens.

If you didn’t look for a possible interaction effect before you examined the main effects, you may have thought no matter how many times you take this drug per day, the effects will be the same. Not so! Always check out the interaction term first in any two-way ANOVA. If the interaction term is significant, you have no way to pull out the effects due to just factor A or just factor B; they’re moot. Checking the main effects of factor A or B without checking out the interaction AB term is considered a no-no in the two-way ANOVA world. Another taboo is examining the factors individually (also known as the main effect) if the interaction term is significant.

Testing the Terms in Two-Way ANOVA

In a one-way ANOVA, you have only one hypothesis test. You use an F-test to determine whether the means of the Y Values are the same or different as you go across the levels of the one factor. In two-way ANOVA you have more items to test besides the overall model. You have the interaction term AB and possibly the main effects of A and B. Each test in a two-way ANOVA is an F-test based on the ideas of one-way ANOVA (see Chapter 9 for more on this).

First, you test whether the interaction term AB is significant. To do this, you use the test statistic F = ,,p , which has an F-distribution with (/ – 1) * ( J - 1) degrees of freedom from MSAB (mean sum of squares for the interaction term

Of A and B) and I * j * (m - 1) degrees of freedom from MSE (mean sum of squares for error), respectively. (Recall that I And J Are the number of levels of A and B, and M Is the sample size at each combination of A and B.) You basically want to see whether more of the total variability in the y’s can be explained by the AB term compared to what is left in the error term. A large value of F Means that the AB term is significant, and you leave it in the model.

If the interaction term isn’t significant, you take the AB term out of the model, and you can explore the effects of factors A and B separately regarding the

Response variable Y. The test for Factor A uses the test statistic F = ,

Which has an F-distribution with I - 1 degrees of freedom from MSA (mean sum of squares for factor A) and I * J* (m - 1) degrees of freedom from MSE (mean

, , . , t, . , , t, R, MS B

Sum of squares for error), respectively. Jesting for factor B uses F = , which has an F-distribution with J - 1 and I * J * (m – 1) degrees of freedom.

The results you can get from testing the terms of the ANOVA model are the same as those represented in Figure 11-1. They’re all provided in Minitab output outlined in the next section, including their sum of squares, degrees of freedom, mean sum of squares, and p-values for their appropriate F-tests.

Running the Two-Way ANOVA Table

The ANOVA table for two-way ANOVA includes the same elements as the ANOVA table for one-way ANOVA (see Chapter 9). But where in the one-way ANOVA you had one line for Factor A’s contributions, now you add lines for the effects of Factor B and the interaction term AB. Minitab calculates the ANOVA table for you as part of the output from running a two-way ANOVA.

In this section, you can figure out how to interpret the results of a two-way ANOVA, assess the model’s fit, and use a multiple comparisons procedure, using the drug-data study.

Interpreting the results: Numbers and graphs

The drug-study example has, say, four people in each treatment combination of three possible dosage levels (10, 20, 30mg per day) and two possible times for taking the drug (one time per day and two times per day). The total sample size is 4 * 3 * 2 = 24. I made up five different data sets in which the analyses represent each of the five scenarios shown in Figure 11-1. Their ANOVA tables, as created by Minitab, are shown in Figure 11-2.

The order of the graphs in Figure 11-1 and the ANOVA tables in Figure 11-2 isn’t the same. Can you match them up? (I promise to give you the answers, so keep reading.)

S = 0.7360

R-Sq = 11.03%

R-Sq(adj) = 0.00%

Figure 11-2:

ANOVA tables for the interaction plots from Figure 11-1.

S = 7.6236

R-Sq = 84.27%

R-Sq(adj) = 79.90%

S = 0.5137

R-Sq = 71.99%

R-Sq(adj) = 64.21%

A

B

C

D

E

Notice that each ANOVA table in Figure 11-2 shows the degrees of freedom for dosage is 3 – 1 = 2; the degrees of freedom for times per day is 2 – 1 = 1; the degrees of freedom for the interaction term is (3 – 1)(2 – 1) = 2; the

Degrees of freedom for total is 3 * 2 * 4 – 1 = 23; and degrees of freedom for error is 3 * 2 * (4 – 1) = 18.

Here are the answers to match the graphs from Figure 11-1 with the output from Figure 11-2:

In the ANOVA table for Figure 11-2a, you see that the interaction term isn’t significant (p-value = 0.526), so the main effects can be studied. The p-values for dosage and times taken are 0.000 and 0.001, indicating both factors A and B respectively are significant; this matches the plot in Figure 11-1a.

In Figure 11-2b, you see that the p-value for interaction is significant (p-value = 0.000) so you can’t examine the main effects of factors A and B (in other words, don’t look at their P-values). This represents the situation in Figure 11-1e.

Figure 11-2c shows nothing is significant (p-value for interaction term is 0.513; p-values for main effects of A (dosage) and B (times taken) are 0.926 and 0.416, respectively). These results coincide with Figure 11-1d.

Figure 11-2d matches Figure 11-1b, with no interaction effect (p-value = 0.899), dosage (factor A) is significant (p-value = 0.000), and times per day (factor B) isn’t (p-value = 0.207).

Figure 11-2e matches Figure 11-c. Dosage * times per day is not significant (p-value = 0.855); times per day is significant with P-value 0.000 but not dosage level (p-value = 0.855).

Assessing the fit

To assess the fit of the two-way ANOVA models, you can use the R2 Adjusted (see Chapter 5). The higher this number is, the better (the maximum is 100 percent or 1.00). Notice that all the ANOVA tables in Figure 11-2 show a fairly high R2 Adjusted except for Figure 11-2c. In this table, none of the terms was significant.

Multiple comparisons

In the case where you find that an interaction effect is statistically significant, you can conduct multiple comparisons to see which combinations of factors A and B create different results in the response. The same ideas hold here as those for Chapter 10 on multiple comparisons, except the tests are performed on all I * j Interactions.

To perform multiple comparisons for a two-way ANOVA by using Minitab, enter your responses (data) in Column 1, your levels of Factor A in Column 2, and your levels of Factor B in Column 3. Choose Stat>ANOVA>General Linear Model. In the Responses box, enter your Column 1 variable. In Model, enter 1 <space> 2 <space> 1*2 (for the main effects and the interaction effect, respectively; here <space> means leave a space where indicated). Click on Comparisons. In Terms, enter columns 2 and 3. Check the Method you want to use for your multiple comparisons (see Chapter 10). Click OK.

In This Chapter

^ Relating the formulas and procedures for one-way ANOVA and regression ^ Making the connection between these two seemingly unrelated procedures

You’re motoring on in your intermediate stat course, working your ^^way through regression (where you estimate Y, Using one or more X Variables — see Chapter 4). Then you hit a new topic, ANOVA, which stands for Analysis of variance — Comparing the means of several populations (see Chapter 9). That seems to be no problem. But wait a minute; now your professor starts talking about how ANOVA is related to regression — suddenly everything starts to spin out of control. How do you reconcile two techniques that appear to be as different as apples and oranges? That’s what this chapter is all about.

Think of this chapter as your bridge across the gap that lies between regression and ANOVA, allowing you to walk smoothly across, answering any questions that a professor may throw into your path. You don’t apply these two techniques in this chapter (you can find that information in Chapters 4 and 9). The goal of this chapter is to determine and describe the relationship between regression and ANOVA so they don’t look quite so much like an apple and an orange.

Seeing Regression through the Eyes of Variation

Every statistical model tries to explain why the different outcomes (y) Are what they are. It tries to figure out what factors or explanatory variables (x) Can help explain that variability in those y’s. In this section, you start with the

Y-values by themselves and see how their variability plays a central role in the regression model. This is the first step toward applying ANOVA (the analysis of Variance) To the regression model.

Verifying Variability in the y’s and looking at x to explain it

No matter what Y Variable you’re interested in predicting, you will always have variability in those y-values. If you want to predict the length of a fish, you may notice that fish have many different lengths (indicating a great deal of variability). Even if you put all the fish of the same age and species together, you still have some variability in their lengths (it will be less than before, but still there nonetheless). The first step to understanding the basic ideas of regression and ANOVA is to understand that variability in the Y‘s is to be expected, and your job is to try to figure out what can explain most of it. This section deals with seeing and explaining variability in the y-values.

Seeing the variability in Internet use

Both regression and ANOVA work to get a handle on explaining the variability in the Y Variable using an X Variable. After you collect your data, you can find the standard deviation in the Y Variable to get a sense of how much the data varies within the sample. From there, you collect data on an X Variable and see how much it contributes to explaining that variability.

Suppose you notice that people spend different amounts of time on the Internet, and you want to explore why that may be. You start by taking a small sample of 20 people and record how many hours per month they spend on the Internet. The results (in hours) are 20, 20, 22, 39, 40, 19, 20, 32, 33, 29, 24, 26, 30, 46, 37, 26, 45, 15, 24, and 31. The first thing you notice about this data is the large amount of variability in it. The Standard deviation (average distance from the data values to their mean) of this data set is 8.93, which is quite large given the size of the numbers in the data set.

Finding an "x-planation" for Internet use

So you figure out that the Y-values (such as amount of time someone uses the Internet from the preceding section) have a great deal of variability in them. What can help explain this? Part of the variability is due to chance. But you

Suspect some variable is out there (call it X) That has some connection to the Y Variable, and that variable can help you make more sense out of this seemingly wide range of y-values.

For example, if you record the calories for five types of candy bars as 100, 200, 300, 400, and 500, you would say "Wow, that’s a lot of variation in calories; I wonder why that is?" Then you notice that the weights of the candy bars are 1, 2, 3, 4, and 5 ounces, respectively. This relationship can be expressed as Y = 100x, where Y Equals calories and X Equals weight.

Now you can look at what before was a bunch of variability in the y-values and say, "Hey, that’s not just random variability; the differing y-values can be explained by the weight of candy bar (x)." You can now use X In a nice regression model to estimate Y. Notice that you’re talking about splitting the total variability in the Y‘s into the part due to X And the part due to chance (error). That’s ANOVA language! Hey, perhaps regression and ANOVA are related after all. . .

To continue with the Internet use example, suppose you have a brainstorm that number of years of education could possibly be related to Internet use. In this case, the explanatory variable (input variable, X) Is years of education, and you want to use it to try to estimate Y, The number of hours on the Internet in a month. You take a larger random sample of 250 Internet users and ask them how many years of education they had (so N = 250). You can check out the first ten observations from your data set containing the (x, y) Pairs in Table 12-1. If a significant connection of some sort exists between the X-values and the Y-values, then you can say that X Is helping to explain some of the variability in the Y‘s. If it explains enough variability, you can place X Into a simple regression model and use it to estimate Y.

(continued)

Getting results with regression

After you have a possible X Variable picked, you collect pairs of data (x, y) On a random sample of individuals from the population, and you look for a possible linear relationship between them. To do this, use Minitab to make a scatterplot of the data and calculate the correlation (r). If the data appear to follow a straight line (as shown on the scatterplot), you go ahead and perform a simple linear regression of the response variable Y Based on the X Variable. The p-value of the X Variable in the simple linear regression analysis tells you whether or not the X Variable does a significant job in predicting Y. Some of the details of getting the regression results are described below (for full information, see Chapter 4).

Looking at the small snippet of 10 out of the 250 person data set in Table 12-1, you can begin to see that you may have a pattern between education and Internet use. It looks like as education increases so does Internet use.

To do a simple linear regression using Minitab, enter your data in two columns: the first column for your X Variable and the second column for your Y Variable (as in Table 12-1). Go to Stat>Regression>Regression. Click on your Y Variable in the left-hand box; the Y Variable then appears in the Response box on the right-hand side. Click on your X Variable in the left-hand box; the X Variable then appears in the Predictor box in the right-hand side. Click OK, and your regression analysis is done. As part of every regression analysis, Minitab also provides you with the corresponding ANOVA results, found at the bottom of the output.

The simple linear regression output that Minitab gives you for the education and Internet example is in Figure 12-1. (Notice the ANOVA output at the bottom; you can see the connection in the upcoming section "Regression and ANOVA: A Meeting of the Models.")

Figure 12-1:

Output for simple linear regression applied to education and Internet use data.

Regression Analysis: Internet versus Education

The regression equation is Internet = —8.29 + 3.15 Education

Predictor Coef SE Coef T P

Constant —8.290 2.665 —3.11 0.002

Education 3.1460 0.2387 13.18 0.000

S = 7.23134

R—Sq = 41.2%

R—Sq(adj) = 41.0%

Analysis of Variance

Source DF SS

Regression 1 9085.6

Residual Error 248 12968.5

Total 249 22054.0

MS F

9085.6 173.75 5 2.3

P

0.000

Looking at Figure 12-1, you see that the p-value on the row marked Education Is 0.000, which means the p-value’s less than 0.001. Therefore the relationship between years of education and Internet use is statistically significant. A scat-terplot of the data (not shown here) also indicates that the data appear to have a positive linear relationship. That means as you increase number of years of education, Internet use also tends to increase (on average).

Assessing the fit of the regression model

Before you go ahead and use a regression model to make predictions for Y Based on an X Variable, you must first assess the fit of your model. One way to get a rough idea of how well your regression model fits is by using a Scatterplot (a graph showing all the pairs of data plotted in the X-y Plane). Use the scatterplot to see whether the data appears to fall in the pattern of a line. If the data appears to follow a straight-line pattern (or even something close to that — anything but a curve or a scattering of points that has no pattern at all), you calculate the correlation, r, to see how strong the linear relationship between X And Y Is (the closer R Is to +1 or -1, the stronger the relationship; the closer R Is to zero, the weaker the relationship). Minitab can do scatterplots and correlations for you; see Chapter 4 for more on simple linear regression, including making a scatterplot and finding the value of R.

If the data doesn’t have a significant correlation, stop the analysis; you can’t go further to find a line that fits a relationship that doesn’t exist.

Next you come to the more general way of assessing not only the fit of a simple linear regression model, but many other models too (for example: multiple, nonlinear, and logistic regression models in Chapters 5, 7, and 8, to name a few). In simple linear regression, the value of R2, as indicated by Minitab and statisticians as a capital R (squared), is equal to the square of the Pearson correlation coefficient, R (indicated by Minitab and statisticians by a small r). In all other situations, R2 Provides a more general measure of model fit. (Note that R Only measures the fit of a straight-line relationship between one X Variable and one Y Variable; see Chapter 4.) Finally, R2 Adjusted modifies R2 To account for the number of variables in the model. R2 Is what statisticians use to assess model fit (see Chapter 5 for more).

The value of R2 Adjusted for the model of using education to estimate Internet use (Figure 12-1) is equal to 41 percent. This value reflects the percentage of variability in Internet use that can be explained by a person’s years of education. This number isn’t great, but it’s not terrible either. Note the square root of 41 percent is 0.64 for R Itself, which in the case of linear regression indicates a moderate relationship.

This evidence gives you the green light to use the results of the regression analysis to estimate number of hours of Internet use in a month by using years of education. The regression equation as it appears in the top part of the Figure 12-1 output is Internet = -8.29 + 3.15 * 16 = 42.11. So if you have 16 years of education, for example, your estimated Internet use is 42.11, or about 42 hours per month (about 10.5 hours per week).

But wait! Look again at Figure 12-1 and zoom in on the bottom part. I didn’t ask for anything special to get this info on the Minitab output, but you can see an ANOVA table there. That seems like a fish out of water doesn’t it? But in the next section you see how an ANOVA table can describe regression results (albeit it in a different way).

Regression and ANOVA: A Meeting of the Models

Okay, here it comes. You’ve already broken down the regression output into all its pieces and parts. The next step toward understanding the connection between regression and ANOVA is to apply the sums of squares from ANOVA to regression (something that is typically not done in a regression analysis). Before you start, think of this process as going to a 3-D movie, where you have to wear special glasses in order to see all the special effects!

In this section, you see the sums of squares in ANOVA applied to regression and how the degrees of freedom work out. You build an ANOVA table for regression and discover how the T-test For a regression coefficient is related to the F-test in ANOVA. I know you can hardly wait, so I won’t keep you in suspense any longer.

Comparing sums of squares

Sums of squares Is a term you may remember from ANOVA (see Chapter 9), but it certainly isn’t a term you normally use when talking about regression (as in Chapter 4). Yet, both types of models can be broken down into sums of squares, and that similarity gets at the true connection between ANOVA and regression. In step-by-step terms, you first partition out the variability in the Y Variable by using formulas for sums of squares from ANOVA (sums of squares for total, treatment, and error). Then you find those same sums of squares for regression — this is the twist on the process because you typically don’t find sums of squares for regression. You compare the two procedures through their sums of squares. This section shows you the details of how this comparison is done.

Partitioning Variability by using SSTO, SSE, and SST for ANOVA

ANOVA is all about partitioning the total variability in the Y-values into sums of squares (see all the info you ever need on one-way ANOVA in Chapter 9). The key idea is that SSTO = SST + SSE, where SSTO is the total variability in the Y-values; SST measures the variability explained by the model (also known as the treatment, or X Variable in this case); and SSE measures the variability due to error (what’s left over after the model is fit).

Y - y J ,

And X CY – Y J Respectively, where Y Is the mean of the Y‘s, Yt Is each observed

Value of Y, And Y Is each predicted value of Y From the ANOVA model. Use these formulas to calculate the sums of squares for ANOVA (Minitab does this for you when it performs ANOVA). Keep these values of SSTO, SST, and SSE. You will use them to compare to the results from regression.

Finding sums of squares for regression

In regression, you measure the deviations in the Y-values by taking each Yt Minus its mean, Y. Square each result and add them all up, and you have SSTO. Next, take the residuals, which represent the difference between each Yt And it’s estimated value from the model, Y. Square the residuals and add them up, and you get the formula for SSE.

Now that you have calculated SSTO and SSE, you need the bridge between them. That is, you need a formula that connects the variability in the y’s (SSTO) and the variability in the residuals after fitting the regression line (SSE). That bridge is SSR (equivalent to SST in ANOVA). In regression, y represents the predicted value of yi based on the regression model. These are the values on the regression line. To assess how much this regression line helps to predict the Y-values, you compare it to the model you would get without any X Variable in it.

Without any other information, the only thing you can do to predict Y Is look at the average, Y. So, SST compares the predicted value from the regression line to the predicted value from the flat line (the mean of the y’s) by subtract -

Ing them. The result is YY _Y ). Square each result and sum them all up, and you get the formula for SST.

Now for one last hoop to jump through (as if you haven’t had enough already). Instead of calling the sum of squares for the regression model SST as is done in ANOVA, statisticians call it SSR For Sum of squares regression. Consider SSR from regression to be equivalent to the SST from ANOVA. The reason this is important is because computer output lists the sums of squares for the regression model as SSR not SST.

To summarize the sums of squares as they apply to regression, you have SSTO = SSR + SSE where

SSTO measures the variability in the observed y-values around their mean. This value represents the variance of the Y-values.

SSE represents the variability between the predicted values for Y (the values on the line) and the observed Y-values. SSE represents the variability left over after the line has been fit to the data.

SSR measures the variability in the predicted values for Y (the values on the line) from the mean of Y. SSR is the sum of squares due to the regression model (the line) itself.

Minitab calculates all the sums of squares for you as part of the regression analysis. You can see this calculation in the section "Bringing regression to the ANOVA table."

Dividing up the degrees of freedom

In ANOVA, you test a model for the treatment (population) means by using an

F-test, which is F = MST. To get MST (the mean sum of squares for treatment),

MSE » v Ji

You take SST (the sum of squares for treatment) and divide by its degrees of

Freedom. You do the same with MSE (that is, take SSE, the sum of squares for error, and divide by its degrees of freedom). The question now is, what do those degrees of freedom represent and how do they relate to regression? This section addresses that issue.

Degrees of freedom in ANOVA

In ANOVA, the degrees of freedom for SSTO is N - 1, which represents the sample size minus one. In the formula for SSTO, X _Yt – y), you see there are N Observed Y-values minus one mean. That in a very general way is where the N - 1 comes from.

X _ Y - y )2

Note that if you divide SSTO by N - 1, you get —1-s-^-, The variance in the

N-1

Y-values. This calculation makes good sense because the variance also measures the total variability in the Y-values.

The degrees of freedom for SSE is N - K. In the formula for SSE, X Y Y – y J ,

You see there are N Observed Y-values, and K Is the number of treatments in the model. In regression, the number of coefficients in the model is K = 2 (the slope and the Y-intercept). So you have degrees of freedom N - 2 associated with SSE when you’re doing regression.

Degrees of freedom in regression

The degrees of freedom for SST in ANOVA equals the number of treatments minus one. How does the degrees of freedom idea relate to regression? The number of treatments in regression is equivalent to the number of parameters in a model (a parameter being an unknown constant in the model that you’re trying to estimate).

When you test a model you’re always comparing it to a different (simpler) model to see whether it fits the data better. In linear regression you compare your regression line Y = B0 + B1x, To the horizontal line Y = Y. This second, simpler model just uses the mean of Y To predict Y All the time, no matter what X Is. In the regression line, you have two coefficients: one to estimate the parameter for the Y-intercept (b0) And one to estimate the parameter for slope (b1) In the model. In the second, simpler model, you have only one parameter: the value of the mean. The degrees of freedom for SSR in simple linear regression is the difference in the parameters of the two models: 2 – 1 = 1.

Putting all this together, the degrees of freedom for regression must add up for the equation SSTO = SSR + SSE. The degrees of freedom corresponding to this equation are (n - 1) = (2 – 1) + (n - 2), which is true if you do the math. So the degrees of freedom for regression, using the ANOVA approach, all check out. Whew!

In Figure 12-1, you can see the degrees of freedom for each sums of squares listed under the DF Column of the ANOVA part of the output. You see SSR has 2 – 1 = 1 degree of freedom, SSE has 250 – 2 = 248 degrees of freedom (because N = 250 observations were in the data set and K = 2 and you find N - K To get degrees of freedom for SSE). The degrees of freedom for SSTO is 250 – 1 = 249.

Bringing regression to the ANOVA table

In ANOVA, you test your model Ho: All K Population means are equal versus Ha: At least two population means are different by using a F-test. You build your F-test statistic by relating the sums of squares for treatment to the sum of squares for error. To do this, you divide SSE and SST by their degrees of freedom (n – K And K - 1, respectively, where N Is the sample size and K Is the number of treatments) to get the mean sums of squares for error (MSE) and mean sums of squares for treatment (MST). In general, you want MST to be large compared to MSE, which would indicate that the model fits well. The results of all these statistical gymnastics are summarized by Minitab in a table called (cleverly) the ANOVA table.

The ANOVA table shown in the bottom part of Figure 12-1 for the Internet-use data represents the ANOVA table you get from using the regression line as your model. Under the Source column, you may be used to seeing treatment, error, and total. For regression, the treatment is the regression line, so you see Regression Instead of treatment. The error term in ANOVA is labeled Residual error, Because in regression, you measure error in terms of residuals. Finally you see Total, Which is the same the world around.

The SS column represents the sums of squares for the regression model. The three sums of squares listed in the SS column are SSR (for regression), SSE (for residuals), and SST (total). These sums of squares are calculated using the formulas from the previous section; the degrees of freedom, DF In the table, are found by using the formulas from the previous section also.

The MS column takes the value of SS "whatever"(you fill in the blank) and divides it by the respective degrees of freedom, just like ANOVA. For example in Figure 12-1, SSE is 12,968.5, and the degrees of freedom is 248. Take the first value divided by the second one to get 52.29 or 52.3, which is listed in the ANOVA table for MSE.

The value of the F-statistic, using the ANOVA method, is F = ^jSE = 9’52 3’6 =

173.7 in the Internet example, which you can see in column five of the ANOVA part of Figure 12-1 (subject to rounding). The F-statistics’s p-value is calculated based on an F-distribution with 2 – 1 = 1 and 250 – 2 = 248 degrees of

Freedom, respectively. (In the Internet example, the p-value listed in the last column of the ANOVA table is 0.000, meaning the regression model fits.) But remember, in regression you don’t use an F-statistic and an F-test. You use a /-statistic and a /-test. What gives? The next section explains.

Relating the F – and t-statistics: The final frontier

In regression, one way of testing whether the best-fitting line is statistically significant is to test Ho: slope = 0 versus Ha: slope ^ 0. To do this, you use a /-test (see Chapter 3). The slope is the heart and soul of the regression line, because it describes the main part of the relationship between X And y. If the slope of the line equals zero (you can’t reject Ho), you’re just left with Y = B1, A horizontal line, and your model Y = B0 + b1x Isn’t doing anything for you.

In ANOVA, you test to see whether the model fits by testing Ho: The means of the populations are all equal, versus Ha: At least two of the population means aren’t equal. To do this you use an F-test (taking MST and dividing it by MSE; see Chapter 10).

The sets of hypotheses in regression and ANOVA seem totally different, but in essence, they’re both doing the same general thing: testing whether a certain model fits. In the regression case, the model you want to see fit is the straight line, and in the ANOVA case, the model of interest is a set of (normally distributed) populations with at least two different means (and the same variance). Here each population is labeled as a treatment by ANOVA.

But more than that, you can think of it this way: Suppose you took all the populations from the ANOVA and lined them up side by side on an X-y Plane (see Figure 12-2). If the means of those distributions are all connected by a flat line (representing the mean of the Y‘s), then you would have no evidence against Ho in the F-test, so you can’t reject it — your model isn’t doing anything for you (it doesn’t fit). This idea is similar to the idea of fitting a flat horizontal line through the Y-values in regression; a straight-line model with a nonzero slope doesn’t work in that case.

The big thing is that statisticians can prove (so you don’t have to) that an F-statistic is equivalent to the square of a /-statistic, and the F-distribution is equivalent to the square of a /-distribution when the SSR has df = 2 – 1 = 1. And when you have a simple linear regression model, the degrees of freedom is exactly one! (Note that F Is always greater than or equal to zero, which is needed if you’re making it the square of something.) So there you have it! The /-statistic for testing the regression model is equivalent to an F-statistic for ANOVA when the ANOVA table is formed for the simple regression model.

Figure 12-2:

Connecting means of populations to the slope of a line.

Indeed (the stat professor’s way of saying "and this is the Really Cool part. . ."), if you look at the value of the /-statistic for testing the slope of the education variable in Figure 12-1, you see that it’s 13.18 (look at the row marked Education And the column marked T). Square that value, and you get 173.71. The F-statistic in the ANOVA table of Figure 12-1 is equal to 173.75. The F-statistic from ANOVA and the t-statistic from regression are equal to each other in Figure 12-2, subject to a little round-off error done by Minitab on the output. (Just like magic! I still get chills just thinking about it.)

In This Chapter

^ Extending the /-test for comparing two means by using ANOVA

^ Discovering and utilizing the ANOVA process

^ Carrying out an F-test

^ Navigating the ANOVA table

Ne of the most commonly used statistical techniques at the intermediate level is Analysis of variance (affectionately known as ANOVA).

Because the name has the word variance in it, you may think that this technique has something to do with Variance — and you would be right. Analysis of variance is all about examining the amount of variability in a Y (response) variable and trying to understand where that variability is coming from.

One way that you can use ANOVA is to compare several populations regarding some quantitative variable, Y. The populations you want to compare constitute different groups (denoted by an X Variable), such as political affiliations, age groups, or different brands of a product. ANOVA is also particularly suitable for situations involving an experiment where you apply certain treatments (x) To subjects, and you measure a response (y).

In this chapter, you start with the /-test for two population means, the precursor to ANOVA. Then you move on to the basic concepts of ANOVA: sums of squares, the F-test, and the ANOVA table. You apply these basics to the one-factor or one-way ANOVA, where you compare the responses based only on one treatment variable. (In Chapter 11, you can see them applied to a two-way ANOVA, which has two treatment variables.)

Comparing Two Means with a t-Test

The Two sample t-test Is designed to test to see whether two population means are different. The conditions for the two sample t-test are the following:

*e The two populations are independent (in other words, their outcomes don’t affect each other).

*e The response variable (y) Is a quantitative variable (meaning that its values represent counts or measurements).

*e The y-values for each population have a normal distribution (however, their means may be different; that is what the t-test determines).

*e The variances of the two normal distributions are equal.

For large sample sizes when you know the variances, you use a Z-test for the two population means. However, a t-test allows you to test two population means when the variances are unknown or the sample sizes are small. This occurs quite often in situations where an experiment is performed and the number of subjects is limited.

Although you have seen t-tests before in your intro stats class, it may be good to review the main ideas. The t-test tests the hypotheses Ho: U\ = |u2 versus Ha: Ui Is <, >, or ^u,2, where the situation dictates which of these hypotheses you use. (Just a note that with ANOVA, you extend this idea to K Different means from K Different populations, and the only version of Ha of interest is ^.)

To conduct the two sample T-test, you collect two data sets from the two populations, using two independent samples. To form the test statistic (the T-statistic), you subtract the two sample means and divide by the standard error (a combination of the two standard deviations from the two samples and their sample sizes). You compare the t-statistic to the t-distribution with ni + n2 – 2 degrees of freedom and find the p-value.

If the p-value is less than the prespecified a level, say 0.05, you have enough evidence to say the population means are different. (For information on hypothesis tests, see Chapter 3.)

For example, suppose you’re at a watermelon seed spitting contest where contestants each put watermelon seeds in their mouths and spit them as far as they can. Results are measured in inches and are treated with the reverence of the shot-put results at the Olympics. You want to compare the watermelon seed spitting distances of female and male adults. Your data set includes ten people from each group.

You can see the results of the T-test In Figure 9-1. The mean spitting distance for females was 47.8 inches; the mean for males was 56.5 inches. The t-statistic for the difference in the two means (females – males) is T = -2.23, which has a p-value of 0.039 (see last line of Figure 9-1 output). At a level of a = 0.05, this difference is significant (because 0.039 < 0.05). You conclude that males and females differ with respect to their mean watermelon seed spitting distance. And you can say males are likely spitting farther because their sample mean was higher.

Figure 9-1:

A f-test comparing mean watermelon seed spitting distances for females versus males.

Two-sample T for females vs males

Estimate for difference: -8.70000

95% CI for difference: (-16.90914, -0.49086)

T-Test of difference = 0 (vs not =): T-Value = -2.23 P-Value

= 0.039 DF = 18

Evaluating More Means with ANOVA

Now that you can compare two independent populations inside and out, at some point two populations will not be enough. Suppose you want to compare more than two populations regarding some response variable (y). This idea kicks the t-test up a notch into the territory of ANOVA. The ANOVA procedure is built around a hypothesis test called the F-test, Which compares how much the groups differ from each other, compared to how much variability is in each group. In this section, I set up an example of when to use ANOVA and show you the steps involved in the ANOVA process. You can then apply the ANOVA steps to the following example throughout the rest of the chapter.

Spitting seeds: A situation just waiting for ANOVA

Before you can jump into using ANOVA, you must figure out what question you want answered and collect the necessary data.

Suppose you want to compare the watermelon seed spitting distances for four different age groups: 6-8, 9-11, 12-14, and 15-17. The hypotheses for this example are Ho: \i1 = U,2 = u,3 = u,4 versus Ha: At least two of these means

Are different, where the population means u, represent those from the age groups, respectively. Over the years of this contest, you have collected data on 200 children from each age group, so you have some prior ideas about what the distances typically look like. This year, you have 20 entrants, 5 in each age group. You can see the data from this year, in inches, in Table 9-1.

Do you think you see a difference in distances for these age groups based on this data? If you just combined all the data, you would see quite a bit of difference (the range of the combined data goes from 38 inches to 47 inches). Perhaps accounting for which age groups each contestant is in does explain at least some of what’s going on. But don’t stop there. In the next section, you see the official steps you need to do to answer your question.

4

Walking through the steps of ANOVA

You have decided on the quantitative response variable (y) You want to compare for your K Various population (or treatment) means, and you collected a random sample of data from each population. Now you’re ready to conduct ANOVA on your data to see whether the population means are different for your response variable, Y.

The characteristic that defines these populations is called the Treatment variable, x. Statisticians use the word Treatment In this context because one of the biggest uses of ANOVA is for designed experiments where subjects are randomly assigned to treatments, and the responses are compared for the various treatment groups. So statisticians oftentimes use the word Treatment Even when the study isn’t an experiment, and they’re comparing regular populations. Hey, don’t blame me! I’m just following the proper statistical terminology.

Just to get a feeling for what an ANOVA procedure involves and to give you a quick reference for a later time, here are the general steps in a one-way ANOVA:

1. Check the ANOVA conditions, using the data collected from each of the K Populations.

See the next section, "Checking the conditions," for the specifics on these conditions.

2. Set up the hypotheses Ho: \i1 = U,2 =. . . = Uk Versus Ha: At least two of the population means are different.

Another way to state your alternative hypothesis is by saying Ha: At least two of U2, .. . U* Are different.

3. Collect data from K Random samples, one from each population.

4. Conduct an F-test on the data from step three, using the hypotheses from step two, and find the p-value.

See the section "Doing the F-test" later in this chapter for these instructions.

5. Make your conclusions: If you reject Ho (when your p-value is less than 0.05 or your prespecified A Level), you conclude that at least two of the population means are different; otherwise, you conclude that you didn’t have enough evidence to reject Ho (you can’t say the means are different).

If these steps look like a foreign language to you, don’t fear — I describe each of these steps in detail in the sections to follow.

Checking the Conditions

Step one of ANOVA is checking to be sure all necessary conditions are met before diving into the data analysis. The conditions for using ANOVA are just an extension of the conditions for a /-test (see the section "Comparing Two Means with a /-Test"). The following conditions all need to hold in order for ANOVA to be conducted:

The K Populations are independent (in other words, their outcomes don’t affect each other).

The K Populations each have a normal distribution. The variances of the K Normal distributions are equal.

I go into more detail about these conditions in the following sections.

Checking off independence

To check the first condition, examine how the data was collected from each of the separate populations. In order to maintain independence, the outcomes from one population can’t affect the outcomes of the other populations. If the data has been collected by using a separate random sample from each population (random Here meaning that each individual in the population had an equal chance of being selected), this factor ensures independence at the strongest level.

In the watermelon seed spitting data (see Table 9-1), the data aren’t randomly sampled from each age group because the data represents everyone who participated in the contest. But, you can argue that the seed spitting distances from one age group don’t affect the seed spitting distances from the other age groups, so the independence assumption is okay here also.

Looking for what’s normal

The second ANOVA condition is that each of the K Populations has a normal distribution. To check this condition, make a separate histogram of the data from each group and see whether it resembles a normal distribution. Data from a normal distribution should look symmetric (in other words, if you split the histogram down the middle, it looks the same on each side) and have a bell-shape. Don’t expect the data in each histogram to follow a normal distribution exactly (remember it’s only a sample), but it shouldn’t be extremely different from a normal, bell-shaped distribution.

Since the data contains only five children per age group, checking conditions can be iffy. But in this case, you have past data for 200 children in each age group, so you can use that to check the conditions. The histograms and descriptive statistics of the seed spitting data for the four age groups are shown in Figure 9-2, all in one panel, so you can easily compare them to each other on the same scale. Looking at the four histograms in Figure 9-2, you can see that each graph resembles a bell shape; the normality condition isn’t being violated here. (Red flags should come up if you see two peaks in the data, or a skewed shape where the peak is off to one side, or if the histogram is flat, for example.)

You can use Minitab to make histograms for each of your samples and have all of them appear on one large panel, all using the same scale. To do this, go to Graph>Histogram and click OK. Choose the variables that represent data from each sample by highlighting them in the left-hand box and clicking Select. Then click on Multiple Graphs, and a new window opens. Under the Show Graph Variables option, check the following box: In separate panels of the same graph. On the Same Scales for Graphs option, check the box for X And the box for Y. This option gives you the same scale on both the X And Y Axes for all the histograms. Then click OK.

Figure 9-2:

Checking ANOVA conditions by using histograms and

Descriptive statistics.

Histogram of Age Group 1, Age Group 2, Age Group 3, Age Group 4

36 39 42 45 48 51

Age Group 1

J]

M.

Age Group 3

—i—R I—I—T-L=—,-

36 39 42 45 48 51

_i_i_i_i_i_

Age Group 2

Age Group 4

EL

20

15

10

5

0

Descriptive Statistics: Age Group 1, Age Group 2, Age Group 3, Age Group 4

Taking note of spread

The third condition for ANOVA is that the variance in each of the K Populations is the same. To check this out on your data, use Minitab to find the variance in each sample and compare them. The variances for each sample should be close. What does Close Mean? A hypothesis test can handle this question; however, it falls outside the scope of most intermediate statistics courses. So you are left with a judgment call. Compare all the variances as a group and look for any glaring differences. If a difference is large enough for you to write home about (say 10 percent or more), this variance indicates a problem. (Not only do you have a problem with the ANOVA conditions, but if you’re writing your mom about your stats problems you might need to get a bit of a life.) If no big differences exist in the variances, you can say that the equal variance condition is met. The variances for the seed spitting data are shown in Figure 9-2 for each age group. They are quite close, so this condition is met.

To find descriptive statistics for each sample, go to Stat>Basic Statistics> Display Descriptive Statistics. Click on each variable in the left-hand box for which you want the descriptive statistics and then click Select. Click on the Statistics option, and a new window appears with tons of different types of statistics. Click on the ones you want and click off the ones you don’t want. Click OK. Then click OK again. Your descriptive statistics are calculated.

Note that you don’t need the sample sizes in each group to be equal to carry out ANOVA; however, in intermediate stats, you’ll typically see what statisticians call a Balanced design, Where each sample from each population has the same sample size. (For more precision in your data, the larger the sample sizes, the better; see Chapter 3.)

Setting Up the Hypotheses

Step two of ANOVA is setting up the hypotheses to be tested. You’re testing to see whether or not all the population means can be deemed equal to each other. The null hypothesis for ANOVA is that all the population means are equal. That is, Ho: u, i = u,2 =. . . = Uk, where \i1 Is the mean of the first population, u,2 is the mean of the second population, and so on until you reach u, k (the mean of the Kth Population).

Now what appears in the alternative hypothesis (Ha) must be the opposite of what is in the null hypothesis (Ho). What’s the opposite of having all K Of the population’s means equal to each other? You may think the opposite is that they’re all different. But that’s not the case. In order to blow Ho wide open, all you need is for at least two of those means to not be equal. The alternative hypothesis, Ha, is that at least two of the population means are different from each other. That is, Ha: At least two of U2, . .. Uk Are different.

Note that Ho and Ha for ANOVA are an extension of the hypotheses for a two sample f-test (which only compares two independent populations). And while the alternative hypothesis in a f-test may be that one mean is greater than, less than, or not equal to the other, you don’t consider any alternative other than ^ in ANOVA. You only want to know whether or not the means are equal — at this stage of the game anyway. After you reach the conclusion that Ho is rejected in ANOVA, you can proceed to figure out how the means are different, which ones are bigger than others, and so on, using multiple comparisons. Those details appear in Chapter 10.

Doing the F-Test

Step three, collecting the data, includes taking K Random samples, one from each population. Step four of ANOVA is doing the F-test on this data, which is

The heart of the ANOVA procedure. This test is the actual hypothesis test of Ho: U4 = U,2 =. . . = \ik Versus Ha: At least two of U,1, u,2, . . . \ik Are different.

You have to carry out three major steps in order to complete the F-test (don’t get these steps confused with the main ANOVA steps; consider the F-test a few steps within a step):

1. Break down the variance of Y Into sums of squares.

2. Find the mean sums of squares.

3. Put the mean sums of squares together to form the F-statistic.

I describe each step of the F-test in detail and apply it to the example of comparing watermelon seed spitting distances (see Table 9-1) in the following sections.

Because data analysts rely heavily on computer software to conduct each step of the F-test, you can do the same. All computer software packages organize and summarize the important information from the F-test into a table format for you. This table of results for ANOVA is called (what else?) the ANOVA table. Because the ANOVA table is a critical part of the entire ANOVA process, I start the following sections out by describing how to run ANOVA in Minitab to get the ANOVA table, and I continue to reference this section as I describe each step of the ANOVA process.

Stacked data Means that you enter all the data into two columns. Column one includes the number indicating what sample the data value is from (1 to k), and the responses (y) Are in column two. To analyze this data, go to Stat>ANOVA>One-Way Stacked. Highlight the response (y) Variable and click Select. Highlight the factor (population) variable and click Select. Click OK.

Unstacked Is the other method of entering data: a separate column for the data in each sample. To analyze the data entered this way, go to Stat>ANOVA>One-Way Unstacked. Highlight the names of the columns where your data are located. Click OK.

Running ANOVA in Minitab

Using Minitab to run ANOVA, you first have to enter the data from the k samples. You can enter the data one in of two ways:

I typically use the unstacked version just because I think it helps visualize the data. However, the choice is up to you, and the results come out the same no matter which one you choose.

Breaking down the Variance into sums of squares

The first step of the F-test is splitting up the variability in the Y Variable into portions that define where the variability is coming from. The term Analysis of variance Is a great description for exactly how you conduct a test of K Population means. With the overall goal of testing whether K Population (or treatment) means are equal, you take a random sample from each of the K Populations. You first put all the data together into one big group and measure how much total variability there is; this variability is called the Sums of squares total, Or SSTO. If the data are really diverse, SSTO is large. If the data are very similar, SSTO is small.

Now the total variability in the combined data set (SSTO) can be split into two parts:

SST: The variability between the groups, known as the sums of squares for treatment

SSE: The variability within the groups, known as the sum of squares for error

This splitting up of the variability in your data results in one of the most important equalities in ANOVA. That equality is SSTO = SST + SSE.

IBE# The formula for SSTO is the numerator of the formula for s2, the variance of a single data set, so SSTO = EE (Xv – x J , where I And J Represent the JTh value in the sample from the I Th population. SSTO represents the total squared distance between the data values and their overall mean. The formula for SST is SST = Nt E (Xi – x) , where ni is the size of the sample coming from the ITh population. SST represents the total squared distance between the means from 2 each sample and the overall mean. The formula for SSE is SSE = EE (Xv – xiJ , where XIj Is the JTh value in the sample from the ITh population and Xi Is the mean of the sample coming from the I Th population. This formula represents the total squared distance between the values in each sample and their corresponding sample means. Using algebra, you can show (with some serious elbow grease) that SSTO = SST + SSE.

The Minitab output for the watermelon seed spitting contest for the four age groups is shown in Figure 9-3. Under the Source column of the ANOVA table, you see Factor Listed in row one. The factor variable (as described by Minitab) represents the treatment or population variable. In column three of the Factor row, you see the SST, which is equal to 89.75. In the Error row (row two), you locate the SSE in column three, which equals 56.80. In row three (Total), column three, you see the SSTO, which is 146.55. Using the values of SST, SSE, and SSTO from the Minitab output, you can verify that SST + SSE = SSTO.

Figure 9-3:

ANOVA Minitab output for the watermelon seed spitting example.

One-Way ANOVA: Age Group 1, Age Group 2, Age Group 3, Age Group 4

Source Factor Error Total

DF 3 16 19

SS 89.75 5 6.80 146.55

MS 2 9.92 3.55

F

8.43

P

0.001

S — 1.884 R-Sq — 61.24% R-Sq(adj) — 53.97%

Now you’re ready to use these sums of squares to complete the next step of the F-test (keep reading).

Locating those mean sums of squares

After you have the sums of squares for treatment, SST, and the sums of squares for error, SSE (see preceding section for more on these), you want to compare them to see whether the variability in the y-values that is due to the model (SST) is large compared to the amount of error left over in the data after the groups have been accounted for (SSE). So you ultimately want a ratio comparing SST to SSE somehow. To make this ratio form a statistic that statisticians know how to work with (in this case, an F-statistic), they decided to find the mean of each of SST and SSE and work with that. Finding the mean sums of squares is the second step of the F-test.

MST Is the mean sums of squares for treatments, which measures the mean variability that occurs between the different treatments (the different samples in the data). What you’re looking for is the amount of variability in the data as you move from one sample to another. A great deal of variability between samples (treatments) may indicate that the populations are different as well. You can find MST by taking SST and dividing by K - 1 (where K Is the number of treatments).

MSE Is the mean sums of squares for error, which measures the mean within-treatment variability. The Within-treatment variability Is the amount of variability that you see within each sample itself, due to chance and/or other factors not included in the model. You can find MSE by taking SSE divided by N - K (where N Is the total sample size and K Is the number of treatments). The values of K - 1 and N - K, Respectively, are called the Degrees of freedom For SST and SSE. Minitab calculates and posts the degrees of freedom for SST and SSE, as well as the values of MST and MSE, in the ANOVA table in columns two and four, respectively.

From the ANOVA table for the seed spitting data in Figure 9-3, you can see that column two has the heading DF, Which stands for degrees of freedom. You can find the degrees of freedom for SST in the Factor row (row two); this value is equal to K - 1 = 4 – 1 = 3. The degrees of freedom for SSE is found to be N - K = 20 – 4 = 16. (Remember you have four age groups and five children in each group for a total of N = 20 data values.) The degrees of freedom for SSTO is N - 1 = 20 – 1 = 19 (found in the Total row under DF.) You can verify that the degrees of freedom for SSTO = degrees of freedom for SST + degrees of freedom for SSE.

The values of MST and MSE are shown in column four of Figure 9-3, with the heading MS. You can see the MST in the Factor row, which is 29.92. This value was calculated by taking SST = 89.75, and dividing it by degrees of freedom, 3. You can see MSE in the Error row, equal to 3.55. MSE is found by taking SSE = 56.80 and dividing that value by its degrees of freedom, 16.

By finding the mean sums of squares, you’ve completed step two of the F-test, but don’t stop here! You need to continue to the next section if you want to complete the process.

Figuring the F-statistic

The test statistic for the test of the equality of the K Population means is

F = . The result of this formula is called the F-statistic. The F-statistic MSE

Has an F-distribution, which is equivalent to the square of a T-test (when the numerator degrees of freedom is 1). All F-distributions start at zero and are skewed to the right. The degree of curvature and the height of the curvature of each F-distribution is reflected in two Degrees of freedom, Represented by K - 1 and N - K. (These come from the denominators of MST and MSE, respectively, where N Is the total sample size and K Is the total number of treatments or populations.) A shorthand way of denoting the F-distribution for this test

Is F(k - 1,n – k).

In the watermelon seed spitting example, you’re comparing four means and have a sample of size five from each population. Figure 9-4 shows the corresponding F-distribution, which has degrees of freedom 4 – 1 = 3 and 20 – 4 = 16; in other words 16).

You can see the F-statistic on the Minitab ANOVA output (see Figure 9-3) in the Factor row, under the column indicated by F. For the seed spitting example, the value of the F-statistic is 8.43. This number was found by taking MST = 29.92 divided by MSE = 3.55. You can then locate 8.43 on the F-distribution in Figure 9-4 to see where it stands. (More on that in the next section.)

F (3, 16)

Making conclusions from ANOVA

If you’ve completed the F-test and found your F-statistic (step four in the ANOVA process), you’re ready for step five of ANOVA: making conclusions for your hypothesis test of the K Population means. If you haven’t already, you can compare the F-statistic to the corresponding F-distribution with K - 1, N - K Degrees of freedom, to see where it stands and make a conclusion. You can make the conclusion in one of two ways: the p-value approach or the critical-value approach. (The approach you use depends primarily on whether you have access to a computer, especially during exams.) I describe these two approaches in the following sections.

Using the p-value approach

On Minitab ANOVA output (see Figure 9-3), the value of the F-statistic is located in the Factor row, under the column noted by F. The associated P-value for the F-test is located in the Factor row under the column headed by P. The p-value tells you whether or not you can reject Ho. If the p-value is less than your prespecified a (typically 0.05), reject Ho. Conclude that the K Population means aren’t all equal and that at least two of them are different. If the P-value is greater than a, then you can’t reject Ho. You don’t have enough evidence in your data to say the K Population means have any differences.

The F-statistic for comparing the mean watermelon seed spitting distances for the four age groups is 8.43. The p-value as indicated in Figure 9-3 is 0.001. That means the results are highly statistically significant. You reject Ho and conclude that at least one pair of age groups differ in its mean watermelon seed spitting distances. (You would hope that a 17-year-old could do a lot better than a 6-year-old, but maybe those 6-year-olds have a lot more spitting going on in their lives than 17-year-olds do.)

Using Figure 94, you see how the F-statistic of 8.43 stands on the F-distribution with (4 – 1, 20 – 4) = (3, 16) degrees of freedom. You can see it’s way off to the right, out of sight. It makes sense that the p-value, which measures the probability of being beyond that F-statistic, is 0.001.

Lf you’ve gotta use critical Values…

If you’re in a situation where you don’t have access to a computer (as is still the case in many statistics courses today when it comes to taking exams), finding the exact p-value for the F-statistic isn’t possible. However, statistical software packages automatically calculate all P-values exactly (so on any computer output you can see them as such).

To approximate the p-value from your F-statistic (in the event you don’t have a computer or computer output available), you find a cutoff value on the F-distribution with (k - 1, N - K) Degrees of freedom that draws a line in the sand between rejecting Ho and not rejecting Ho. This cutoff (also known as the Critical value) Is determined by your prespecified a (typically 0.05). You choose the critical value so that the area to its right on the F-distribution is equal to a.

Table A-5 in the Appendix shows the critical values of the F-distribution with various degrees of freedom, all using a = 0.05. Other F-distribution tables are available in various statistics textbooks and Internet links for other values of a; however, a = 0.05 is by far the most common a level used for the F-distribution and is sufficient for your purposes.

This table of values for the F-distribution is called the F-table (students are typically given these with their exams). For the seed spitting example, the F-statistic has an F-distribution with degrees of freedom (3, 16), which I calculate in a previous section. To find the critical value, go to Table A-5 in the Appendix. Because the degrees of freedom are (3, 16), go to column 3 and row 16 on the F-table. The critical value is 3.2389 (or 3.24). Your F-statistic for the seed spitting example is 8.43, which is well beyond this critical value (you can see how 8.43 compares to 3.24 by looking at Figure 9-4). Your conclusion is to reject Ho at level a. At least two of the age groups differ on mean seed spitting distances.

With the critical value approach, any F-statistic that lies beyond the critical value results in rejecting Ho, no matter how far or close to the line it is. If your F-statistic is beyond the value found in Table A-5, then you reject Ho and say at least two of the treatments (or populations) have different means.

What’s next?

After you’ve rejected Ho in the F-test and concluded that not all the populations means are the same, your next question may be: Which ones are different? You can answer that question by using a statistical technique called Multiple comparisons. Statisticians use many different multiple comparison procedures to further explore the means themselves after the F-test has been rejected. I discuss and apply some of the more common multiple comparison techniques in Chapter 10.

Checking the Fit of the ANOVA Model

As with any other model, you must determine how well the ANOVA model fits before you can use its results with confidence. In the case of ANOVA, the model basically boils down to a treatment variable (also known as the population you’re in) plus an error term. To assess how well that model fits the data, see the values of R2 And R2 Adjusted on the last line of the ANOVA output below the ANOVA table. For the seed spitting data, you see those values at the bottom of Figure 9-3.

The value of R2 Measures the percentage of the variability in the response variable (y) Explained by the explanatory variable (x). In the case of ANOVA, the X Variable is the factor due to treatment (where the treatment can represent a population being compared). A high value of R2 (say above 80 percent) means this model fits well. The value of R2 Adjusted, the preferred measure, takes R2 And adjusts it for the number of variables in the model. In the case of one-way ANOVA, you have only one variable, the factor due to treatment so R2 And R2 Adjusted won’t be very far apart. For more on R2 And R2 Adjusted, see Chapter 5.

For the watermelon seed spitting data, the value of R2 Adjusted (as found in the last row of Figure 9-3) is only 53.97 percent. That means age group (while shown to be statistically significant by the F-test; see the section "Making conclusions from ANOVA") explains just over half of the variability in the watermelon seed spitting distances. Because age group alone explains only a little over half of what’s going on in the seed spitting distances, you may find other variables you can examine in addition to age group, making an even better model.

The results of the T-test Done to compare the spitting distances of males and females in the section "Comparing Two Means with a t-Test" (see Figure 9-1) showed that males and females were significantly different on mean seed spitting distances. So I would venture a guess that if you include gender as well as age group thereby creating what statisticians call a Two-factor ANOVA (or two-way ANOVA), The resulting model would fit the data even better, resulting in higher values of R2 And R2 Adjusted. (See Chapter 11 for two-way ANOVA.)

Many medical and psychological studies use designed experiments to compare the responses of several different treatments, looking for differences. A Designed experiment Is a study in which subjects are randomly assigned to treatments (experimental conditions) and their responses are recorded. The results are used to compare treatments to see which one(s) work best, which ones work equally well, and so on.

One example of one such experiment that employs ANOVA is from The Ohio State University research press release Web site. The experiment tested three traditional principles of writing refusal letters:

Using a buffer — a neutral or positive sentence that delays the negative information

Placing the reason before the refusal

Ending the letter on a positive note as a way of reselling the business

Subjects were randomly assigned to treatments, and their responses to the rejection letters were compared (likely on some sort of scale such as 1 = very negative to 7 = very positive with 4 being a neutral response).

This scenario can be analyzed by using ANOVA. It compares three treatments (forms of the rejection letters) on some quantitative variable (response to the letter). You can argue that this isn’t a continuous variable, because it has

Enough possible values that ANOVA isn’t unreasonable. The data were also shown to have a bell shape.

The null hypothesis would be Ho: Mean responses to the three types of rejection letters are equal, versus Ha: At least two forms of the rejection letter resulted in different mean responses.

In the end, the researcher did find some significant results. In other words, the different ways the rejection letter was written affected the participants in different ways. Using multiple comparison procedures (see Chapter 10), you would be able to go in and determine which forms of the rejection letters gave different responses and how the responses differed.

So in case you have to write a rejection letter at some point, the researcher recommends the following guidelines for writing it:

Don’t use buffers to begin negative messages.

Give a reason for the refusal when it makes the sender’s boss look good.

Present the negative positively but clearly; offer an alternative or compromise if possible.

A positive ending isn’t necessary.