In This Chapter

^ Deciphering the difference between qualitative and quantitative variables ^ Choosing appropriate statistical techniques for the task at hand ^ Evaluating bias and precision levels ^ Interpreting the results properly

Ne of the most critical elements of statistics and data analysis is the ability to choose the right statistical technique for each job. Carpenters and mechanics know the importance of having the right tool when they need it and the problems that can occur if they use wrong tool. They also know that the right tool helps to increase their odds of getting the results they want the first time around, using the "work smarter not harder" approach.

In this chapter, you look at the some of the major statistical analysis techniques from the point of view of the mechanics and carpenters — knowing what each statistical tool is meant to do, how to use it, and when to use it. You also zoom in on mistakes some number crunchers make in applying the wrong analysis or doing too many analyses. Knowing how to spot these problems can help you avoid making the same mistakes, but it also helps you to steer your way through the ocean of statistics that may await you in your job and in everyday life.

If many of the ideas you find in this chapter seem like a foreign language to you and you feel like you need more background information, don’t fret. Before continuing on in this chapter, head to your nearest intro stats book or check out another one of my books, Statistics For Dummies (Wiley).

Qualitative Versus Quantitative Variables in Statistical Analysis

After you’ve collected all the data you need from your sample, you want to organize it, summarize it, and analyze it. Before plunging the data in to do all the number crunching though, you need to first identify the type of data you’re dealing with. The type of data you have points you to the proper types of graphs, statistics, and analyses you’re able to use.

Before I begin, here’s an important piece of jargon: Statisticians call any quantity or characteristic you measure on an individual a Variable; The data collected on a variable is expected to vary from person to person (hence the creative name).

The two major types of variables are the following:

Qualitative: A qualitative variable classifies the individual based on categories. For example, political affiliation may be classified into four categories: Democrat, Republican, Independent, and other; gender as a variable takes on two possible categories: male and female. A person may be categorized as a female Republican, which means that, regarding the gender variable, she falls into the female category, and regarding the political affiliation variable, she falls into the Republican category. Another name for a qualitative variable is a Categorical variable.

Quantitative: A quantitative variable measures or counts a quantifiable characteristic, such as height, weight, number of children you have, your GPA in college, or the number of hours of sleep you got last night. The quantitative variable value represents a quantity (count) or a measurement and has numerical meaning. That is, you can add, subtract, multiply, or divide the values of a quantitative variable, and the results make sense as numbers. This characteristic isn’t true of qualitative variables, which can take on numerical values only as placeholders.

Because the two types of variables represent such different types of data, it makes sense that each type has its own set of statistics. Qualitative variables, such as gender, are somewhat limited in terms of the statistics that can be performed on them. For example, suppose you have a sample of 500 classmates classified by gender — 180 of them are male, and 320 are female. How can you summarize this information? You already have the total number in each category (this statistic is called the Frequency). You’re off to a good start, but frequencies are hard to interpret because you find yourself trying to compare them to a total in your mind in order to get a proper comparison. In the previous example, you may be thinking "One hundred and eighty males out of what? Let’s see, it’s out of 500. Hmmm. . . what percentage is that? I can’t think."

The next step is to find a means to relate these numbers to each other in an easy way. You can do this by using what is called a relative frequency. The Relative frequency Is the percentage of data that falls into a specific category of a qualitative variable. You can find a category’s relative frequency by dividing the frequency by the sample total (500, using this example) and multiplying

By 100. In this case, you have i80 = 0.36 * 100 = 36 percent males and

500

320 = 0.64 *100 = 64 percent females. 500

You can also express the relative frequency as a proportion in each group by leaving the result in decimal form and not multiplying by 100. This statistic is called the Sample proportion. If you continue with the same example, the sample proportion of males is 0.36, and the sample proportion of females is 0.64.

You mainly summarize qualitative variables by using two statistics — the number in each category (frequency) and the percentage (relative frequency) in each category.

Statistics for Qualitative Variables

The types of statistics done on qualitative data may seem to be limited; however, the wide variety of analyses you can perform using frequencies and relative frequencies offers answers to an extensive range of possible questions you may want to explore.

In this section, you see that the proportion in each group is the number-one statistic for summarizing qualitative data. Beyond that, you see how you can use proportions to estimate, compare, and look for relationships between the groups that compose the qualitative data.

Comparing proportions

Researchers, the media, and even everyday folk like you and me love to compare groups (whether you like to admit it or not). For example, what proportion of Democrats support oil drilling in Alaska, compared to Republicans? What percentage of women watch college football versus men? What proportion of readers of Intermediate Statistics For Dummies Pass their stats exams with flying colors, compared to nonreaders? To answer these questions, you need to compare the sample proportions using a hypothesis test for two proportions (see Chapter 3 or your intro stat textbook).

Suppose you’ve collected data on a random sample of 1,000 United States voters. You may want to compare the proportion of female voters to the proportion of male voters and find out whether they’re equal. Suppose in your sample you find that the proportion of females is 0.53, and the proportion of males is 0.47. So for this sample of 1,000 people, you have a higher proportion of females than males. But here’s the big question: Are these sample proportions different enough to say that the entire population of U. S. voters has more females in it than males? After all, sample results vary from sample to sample. The answer to this question requires comparing the sample proportions by using a hypothesis test for two proportions. I demonstrate and expand on this technique in Chapter 3.

Estimating a proportion

You can also use relative frequencies (check out the section "Qualitative versus Quantitative Variables in Statistical Analysis") to make estimates about a single population proportion.

Say, for example, you want to know what proportion of females in the United States are Democrats. According to a sample of 29,839 female voters from the U. S. conducted by the Pew Research Foundation in 2003, the percentage of female Democrats was 36. Now because the Pew researchers based these results on only a sample of the population and not on the entire population, these results may vary from sample to sample. The amount of variability is measured by the Margin of error (the amount that you add and subtract from your sample statistic), which for this sample is only about 0.5 percent. (To find out how to calculate margin of error, explore Chapter 3.) That means that the estimated percentage of female Democrats in the U. S. voting population is estimated to be somewhere between 35.5 percent and 36.5 percent.

The margin of error, combined with the sample proportion, forms what statisticians call a confidence interval for the population proportion. Recall from intro stats that a Confidence interval Is a range of likely values for a population parameter, formed by taking the sample statistic plus or minus the margin of error. (For more on confidence intervals, see Chapter 3.)

Looking for relationships between qualitative Variables

Suppose you want to know whether two qualitative variables are related (for example, is gender related to political affiliation?). Answering this question requires putting the sample data into a two-way table (using rows and

Columns to represent the two variables), and analyzing the data by using a Chi-square test (see Chapter 14). By following this process, you can determine whether two categorical variables are independent (unrelated) or whether a relationship exists between them. If you find a relationship, you can use percentages to describe it.

Table 2-1 shows an example of data organized in a two-way table. The data was collected by the Pew Research Foundation.

Notice that the percentage of male Republicans in the sample is 32 and the percentage of female Republicans in the sample is 29. These percentages are quite close in relative terms. However, the percentage of female Democrats seems much higher than the percentage of male Democrats (36 percent versus 27 percent); also, the percentage of males in the "Other" category is quite a bit higher than the percentage of females in the "Other" category (41 percent versus 35 percent). These large differences in the percentages indicates that gender and political affiliation are related in the sample. But do these trends carry over to the population of all U. S. voters? This question requires a hypothesis test to answer. The particular hypothesis test you need in this situation is a Chi-square test, which I discuss in detail in Chapter 14.

To make a two-way table from a data set by using Minitab, first enter the data in two columns, where column one is the row variable (continuing with the previous example, this variable would be gender) and column two is the column variable (in this case, political affiliation). For example, suppose the first person is a male Democrat. In row one of Minitab, enter M (for male) in column one and D (Democrat) in column two. Then go to Stat>Tables>Cross Tabulation and Chi-square. Highlight column one and click Select to enter this variable in the For Rows line. Highlight column two and click Select to enter this variable in the For Columns line. Click on OK.

People often use the word Correlation To discuss relationships between variables, but in the statistical world, you can use correlation only to discuss the relationship between two quantitative (numerical) variables, not two qualitative (categorical) variables. Correlation measures how closely the relationship between two quantitative variables, such as height and weight, follows a

Straight line and tells you the direction of that line as well. In total, for any two quantitative variables, X And Y, The correlation measures the strength and direction of their linear relationship. As one increases, what does the other one do?

Because qualitative variables don’t have a numerical order to them, they don’t increase or decrease in value. For example, just because male = 1 and female = 2 doesn’t mean that a female is worth twice a male. (Although some women may want to disagree.) These numbers represent categories, not values. Therefore, you can’t use the word Correlation To describe the relationship between, say, gender and political affiliation. The appropriate term to describe the relationships of qualitative variables is Association. You can say that political affiliation is associated with gender, and explain how. (For full details on association, see Chapter 13. For more information on correlation, see Chapter 4.)

Building models to make predictions

You can also build models to predict the value of a qualitative variable based on other related information. In this case, building models is more than a lot of little plastic pieces and some irritatingly sticky glue. When you build a model, you look for variables that help explain, estimate, or predict some response you’re interested in (the variables that do this are called Explanatory variables). You sort through the explanatory variables and figure out which ones do the best job of predicting the response, and you put them together into a type of equation like Y = 2x + 4 where X = shoe size and Y = length of your calf. That equation is a Model.

For example, what if you want to know which factors or variables can help you predict someone’s political affiliation? Is a woman without children more likely to be a Republican or a Democrat? What about a middle-aged man who proclaims Hinduism as his religion? In order for you to compare these complex relationships, you must build a model to evaluate each group’s impact on political affiliation (or some other qualitative variable). This kind of model building is explored more in-depth in Chapter 8, where I discuss the topic of logistic regression.

Logistic regression builds models to predict the outcome of a qualitative variable, such as political affiliation. If you want to make predictions about a quantitative variable, such as income, you need to use the standard type of regression (check out Chapters 4 and 5).

In 2003, the Pew Research Foundation studied the following variables in terms of their relationship with political affiliation: gender, race, state of residence, income level, age, education, religion, marital status, and whether or not you have children. While you can do individual Chi-square analyses to examine possible connections between each of these variables and political affiliation separately, you can’t find out which combinations of these variables increase the likelihood of someone being a Democrat, Republican, or other.

For example, the Foundation found that women are more likely to be Democrats than men, but age is also a factor. Younger people tend to be more inclined to be Republican, and older people lean toward being Democrat. However, if you look at the combination of gender and age, you can see mixed results; males who are older are more likely than young females to be Democrat rather than Republican, for example. This kind of result is called an Interaction effect Between gender and age group. An interaction effect occurs when certain combinations of variables produce different results than other combinations. The only way to look for these kinds of more-complex relationships is to do model building, which allows you to examine the combinations of variables and their impact on political affiliation. The Pew Foundation was able to make conclusions about the United States population based on its model linking political affiliation, age and gender, as well as their interactions.

Statistics for Quantitative Variables

Quantitative variables, unlike qualitative variables, have a wider range of statistics that you can do, depending on what questions you want to ask. The main reason for this wider range is that Quantitative data Are numbers that represent measurements or counts, so it makes sense that you can order, add or subtract, and multiply or divide them — and the results all have numerical meaning. Examining quantitative date opens up a whole world of possibilities for analysis. In this section, I present the major data-analysis techniques for quantitative data. I further expand each technique in later chapters of this book.

Making comparisons

Suppose you want to look at income (a quantitative variable) and how it relates to a qualitative variable, such as gender or region of the country. Your first question may be: Do males still make more money than females? In this case, you can compare the mean incomes of two populations — males and

Females. This assessment requires a hypothesis test of two means (oftentimes called a /-test for independent samples). I present more information on this technique in Chapter 3.

When comparing the means of More Than two groups, don’t simply look at all the possible /-tests that you can do on the pairs of means, because you have to control for an overall error rate in your analysis. Too many analyses can result in errors — adding up to disaster. For example, if you conduct 100 hypothesis tests, each one with a 5 percent error rate, then 5 of those 100 tests give wrong results on average, just by chance.

If you want to compare the average wage in different regions of the country (the East, the Midwest, the South, and the West, for example), this comparison requires a more sophisticated analysis, because you’re looking at four groups rather than just two. The procedure you can use to compare more than two means is called Analysis of variance (ANOVA), and I discuss this method in detail in Chapters 9 and 10.

Finding connections

Suppose you’re an avid golfer and you want to figure out how much time you should spend on your putting game. The question is this: Is the number of putts related to your total score? If the answer is yes, then spending time on your putting game makes sense. If not, then you can slack off on it a bit. Both of these variables are quantitative variables, and you’re looking for a connection between them. You collect data on 100 rounds of golf played by golfers at your favorite course over a weekend. Table 2-2 shows the first few lines of your data set.

29

82

The first step in looking for a connection between putts and total scores (or any other quantitative variables) is to make what is called a Scatterplot Of the data, which graphs your data set in two-dimensional space by using an X And Y Plane. You can take a look at the scatterplot of the golf data in Figure 2-1. Here, X Represents the number of putts, and Y Represents the total score. For example, the point in the lower-left corner of the graph represents someone who had only 23 putts and a total score of 75. (For instructions on making a scatterplot by using Minitab, see Chapter 4.)

Figure 2-1:

A scatterplot is a two-dimensional graph you can use to look for relationships in data.

According to Figure 2-1, it appears that as the number of putts increases, so does total score. The relationship seems pretty strong — the number of putts plays a big part in determining the total score.

Now you need a measure of how strong the relationship is between X And Y And whether it goes uphill or downhill. Correlation is the number that measures how close the points follow a straight line. Correlation is always between -1.0 and +1.0, and the more closely the points follow a straight line,

The closer the correlation is to -1.0 or +1.0. A positive correlation means that as X Increases on the x-axis, Y Also increases on the y-axis. Statisticians call this type of relationship an Uphill relationship. A negative correlation means that as X Increases on the X-axis, Y Goes down. Statisticians call this type of relationship — you guessed it — a Downhill relationship.

For the golf data set, the correlation is 0.896 = 0.90, which is extremely high as correlations go. This strong correlation (close to +1.0) is a good thing because it means number of putts can do a great job of predicting total score. Because the sign of the correlation is positive, it means as you increase number of putts, your total score increases (an uphill relationship). For instructions on calculating a correlation in Minitab, see Chapter 4.

Making predictions

If you want to predict some response variable (y) Using one explanatory variable (x), And you want to use a straight line to do it, you can use Simple linear regression (see Chapter 4 for all the fine points on this topic). Linear regression finds the best-fitting line that cuts through the data set, called the Regression line. After you get the regression line, you can plug in a value of X And get your prediction for Y. (For instructions on using Minitab to find the best-fitting line for your data, see Chapter 4.)

To use the golf example from the previous section, suppose you want to predict the total score you can get for a certain number of putts. In this case, you want to calculate the linear regression line. By using the data set shown in Table 2-2, and running a regression analysis, the computer tells you that the best line to use to predict total score using number of putts is the following:

Total score = 39.6 + 1.52 * Number of putts

So if you have 35 putts in an 18-hole golf course, your total score is predicted to be about 39.6 + 1.52 * 35 = 92.8, or 93. (Not bad for 18 holes!)

Notice that the slope of the regression line tells you what you really want to know — how much does your total score increase with every additional putt? In other words, how much damage is done when you miss the hole on your first, or second, or third putt? The slope of the regression line for the golf data set is 1.52. Because the slope of a line is the ratio of the change in Y (total score) to the change in X (number of putts) this means that every additional putt you need results in an overall increase in total score by 1.52. Maybe that’s why Tiger Woods spends so much time on his short game.

Don’t try to predict Y For x-values that fall outside the range of where the data was collected; you have no guarantee that the line still works outside of that range, or that it will even make sense. For the golf example, you can’t say that if X (the number of putts) = 0 the total score would be 39.6 + 1.52 * 0 = 39.6 (unless you just call it good after your ball hits the green). This mistake is called Extrapolation.

You can discover more about simple linear regression, and expansions on it, in Chapters 4 and 5.

AvoIDINg BIAs

Bias is the bane of a statistician’s existence; it’s easy to create and very hard to deal with, if not impossible in most situations. The statistical definition of Bias Is the systematic overestimation or underestimation of the actual value. In language the rest of us can understand, it means that the results are always off by a certain amount in a certain direction. For example, a bathroom scale may always report a weight that’s five pounds more than it should be (I’m convinced this is true of my doctor’s office scale); this consistent adding of five points to every outcome represents a systematic overestimation of the actual weight.

The most important idea when dealing with bias is prevention, or at least minimizing it. Bias is like weeds in a garden: After they’re present, they’re very hard to deal with, and it’s always better to eliminate them from the start. In this section, you see ways bias can creep into a data set, or even into a statistic, and what you can do about it.

LookINg at bIAs through statIStICal glasses

Bias can show up in a data set a variety of different ways. Here are some of the most common ways bias can creep into your data:

Selecting the sample from the population: Bias occurs when you leave some intended groups out of the process, and/or give certain groups too much weight.

For example, TV surveys (the ones where they ask you to phone in your opinion) are biased because no one has selected a prior sample of people to represent the population — people call in on their own. When people participate in a survey on their own, they’re more likely to have stronger opinions than those who don’t choose to participate. Such samples are called Self-selected samples And are typically very biased.

Designing the data-collection instrument: Poorly designed instruments (including surveys) can result in inconsistent or even incorrect data.

For example, a survey question’s wording plays a large role in whether or not results are biased. A leading question can make people feel like they should answer a certain way. For example: "Don’t you think that the president should be allowed to have a line-item veto to prevent government spending waste?" Who would feel they should say No To that?

Collecting the data: In this case, bias can infiltrate the results if someone makes errors in the recording of the data or if interviewers deviate from the script.

Ur Deciding how and when the data is collected: The time and place you collect data can affect whether your results are biased. For example, if you conduct a telephone survey during the middle of the day, people who work from nine to five aren’t able to participate. Depending on the issue, the timing of this survey could lead to biased results.

Bias can creep into a data set very easily. The best way to deal with bias is to avoid it in the first place. You can do this in two major ways:

Use a random process to select the sample from the population. The

Only way a sample is truly random is if every single member of the population has an equal chance of being selected. Self-selected samples aren’t random.

Make sure that the data is collected in a fair and consistent way. Be

Sure to use neutral question wording and time the survey properly.

SeTtliNg ThE VARiAnce conTrOveRSy: The battle of n-1 Versus n

Not all statistical formulas are free of bias. In other words, some statistics have good characteristics (like offering great precision) and some not-so-good characteristics (like not giving the best possible result in all situations). Statisticians definitely prefer statistics that are both precise and unbiased, and the techniques you find in this book have both qualities. However, precise and unbiased statistics doesn’t always happen naturally; sometimes the basic idea requires a little tweaking to get a statistic that actually meets the standards of the statistical powers that be (of which I am not one). The classic example of this need to fine-tune is the formula for the variance of a data set, which I describe in the following section.

The problem

Statistics textbooks sometimes show two formulas for the variance of a data

! _ Xi – X)

Set. One formula shown for the variance is S2 = ——n-, where N Is the

Sample size, the values of X Are the data values, and the sample mean (or the

N

Average of all the values of the data set) is X = ■L=n—. This formula for variance, you may note, contains an N All by itself in the denominator. The fact that the denominator is N And not N - 1 makes a teacher’s job of explaining variance a whole lot easier, because it represents the average squared distance from the mean. In this case, the values being squared are the differences between the data values and their mean. You get the average of these squared values by summing them up and dividing by N, The sample size.

However, this version of a formula for variance, as it’s written, is biased. That means in a statistical sense, you know that in the long term, the results are always off by a very small amount from their target value. If you take repeated samples, find the variance, and do this over and over, the results on average are a little smaller than they should be. (Statisticians can prove this, but you don’t have to worry about that. I’m sure you have better things to do.)

The Solution

Because statisticians prefer results being correct to results that can be more easily explained, they decided to do something about this bias problem in the formula for the sample variance. A group of stat big wigs figured out that dividing by N Was the problem, and if you divide by N - 1 rather than N, You can get answers that are right on target. That’s how the following commonly used formula for sample variance came into being:

N

! _Xi – X)

S 2 = —-

N-1

Notice that an N - 1 rather than an N Is now in the denominator. However, trying to explain why the formula isn’t dividing by N Does tend to open up a can of worms for statistics professors (and explains why biased statistics are a topic left for the intermediate-level students, like you!).

Because statistics can be biased too, in terms of the results they create through their formulas alone, it’s always a good idea to check with a statistician or someone else in the know whether a particular statistic is unbiased before you use it.

An animal science researcher came to me one time with a data set he was so proud of. He was studying cows and the variables involved in helping determine their longevity. He came in with a super-mega data set that contained over 100,000 observations. He was thinking "Wow, this is gonna be great! I’ve been collecting this data for years and years, and I can finally have it analyzed. There’s got to be loads of information I can get out of this. The papers I’ll write, the talks I’ll be invited to give. . . the raise I’ll get!" He turned his precious data over to me with an expectant smile and sparkling eyes.

But after looking at his data for a few minutes I made a terrible realization — all of his data came from exactly one cow. With no other cows to compare with and a sample size of just one, he had no way to even measure how much those results would vary if he wanted to apply them to another cow. His results were so biased toward that one animal that I couldn’t do anything with the data. After I summed up the courage to tell him, it took a while to peel him off the floor. The moral of the story, I suppose, is to find a statistician and check out your big plans with her before you go down a cow path like this guy did.

GettINg Good PREcISIOn

Precision Is the amount of movement you expect to have in your sample results if you repeat your entire study again with a new sample. Low precision Means that you expect your sample results to move a lot (not a good thing). High precision Means you expect your sample results to remain fairly close in the repeated samples (a good thing). In this section, you find out what precision does and doesn’t measure, and you see how to measure the precision of a statistic in general terms.

Understanding precision from a statistical point of view

You may think that precision means the level of correctness you have in your statistical results. But precision only measures the Level of consistency In the results from sample to sample. Your results can be consistently correct or consistently incorrect.

For example, a field-goal kicker on a football team may consistently kick the ball two feet to the right of the goalposts every single time. Even though he’s consistent, he never gets to score, because his results are systematically off by the same amount each time. In other words, his results are biased, even though they’re precise.

IBE# A statistic can be precise with or without bias, and vice versa. The best situation is when your results are both precise (consistent) as well as unbiased (on target). That goal is what statisticians always strive for. How often does it happen? You can have a lot of control of the precision part by simply taking a larger sample. However, the goal of completely unbiased results is rarely achieved, but that doesn’t stop statisticians from trying. And you do have ways to minimize it (keep reading).

Measuring precision with margin of error

You can measure precision by the margin of error. The Margin of error Is the amount that you expect your statistical results to change from one sample to the next. While you always hope, and may even assume, that statistical results shouldn’t change much with another sample, that’s not always the case. It’s like a commercial that tries to sell a weight-loss product by showing a person who lost 50 pounds in a single weekend; then in small letters at the bottom of the screen, you see the words "results will vary." Before you report or try to interpret any statistical results, you need to have some measurement of how much those results are expected to vary from sample to sample.

The following sections show how to calculate the precision of your statistic and how to come up with a margin of error.

Calculating precision

The exact formulas for margin of error differ depending on the type of data that you’re analyzing; however, they all contain two major components:

Confidence coefficient Standard error of the statistic

The general structure of a formula for margin of error is the following, where standard error is the standard deviation of the population divided by the square root of the sample size (you can see all the details on margin of error in Chapter 3):

Margin of error = ± Confidence coefficient * Standard error

The big idea is that the confidence coefficient tells you the number of standard errors you’re willing to add and subtract in order to have a certain level of confidence in your results. If you want to be more confident in your results, you add or subtract more standard errors. If you don’t have to be as confident, you don’t have to add or subtract as many standard errors. Typically, you add and subtract about two standard errors if you want to be 95 percent confident and three standard errors if you want to be more than 99 percent confident. This rule of thumb follows a statistical result called the Empirical Rule, Also known as the 68-95-99.7 Rule.

The Standard error Is the average amount of movement in the statistic you’re using. It’s a function of two quantities:

Sample size: Sample size is perhaps the most important factor in controlling margin of error. The sample size is in the denominator of the standard error, meaning that as your sample size increases, the standard error goes down, and that’s why the margin of error goes down.

This result makes sense, because having a larger sample means having more information in your analysis, which should lead to greater precision. As the sample size decreases, the margin of error goes up, because you have less information to work with and that makes for less-precise results.

Standard deviation in the population: Standard deviation is close to the average distance from the mean. If the population you took your sample from has a large amount of variability, the standard deviation is large, and the margin of error for your statistic goes up (because standard deviation is in the numerator of the margin of error). If the population is more homogeneous, your sample results are more homogeneous as well, and the margin of error goes down (because the standard error gets smaller).

The Gallup Organization states its survey results in a universal, statistically correct format. Using a specific example from a recent survey it conducted, you can see the language it uses to report its results:

"These results are based on telephone interviews with a randomly selected national sample of 1,002 adults, aged 18 years and older, conducted June 9-11, 2006. For results based on this sample, one can say with 95% confidence that the maximum error attributable to sampling and other random effects is ±3 percentage points. In addition to sampling error, question wording and practical difficulties in conducting surveys can introduce error or bias into the findings of public opinion polls."

The first sentence of the quote refers to how the Gallup Organization collected the data, as well

As the size of the sample. As you can guess, precision is related to the sample size, as seen in the section "Calculating precision."

The second sentence of the quote refers to the precision measurement: How much did Gallup expect these sample results to vary? The fact that Gallup is 95 percent confident means that if this process were repeated a large number of times, in 5 percent of the cases the results would be wrong, just by chance. This inconsistency occurs if the sample selected for the analysis doesn’t represent the population — not due to biased reasons, but due to chance alone (more on this in Chapter 3).

(Check out the section "Bias not included" to get the info on why the third sentence is included in this quote.)

For more details on how to calculate margin of error in various statistical techniques, see Chapter 3.

Interpreting margin of error

Finding the margin of error is one thing — figuring out what it means is a whole other ball o’ wax. But don’t fear; it’s actually not so bad. To interpret the margin of error, just think of it as the amount of play you allow in your results to cover most of the other samples you could have taken.

Suppose you’re trying to estimate the proportion of people in the population who support a certain issue, and you want to be 95 percent confident in your results. You sample 1,002 individuals and find that 65 percent support the issue. The margin of error for this survey turns out to be plus or minus 3 percentage points (you can find the details of this calculation in Chapter 3). That result means that you can expect the sample proportion of 65 percent to change by as much as 3 percentage points either way if you took a different sample of 1,002 individuals. In other words, you believe the actual population proportion is somewhere between 65 – 3 = 62 percent and 65 + 3 = 68 percent. That’s the best you can say.

Bias not included!

Realizing that the margin of error measures the consistency (precision) of a statistic only, not its level of bias is extremely important. In other words, a margin of error can appear on paper to be very small yet actually be way off target because of bias in the data that was collected. (In the nearby sidebar, you can see that Gallup discusses margin of error and bias separately.)

Any reported margin of error was calculated on the basis of having zero bias in the data. However, this assumption is rarely true. Before interpreting any margin of error, check first to be sure that the sampling process and the data-collection process don’t contain any obvious sources of bias. If a great deal of bias exists, you should ignore the results, or take them with a great deal of skepticism.

Making Conclusions and Knowing Your Limitations

The most important goal of any data analyst is to remain focused on the big picture — the question that you or someone else is asking — and make sure that the data analysis used is appropriate and comprehensive enough to answer that question correctly and fairly.

Here are some tips for analyzing data and interpreting the results, in terms of the statistical procedures and techniques that you may use — at school, in your job, and in everyday life. These tips are implemented and reinforced throughout this book:

Be sure that the research question being asked is clear and definitive.

Some researchers don’t want to be pinned down on any particular set of questions because they have the intent of mining the data (looking for any relationship they can find, and then stating their results after the fact). This can lead to overanalyzing the data, making the results subject to skepticism by statisticians.

Double-check that you clearly understand the type of data being collected. Is the data qualitative or quantitative? The type of data used drives the approach that you take in the analysis.

Make sure that the statistical technique you use is designed to answer the research question. If you want to make comparisons between two groups and your data is quantitative, use a hypothesis test for two means. If you want to compare five groups, use analysis of variance (ANOVA). You can use this book as a resource to help you determine the technique you need.

Look for the limitations of the data analysis. For example, if the researcher wants to know whether negative political ads affect the population of voters, and she bases her study on a group of college students, you can find severe limitations here. For starters, student reactions to negative ads don’t necessarily carry over to all voters in the population. And even if the population were limited to all student voters, the students from this particular class don’t represent all students. In this case, it’s best to limit the conclusions to college students in that class (which no researcher would ever want to do). Ultimately what needs to be done is design the study so the sample contains a representation of the intended population of all voters in the first place (a much more difficult task, but well worth it).

One of the hardest parts of my job as a statistical consultant is dealing with analyses after the design was already done — and done incorrectly. It’s much better to put in a little work to get a good design together first, and then the analysis will take care of itself.

In This Chapter

^ Realizing your role as a data analyst

^ Avoiding statistical faux pas

^ Delving into the jargon of intermediate statistics

Ecause you’re reading this book, you’re likely familiar with the basics of statistics. You’re now ready to take it up a notch. That next level

Involves using what you know, picking up a few more tools and techniques at the intermediate level, and finally putting it all to use to help you answer more realistic questions by using real data.

In statistical terms, you’re ready to enter the world of the Data analyst. This world’s an exciting one, with many options to explore and many tools available. But, as you may have guessed, you have to navigate this world very carefully, choosing the right methods for each situation. In this book, you can see that I’m including the underlying theories and ideas behind the methods where necessary to help you make good decisions — and not just get into the point-and-click mode that today’s software packages offer.

In this chapter, you review the terms involved in statistics as they pertain to data analysis at the intermediate level. You get a glimpse of the impact that your results can have by seeing what these analysis techniques can do. You also gain insight into some of the common misuses of data analysis and their effects.

Data Analysis: It’s Not Just for Statisticians Anymore

It used to be that statisticians were the only ones who really analyzed data. The reason for this is because the only computer programs that were available

Then were very complicated to use, requiring a great deal of knowledge about statistics to set up and carry out. The calculations were tedious and at times unpredictable and required a thorough understanding of the theories and methods behind the calculations to get correct and reliable answers.

Today, anyone who wants to analyze data can do it easily. Many user-friendly statistical software packages are made expressly for that purpose — Microsoft Excel, Minitab, SAS, and SPSS, just to name a few. Free online programs are even available, such as Stat Crunch, to help you do just what it says — crunch your numbers and get an answer. As you see in this section, the modern easy-to-use statistical packages are good in some ways, and not-so-good in other ways.

The most important idea when applying statistical techniques to analyze data is to know what’s going on behind the number crunching, so you (not the computer) are in control of the analysis. That’s why knowledge of intermediate statistics is so critical.

Remembering the old days

In the old days, in order to determine whether methods gave different results, you had to write a computer program to do it, using code that you had to take a class to learn. You had to type in your data in a specific way that the computer program demanded, and you had to submit your program to a mainframe computer and wait for the printer to print out your results. This method was time consuming and a general all-around pain.

I remember the day in college when I reached bottom. I was just learning to write those sophisticated programs you needed to do the simplest analysis. No matter how hard I tried to write the perfect program, the computer kept spitting my work back at me without doing my analysis, noting error after error in the way I typed the commands. The last straw came when I gave my program to the computer for the umpteenth time: At the end of the printout, the computer told me on the very last line: "Error #34410: Too many errors."

Now, don’t get the idea that your author doesn’t know what she’s doing. I had all the statistical methods right; I just wasn’t very good at writing computer programs. So for anyone out there who’s ever been frustrated by a computer, I feel your pain, and I try to minimize your troubles throughout this book.

Enough lamenting about having to walk to school uphill both ways in the snow with plastic bags on my feet instead of boots. The point is, statistical software packages have undergone an incredible evolution in the last 10 to 15 years, to the point where you can now enter your data quickly and easily in almost any format. Moreover, the choices for data analysis are well organized

And listed in pull-down menus. Now almost anyone (even me) can quickly see how to find the necessary procedure and tell the computer what to do. The results come instantly and successfully, and you can cut and paste them into a word-processing document without blinking an eye. For example, comparing the weight loss for people on different weight-loss programs now takes less than three clicks of the mouse to perform, which is great news for folks like me.

Many very useful and efficient statistical software packages exist, including SAS, SPSS, Data Desk, Stat Crunch, MS Excel, and Minitab, and each one has its own pros and cons (and its own users and protesters). My software of choice, and the one I reference throughout this book, is Minitab, because it’s very easy to use, the results are correct, the output is very clear and professional looking, and the software’s loaded with all the data-analysis techniques that are used in intermediate statistics as well as in this book. While a site license for Minitab can be expensive, the downloadable student version is available for rent for only a few bucks a semester.

The downside of today’s statistical software

You may be wondering where the downside is in all of this. Is it too good to be true that what was once a tedious, complicated process for analyzing data has now become as easy as checking your e-mail on your cell phone? Yes and no. Yes, it’s too good to be true that the software practically does everything for you — if you don’t pay attention to what the programs are really doing. Yes, it’s too good to be true if you don’t understand that conditions need to be checked in every situation before an analysis should be applied. Yes, it’s too good to be true if you take all the results as complete and utter gospel (as too many statistician wannabees do).

Bottom line: Today’s software packages are too good to be true if you don’t have a clear and thorough understanding of the intermediate level of statistics that lie underneath them.

Here’s the good news, though. By reading this book, you gain the understanding you need to set you up for success. You get enough of the underlying intermediate statistical concepts to be empowered, but not be dangerous. You find out what conditions need to be checked on the data before applying an analysis and how to check them. You get a good feel for which analyses to use to answer your question (and which ones can cause you trouble), and you become aware of the kinds of results you can expect. Most importantly, you discover what’s possible and appropriate to conclude from your analysis and what limitations and caveats you need to make.

Rule #1: Look Before You CRUnch

Many people don’t realize that statistical software can’t tell you when to use and not to use a certain statistical technique. You have to determine that on your own. As a result, people think they’re doing their analyses correctly, but they can end up making all kinds of mistakes. Statistical software packages are centered on mathematical formulas, and mathematical formulas aren’t smart enough to know how you’re applying them or to warn you when you’re doing something wrong (that’s where this book comes in).

In this section, I give some examples of some of the major situations where innocent data analyses can go wrong and why it’s important to know what’s happening behind the scenes from a statistical standpoint before you start crunching numbers.

Nothing (even a straight line) lasts forever

After you get a statistical equation, or Model, That tries to explain or predict some random phenomena, you need to specify for what values the equation applies and for what values the equation doesn’t apply. Equations don’t know when they work and when they don’t; it’s up to the data analyst to determine that. This idea is the same for applying the results of any data analysis that you do.

Bill Prediction is a statistics student, studying the affect of study time on exam score. Based on his experience, and that of a few friends, Bill comes up with the equation Y = 10x + 30, where Y Represents the test score you get if you study a certain number of hours (x). This equation is Bill’s model for predicting exam score using study time. Notice that this model is the equation of a straight line with a y-intercept of 30 and a slope of 10.

So Bill predicts, using this model, that if you don’t study at all, you’ll get a 30 on the exam (plugging X = 0 into the equation and solving for Y; This point represents the Y-intercept of the line). And he predicts, using this model, that if you study for five hours, you’ll get an exam score of Y = 10 *5 + 30 = 80. So, the point (5, 80) is also on this line. (I won’t talk in detail at this point about how well Bill’s model does at predicting exam score, but you can just say he’s got some work to do on this and leave it at that for now.)

I’m sure you would agree that because X Is the amount of study time, that X Can never be a number less than zero. If you plug a negative number in for X, Say X = -10, you get Y = 10 * -10 + 30 = -70, which makes no sense. The worst possible score, according to Bill’s model, is 30, which occurs when X Equals 0.

And, you can’t study a negative number of hours, so a negative number for X Itself isn’t even possible.

On the other side of the coin, X Probably isn’t a number in the two-digit range (10 or more). Why is this? Say someone did study ten hours for this exam. Plugging in 10 for X In Bill’s equation, you get Y = 10 * 10 + 30, which equals 130. Remember, Y Is the predicted exam score. Because most exams are out of 100 possible points, a score of 130 isn’t possible. (I’m all for extra credit on exams, but 30 points of extra credit is too much, even for me.)

The point is that there are limits on the values of X That make sense in this equation. However, the equation itself, Y = 10x + 30, doesn’t know that, and if you graph this line, it’ll go on forever in both the positive and negative directions (see Figure 1-1).

Y

200 150^ 100 50

-20 -15 -10 -5 5 10 15 20

-50–

Figure 1-1:

The line y= 10x + 30, for all possible values of X.

(-10,-70) ,

-100 -150 -200

(10,130)

Y=10x+30

V – (0,30)

111111111111111 \"__\ 1111111111111111111 > X

Data snooping isn’t cool

Statisticians have come up with a saying that you may have heard of: "Figures don’t lie. Liars figure." Make sure that you find out about all the analyses that were performed on a data set, not just the ones reported as being statistically significant.

Suppose Bill Prediction tries to apply his simple model (from the preceding section) to predict exam scores for his whole class, based on their reported amounts of study time, and he finds out that his results fall flat. He figures out

That he needs more information, so he tries to uncover what other factors help determine exam score on a statistics test besides study time. Bill measures everything from soup to nuts. His set of possible variables includes study time, GPA, previous experience in statistics, math grades in high school, attitudes toward statistics, whether you listen to classical music while studying, shoe size, whether you chew gum during the exam, and even what your favorite color is (after all, you never know, he figures). For good measure, he includes 11 other variables, for a total of 20 possible factors that he thinks may relate to exam score.

Bill starts out by looking for relationships between each of these variables and exam score, so he does 20 correlations. (Correlation Is a measure of the linear relationship between two variables; see the section on correlation later in this chapter). He finds out that four variables have a statistically significant relationship with exam score (that means the results are supposed to be correct with a 95 percent chance — but only if he collected the data properly and did the analysis correctly).

The variables that Bill found to be related to exam score were study time, math grades in high school, GPA, and whether the person chews gum during the exam. It turns out that his new model fits pretty well (by criteria I discuss in Chapter 5 on multiple linear regression models). Bill now thinks he’s scored a home run and has answered that all-elusive question: How can I do better on my statistics test?

But as they said in Apollo 13, "Houston, we have a problem." By looking at all possible correlations between his 20 variables and exam score, Bill is actually doing 20 separate statistical analyses. Under typical conditions (I describe these conditions in Chapter 3), each statistical analysis has a 5 percent chance of being wrong just by chance (this value of 5 percent is called the Significance level Of the test).

Because 5 percent of 20 analyses is equal to one, you can expect that when you do 20 statistical analyses, one of them will give the wrong result, just by chance, over the long term. I bet you can guess which one of Bill’s correlations likely came out wrong in this case. Of course, study time has Nothing To do with exam score, and gum-chewing is the answer to all of our problems, right? (If that were the case, all statisticians would be out of business and working for chewing-gum companies instead.)

What Bill is doing is called Data snooping In the data-analysis business. Bill looks around until he finds something, and then he believes the result. This strategy is dangerous, but one that’s done all too often in the real world. One of the reasons data snooping is running rampant today is because everyone and his brother is out there collecting data and analyzing it — and everyone wants to find something. They’re using statistical software that allows them

To just point and click to do as many analyses as they want, without any warning about what statisticians call the Overall error rate (that is, the probability of making an error due to chance during any step of the entire analysis, not just the probability of making an error due to chance on any single analysis).

No (data) fIShINg allowed

Redoing analyses in different ways to try to get the results you want is called Data fishing In the statistics business, and folks in the stat biz consider it to be a major no-no (however, people unfortunately do it all too often in the name of research).

For example, Ellen Go-getter is convinced that dissolving sugar in the water helps cut flowers last longer. She performs an experiment to prove her hypothesis. She cuts two dozen roses and puts one rose in each vase. She fills each vase with 3 cups of water, but in 12 of the vases she adds 1 tablespoon of sugar (the other 12 vases constitute the control group, meaning that Ellen doesn’t apply any new treatment to them to show what happens if she adds nothing). In the next sections, you follow Ellen through her experiment, keeping an eye on the statistical analyses that pop up along the way.

Examining Ellen’s data

Ellen counts how many days the flowers still look nice and uses the same criteria for each flower. After ten days, all the flowers have withered to the point where they need to be thrown away, so the experiment is over. You can see Ellen’s data in Table 1-1.

(continued)

Setting the hypothesis

Ellen wants to compare the two methods, water and sugar, to see whether the roses that had sugar added lasted longer than the regular water group. She needs to conduct a hypothesis test whose null hypothesis is Ho: There is no difference in days lasted for sugar group versus control group. Her alternative hypothesis, which she hopes to show, is Ha: The roses in the sugar group lasted longer than the control group. She figures a two-sample /-test is in order here. (I discuss hypothesis tests in Chapter 3.)

Checking the conditions

Ellen has taken a few statistics classes before and knows that before she plunges into an analysis, she needs to check the proper conditions. For a comparison of two groups, she has to plot the data from each group on a Histogram (a bar graph showing the number of days the flowers lasted, organized into groupings in numerical order versus the number of flowers that lasted each number of days). According to what she knows about a two-sample T-test, the data in each group has to have a normal distribution before she starts. That is, the data has to have a bell-shaped curve when you look at the histogram. Ellen plots the data in histograms for the two groups and gets the following results (see Figures 1-2a and 1-2b).

Figure 1-2:

Histograms showing number of days roses lasted, using water only versus sugar added.

Histogram of Days Lasted: Water Only

2 3 4 5

Days Lasted: Water Only

Histogram of Days Lasted: Sugar Group

2345 Days Lasted: Sugar Group

B

Getting the bad news

As you can see in Figures 1-2a and 1-2b, Ellen’s data doesn’t follow the typical bell-shaped curve. One of the problems is her data only takes on values that are positive whole numbers, so numbers like 1.2, 2.3, and the like aren’t possible. (Normal distributions are supposed to have many possible values.) The other problem is that the data has no values outside the typical two-, three-, four-, or five-day range, so the histogram doesn’t have a chance to take on a bell shape. Perhaps more data would have curbed this problem. At any rate, Ellen knows that the conditions for a two-sample /-test aren’t met here; namely that the data doesn’t have a normal distribution and is, in fact, Skewed (meaning set off to one side or the other).

Going nonparametric

Undaunted by this turn of events, Ellen employs a nonparametric test of her data, which is the right thing to do. Statisticians use Nonparametric statistics In situations where the assumptions of the typical analyses aren’t met (like not having a normal distribution). However, nonparametric stats often give more conservative (albeit more accurate) results than the typical (parametric) procedures you’re used to using. (I discuss nonparametrics a bit more in the last section of this chapter. Nonparametric procedures are discussed in full detail in Chapters 16-19.)

Because Ellen’s data doesn’t have a normal distribution or even a Symmetric distribution (meaning one that looks the same on each side when you split it down the middle), the mean (or average) isn’t a good measure of the center of the data, so a two-sample T-test isn’t possible. As an alternative, she can test whether the two histograms are the same or not, if she compares the histograms of the two populations in question (all roses given water, versus all roses given sugar water).

Because she’s comparing two groups, Ellen uses a Wilcoxon Rank Sum test, also known as the Mann-Whitney test (see Chapter 19). The Wilcoxon Rank Sum test checks whether two populations have the same distribution (meaning whether the two histograms look the same) versus one of the populations shifting to the right or left. Ellen’s theory is that the sugar group lasts longer, so she tests Ho: Sugar group and control group have the same distribution versus Ha: Sugar group is shifted to the right of the control group.

Ellen strikes out

To cut to the chase, the Wilcoxon Rank Sum test unfortunately fails to reject Ellen’s null hypothesis. She didn’t prove what she wanted to confirm by her experiment. Not enough roses in the sugar group lasted longer than those roses in the control group. You can see the underlying reason for this result by comparing the medians of the two groups. When you find the median of each of the data sets in Table 1-1, you get the value of 4 in each case. Because the medians of the two data sets are equal, it’s unlikely that Ellen can find a statistically significant result by using this test.

Breaking the rules

According to the rules that all good statisticians live by, Ellen’s story should end there. She may still be convinced that sugar indeed helps roses last longer. She may use sugar with her roses for the rest of time and tell her friends to use it too. But, she isn’t allowed to say that sugar water gives statistically different results than water alone; her analysis failed to show that.

But remember, Ellen’s last name is Go-getter, so she’s out to get those results. She knows that nonparametric tests usually give more conservative results than regular tests, and despite the fact that the conditions aren’t met, she decides to analyze her data again, this time using the two-sample /-test.

Putting her data into a two-sample /-test takes only two more clicks of the mouse, and Ellen’s results give her a p-value of 0.043. Using the usual significance level used for hypothesis tests, 0.050, her p-value is less than this number, so she can reject Ho. (In a two-sample /-test, Ho is that there’s no difference in the means of the two groups. And her Ha in this case is that the mean of the sugar group is larger than the mean of the control group.) So Ellen gleefully cheers herself on for getting the results she wanted and decides there’s no harm in trying a different analysis when all else fails.

Seeing the error of Ellen’s Ways

But again, "Houston. . ." — you know the rest. Ellen’s problem is that she cheated her way to getting a result that’s incorrect. She knew that the conditions for the two-sample /-test weren’t met, but when the correct analysis failed to get the results she wanted, she found an analysis that did. The trouble is, the results of the two-sample /-test are bogus.

Now it may not be a life-and-death situation whether your roses actually do last a little bit longer on sugar or not. (Incidentally, the gardening crowd says they don’t, and that sugar in fact can encourage the growth of stem-clogging bacteria so the flower can’t take in water.) But imagine a situation where doctors are trying to test to see whether a certain medication helps people get over an illness faster or whether some procedure helps cancer patients live longer. Now you’re talking about results with a very serious impact.

Using the wrong data analysis for the sake of getting the results you desire results in two major problems:

You mislead your audience into thinking that your hypothesis is actually correct, which it may not be.

Sooner or later someone is going to try to replicate those results and will find out that they can’t be replicated. This discovery will result in a loss of your credibility Big /ime. And unfortunately, you mislead many people in the meantime.

Getting the Big Picture: An OverView of Intermediate Statistics

Because of the dangers and lingering effects of using the wrong techniques in the wrong situation to analyze data to answer questions, knowing what’s happening behind the scenes of any data analysis and staying within the rules of well-chosen techniques and appropriate practices is very important. In other words, it’s crucial for you to take your knowledge of statistics to the next level.

Intermediate statistics is an extension of introductory statistics, so the jargon follows suit and the techniques build on what you already know. If you’ve been able to grasp the ideas from the first course, you’ll find no trouble with the terminology for intermediate statistics. If you’re still unsure about some of the terms from introductory statistics, you can consult your textbook from your first course or see my other book, Statistics For Dummies (Wiley), for a complete rundown.

In this section, you get an introduction to the terminology you use in intermediate statistics, and you get a broad overview of the techniques that statisticians use for the purpose of analyzing data and the big picture behind them.

Population parameter

A Parameter Is a number that summarizes the population (the entire group you’re interested in investigating). Examples of parameters include the mean of a population, the median of a population, or the proportion of the population that falls into a certain category.

Suppose you want to determine the average length of a cell-phone call among teenagers (ages 13 to 18). You’re not interested in making any comparisons; you just want to make a good guesstimate as to what the average time is. So you want to estimate a population parameter (such as the mean or average). The population is all cell-phone users between the ages of 13 and 18 years old. The parameter is the average length of a phone call this population makes.

Sample statistic

You normally can’t study every member of an entire population (how would you like to measure and record the length of every single cell-phone call made by all teenagers?). So you can’t determine population parameters exactly; you can only estimate them. But all is not lost; by taking a sample (a subset of individuals) from the population and studying them, you can come

Up with a good guess (estimate) of the population parameter, if you play your cards right. A subset of this population is called a Sample. A Sample statistic Is a single number that summarizes that subset of the population.

For example, in the cell-phone scenario, you select a sample of teenagers and measure the length of their cell-phone calls over a period of time (or look at their cell-phone records if you can gain access legally). You take the average of the cell-phone call lengths. For example, the average length of 100 cellphone calls may be 12.2 minutes — this average is a statistic. This particular statistic is called the Sample mean, Because it’s the average value from your sample data.

You can also find a statistic called the Sample proportion (the proportion of individuals in the sample that have a certain characteristic — for example, the percentage of female teens who use cell phones). Many different statistics are available (which you probably picked up in intro stats) to study different characteristics of a sample, such as the median, variance, and standard deviation.

Confidence interval

A Confidence interval Is a range of values that provides reasonable estimates for a population parameter. A confidence interval is based on a sample and the statistics that come from that sample. The main reason you want to provide a range of possible values rather than a single number is that sample results vary from sample to sample.

For example, say you want to estimate the percentage of people who eat chocolate. According to the Simmons Research Bureau, 78 percent of adults reported eating chocolate, and of those, 18 percent admitted to eating sweets frequently. What’s missing in these results? These numbers are only a single sample of people, and those sample results are guaranteed to vary from sample to sample. You need some measure of how much you can expect those results to move if you were to repeat the study.

This expected movement in your statistic is measured by the Margin of error, Which reflects a certain number of standard deviations of your statistic you add and subtract to have a certain confidence in your results (see Chapter 3 for more on margin of error). If the chocolate-eater results were based on 1,000 people, the margin of error would be approximately 3 percent, meaning the actual percentage of people who eat chocolate in the entire population is expected to be 78 percent, plus or minus 3 percent. In other words, it’s somewhere between 75 percent and 81 percent. Now if you only base these results on a sample of 100 people, the margin of error balloons to 10 percent, meaning the percentage of chocolate eaters can only be reported to be between 68 and 88 percent. Notice how much wider the interval becomes when a smaller sample size is used. This result confirms that more data means more precision in your results (provided the data is collected properly).

Hypothesis test

0

A Hypothesis test Is a statistical procedure that you use to test an existing claim about the population, using your data. The claim is noted by Ho (the null hypothesis). If your data support the claim, you fail to reject Ho. If your data don’t support the claim, you reject Ho and conclude an alternative hypothesis, Ha. The reason most people conduct a hypothesis test is not to merely show that their data support an existing claim, but rather to show that the existing claim is false, in favor of the alternative hypothesis.

The Pew Research Center studied the percentage of people who go to ESPN for their sports news. Their statistics, based on a survey of about 1,000 people, found that in 2000, 23 percent of people said they go to ESPN; while in 2004, only 20 percent reported going to ESPN. The question is this: Does this 3-percent reduction in viewers from 2000 to 2004 represent a significant trend that ESPN should worry about?

To test these differences formally, you can set up a hypothesis test. You set up your null hypothesis as the result you have to believe without your study, Ho = no difference exists between 2000 and 2004 data for ESPN viewership. Your alternative hypothesis (Ha) is that a difference is there.

In very general terms, here’s what’s happening with a hypothesis test. You have the sample data, and you find the statistics that are relevant. In this case, you have two sample percentages, one for 2000 and one for 2004. You take the difference between the two samples (3 percent), and divide it by the standard error for the difference. The standard error measures how much the difference in the statistics is expected to change from sample to sample. In this case, the standard error comes to about 1.8 percent (for specific calculations see Chapter 3).

Taking the difference in the statistics (3 percent = 0.03) divided by the standard error (1.8 percent = 0.018) gives you the value of 1.67 (called the Test statistic). This value represents the difference between the two statistics, in terms of number of standard errors. This result has a universal interpretation. Roughly speaking, if your test statistic falls between -2.00 and +2.00, that means the results you found don’t differ enough to get excited about, because 95 percent of the time, this outcome happens just by chance. (And this example falls right into that situation.) After you take the variability of the sample results into account, the difference in these particular samples doesn’t transfer over to the populations they represent. So, because you can’t reject Ho, you have to say the percentage of viewers of ESPN in the entire population probably didn’t change from 2000 to 2004.

Because you have a 95 percent confidence level, this test uses a significance level (a level) of 1 – 0.95 = 0.05 or 5 percent. This percentage measures how likely your results would have been just by chance.

The trouble is that people often just report the sample statistics and give no regard to the expected amount of change with a new sample. This disregard leads to big mistakes in the conclusions (more on hypothesis testing in Chapter 3).

Analysis of Variance (ANOVA)

ANOVA is the acronym for Analysis of variance. You use ANOVA in situations where you want to compare the means of more than two populations. For example, you want to compare the lifetime of four brands of tires, in number of miles. You take a random sample of 50 tires from each group, for a total of 200 tires, and set up an experiment to compare the lifetime of each tire, and record it. You have four means and four standard deviations now, one for each data set. But you have different types of variability in your data, each measured by using various sums of squares. (Remember from your intro stats that the variance of a data set is the total of all the squared distances between the data and the mean, all divided by N - 1.)

One of the types of variability in your data is called the variability Between Treatments (also known as SST, The treatment sums of squares). SST measures the variation in the average lifetimes of each brand of tire, compared to the overall average lifetime. If SST is large, you have a chance that there’s a difference in lifetimes due to the treatment (in this case, the brand of tire).

Next, you have the variability Within The treatments (also known as SSE, The error sums of squares). SSE measures the overall average amount of variability of the tire lifetimes within each particular brand (after all, not all tires are created equal, even if they’re of the same brand). If SSE is large, you have so much variability within the tire brands themselves, that it will be harder to see any real difference between the brands, even if it actually exists.

And finally, you have the Total Overall variability in the data values if you just put them all together into one big data set. This variability is known as SSTO, The total sums of squares. ANOVA splits up the total variability (SSTO) into the between-groups variability (SST) plus the within-groups variability (SSE).

Then, to test for differences in average lifetime for the four brands of tires, you compare the mean sums of squares for treatments (MST) to the mean sums of squares for error (MSE) in a ratio called the F-statistic. If this ratio is large, then the variability between the brands is more than the variability within the brands, giving evidence that not all the means are the same for the different tire brands. If the F-statistic is small, that means not enough difference was between the treatment means, compared to the general variability within the treatments themselves. In this case, you can’t say that the means are different for the groups. (I give you the full scoop on ANOVA in Chapters 9 and 10.)

Multiple comparisons

Suppose you conduct ANOVA, and you find a difference in the average lifetimes of the four brands of tire (see preceding section). Your next questions would probably be, which brands are different, and how different are they? To answer these questions, you use multiple-comparison procedures.

A Multiple-comparison procedure Is a statistical technique that compares means to each other and finds out which ones are different and which ones aren’t. You’re then able to put the groups in order, from those with the largest mean to those with the smallest mean, realizing that sometimes two or more groups were too close to tell and so you put them in the same group.

Suppose you compare the exam scores of four different classes (call them class one, class two, class three, and class four), and your ANOVA procedure finds out that not all the means were the same. That means the F-statistic is large. Next, you use multiple-comparison procedures in order to make separate comparisons and figure out which classes were about the same and which ones were different, and come up with an ordering of the classes. It may be, for example, that class four was statistically higher than all the others; classes one and two were statistically equivalent, but both were lower than class four. And class one was in a group all by itself at the bottom. The ordering is: class four (highest average), classes two and three (tied for second highest), and class one (the lowest average).

Never take that second step to compare the means of the groups if the ANOVA procedure doesn’t find any significant results during the first step. (See Chapter 11 for more information.)

Many different multiple-comparison procedures exist to compare individual means and come up with an ordering in the event that your /-statistic does find that some difference exists. Some of the multiple-comparison procedures include Tukey’s test, LSD, and pairwise t-tests. (While these tests’ names may cause you to raise an eyebrow, don’t worry. They’re legitimate statistical tests.) Some procedures are better than others, depending on the conditions and your goal as a data analyst. I discuss multiple-comparison procedures in detail in Chapter 11.

Interaction effects

An Interaction effect In statistics operates the same way that it does in the world of medicine. Sometimes if you take two different medicines at the same time, the combined effect is much different than if you take the two individual medications separately.

Interaction effects come up when you have a model that includes two or more variables, and you’re using those variables to explain differences or to make comparisons regarding some outcome. When you have two or more variables in a model, you can’t automatically study the effect of each variable separately; you also have to take into account the way those variables interact in terms of the outcome. In other words, you have to examine whether or not an interaction effect is present.

For example, suppose medical researchers are studying a new drug for depression and want to know how this drug affects the change in blood pressure for a low dose versus a high dose of the drug. They also compare the effects for children versus adults. In total, the model being studied has one response variable, an increase in blood pressure, and two factors that may possibly explain changes in the outcome, namely age group (adults versus children) and dosage level (low versus high). It could be that dosage level affects the blood pressure of adults differently than the blood pressure of children. This type of model is called a Two-way ANOVA model, With a possible interaction effect between the two factors (age group and dosage level). See Chapter 11 for more.

One of the first things statisticians do when they have a two-way ANOVA is to plot the mean outcomes for each group they’re comparing and look for patterns. This is called an Interaction plot. One interaction plot for the drug-study scenario is in Figure 1-3.

Age group and dosage level when studying the effect on blood pressure.

CD CD

Oo

CD ^

03 00

- 00

O CD

Figure 1-3: .= Q.

Interaction between

CD O CD o

Low

High

Dosage Level

As you can see by Figure 1-3, the lines cross. If you look at the line representing children, you can see that the mean increase in blood pressure is low for the low dose of the drug, but then for the high dose of the drug; the increase in blood pressure goes way up. Alternatively, the reaction is the exact opposite for adults; on the low dose, the mean increase in blood pressure is very high, but for the high dose, the increase is very low. If the doctors neglected to study children as well as adults, the results of this study could be extremely damaging to children if doctors applied the rules for adults to children. This example shows that interaction effects are very important to look at.

Figure 1-4 shows the situation where you have no interaction effect for this drug. The lines are parallel, which tells you that the mean blood pressure increases more on a higher dosage of the drug for both adults and children. Because the line for the adults is higher up than the line for children, that means that overall, the increase in blood pressure is more for adults than the increase in blood pressure for children, no matter what the dosage level is.

Figure 1-4:

No

Interaction between age group and dosage level when studying the effect on blood pressure.

Low

High

Dosage Level

Correlation

The term Correlation Is often misused. Statistically speaking, the correlation measures the strength and direction of the linear relationship between two quantitative variables (variables that represent counts or measurements only).

You aren’t supposed to use the word Correlation To talk about relationships of any other kind. For example, it’s wrong to say that a correlation exists between eye color and hair color. While these variables may be related in

Some way, they’re not quantitative variables, so you can’t discuss their relationship in terms of a correlation. (In this case, you would use the term Association; In Chapter 14, you see how to test for association of two categorical variables.)

The long and short of correlation is the following: Correlation Is a number between -1.0 and +1.0. Positive one indicates a perfect positive relationship; in other words, as you increase one variable, the other one increases in perfect sync. On the other side of the coin, a correlation that is -1.0 indicates a perfect negative relationship between the variables. As one variable increases, the other one decreases in perfect sync. A correlation of zero indicates that you found no linear relationship at all between the variables. Most correlations in the real world aren’t exactly +1.0, -1.0, or 0 — they fall somewhere in between. The closer to +1.0 or -1.0, the stronger the relationship is; the closer to 0, the weaker the relationship is.

Figure 1-5 shows an example of a plot showing the number of coffees sold at football games in Buffalo, New York, as well as the air temperature (in Fahrenheit) at each game. This data set seems to follow a downhill straight line fairly well, indicating a negative correlation. When you calculate the correlation, you get the value of -0.741. This value says that coffees sold has a fairly strong negative relationship with the temperature of the football game. This makes sense, because on days when the temperature is low, people will get cold and want more coffee. On days when the temperature is higher, people will tend to drink less coffee and perhaps tend more toward soft drinks, which are cold. I discuss correlation further, as it applies to model building, in Chapter 4.

Figure 1-5:

Coffees sold at various air temperatures on football game day.

Linear regression

After you’ve determined that two variables have a fairly strong linear relationship, you may want to try to make predictions for one variable based on the value of the other variable. For example, if you know that a fairly strong negative linear relationship exists between coffees sold and the air temperature at a football game, you may want to use this information to predict how much coffee is needed for a game, just by knowing the temperature. This method of finding the best-fitting line is called Linear regression.

In the coffees and temperature example (see Figure 1-5), the best-fitting line has the equation Y = 49,337 – 554 * X , where X Is temperature and Y Is the number of coffees sold. So when the temperature (x) Is zero degrees, you can expect to sell around 49,337 coffees (this is how you interpret the y-intercept of the line). To interpret the slope of this line, think of -554 as -554 divided by one and use the old rise-over-run idea using coffees and degrees of temperature. Applied here, it means that for every one degree increase in temperature, you can expect the coffee sales to decrease by 554. You can use this line to make predictions for reasonable values of the temperature (x). For example, if the temperature is a cold 20-degrees Fahrenheit, you can predict that the number of coffees sold will be around 49,337 – 554 * 20 = 38,257.

When you use only one variable to predict the response, the method of regression is called Simple linear regression. (I review the basics of simple linear regression in Chapter 4. But many other types of regression are out there, many of which I discuss in this book.)

Most researchers use more than one variable to predict a response; this technique is called Multiple linear regression. (Check out Chapter 5 for the details about multiple linear regression.) Multiple linear regression has many issues of its own because some variables you can use in the model may be related to each other, making overlapping contributions to the response. That possibility of overlapping makes their individual contributions hard to track. You also have to watch for interaction effects when using more than one variable to predict a response.

Simple and multiple linear regression assume that the response variable (the one being studied) is quantitative in nature (that is, it measures or counts something). However, you may be interested in making predictions about a variable that has only two outcomes: yes or no. For example, whether or not a certain horse will win a race; whether a baby will be a girl or a boy; or whether or not a tropical storm is going to make landfall. These situations require a different kind of regression called Logistic regression. (I discuss logistic regression in Chapter 8.)

Finally, you may be interested in building a model for which a straight line doesn’t fit. For example, you may want to predict miles per gallon, using the speed of the car. While high speeds get low miles per gallon, low speeds can get low miles per gallon as well. So the relationship between speed and miles per gallon actually follows that of a Parabola (an upside-down bowl, in this case). This kind of relationship is called a Quadratic relationship. More generally speaking, relationships that don’t follow a straight line are called Nonlinear relationships, And the technique you use to handle these situations is called (no surprise) Nonlinear regression. I get into the meat of this technique in detail in Chapter 7.

Chi-square tests

Correlation and regression techniques all assume that the variable being studied in most detail (the response variable) is quantitative. That is, the variable measures or counts something. However, you can run into many situations where the data being studied isn’t quantitative, but rather qualitative. In other words, the data themselves represent categories, not measurements or counts.

For example, suppose you want to compare the views of the president by political affiliation. Say that in this particular year, the president is a Republican, and you select a random sample of 150 Republicans, 150 Democrats, and 150 Independents to find out their views on the president. The data may look like Table 1-2.

In looking at how the numbers appear across the columns for various rows in Table 1-2, you may suspect that something is up. It appears that Republicans tend to approve of the president, while Democrats tend to disapprove, and Independents are split down the middle. (So much for the spirit of bipartisanship. . . .)

Now does this association you found in the data set for this sample of 450 people carry over to the entire population? In order to answer this question, you need to conduct a hypothesis test. And not just any hypothesis test — a Chi-square test for independence. You’re testing to see whether the two qualitative variables, political affiliation and views on the president, are related or not. If they are related, the variables are deemed not independent; if they are unrelated, the variables are independent.

A Chi-square test basically does the following: It figures out the number of values that you expect to see in each cell of the table if the variables are independent (these values are brilliantly called the Expected cell counts). The Chi-square test then compares these expected cell counts to what you actually saw in the data (called the Observed cell counts) And compares them to each other in a Chi-square statistic (see Chapter 14).

If the Chi-square test statistic is large, you’re likely to find an association between the two variables, because the total differences are large between the observed and expected cell counts. In other words, the variables are not independent, and you can look at the observed cell counts to discuss the relationship you see. If the Chi-square test statistic is small, then you can’t conclude you’ve found a relationship, and the two variables are independent.

In the case of political affiliation and views on the president, the Chi-square test statistic is huge, and you conclude a relationship is there somewhere. You can say that in the population, Republicans tend to support the president, Democrats tend to oppose the president, and the Independents are split down the middle. (You can find the details of how to find the expected counts and conduct the Chi-square test in Chapter 14.)

You can also use the Chi-square test to see whether your theory about what percent of each group falls into a certain category is true or not. For example, can you guess what percentage of M&Ms fall into each color category? More on these Chi-square variations, as well as the M&Ms question, in Chapter 15.

Nonparametrics

Nonparametrics Is an entire area of statistics that provides analysis techniques to use when the conditions for the more traditional and commonly used methods aren’t met. For example, in order to use a T-test, the data needs to be collected from a population that has a normal distribution (that is, it has to have a bell-shaped curve). In order to do a hypothesis test for two means, the data from each group must be from its own normal population. In fact, most all of the commonly used data-analysis procedures have conditions that must be met in order to use them.

The trouble with these requirements is that many times people forget or just don’t bother to check those conditions, and if the conditions are actually not met, the entire analysis goes out the window, and the researcher doesn’t even know it. Or, someone finds out that the conditions aren’t being met, yet she still goes ahead and uses the procedures anyway (for more on this faux pas, see the section in this chapter "No [data] fishing allowed").

While many of the traditional methods are what statisticians call Robust, With respect to violations of their conditions (that’s fancy terminology for the fact that they’re pretty forgiving), you can only push the window so far. Proceeding to use a statistical procedure that isn’t appropriate causes a great deal of trouble with respect to the correctness of the conclusions and the credibility of the researcher.

Have no fear, nonparametrics comes to your rescue. If the conditions aren’t met for a data-analysis procedure that you want to do, chances are that an equivalent nonparametric procedure is waiting in the wings. And the good news is that they’re generally pretty tame, in terms of formulas, and most statistical software packages can do them just as easily as the regular (parametric) procedures.

Conditions aren’t checked automatically by statistical software packages, before doing a data analysis. It’s up to the user to check any and all appropriate conditions, and if they’re seriously violated, to take another course of action. Many times a nonparametric procedure is just the ticket. For much more information on different nonparametric procedures, see Chapters 16 through 19.

In This Chapter

^ Utilizing confidence intervals to estimate parameters

^ Testing models by using hypothesis tests

^ Finding the probability of getting it right and getting it wrong

^ Discovering power in a large sample size

Ne of the major goals in statistics is to use the information you collect from a sample in order to get a better idea of what’s going on in the

Entire population you’re studying (because populations are generally large and exact info is often unknown). The most common items to study are the mean of the population, the proportion of the population that has a certain characteristic, or a comparison of the means or proportions from two different populations. These unknown values that summarize the population are called Population parameters. Researchers typically either want to get a handle on what those parameters are, or they want to test a hypothesis about the population parameters. In introductory statistics, you typically go over confidence intervals and hypothesis tests for one and two population means and one and two population proportions. Your instructor hopefully emphasized that no matter which parameters you’re trying to estimate or test, the general process is the same. If not, don’t worry; that’s what this chapter’s all about.

The most important idea you can gain from this chapter is that intermediate statistics focuses on building and testing models. You’re typically faced with some random phenomena, and you’re trying to build a model that explains or predicts that phenomena. The situation is more complex than it was in intro stats, where you used one variable to predict another variable in simple linear regression. Intermediate statistics takes it up a notch to using many variables to predict another one. But as long as you keep the big picture of how the process works in your mind, you’ll be okay.

It all comes down in the end to testing hypotheses to see whether certain models fit, and if they do, to using confidence intervals to estimate certain values in the population or to make predictions based on the model that you built.

This chapter reviews the basic concepts of confidence intervals and hypothesis tests, including the probabilities of making errors by chance. I also discuss how statisticians measure the ability of a statistical procedure to do a good job — of detecting a real difference in the populations, for example. Hang on — you’re in for quite a ride.

Estimating Parameters by Using Confidence Intervals

Confidence intervals are a statistician’s way of covering themselves when it comes to estimating a population parameter. For example, instead of just giving a one-number guess as to what the average household income is in the United States, a statistician would give a range of likely values for this number. Statisticians do this for two reasons:

All good statisticians know sample results vary from sample to sample, so a one-number estimate isn’t any good.

Statisticians have developed some awfully nice formulas you can use to give a range of likely values, so why not use them?

In this section, you get the general formula for a confidence interval, including the margin of error, and a good look at the common approach to building confidence intervals. I also discuss interpretation and the chance of making an error.

Getting the basics: The general form of a confidence interval

The big idea of a confidence interval is coming up with a range of likely values for a population parameter. The Confidence level Represents the chance that if you repeated your sample-taking over and over, you’d get a range of likely values that actually contains the actual population parameter. In other words, it’s the long-term chance of being correct.

The general formula for a confidence interval is the following: Confidence interval = Sample statistic ± Margin of error

The confidence interval has a certain level of precision (measured by the margin of error). Precision calculates how close you expect your results to be to the truth.

For example, you want to know the average amount of time a student at Ohio State University spends listening to music per day, using an MP3 player. The average time for the entire population of OSU students that are MP3-player users is the parameter you’re looking for. Certain that you can’t ask every student who uses an MP3 player at OSU this question, you take a random sample of students and find the average from there.

Suppose the average time a student uses an MP3 player per day to listen to music based on a random sample of 1,000 OSU students is 2.5 hours, and the standard deviation is 0.5 hours. Is it right to say that the population of all OSU-student MP3-player owners use their players an average of 2.5 hours per day for music listening? No. You hope and may assume that the average for the whole population is close to 2.5, but it probably isn’t exact. After all, you’re only sampling a tiny fraction of the 60,000 member population of all OSU students. The fact is that sample results vary from sample to sample.

What’s the solution to this problem? The solution is to not only report the average from your sample, but along with it, report some measure of how much you expect that sample average to vary from one sample to the next, with a certain level of confidence. You want to cover your bases, so to speak (at least most of the time). The number that you use to represent this level of precision in your results is called the Margin of error. You take your sample average and add and subtract the margin of error (to get that plus-or-minus factor going), which gives you a confidence interval for the average time all OSU students use their MP3 players.

Finding the confidence interval for a population mean

The sample statistic part of the confidence-interval formula is fairly straightforward. If you want to estimate the population mean, you use the sample mean. If you want to estimate the population proportion, use the sample proportion. If you want to find the difference of two population means, take two samples, find their sample means, and subtract them.

In the case of the population mean, you use the sample mean to estimate it. The sample mean has a standard error of -?=■. In this formula, you can see the population standard deviation (o) And the sample size (n).

If you think about it though, why would you know the standard deviation of the population, O, When you don’t even know the mean (recall that the mean is what you’re trying to estimate)? To handle this additional unknown, do what statisticians always do — estimate it and move on. So you estimate O, The population standard deviation, using (what else?) the standard deviation of the sample, denoted by S. So you replace O By S In the formula for the standard error of the mean.

To estimate the population mean by using a confidence interval when O Is

Unknown, you use the formula X -.

. This formula contains the sample

S

1 JN

Standard deviation (s), The sample size (n), And a /-value representing how many standard errors you want to add and subtract to get the confidence you need. To get the margin of error for the mean, you see the standard error,

-4=, is being multiplied by a factor of T. Notice that T Has N - 1 as a subscript N

To indicate which of the myriad T-distributions you use for your confidence interval. The N - 1 is called Degrees of freedom, Where N Is the sample size.

The value of T In this case represents the number of standard errors you add and subtract to or from the sample mean to get the confidence you want. If you want to be 95 percent confident, for example, you add and subtract about two of those standard errors. If you want to be 99.7 percent confident, you add or subtract about three of them. (Table A-1 in the Appendix presents the t-distribu-tion from which you can find T-values for any confidence level you want.)

If you do know the population standard deviation for some reason, you would certainly use it. In that case, you use the corresponding number from the Z-distribution (standard normal distribution) in the confidence interval formula. (The Z-distribution from your intro stat book can give you the numbers you need.) Or if you know O And have a large sample size, you can simply use the bottom line of the T-distribution, because a T-distribution with a large number of degrees of freedom gives very similar values to the Z-distribution.

For the MP3 player example from the preceding section, a random sample of 1,000 OSU students spends an average of 2.5 hours using their MP3 players to listen to music. The standard deviation is 0.5 hours. Plugging this information

Into the formula for a confidence interval, you get 2.5 ± 1.96

0.5

= 2.5 ±

/ 1,000

0.03 hours. You can conclude that OSU MP3-player owners spent an average of between 2.47 and 2.53 hours listening to music on their players. (The value for T In this example came from the last line of Table A-1 in the Appendix, because this line represents the situation where N Is large.)

What cHAnges ThE maRGIN of eRrOR)

What do you need to know in order to come up with a margin of error? Margin of error, in general, depends on three elements:

The standard deviation of the population, O (or an estimate of it, denoted by S, The sample standard deviation)

The sample size, N

The level of confidence you need

You can see these elements in action in the following formula for margin of

Error of the sample mean: Tn_J* -4=-. Here I assume that O Isn’t known; Tn_ 1

N

Represents the value on the /-distribution (Table A-l in the Appendix) with N _ l degrees of freedom.

Each of these three elements has a major role in determining how large the margin of error will be when you estimate the mean of a population. At times it may seem that different elements work against each other (and they do!), but you can find ways around that. In the following sections, I show how each of the elements of the margin of error formula work separately and together to affect the size of the margin of error.

The population standard deviation’s affect on margin of error

The standard deviation of the population is typically combined with the sample size in the margin of error formula, with the population standard deviation on top of the fraction, and N In the bottom. (In this case, the standard error of the population, O, Is estimated by the standard deviation of the sample, S, Because O Is typically unknown.)

This combination of standard deviation of the population and sample size is known as the Standard error Of your statistic. It measures how much the sample statistic deviates from its mean in the long term.

How does the standard deviation of the population (o) Affect margin of error? As the standard deviation of the population (or its estimate, S) Gets larger, the margin of error increases, so your range of likely values is wider. That’s why you typically see the population standard deviation in the numerator of margin of error formulas. The formula for the margin of error for one population is an example of this.

Suppose you have two gas stations, one on a busy corner (gas station #1) and one farther off the main drag (gas station #2). You want to estimate the average time between customers at each station. At the busy gas station #l,

Customers are constantly using the gas pumps, so you basically have no time between customers, and that model holds day after day. At gas station #2, customers sometimes come all at once, and sometimes you don’t see a single person for an hour or more. So the time between customers varies quite a bit.

For which gas station would it be easier to estimate the overall average time between customers as a whole? Gas station #1 has much more consistency, which represents a smaller standard deviation of times between customers. Gas station #2 has much more heterogeneity of times between customers, so that one is harder to get a handle on. That means O For gas station #1 is smaller than O For gas station #2.

Sample size and margin of error

Sample size affects margin of error in a very intuitive way. Suppose you’re trying to estimate the average number of pets per household in your city. Which sample size would give you better information: 10 homes or 100 homes? You’d agree that 100 homes would give more precise information (as long as the data on those 100 homes was collected properly).

If you have more data to base your conclusions on, and that data is collected properly, your results will be more precise. Precision is measured by margin of error; so as the sample size increases, the margin of error of your estimate goes down. That’s why you typically see an N (sample size) in the denominator of margin of error formulas. In the formula for the margin of error of the sample mean, you can see N In the denominator.

Bigger is only better in terms of sample size if the data is collected properly. That is, you should find no bias in the way the members of the sample were selected or in the way the data was collected on those subjects. If the quality of the data can’t be maintained with a larger sample size, it does no good to have it.

Confidence level and margin of error

The amount of confidence you need to have differs from problem to problem. Suppose you’re estimating the mean weight that an elevator can hold. You would want to be pretty confident about your results, right? But, if you wanted to estimate the percentage of females that may come to your party on Saturday night, you may not need to be so confident (despite the desperation you see in your single buddies’ eyes). For each problem at hand, you have to address how confident you need to be in your results over the long term, and, of course, more confidence comes with a price in the margin of error formula. This level of confidence in your results over the long term is reflected in a number called the confidence level, reported as a percentage. In general, more confidence requires a wider range of likely values. Ninety-five percent is the most common confidence level statisticians use.

Every margin of error is interpreted as plus or minus a certain number of standard errors. The number of standard errors added and subtracted is determined by the confidence level. If you need more confidence, you add and subtract more standard errors. If you need less confidence, you add and subtract fewer standard errors. The number that represents how many standard errors to add and subtract is different from situation to situation. For one population mean, you use a value on the /-distribution, represented by Tn_ !, where N Is the sample size. See Table A-1 in the Appendix.

Here’s an example. Suppose you have a sample size of 20, and you want to estimate the mean of a population. The number of standard errors you add and subtract is represented by Tn _ 1, which in this case is t19. Suppose your confidence level is 90 percent. To find the value of T, You look at row 19 in the t-distribution table (Table A-1 in the Appendix). The table uses the area to the right, so that area in this case is 0.05. (You get this value because 90 percent is within the confidence interval, so 10 percent is outside of it. Half of that 10 percent lies above the confidence interval, and the other half lies below it.) So look at row 19 and the column headed by the value 0.05. You get the value of T = 1.73. So to be 90 percent confident with a sample size of 20, you need to add and subtract 1.73 standard errors.

Now suppose you want to be 95 percent confident in your results, with the same sample size of N = 20. The area above the interval is now half of 5 percent, which is 2.5 percent or 0.025. Row 19 and column 0.025 in Table A-1 gives you the value of t19 = 2.09. Notice that this value of T Is larger than the value of T For 90 percent confidence, because in order to be more confident, you need to go out more standard deviations on the T-distribution table to cover more possible results.

Large confidence, narrow intervals — just the right size

A narrow confidence interval is much more desirable than a wide one. For example, if you said that you think the average cost of a new home is $150,000 plus or minus $100,000, that wouldn’t be helpful at all because this makes your estimate anywhere between $50,000 and $250,000. (Who has an extra hundred grand to throw around?) But you Have To be 99 percent confident, so your statistician has to add and subtract more standard errors to get there, which makes the interval that much wider (a downer). She tells you to be happy with 95 percent confidence, but no!

Wait, don’t panic — you can have your cake and eat it too! If you know you want to have a high level of confidence, but you don’t want a wide confidence interval, just increase your sample size to meet that level of confidence. The effect of sample size and the effect of confidence level cancel each other out, so you can have a precise (narrow) confidence interval and a high level of confidence at the same time. It all depends on sample size, something you can control (up to the size of your pocketbook of course).

For example, say the standard deviation of the house prices from a previous study is S = $15,000, and you want to be 95 percent confident in your estimate of average house price. Using a large sample size, your value of T (from the last row of Table A-1 in the Appendix) would be 1.96. With a sample of 100 homes, your margin of error would be plus or minus 1.96 times $15,000 divided by the square root of 100, which comes out to $2,940. If this is too large for you but you still want 95 percent confidence, crank up your value of N. If you sample 500 homes, the margin of error decreases to plus or minus 1.96 times $15,000 divided by the square root of 500, which brings you down to $1,314.81.

I&M-STfy. You can actually use a formula to find the sample size you need to meet a

Desired margin of error. That formula is N

.( tn -1 S

, where MOE is the

\MOE/

Desired margin of error (as a proportion), S Is the sample standard deviation, and T Is the value on the T-distribution that corresponds with the confidence level you want. (You can use the last line of Table A-1 in the Appendix, which will work fine, assuming that your sample size is fairly beyond 30.)

Interpreting a confidence interval

Interpreting a confidence interval involves a couple of subtle but important issues, which I discuss in this section. The big idea is that a Confidence interval Presents a range of likely values for the population parameter, based on your sample. It includes this range because your sample results are going to vary, and you want to address that. A 95 percent confidence interval, for example, provides a range of likely values for the parameter such that the parameter is included in the interval 95 percent of the time in the long term.

AjV\NG/ A 95 percent confidence interval doesn’t mean that your particular confidence interval has a 95 percent chance of capturing the actual value of the

( ] parameter; after the sample has been taken, it’s either in the interval or it

J Isn’t. A confidence interval represents the long-term chances of capturing the actual value of the population parameter over many different samples.

Suppose a polling organization wants to estimate the percentage of people in the United States who drive a car with more than 100,000 miles on it, and it wants to be 95 percent confident in its results. The organization takes a random sample of 1,200 people and finds that 420 of them (35 percent) drive a much-driven car.

The meaty part of the interpretation lies in the confidence level — in this case, the 95 percent. Because the organization took a sample of 1,200 people in the U. S., asked each of them whether his or her car has more than 100,000 miles on it and made a confidence interval out of it, the polling organization is, in

Essence, accounting for all of the other samples out there that it could have gotten by building in the margin of error (±3 percent). The organization wants to cover its bases on 95 percent of those other situations, and the ±3 percent satisfies that.

Another way of thinking about the confidence interval is to say that if the organization sampled 1,200 people over and over again and made a confidence interval from its results each time, 95 percent of those confidence intervals would be right. (You just have to hope that yours is one of those right results.)

Using stat notation, you can write confidence levels as 1 – a. So if you want 95 percent confidence, you write it as 1 – 0.05. The number that a represents is the chance that your confidence interval is one of the wrong ones. This number, a, is also related to the chance of making a certain kind of error with a hypothesis test, which I explain in the hypothesis-testing section.

Setting Up and Testing Models

A Model Is an equation that attempts to describe how a population behaves. It can be a claim that’s made about a population parameter; for example, a shipping company might say that its packages are on time 95 percent of the time, or a campus official claims that 75 percent of students live off campus. It is important to test these models to see whether they actually hold up in the population, which you can do by using hypothesis tests.

In this section, you see the big ideas of hypothesis testing that are the basis for the data-analysis techniques in this book. You review and expand on the concepts involved in a hypothesis test, including the hypotheses, the test statistic, and the p-value.

What do Ho and Ha represent — really?

The big idea here is that you set up a hypothesis test to see whether your model fits the population, based on your data. In the intro stat course, you tested simple hypotheses — like whether the population mean is equal to ten. At the intermediate statistics level, you get to look at much more sophisticated and relevant models that involve several variables and/or several different populations in a variety of situations. The good news, though, is that the basic ideas from intro stats apply here as well. (If you need a brief refresher before barreling through this section, feel free to flip through your intro stats book or check out my other book Statistics For Dummies [Wiley].)

You use a hypothesis test in situations where you have a certain model in mind, and you want to see whether that model fits your data. Your model may be one that just revolves around the population mean (testing whether that mean is equal to ten, for example). Your model may be testing the slope of a regression line (whether or not it’s zero, for example, with zero meaning you find no relationship between X And y). You may be trying to use several different variables to predict the marketability of a product, and you believe a model using customer age, price, and shelf location can help predict it, so you need to run one or more hypothesis tests to see whether that model works. (This process is called multiple regression; more info on this in Chapter 5.)

A hypothesis test is made up of two hypotheses:

The null hypothesis (Ho): Ho symbolizes the current situation — the one that everyone assumed was true until you got involved.

The alternative hypothesis (Ha): Ha represents the alternative model that you want to consider. It stands for the researcher’s hypothesis, and the burden of proof lies on the researcher to prove it.

Ho is the model that’s on trial. If you get enough evidence against it, you conclude Ha, which is the model you’re claiming is the right one. If you don’t get enough evidence against Ho, then you can’t say that your model (Ha) is the right one.

Gathering your evidence into a test statistic

A Test statistic Is the statistic from your sample, standardized so you can look it up on a table, basically. While each hypothesis test is a little different, the main thought is the same. For whatever model you’re trying to test, you come up with a statistic that you use to test that model. Take that statistic, standardize it (take the statistic minus its expected value from Ho and divide all that by the standard error). Then look up your test statistic on a table to see where it stands. That table may be the t-table (Table A-1 in the Appendix), it may be the Chi-square table (Table A-3 in the Appendix), or it may be a different table. The type of test you need to you use on your data dictates which table you use.

In the case of testing a hypothesis for a population mean, u,, you use the sample mean, X, as your statistic. To standardize it, you take X And convert it to a

Value of T By using the formula Tn _ 1 = X s U 0, where u,0 is the value in Ho. This

Value is your test statistic. You compare your test statistic to the T-distribution (check out Table A-1 in the Appendix).

Determining strength of evidence with a p-value

If you want to know whether your data has the brawn to stand up against Ho, you want to figure out the p-value and compare it to a prespecified cutoff, a (typically 0.05). The P-value Is a measure of the strength of your evidence against Ho. You can calculate the P-value by doing the following:

1. Calculate the test statistic. See the preceding section for more info on this.

2. Look up the test statistic on the appropriate table (such as the f-table, A-1 in the Appendix).

3. Find the percentage of values on the table that fall beyond your test statistic. This percentage is the P-value.

Suppose you’re conducting a hypothesis test and have already decided you will reject Ho at level a = 0.05. You collect your data and find the test statistic (see preceding section). If your test statistic is extremely high or extremely low compared to other values on the table (whatever that table is), then you reject Ho.

For example, say the cutoff value for rejecting Ho at a level a = 0.05 is 1.645, where you’re testing for the mean of one population. If you get a test statistic of 1.7, you reject Ho. If you get a test statistic of 2.7, you Really Reject Ho. That is, you have more evidence against Ho with a test statistic of 2.7 than with a test statistic of 1.7. The two P-values of 1.7 and 2.7 are what statisticians call Marginally significant And Highly significant Results respectively, to use proper terms.

IBE# Your friend, a, is the cutoff for your p-value — and the star of this chapter.

(a is typically set at 0.05 — sometimes 0.10.) If your p-value is less than your predetermined value of a, reject Ho, because you have sufficient evidence against it. If your p-value is greater than or equal to a, you can’t reject Ho.

For example, if your P-value is 0.002, then your test statistic is so far away from Ho that the chance of getting this result only by chance is only 2 out of 1,000. So, you conclude that Ho is very likely to be false. However, if your p-value turns out to be 0.30, then this result happens 30 percent of the time anyway, so you see no red flags there, and you can’t reject Ho. You don’t have enough evidence against it. It doesn’t mean Ho is true, but you don’t have the evidence to say it’s false — a subtle, but important, difference.

When I compare the p-value to the a (the cutoff value), I like to think of a football analogy, assuming that Ho is "the opposing team can’t make a touchdown." The burden is on the other team to show enough evidence to reject

Ho. Now, imagine that their running back makes a touchdown by pushing the ball just barely over the goal line, so close that his team needs to have a referee review the film before calling it a touchdown. This situation is equivalent to rejecting Ho with a p-value just below your prespecified value of a = 0.05. In this case, the p-value is close to the borderline, say 0.045. But, if their team makes a touchdown by catching a pass deep in the end zone, no one has any doubt about the result because the ball was obviously past the goal line, which is equivalent to the p-value being very small, say something like 0.001. The opposing team’s showing a lot of evidence against Ho (and your team could be in a lot of trouble).

Deconstructing Type I and Type II errors

Any technique you use in statistics to make a conclusion about a population based on a sample of data has the chance of making an error. The errors I am talking about, Type I and Type II errors, are due to random chance.

For example, you could flip a fair coin ten times and get all heads, making you think that the coin isn’t fair at all. This thinking would result in an error, because the coin actually was fair, but the data just wasn’t confirming that due to chance. On the other hand, another coin may be unfair, and, just by chance, you flip it ten times and get exactly five heads, which makes you think that particular coin is equally balanced and doesn’t present any problem. (This tells you strange things can happen, especially when the sample size is small.)

The way you set up your test can help to reduce these kinds of errors, but they are always out there. As a data analyst, you need to know how to measure and understand the impact of the errors that can occur with a hypothesis test and what you can do to possibly make those errors smaller. In the following sections, I show you how you can do just that.

Making false alarms with Type I errors

A Type I error represents the situation where the coin was actually fair (using the example from the preceding section), but your data led you to conclude that it wasn’t, just by chance. I think of a Type I error as a false alarm: You blew the whistle when you shouldn’t have.

To include a definition that makes all those stat experts happy, a Type I error is the conditional probability of rejecting Ho, given that Ho is true.

The chance of making a Type I error is equal to a, which is predetermined before you begin collecting your data. This a is the same a that represents the chance of missing the boat in a confidence interval. It makes some sense

That these two probabilities are both equal, because the probability of rejecting Ho when you shouldn’t (Type I error) is the same as the chance that the true population parameter falls out of the range of likely values when it shouldn’t. That chance is A.

Say someone claims that the mean time to deliver packages for a company is 3.0 days on average (so Ho is U, = 3.0), but you believe it’s not equal to that (so Ha is 3.0). Your alpha level is 0.05, and because you have a two-sided test, this means you have 0.025 on each side. Your sample of 100 packages has a mean of 3.5 days with a standard deviation of 1.5 days. You find the test

. . X – U 0 3 5 – 3 0 … 1 O OO rr,, . , , , ,

Statistic Tn-1 = —S— = ‘ 15 ‘ , which equals 3.33. Ihis value falls beyond

1.96 (the value on the last row and the 0.025 column of the /-distribution, Table A-1 in the Appendix). So you don’t think 3.0 is a likely value for the mean time of delivery, over all possible packages, and you reject Ho. Your data led you to that decision and you stick to it.

But suppose your sample just by chance contained some longer than normal delivery times, and that in reality, the company’s claim is right. You just made a Type I error. You made a false alarm about the company’s claim.

To reduce the chance of a Type I error, reduce your value of a. However I wouldn’t recommend reducing a too far. On the positive side, this reduction makes it harder to reject Ho, because you need more evidence in your data to do so. On the negative side, by reducing your chance of a Type I error, you increase the chance of another type of error — the Type II error. To tackle Type II errors, keep reading!

Missing an opportunity with a Type 11 error

A Type II error represents the situation where (continuing with the coin example) the coin was actually unfair, but your data didn’t have enough evidence to catch it, just by chance. You can think of a Type II error as a missed opportunity — you didn’t blow the whistle when you should have. In statistical terms, a Type II error is the conditional probability of not rejecting Ho, given that Ho is false. I call it a missed opportunity, because you were supposed to be able to find a problem with Ho and reject it, but you didn’t.

The chance of making a Type II error depends on a couple of things:

Sample size: If you have more data, you’re less likely to miss something that’s going on. For example, if a coin actually is unfair (and you don’t know it), flipping the coin only ten times may not reveal the problem, because results can go all over the place when the sample size is small. But if you flip the coin 1,000 times, you have a good chance of seeing a pattern that favors heads over tails or vice versa.

S FN

Actual value of the parameter: A Type II error is also related to how big the problem is that you’re trying to uncover. For example, suppose a company claims that the average delivery time for packages is 3.5 days. If the actual average delivery time is 5 days, you won’t have a very hard time detecting that with your sample (even a small sample). Evidence will mount up fast for rejecting Ho, which is exactly what you’re supposed to do in this situation. But if the actual average delivery time is 4.0 days, you have to do more work to actually detect the problem. Note that you never do know the actual value of a parameter, but you want to protect yourself against the different possibilities, which is why you consider them.

To reduce the chance of a Type II error, take a larger sample size. A greater sample size makes it easier to reject Ho, but increases the chance of a Type I error. Type I and Type II errors sit on opposite ends of a seesaw — as one goes up, the other goes down. To try to meet in the middle, choose a large sample size (the bigger, the better; see Figures 3-1 and 3-2) and a small a level (0.05 or less) for your hypothesis test.

Getting empowered by the power of a hypothesis test

Type II errors (see preceding section) show the downside of a hypothesis test. Statisticians, despite what many may think, actually try to look on the bright side once in a while, and this case is one of those times. Instead of looking at the chance of Missing A difference from Ho that actually is there, you can look at the chance of Detecting A difference that really is there. This detection is called the Power of a hypothesis test.

BER The power of a hypothesis test is one minus the probability of making a Type II error. So Power Is a number between 0 and 1 that represents the chance that you rejected Ho when Ho was false. (You can even sing about it "If Ho is false and you know it, clap your hands. . . .") Remember that power (just like Type II errors) depends on two elements: the sample size and the actual value of the parameter (see the preceding section for a description of these elements).

In the following sections, you discover what Power Means in statistics (not being one of the big wigs, mind you); you also find out how to quantify power by using a power curve.

Quantifying power with a power curve

The specific calculations for the power of a hypothesis test are beyond the scope of this book (so, take that sigh of relief), but computer programs and graphs are available online to show you what the power is for different hypothesis tests and various sample sizes (just type "power curve for the [blah blah blah] test" into an Internet search engine). These graphs are called Power curves For a hypothesis test. A power curve is a special kind of graph. It gives you an idea of how much of a difference from Ho you can detect with the sample size that you have. Because the precision of your test statistic increases as your sample size increases, sample size is directly related to power. But it also depends on how much of a difference from Ho you’re trying to detect. For example, if a package delivery company claims that its packages arrive in 2 days or less, do you want to blow the whistle if it’s actually 2.1 days? Or wait until it’s 3 days? You need a much larger sample size to detect the 2.1-days situation versus the 3-days situation just because of the precision level needed.

In Figure 3-1, you can see the power curve for a particular test of Ho: U, = 0 versus Ha: U, > 0. You can assume that O (the standard deviation of the population) is equal to two (I give you this value in each problem) and doesn’t change. I set the sample size at ten throughout.

The horizontal (x) Axis on the power curve shows a range of actual values of U . For example, you hypothesize that U Is equal to 0, but it may actually be 0.5, 1.0, 2.0, 3.0, or any other possible value. If U, Equals 0, then Ho is true, and the chance of detecting this (rejecting Ho) is equal to 0.05, the set value of A. You work from that baseline. So, on the graph in Figure 3-1, when X = 0, you get a y-value of 0.05.

1.0

Suppose that u, is actually 0.5, not 0, as you hypothesized. A computer tells you that the chance of rejecting Ho (what you’re supposed to do here) is 0.197 = 0.20, which is the power. So, you have about a 20 percent chance of detecting this difference with a sample size of ten. As you move to the right, away from zero on the horizontal (x) Axis, you can see that the power goes up, and the y-values get closer and closer to 1.0.

For example, if the actual value of u is 1.0, the difference from 0 is easier to detect than if it’s 0.50. In fact, the power at 1.0 is equal to 0.475 = 0.48, so you have almost a 50 percent chance of catching the difference from Ho in this case. And as the values of the mean increase, the power gets closer and closer to 1.0. Power never reaches 1.0, because statistics can never prove anything with 100 percent accuracy. But you can get close to 1.0 if the actual value is far enough from your hypothesis.

Controlling the sample size

You don’t have any control over what the actual value of the parameter is, though, because that number is unknown. So what do you have control over? The sample size. As the sample size increases, it becomes easier to detect a real difference from Ho.

Figure 3-2 shows the power curve with the same numbers as Figure 3-1, except for the sample size (n), Which is 100 instead of 10. Notice that the curve increases much more quickly and approaches 1.0 when the actual mean is 1.0, compared to your hypothesis of 0. You want to see this kind of curve — one that moves up quickly toward the value of 1.0, while the actual values of the parameter increase on the X-axis.

1.0

Ho: U = 0 versus Ha:

U > 0, for N = 100 and

Figure 3-2:

Curve for

Power

Power (n=100)

0.5 1.0 1.5 2.0 2.5 3.0 Actual Value of the Parameter

O = 2.

If you compare the power of your test when U, Is 1.0 for the N = 10 situation (in Figure 3-1) versus the N = 100 situation (in Figure 3-2), you see that the power increases from 0.475 to more than 0.999. Table 3-1 shows the different values of power for the N = 10 case versus the N = 100 case, when you test Ho: U = 0 versus Ha: U, > 0, assuming a value of O = 2.

You can find power curves for a variety of hypothesis tests under many different scenarios. Each has the same general look and feel to it: starting at the value of a when Ho is true, increasing in an S-shape as you move from left to right on the x-axis, and finally approaching the value of 1.0 at some point. Power curves with large sample sizes approach 1.0 faster than power curves with low sample sizes.

OjtXNG/ You can have too much power. For example, if you make the power curve for N = 10,000 and compare it to Figures 3-1 and 3-2, you can find that it’s practically at 1.0 already for any number other than 0.0 for the mean. In other words, the actual mean could be 0.05 and with your hypothesis Ho: u = 0.00, you would reject Ho, because of the huge sample size you’ve got. If you zoom in enough, you can always detect something, even if that something makes no practical difference. If the sample size is incredibly large, it can inflate power to the point where you can detect differences from Ho that are smaller than you really want, from a practical standpoint. Beware of surveys and experiments that have what appears to be an excessive sample size — for example, in the tens of thousands. They may be reporting "statistically significant" results that don’t mean diddly.

The power of a test plays a role in the manufacturing process. Manufacturers often have very strict specifications regarding the size, weight, and/or quality of their products. During the manufacturing process, manufacturers want to be able to detect deviations from these specifications, even small ones, so they must think about how much of a difference from Ho they want to detect, and then figure out the sample size they need in order to detect that difference when it appears. For example, if the candy bar is supposed to weight 2.0 ounces, the manufacturer may want to blow the whistle if the actual

Average weight shifts to, say, 2.5 ounces. Statisticians can work backwards in calculating the power and find the sample size they need to know to stop the process.

Medical scientists also think about power when they set up their studies (called clinical trials). Suppose they’re checking to see whether an antidepressant adversely affects blood pressure (as a side effect of taking the drug). Scientists need to be able to detect small differences in blood pressure, because for some patients, any change in blood pressure is important to note.