3 Study Designs

3.1 Measurement Through Study

There are two primary categories of study designs (figure 3.1), and the primary difference between the two is whether or not we control the study factors.

In observational studies, we do not manipulate any study factors and do not randomize. We observe what happens in a particular group of people—for example, factory workers, children in a preschool, or patients seen in a clinic for primary care. When we say manipulate, we do not mean that we make things up. What we do mean is that we can set the parameters of the study (i.e., control study factors) such as who gets the exposure (e.g., a medication) or who does not (e.g., the placebo or standard of care) in order to see causal effects, if they exist between an exposure and an outcome. When we do this, it is called an experimental study.

In experimental studies, we do control factors and often use randomization to create fairly perfect conditions to see the influence of an exposure on an outcome. For example, we might enroll some cancer patients in a trial to see how a new medication works, or we might test how different the health is in communities with fluoridated water compared to those without fluoridated water. Randomization means that we use some sort of objective criteria to put study participants in whatever groups we establish for our study. For example, we may have one group that gets a sugar pill (i.e., a placebo), one group that gets the standard of care, and one group that gets the drug we are testing. In this scenario, we might assign patients to a group based on the order in which they come to the clinic. We might also choose to assign all patients a number and randomly allocate them to a group using a random number generator. No matter the assignation, we use an objective method to put patients in a study group. This helps us reduce the chance of a biased study result.

As you consider each study design, pay attention to these details:

• Number of observations made
• Directionality of exposure
• Data collection methods
• Timing of data collection
• Unit of observation
• Availability of subjects

All study designs are not created to be equal, but each has a specific purpose. Each study design helps us move closer to an understanding of causality (section 1.2). As you move from in-vitro studies to meta-analyses (figure 3.2), you can see that the evidence each study design provides becomes stronger. It does not mean the designs at the top are weaker or useless, they just provide a different type of evidence. Though there is a general consensus about how valid or strong the evidence is from any particular type of study, the evidence from each design builds on the others.

Figure 3.2: How much can we rely on the answers from your study when determining the etiology (cause) of disease or conditions?

3.2 Study Designs

Beyond the measures presented in chapter 2, epidemiologic studies allow us to create and compare measures across individuals and groups. Because we are really examining the relationship between factors, exposures, and outcomes, we call the majority of these measures of association. Figures 3.3 and 3.4 lay out the different types of studies and some overview details about them. The rest of this chapter is dedicated to explaining and discussing temporality, measures of association, measures of effect, and sampling.

Figure 3.3 describes five types of observational study designs: case series, ecologic studies, cross-sectional studies, case-control studies, and cohort studies. From left to right, the designs are listed in order of the strength of their evidence (weakest to strongest).

Details	Observational designs (in order of strength)
	Case series	Ecologic	Cross-sectional	Case control	Cohort
Also known as	Case study	Correlational study	Prevalence study	Case-referent study	Follow-up study
Descriptive or analytic?	Descriptive	Primarily descriptive	Descriptive	Analytic	Analytic
Can temporality be determined?	No	No	No	No	Yes
Unit of observation	Individual	Group	Individual	Individual	Individual
Major uses of the design/What is the design good for?	Describe interesting cases of disease, injury, or other health issues.	Test or develop etiologic hypotheses (hypotheses about the population). Create hypotheses about causation or identify methods of prevention.	Present the burden of disease, injury, or other health issues (morbidity or mortality). Generate hypotheses. Supports planning health services.	Outbreaks, studying diseases of low prevalence, testing hypotheses.	Studying etiology, providing direct measures of risk, testing hypotheses, showing temporal relationships, looking at rare exposures.
What measures or measures of association are used with the design?	None	Correlation, chi-square	Prevalence estimates, prevalence rate ratio (AKA prevalence relative risk)	Odds ratio	Relative risk (most often), odds ratio (sometimes)
Formula	N/A	Depends on the design	(Uses prevalence not incidence like the RR)	This cross-product ratio is the derivative of:	(Direct Measurement of Risk) (Indirect Measurement of Risk) This cross-product ratio is the derivative of:
Advantages	Able to share information with others to then develop hypotheses or plan studies.	Quick and easy to conduct. Inexpensive.	Sometimes quick and relatively easy to conduct (if using secondary vs primary data). Inexpensive.	Great with rare outcomes. Cheap, efficient. Can be completed rather quickly.	Great with rare exposures. Can show temporal relationships between exposure and disease retrospectively, prospectively, or a combination of the two.
Disadvantages	Not enough details to make decisions for treatment.	Ecologic fallacy. Imprecise measurement.	Not good for rare diseases. Shouldn’t be used for etiologic studies.	Not good for rare exposures. Cannot provide a direct measure of risk. Recall bias.	Can cost a lot of money and take a lot of time to complete. Difficult to execute. Selection bias. Not good for rare diseases.
Example	We had five patients with hallucinations after taking NSAID A that is not known for causing hallucinations. We will describe their clinical presentation here.	The rate of premature births decreased in West Virginia when Medicaid was expanded.	The prevalence of high adiposity increased in New Mexico during the COVID-19 pandemic amongst people 60 to 65 years of age that were retired.	People that worked in grocery stores during the first four months of the COVID-19 pandemic were more likely to be hospitalized with COVID-19 than the general population.	Soldiers that entered boot camp in 1980 and stayed in the military for 20 years had a higher risk of osteoarthritis at 60 than soldiers that entered at the same time and stayed in the military less than 7 years.
PRR: Prevalence Rate Ratio OR: Odds Ratio RR: Relative Risk

Figure 3.3: Epidemiological study designs.

Figure 3.4 describes two types of experimental study designs: community trials and clinical trials.

Figure 3.4: Community trials and clinical trials.

One tool that is used to calculate a number of epidemiological measures is the 2×2 table (figure 3.5). This table is repeated many times in the following text. The primary columns represent the presence (e.g., outcome +) or absence (e.g., outcome –) of the outcome or event of interest (e.g., ACL injury). The primary rows represent the presence (e.g., exposed +) or absence (e.g., exposed –) of the exposure of interest (e.g., being hit). In this example table we also show the total number of those exposed and the total number of those with the outcome. These totals are sometimes needed for different calculations.

The letters A, B, C, and D represent the number of observations that meet different criteria.

A = The count of observations that have both the outcome and the exposure

B = The count of observations that have the exposure but not the outcome

C = The count of observations that have the outcome but not the exposure

D = The count of observations that have neither the outcome nor the exposure

Using this same logic, the sum of A and B gives us the total number of observations with the exposure and the sum of C and D gives us the total number of observations without the exposure. The sum of A and C gives us the total number of observations with the outcome and the sum of B and D gives us the total number of observations without the outcome.

3.3 Temporality

In order to establish causality, it is important to be able to establish a temporal—or time—relationship between factors. As seen in figures 3.3 and 3.4, all studies are not good at measuring temporality. All studies also are not intended to measure temporality. Studies such as cohort studies or RCTs are the most used when trying to answer questions such as “Did the chicken come first or did the egg?” [Answer to that question!^[1]]. Figure 3.6 displays at what point in time data collection for different studies starts, the directionality of data collection, and the minimum number of time points captured by the study. For example, in a cross-sectional study all data is captured at the same point in time (the present day) and shows what is happening right now. Cross-sectional studies can be thought of as a snapshot in time, and the time period could vary from something such as a patient’s last visit to patient outcomes over the last year. Because all questions get asked at once and typically involve recalling events from the past, we cannot determine temporality. A cross-sectional study can, however, give us a great perspective about the prevalence of a particular health issue. A retrospective cohort study, on the other hand, starts in the present day but looks backwards to capture information from the past. Oftentimes, there can be confusion about the difference between a retrospective cohort study and a case-control study. Even in a retrospective cohort study, our goal is to determine if a known exposure leads to a disease, such as when we are trying to determine whether playing football leads to developing chronic traumatic encephalopathy (CTE). We have information about the population both before and after diagnosis, which allows us to observe whether the exposure led to the disease. In a case-control study, we are looking to find what exposures could have led to known disease. It is most often used when we need an answer quickly, such as in an outbreak; for example, what caused an outbreak of ringworm in wrestlers. While we start with people we know have the outcome, we have to determine what possible exposures are of interest and then narrow down which one had the higher probability of causing the outcome. We cannot definitively determine temporality. One of the main differences between a prospective cohort study and a randomized control trial is that instead of seeing the natural course of exposure (e.g., choice to smoke or not smoke), we instead randomly allocate participants into our study groups—we choose for them. This means we may give one group the standard of care for an ankle injury and give the other group a new cryotherapy plus standard of care to see the effect the cryotherapy has on the outcome of the injury.

3.4 Observational Study Designs

Ecological studies use group summary measures for exposure and outcome rather than measures about individual people. We would use this type of study to compare populations, such as the rate of disease in France compared to the United States or the rate of disease in the United States in 1950 compared to 2000. Because this type of study compares groups, we cannot assume that the results from this study should apply to individuals. It also means that studies we do using data on individuals should not be assumed to apply to groups. If we were to do that, we would be committing the ecologic fallacy.

Example: Ecologic fallacy

If we find out that the rate of heat-related illnesses during track and field is high in states in the southern United States, that does not automatically mean that individuals in the southern United States have higher risks of heat-related illness than people living other places. It just means that on a group basis, their rates are higher. If we find out that 80 of 100 individual people with heat illness at a track meet are from the southern United States, it does not mean that 80 percent of all heat-related illnesses occur in the southern United States. If we do make these incorrect assumptions, we have just been guilty of the ecological fallacy. We need to do a better job being correct in our inferences, or the meaning we assign to the data that we see. It would be a fallacy to assume that people from the southern United States will experience heat illness based on the presentation of data.

In this example, we see that there is a positive relationship between the number of Professional Australian Football matches played and the number of concussions that were diagnosed.^[2] However, we would not want to assume that every player with more matches will have any concussions. As we can see, at least some players with a high number of matches have no concussions. We also can see that some players with few matches have a higher number of concussions than players with more matches. We can only infer what we see, which is the probability (chance) of an increased risk of concussion with more matches.

In a 2006 TED Talk, statistics expert and physician Hans Rosling provided an excellent example of the importance of ecological studies. You can see it in the first 7 minutes of this video.^[3]

Cross-sectional studies measure the prevalence of disease and of exposures (i.e., risk factors) at one point in time. Cross-sectional studies are also known as prevalence studies. When we think about what is being measured in a cross-sectional study, we should think about taking a photo or a snapshot: it is a photo of you right now, not what you looked like in the past or what you will look like in the future. We do not know when an exposure happened or when a disease started, we just know they are present right now.

Case-control studies are used to find out whether a particular exposure could have been the source or cause of a disease, particularly in urgent health situations. We start by identifying who already has the disease (cases), then we find a set of people who are like the cases in every respect except they do not have disease. These are called controls. We ask these cases and controls questions about their past exposures. Because we start with people who are diseased, case-control studies are great when you are interested in studying people who have rare diseases. This design is explored more in the next section on Outbreak Investigations.

Cohort studies start with a group of individuals based on their exposure status. They are used to find out whether a particular disease comes after a particular exposure or development of a risk factor. If someone does not have the chance of being exposed, they would not be a good selection for a cohort study. You want everyone to have the potential of getting the outcome because of the exposure. Because of this, cohort studies are great when you’re interested in studying people who have rare exposures. Once the exposure status is identified, researchers then identify whether or not the subjects have the outcome of interest already. If they do, they would be removed from a prospective study because our goal is to see if the outcome happens after the exposure, and if they already have both, how would we know? There are roughly three types of cohort studies: prospective, retrospective, and historical. Every cohort study has at least two data collection points and they do not overlap. Prospective means we are setting up the study today and actively following forward into the future. Retrospective means we are setting up the study today but we are looking at information that was previously gathered. So how are retrospective cohort studies different from case-control studies? (See figure 3.6.)

In our next example, we explore how we might approach hospital-acquired infections after anterior cruciate ligament (ACL) reconstruction surgery compared to ACL repair surgery with a cohort study or a case-control study.

3.5 Measures of Association

As noted in section 3.2, we often use a 2x2 table to analyze data from an epidemiological study (figure 3.5). This table is repeated many times in the following text.

Beware! While one side of the table above has exposure (or risk factors) and the other side has outcomes (such as disease), everyone does not set their table up the same way (see figure 3.10). Before doing any calculations with data from a 2x2 table, pay attention to how it is set up. All examples in this book use the version showing exposure in rows and outcome in columns.

When we calculate our measures of association, we refer to the needed components by referring to different boxes of our 2x2 table using letters.

• A – Has the outcome and is exposed
• B – Does not have the outcome and is exposed
• C – Has the outcome and is not exposed
• D – Does not have the outcome and is not exposed

Examples of the measures of association are the odds ratio and the relative risk. A measure used in cross-sectional studies is the prevalence rate ratio.

Figure 3.11: The variety of measures that can be calculated from different study designs.

3.5.1 Odds Ratio

The only measure of association that can be calculated in a case-control study is the odds ratio (OR) [the probability of being exposed among cases compared to the probability of being exposed among controls]. This particular odds ratio is referred to as the odds ratio of exposure.

 is the ratio of those with the outcome and exposure (A) to those with the outcome but no exposure (C). In other words, what proportion of those that have the outcome have the exposure?

is the ratio of those without the outcome but with the exposure (B) to those without the outcome and with no exposure (D). In other words, what proportion of those that do not have the outcome have the exposure?

The shortcut, , is called the cross-product ratio.

The resulting answer is a direct comparison of the ratio of the proportion of those with the exposure who have the outcome to proportion of those with the exposure without the outcome. If this number is equal to 1 (roughly, 0.9 to 1.1), there is no difference in the probability of having the exposure between the outcome groups. If this number is greater than 1 (roughly, greater than 1.1), the group with the outcome is more likely to have the exposure than the group without the outcome. If this number is less than 1 (roughly, less than 0.9), the group with the outcome is less likely to have the exposure than the group without the outcome.

Always be specific when drawing comparisons. Just saying, for example, “Cases are 3.2 times more likely to have the exposure” is an incomplete interpretation of the OR. “Cases are 3.2 times more likely to have the exposure compared to controls” is clear about what you are comparing the odds of cases to. This applies to relative risk interpretations as well.

We can also calculate an OR (of exposure or disease) in other study designs, including cross-sectional, cohort, and RCTs. How it gets interpreted in these cases is often different than how we interpret it in a case-control based on the nature of the study and the difference in the full calculation.

3.5.2 Relative Risk

The primary measure of association that is calculated in a cohort study is the relative risk (the risk or incidence of the outcome in the exposed compared to the risk or incidence of the outcome in the unexposed).

is the incidence (or risk) of disease (A) in the exposed (A+B). is the incidence (or risk) of disease (C) in the unexposed (C+D). The relative risk is the ratio of the incidence of disease in the exposed to the incidence of disease in the unexposed. In other words, how is the risk of disease in the exposed different than the risk of disease in the unexposed?

If this number is equal to 1 (roughly, 0.9 to 1.1), there is no difference in the risk between exposure groups. If this number is greater than 1 (roughly, greater than 1.1), the group with the exposure is more likely to have the disease than the group without the exposure. If this number is less than 1 (roughly, less than 0.9), the group with the exposure is less likely to have the disease than the group without the exposure.

Calculating the odds ratio in a cohort study means that we are calculating the odds ratio of disease. This is calculated differently than the odds ratio of exposure that we calculate in a case-control study (see above). While both formulas result in the cross-product ratio, because they were calculated differently we interpret them differently. Remember that cohort studies are to identify the risk of disease in the exposed compared to the risk of disease in the unexposed.

3.5.3 Prevalence Rate Ratio

As noted earlier, prevalence is:

In cross-sectional studies, a common measure of association we calculate is the prevalence rate ratio. While the name is a misnomer (prevalence is a proportion, not a rate), it still uses a familiar formula to compare things like the prevalence between either two separate groups (e.g., injury prevalence in Oklahoma compared to injury prevalence in Texas) or the same group at different points in time (e.g., injury prevalence in Virginia in 2015 compared to injury prevalence in Virginia in 2020).

3.6 Outbreak Investigations

An outbreak is the occurrence of disease in an area at a level exceeding the normally expected number of cases. An outbreak technically differs from an epidemic because an outbreak occurs in a more limited geographic area. Epidemics are declared by country-specific health bodies (e.g., the US Centers for Disease Control and Prevention). A disease is endemic if it is occurring at a level expected. It is normally occurring in that place. An epidemic becomes a pandemic when the World Health Organization decides it has become one. A pandemic is an epidemic that is spread over multiple countries or continents. Epidemics and pandemics can have variable time patterns, as seen in section 2.4.

One of the most common ways that outbreaks are identified is through clinicians paying attention to changes in what they are treating and who they are treating. Figure 3.14 displays the 11 steps to solving an outbreak.^[6]

Step 1: Establish the existence of an outbreak.

Step 2: Verify the diagnosis.

Before we expend too many resources and too much time, we want to be sure that we are actually observing an outbreak. Things that could look like an outbreak but are not:

• Misdiagnosis/false report:
  • False positive (specificity)
  • Laboratory error
  • Change in case definition
  • Incorrect time or place
  • False report
• Changes in:
  • Awareness
  • Record keeping
  • Observation
  • Population composition

Sometimes we improve our surveillance systems or other tracking methods and pick up more cases because we are doing a better job. This does not mean we actually have more cases, we just are doing a better job at seeing them. Other times, we simply make mistakes in identification that could make it appear like we have more cases. Besides these things, we start calculating our prevalence and incidence, as well as if there are reasonable explanations for changes in these numbers, to determine whether to proceed. We should calculate prevalence if we need to know the total burden of the problem. We should calculate incidence if we are trying to find the risk of developing a disease in a given time. Sometimes we need to do both. The most important part of steps 1 and 2 is that we must verify that the diagnosis we think is the problem is in fact the correct diagnosis. For example, if we think that we are having an outbreak of meningitis A, we should confirm that all of the people who are sick actually have meningitis A.

Our goal is to identify all of the following:

• Individual: Who is affected?
• Place: Where are they affected?
• Time: When did this start or change?
• Connections: What factors are related?

Moving forward in an outbreak investigation is all about what we think, what we know, and what we can prove.

Step 3: Construct a working case definition.

Taking this information, we move into Step 3 and create a working case definition. Many times, this definition stays in flux. Using our case definition, we identify the individual cases, controls, and possible/suspected cases.

Case definitions include a standard set of criteria used to determine if an individual should be classified as a case. Depending on the condition or disease in question, case definitions may already be established. In other situations, this needs to be developed as the investigation progresses. Sometimes the disease or condition in question is required to be reported to the health department or the Centers for Disease Control and Prevention. Nationally notifiable conditions are reported to the National Notifiable Diseases Surveillance System.^[7] Each state also has a separate list of notifiable conditions. For example, Virginia’s conditions are reported to the Virginia Department of Health and the State Board of Health.^[8]

A case definition usually includes both:

• Clinical criteria and/or lab test
• Restrictions by time, place, and/or person

When developing the case definition, we tend to emphasize sensitivity (to identify all possible cases) over specificity (to identify only “true” cases). Part of this is because it is better to err with caution and include too many people than not all cases, especially in the beginning of the investigation. Sensitivity and specificity are discussed in more detail in chapter 4. 

Example: Case definition

In figure 3.15, we see the diagnostic criteria for hemophagocytic lymphohistiocytosis (HLH), a rare syndrome of excessive immune response. In order to be considered someone who has HLH, a person must have most but not all diagnostic criteria. However, sometimes not all patients will have all tests that are required to be considered a case. If they meet several criteria, they are instead what is known as a possible or probable case.

Step 4: Find cases systematically and record information.

Once we have a case definition, we can then work to find all cases (Step 4). We must do this in a systematic fashion and record data on any cases or potential cases we find. We make every effort to find cases that occurred earlier than when we first realized something might be amiss. We use a line listing to organize the data about our cases. In figure 3.16, we see an example of a line listing from an anthrax outbreak. Each row corresponds to a different case, and we include all the possible details relevant to the case status and demographic information.

Figure 3.16: Example line listing.

Step 5: Perform descriptive epidemiology.

In Step 5, we perform descriptive epidemiology on the data we have gathered from clinical records, questionnaires, interviews, and so on. Just as with several other conditions, if it is a suspected foodborne outbreak, we can use tools from CDC^[9] to gather all the pertinent details. We are specifically looking for patterns and associations between risk factors and disease. All the information we will compare is in our line listing. Our measures of association and effect are very useful at this step.

If it is a foodborne outbreak, instead of calculating incidence as we learned before, we usually reframe risk as the attack rate (figure 3.17).

We can use this to find what percentage of those at risk are actually ill.

We interpret the attack rate as the percentage of those with the exposure that are sick. In the above example, 70.6 percent of those that ate salad are sick. We would compare attack rates to determine which exposures deserve more attention as possible causes.

Step 6: Develop hypotheses.

Step 7: Evaluate hypotheses epidemiologically.

In Steps 6 and 7, we form hypotheses based on our existing data and test them. Among other things, our hypotheses may relate to:

• Cause of the outbreak
• Risk factors for disease
• Risk factors for infection
• Intervention to stop spread: quarantine and vaccinate
• Treatment of affected individuals

Step 8: As necessary, reconsider, refine, and reevaluate hypotheses.

Step 9: Compare and reconcile with laboratory and/or environmental studies.

Step 10: Implement control and prevention measures.

Step 11: Initiate or maintain surveillance findings.

Our final steps of an outbreak investigation are to continue refining our hypotheses, compiling more data to support or refute our hypotheses, controlling the outbreak, and performing surveillance to keep an eye on the problem. Sometimes we find the source of the problem but cannot just “solve” it. The cost of treating the problem, the cost of the intervention to fix the problem, and the existence of other alternatives all play into our decision about what to do. Controlling the problem might include vaccine development and distribution, it might be stopping access to a dangerous substance, or recalling food products.

In the case of some problems, like COVID-19 or sickle cell disease, we initiate and maintain an ongoing systematic data collection system. This is known as disease surveillance. The US Centers for Disease Control and Prevention reports on the surveillance of notifiable diseases in both the Morbidity and Mortality Weekly Report^[10] (MMWR) and CDC WONDER.^[11]

3.7 Measures of Effect

When we are comparing results from our study, we compare the measures that we found. Often, we look at:

1. 1. How big is the difference between groups or individuals with and without a particular risk factor? (Magnitude of effect; ratio, difference)
   2. Could the difference we found be just due to chance variation? (Significance of effect; p values)
   3. How certain are we of the size of the effect? (Precision of, or uncertainty in, estimate; confidence intervals)

We specifically discuss A in this book. More details on B and C can be found in many books on biostatistics.

We already looked at whether one factor was associated with (or related to) another factor or whether an outcome was associated with an exposure. But in the grand scheme of things, what does that really mean for the population we are focused on?

Measures of effect (how big is the effect of an exposure or risk factor) include the attributable risk (attributable fraction) and the population attributable risk (population attributable fraction). Sometimes epidemiologists and others will refer to these as more measures of association rather than separating them into their own category. Because they are very interrelated, it does not matter whether you refer to them as measures of effect or measures of association but rather when and how to use them. When we’re focused on population health, looking at relative differences like the odds ratio or relative risk is extremely useful to decide where we want to make a difference and what factors we should spend our time and energy on. But when we’re trying to figure out how to approach the problem at the individual level (for your patient for example), absolute measures can be much more useful.

3.7.1 Attributable Risk

Of everyone that has the exposure, how much of the occurrence of the disease is due to the exposure in question? That’s the attributable risk. In other words, what’s the difference in how much disease we could already expect without the exposure (risk in the unexposed) and how much disease we have if the exposure is present (the risk in the exposed)? This could also be called the risk difference. The risk in the unexposed is often referred to as the baseline risk.

Example: Attributable risk

In our example, the risk of disease in the exposed group is (0.625). The risk of disease in the unexposed group is (0.500). The attributable risk is the difference between these two numbers:

AR = 0.625 - 0.500 = 0.125

The number we get—0.125—is called an absolute number that tells us how different the risk is for the exposed than the risk for the unexposed. For improved understanding, we tend to make it relative by turning it into a percentage.

The AR percent tells us what percent of the risk of disease in the exposed group is attributable to the exposure itself. In this case, 20 percent of the risk of an ankle sprain in those that play racquet sports is due to those people playing racquet sports.

When we use attributable risk to see how well a clinical intervention (e.g., a vaccination) performs, we know that the relative risk correlates to how well the intervention will perform. If the relative risk is < 1 (lower risk of the outcome due to the intervention), then the AR will be negative. This is what happens if the intervention works! If the relative risk is > 1 (higher risk of the outcome due to the intervention), then the AR will be positive. This is what happens when the intervention is not that great.

3.7.2 Measures Especially Important in Clinical Medicine

If we can figure out the attributable risk, we can also identify the relative risk reduction, the absolute risk reduction, the number needed to treat, and the number needed to harm.

Measure	Equation	Which way to round
Relative risk reduction (RRR)		-
Absolute risk reduction (ARR)		Neither. Take the absolute value.
Number needed to treat (NNT)		Up
Number needed to harm (NNH)		Down

Figure 3.20: Summary of important clinical medicine measures.

The relative risk reduction: If there is a reduction in the risk of the outcome when a particular intervention is used, how much of that is due to the intervention compared to the control?

The absolute risk reduction (also known as the risk difference): While the ARR and the AR can both be referred to as the risk difference, there is a distinct difference between the two. AR refers to the difference in risk for the outcome among the exposed due to the exposure itself. The ARR is broader and refers to the difference in risk for the outcome in the group that did not have the intervention and the risk for the outcome in the group that did have the intervention.

The number needed to treat: How many patients have to be treated in order to make a difference for one patient?

Always round the result of the NNT formula up.

The number needed to harm: How many patients have to be exposed to a risk factor in order to harm one patient?

Always round the result of the NNH formula down.

These four measures (NNH, NNT, ARR, and RRR) are very important in clinical medicine.^[13] Figure 3.22 provides an example of how to calculate these statistics.

Example: NNH, NNT, ARR, RRR

Female athletes have a greater risk for ACL injury than male athletes for a variety of reasons. Some 70 percent of ACL injuries in female athletes are due to reasons other than coming in contact with an object or a person. Basketball players are at risk for ACL injury due to the movements they make during play. A study was conducted by Omi et al.^[14] to identify the effectiveness of an intervention that aimed to alter risk factors like landing mechanics, muscular strength, postural control, and hip joint control.

The graphic shown (figure 3 from the manuscript^[15]) shows the following rates:

• Incidence rate of noncontact ACL injury for 309 athletes who did not receive an intervention (the initial observation period) [Total of 13 injuries]
• Incidence rate for 268 athletes who received Intervention I (players used a ball to simulate basketball rebounding motions and worked to have appropriate knee alignment during landing) [Total of five injuries]
• Incidence rate for 268 athletes who received Intervention II (an upgrade to Intervention I that included [a] application of a flexible band at the thigh level in all jump-landing maneuvers except for contact jump to reduce hip adduction, hip internal rotation, and knee valgus; [b] implementation of hip external rotation strengthening in addition to hip abduction strengthening; and [c] enhancement in quality of balance exercises such as cross-leg hop forward and side hop) [Total of three injuries]
• Combined incidence rate for Interventions I and II

If you need more numbers to follow along, download the manuscript17 from PubMed. Remember that rounding differently and using the rates per 1000 athlete-exposures (aka, person-time) as opposed to incidence per total in the group results in differences in numbers during calculations.

For the purpose of our example, we’ll refer only to the Observation, Intervention I, and Intervention II parts of the graphic.

• Risk of noncontact ACL injury during Observation = 0.21
• Risk of noncontact ACL injury during Intervention I = 0.09
• Risk of noncontact ACL injury during Intervention II = 0.08

How much of the risk of noncontact ACL injury during Intervention I is due to participating in Intervention I?

If we are comparing Intervention I to the Observation (which can be considered baseline since no intervention has taken place):

Relative risk = = 0.43

Athletes who participate in Intervention I have 0.43 times the risk of a noncontact ACL injury compared to athletes at baseline. Intervention I seems to reduce the risk of noncontact ACL injury.

Attributable risk (risk difference) = 0.09 – 0.21 = -0.13

Our risk difference is negative. The risk of a noncontact ACL injury is reduced by 13 percent in those who participate in Intervention I.

Relative Risk Reduction = 1 – 0.43 = 0.57

The intervention reduces the risk of noncontact ACL injuries by 57 percent.

Absolute Risk Reduction = – = 0.04 – 0.02 = 0.025

The intervention reduces the risk of noncontact ACL injury 2.5 percent compared to baseline.

Number Needed to Treat = = = 40

To prevent a noncontact ACL injury in just 1 athlete, 40 athletes must participate in the intervention.

Number Needed to Harm = N/A [There is a positive NNT, so there is no NNH for Intervention I]

If we are comparing Intervention II to the Observation:

Relative risk = = 0.38

Athletes who participate in Intervention II have 0.38 times the risk of a noncontact ACL injury compared to athletes at baseline. Intervention II seems to reduce the risk of noncontact ACL injury.

Attributable risk (risk difference) = 0.08 – 0.21 = -0.14

Our risk difference is negative. The risk of a noncontact ACL injury is reduced by 14 percent in those who participate in Intervention II.

Relative Risk Reduction = 1 – 0.38 = 0.62

The intervention reduces the risk of noncontact ACL injuries by 62 percent.

Absolute Risk Reduction = – = 0.03

The intervention reduces the risk of noncontact ACL injury 3 percent compared to baseline.

Number Needed to Treat = 1/0.03 = 34

To prevent a noncontact ACL injury in just 1 athlete, 34 athletes must participate in the intervention.

Number Needed to Harm = N/A [There is a positive NNT, so there is no NNH for Intervention II]

Both the relative risk of noncontact ACL injury after Intervention I and after Intervention II are less than half the risk of noncontact ACL injury when no intervention was used. Intervention II had a slight improvement over Intervention I for how much it reduced the risk of noncontact ACL injury when comparing the absolute risk reductions vs baseline (3 percent vs 2.5 percent).

Attributable risk and its derivatives are important when we are considering a specific population, but often when we develop medications or create other interventions we are considering how much impact they will have on the overall burden of a health problem. Extending our example (figure 3.22), how many noncontact ACL injuries could we have eliminated from the entire population if we eliminated them from women’s basketball? The answer to this question is the population attributable risk. The population attributable risk is the absolute level of risk of the outcome in the whole population due to the exposure. The difference between this and the attributable risk is that this applies to the risk reduction even in those that do not have the exposure. One way to calculate this is:

3.8 Reporting Results of Epidemiologic and Clinical Studies

There are various standards for the reporting of study results and methods. Figure 3.24 provides an example list of different standards. You can find additional standards for various disciplines and different types of studies at the EQUATOR network website.^[20]

Figure 3.24: Standards for study design and reporting.

In addition to reporting study results, it is also normal and helpful to report on how studies were designed and implemented. This reporting of methods helps others better understand all the work that goes into obtaining results as well as potential roadblocks to watch out for when designing a similar study to expand what is known about a topic. The CARE Consortium published a journal article in 2017 about how they built a national study of concussion in service academy students and collegiate athletes with the Department of Defense.^[21]

Figure Descriptions

Figure References