Why is it important to understand research methods? Simply put, if we want answers to certain questions, gathering information pertinent to those questions and reviewing the data can give us the best answers. Research, then, is no more than a process of acquiring information to answer questions. Education research helps answer questions we're all interested in, effective strategies for raising student achievement. Some people may be baffled by that the various forms of research—what they are, how they’re conducted, and what they say. With this guide, we set forth the basic concepts to unravel the mystery.
In a nutshell, research applies scientific thinking to questions. This means that the questions themselves must be constructed very carefully so that the data gathered can actually help to answer the question. Moreover, the data must be gathered and analyzed in the most objective manner possible. What we’re after is solid research, which provides answers with facts and data that can be trusted.
What types of research methods are used?
Research methods fall into two main categories—quantitative and qualitative—but at times they are combined to create a mixed-method research design.
Quantitative methods include:
Experimental research
Quasi-experimental research
Relational research (also known as cross-sectional or correlational)
Descriptive studies and surveys
Qualitative methods include:
Ethnographic research
Grounded theory research
Case studies
Mixed methods include:
Quantitative and qualitative methods that are combined in a mixed-method research design.
Quantitative methods
The goal of quantitative research is to determine the what; what effect does an intervention have on a specific group or what relationship exists between two factors. Quantitative methods rely on statistical analysis of numeric data (numbers) and are most often used to study large populations. More rigorous forms of quantitative research, such as experimental research, evaluate whether a specific intervention—or deliberate strategy—causes a predicted outcome (a good example is the STAR class size research).
While most researchers would like to have the rigor of experimental or quasi-experimental design, it is rarely possible in the social sciences, which includes the field of education. Students and classrooms aren’t easy to compare, no matter how strictly researchers try to match their sample populations. To further complicate the matter, if an experimental intervention is expected to provide benefits to students who receive it, withholding the intervention from some students purely for research purposes may be unethical.
Fortunately, there are other quantitative research methods, such as relational research and descriptive studies and surveys, that produce useful information for decision making even though they lack the rigor of experimental or quasi-experimental design. Relational research, for example, can shed light on whether two or more measurable factors are related (e.g., whether teacher credentials are related to student achievement) and the strength of the relationship. Quantitative methods also are useful for illustrating a situation with data points (e.g., graduation rates, test scores, or survey responses).
Experimental research
The most rigorous research method, experimental research, is considered the “gold standard” for investigating cause-and-effect relationships.
Experimental research in education compares two (or more) groups typically comprised of students or teachers. One group serves as the control group (the participants who do not get the intervention) and one (or more) serves as the experimental group (the participants who do get the intervention). To isolate the effect of an intervention (e.g., a new math curriculum), participants are randomly assigned into either the control or experimental group. If assignments are done correctly and the groups are large enough, the members of each group will have similar characteristics relevant to the study, such as socio-economic status, race and ethnicity, and age. The groups must be similar so that differences in outcomes between the groups can be attributed solely to the intervention and not to any differences in the characteristics of the participants.
Project STAR (Student/Teacher Achievement Ratio) is an example of an experimental research study in education. It was a monumental four-year study involving almost eighty schools from forty-two Tennessee school districts in which seven thousand K–3 students were randomly assigned to small or large classes. The results of this study were so rock-solid that its major conclusion—small classes in grades K–3 produced improved student achievement, especially for minority children—has become policy in many districts across the United States. (View the Center's review of research on class size.)
[back to top]
Quasi-experimental research
The high quality of experimental design is difficult to achieve in education because it often is not possible to randomly assign students to schools or students within schools to certain classes. Also, the population needs to be large enough to ensure the groups are similar. Therefore, educational researchers are more likely to use a quasi-experimental design to determine cause-and effect-relationships.
Just as in experimental design, quasi-experimental design compares a control group with an experimental group or groups. But, in a quasi-experimental design, students are not randomly assigned to groups. Nevertheless, the groups are carefully constructed so that their characteristics are similar. This method is not as strong as an experimental design because other factors can “confound” or cloud the effects of the intervention, but in most cases a well-executed quasi-experimental design will effectively isolate the effect of an intervention.
[back to top]
Relational research
Relational research design is also often referred to as correlational or cross-sectional research. The studies cited in the Center’s teacher quality research review are examples of relational research design because they show a correlation, or association, between factors (e.g., teacher credentials and student achievement). Identifying a correlation is not the same as finding cause and effect. For example, relational research on teacher quality finds that certified teachers have higher-achieving students than non-certified teachers. But determining whether certification itself causes higher student achievement requires an experimental or quasi-experimental research design. Nonetheless, knowing that teacher certification is positively correlated, or associated, with higher student achievement can be useful for informing policy making.
[back to top]
Descriptive studies and surveys
For you, and most people, descriptive studies and surveys are probably the most familiar type of education research. Descriptive studies and surveys do not evaluate causes and effects or relationships among factors. Instead, they report facts about a situation.
A good example of a descriptive study is the periodic release of scores from the National Assessment of Educational Progress (NAEP). NAEP reports the average scores of students at the time a specific test was administered, as well as changes in scores over time. However, NAEP simply reports scores, since they are not linked to other measures to determine what factors may have influenced them.
MetLife’s annual "The American Teacher" report is a good example of a survey. The American Teacher survey queries teachers on how they feel about various aspects of their jobs but does not link their responses to any measure, such as student performance.
It is important to take extra care when interpreting studies like surveys that are based on self-reported data. Respondents may report doing one thing but actually do another in practice. For example, a teacher may respond in a survey that his or her primary instructional method is cooperative learning, while observations of the teacher may show he spends most of the class time lecturing.
Relational research and descriptive studies and surveys provide snapshots of a particular situation, but neither offers evidence about cause. However, by knowing the current conditions, researchers can use other designs to come closer to identifying cause and effect.
[back to top]
Qualitative methods
Qualitative research is much different from its quantitative cousin. The goal here is to determine why an intervention or a factor has a certain effect. While quantitative methods collect, analyze, and report numbers, qualitative methods typically collect, analyze, and report words. These words come from participants who explain how the treatment affected them. Depending on the purpose of the study, a qualitative researcher will choose from a variety of methods (e.g., ethnography, grounded theory, case studies) to describe how a treatment affected the participants.
Because of the type of data used and the way it is collected, qualitative methods are usually considered more susceptible to bias and subjectivity than quantitative methods. Although researchers collect qualitative data themselves through observations, interviews, and document searches, a researcher’s involvement can subject the data to bias. Many qualitative studies minimize bias by creating protocols, or detailed rules, for how data is collected and analyzed, much the same as quantitative methods.
The choice of using a quantitative or qualitative method depends on what type of population you are studying and what kinds of questions you are asking; both are used for different types of populations and questions. While quantitative methods are usually preferred when studying large populations, qualitative methods typically focus on small populations such as a school or even a single student, because this method of data collection and analysis is time consuming and expensive. Because qualitative studies involve small numbers, the results typically cannot be generalized to represent a larger population. Nonetheless, they are in-depth in nature and, therefore, can provide valuable insight into different practices and capture nuances that can be missed in quantitative studies.
Ethnographic research
The primary purpose of ethnographic research is to describe, analyze, and interpret the shared patterns of behaviors and beliefs of a specific group over time. Ethnography (a type of field study) has its roots in anthropology and uses on-site observations of a population engaging in a certain activity, such as a group of students participating in a specific reading group. Often, it also uses in-depth interviews of participants and analysis of artifacts such as documents. For example, ethnographic studies may be conducted by professional development evaluators to understand teachers’ needs, viewpoints, experiences, and goals to help them design programs to better meet the teachers’ needs.
[back to top]
Grounded theory research
Grounded theory is an approach to identifying and explaining social phenomena—for example, how teachers integrate professional development into classroom practice. Grounded theory uses a particular method of coding and categorizing qualitative data (such as transcripts of interviews or documents) to capture the experiences of participants and generate theories (grounded in the data) about why the phenomena occur. Grounded theory is useful when existing theories do not address the problem or population of interest and when studying processes (such as how teachers implement a new curriculum).
[back to top]
Case studies
Of all the qualitative methods, case studies are probably the most common in education research. A case study examines in great detail a program, event, activity, or person fixed in time or location. Case studies examine the problem, the context, the issues, and the lessons learned within the case being studied. In education, a researcher may conduct a case study to examine what a particular urban high school is doing to prepare its students for college. In this case, the study would take an in depth approach to examining and explaining the policies, practices, and culture of the school as they relate to preparing students for college.
[back to top]
Mixed methods
To gain a better understanding of how an intervention is working, researchers may adopt a mixed-methods research design, using both quantitative and qualitative methods. Mixed methods allow researchers to take advantage of the strengths of each approach: a quantitative method to determine what effect the treatment has on a large population complemented by a qualitative study of a smaller sample of the population to gain a better sense of why the treatment is having the effect. Thus mixed methods help researchers and policymakers obtain more conclusive evidence on the effects of an intervention or the relationship between two factors.
For example, an education researcher may conduct a relational study to determine what characteristics (e.g., curriculum and instructional techniques) high schools with high ACT scores have in common. The researcher may then conduct a case study on one of the high schools to better understand not only how the curriculum and instructional techniques affect ACT scores but how the curriculum and instructional techniques are implemented at the school and classroom levels.
[back to top]
Putting it all together
Now that you understand the different types of research methods and what they actually measure you can become a more critical reader of education research. Know that when you hear about new research that proves teachers with a masters degree are better teachers, you can read the research and determine if the study actually proved teachers with masters degrees increase the achievement scores of their students, if there is a relationship between teachers with masters degrees and student achievement, or if parents reported in a survey that they believed teachers with masters degrees were more effective. Now that you are able to notice these differences you can feel confident that the decisions you make to improve your schools are informed and knowledgeable.
Resources
Educational research. (1995-2005). Pearson Education, Inc., publishing as Pearson Prentice Hall: Ethnographic designs.
North Central Regional Educational Laboratory. (1979): Ethnographic research.
Posted: June 20, 2007
