Learn About: Evaluating Performance | Common Core
Home > Policies > Consumer’s guide to education research > Common definitions for technical research terms
| print Print


Common definitions for technical research terms

Intervention (Treatment)
 Definition: A policy, program or process whose effect is being studied.

Example:  In a study to determine whether the CPE Reading Curriculum improves fourth grade reading scores the CPE Reading Curriculum would be the intervention that is being studied.

Variable
Definition: A factor, trait, or condition that can exist in differing amounts or types. An experiment usually has three kinds of variables: Dependent, independent, and controlled.

Example: In a study to determine the effect of IQ on SAT scores, the SAT scores is the dependent variable because SAT scores may change based on the student’s IQ. The IQ is the independent variable; students with low, average, and high IQ scores take the test. If another factor such as a student’s gender may influence SAT scores, gender could be added as a control variable to better isolate the effect of IQ on SAT scores.

Dependent Variable
Definition: The “thing” that the researcher believes will be influenced.
 
Example: In a study to determine the effect of IQ on SAT scores the dependent variable is SAT scores. Do people with higher IQs have higher SAT scores? Do people with lower IQs have lower SAT scores? In other words, the SAT score could change depending on a person’s IQ.

Independent Variable
Definition: The “thing” that is used to predict the effect on the dependent variable. As the researcher changes this thing he or she observes what happens to the dependent variable.

Example: In a study to determine the effect IQ has on SAT scores, the independent variable is IQ. Is it more likely a person who has an above average IQ will score higher on the SAT than someone who has a lower IQ? In other words, does an increase in a person’s IQ score increase their SAT score?

Controlling Variable
Definition: An outside factor included in the study meant to minimize its effect on the dependent variable. These are the variables researchers include to ensure that any change in the dependent variable is due to the factor(s) being studied.

Example: To isolate the effect IQ has on SAT scores the researcher will include other variables that would likely influence SAT scores besides IQ. To ensure that results are due only to the student IQ and not any other student characteristics, the researcher may include variables to represent the student’s socioeconomic status or whether the student is an English Language Lerner which have been shown to influence SAT scores.

Random Sample
Definition: A part (subset) of a larger population that is being studied where the subjects in the subset (sample) are selected at random. Each person in the larger population has an equal chance of being selected to be part to the sample.

Example: The American Teacher Survey did not ask every teacher in the United States to take part in their survey. Instead they selected a sample of teachers from around the country at random to complete the survey.

Control Group
Definition: Subjects in the study who do not receive the intervention being studied. The subjects will have similar characteristics as the subjects in the experimental group except for the fact they did not receive the intervention.

Example: To determine whether the change in fourth grade reading scores are attributable to the CPE Reading Curriculum and not to other factors, the fourth grade reading scores of students who receive the CPE Reading Curriculum are compared to students who did not receive the CPE Reading Curriculum. In this case the students who did not receive the CPE Reading Curriculum are the students in the control group.

Experimental Group
Definition: Subjects in the study who do receive the intervention being studied. The subjects will have similar characteristics as the subjects in the control group except for the fact they did receive the intervention.

Example: To determine whether the change in fourth grade reading scores are attributable to the CPE Reading Curriculum and not to other factors, the fourth grade reading scores of students who receive the CPE Reading Curriculum are compared to students who did not receive in the CPE Reading Curriculum. In this case the students who received the CPE Reading Curriculum are the students in the experimental group.

Confounding Factors
Definition: Outside factors that are not being studied or controlled for that influence the dependent variable.

Example:  Factors that could influence fourth grade reading scores besides the CPE Reading Curriculum are confounding factors. If they are not included as a control variable in the study, confounding factors could be student characteristics such as socioeconomic status, teacher effectiveness or even the amount of sleep the students got the previous night.   

Reliability
 Definition:  Scores from measuring variables that are stable and consistent.

Example: If the outcome of the IQ/SAT study is reliable, the researcher will get similar results on repeated trials (using the same variables).

External Validity
Definition: Conclusions that generalize to a larger population than just the sample population being studied.

Example: The American Teacher Survey: This survey was given to a random sample of teachers around the country, but if every teacher in the United States was surveyed the results would be similar.

Internal Validity
Definition: Changes in the independent variable caused the changes in the dependent variable.

Example: The change in weight as measured by the bathroom scale is due to a change in eating habits and exercise since the last time you stepped on the scale and not due to using a different scale.

Type I Error
Definition: The hypothesis is incorrectly ‘rejected’ which gives a false positive result. 

Example: A jury rejects the hypothesis that a defendant is innocent even though he did not commit the crime. The defendant is convicted of a crime he did not commit.

Type II Error
Definition: The hypothesis is incorrectly ‘not rejected’ which gives a false negative result.

Example: The jury accepts (fails to reject) the hypothesis that a defendant is innocent even though he did commit the crime. The defendant is found not guilty of a crime he actually did commit.

Statistical Significance
Definition: Statistical evidence that a difference did not likely happen by chance (typically ninety-five percent likely). However, it does not refer to the size or the importance of the difference.

 Example: Russian eighth graders outscored U.S. eighth graders in math 508 to 504; however, the difference in scores is not significantly different. Why? If the test was given again it would be quite possible that U.S. eighth graders would score the same or better than their Russian counterparts. However, Singapore’s score of 605 is significantly higher than the United States score, meaning it is likely (ninety-five percent likely) that if the test was taken over and over Singapore would continually outscore the United States.


This piece was written by Jim Hull, Center for Public Education Policy Analyst.

Posted: June 20, 2007

© 2007 Center for Public Education


Home > Policies > Consumer’s guide to education research > Common definitions for technical research terms