Research Tools and Techniques

The Research Process: Step 6 (Research Design for Experiments Part B) Lecture 23

Lecture Topics Covered Previously in the Last Lecture

•

Introduction to Experimental Designs

•

Control and Manipulation of Independent Variable

•

Techniques for Controlling Exogenous Variables

•

Internal and External Validity

What we are going to Cover in this Lecture

•

Factors Effecting the Internal Validity of Experiments

THE RESEARCH PROCESS

(1).

Observation The Broad Problem Area (3).

Problem Definition (4).

Theoretical Framework Variables Identification (5) Generation of Hypothesis

(6).

Scientific Research Design (2).

Preliminary Data Gathering Interviews and Library Search (9).

Report Writing (10).

Report Presentation (11).

Managerial Decision Making (7).

Data Collection and Analysis (8) Deduction

THE ELEMENTS OF RESEARCH DESIGN 1. Purpose of Study 2. Type of Investigation

•

Exploratory

•

Descriptive

•

Hypothesis Testing

•

Case Study Establishing: Causal Relationship or Co-relational 3. Extent of Researcher Interference

•

Minimal

•

Moderate

•

Excessive 4. Study Setting 5. Measurement & Measures

•

Contrived

•

Non-Contrived

•

Operational Definition

•

Scaling

•

Categorizing

•

Coding 6.Unit of Analysis (Population to be studied) Individuals Dyads Groups Organizations Machines etc.

7. Sampling Design Probability Non-probability Sample Size (n) 8. Time Horizon One-Shot (Cross-Sectional) or Longitudinal 10. Test Application Feel for Data Goodness of Data 9. Data Collection Methods Hypotheses Testing Observation Interviews Questionnaire Physical Measurement

FACTORS AFFECTING INTERNAL VALIDITY

Even the best designed lab studies could be influenced by factors that might affect the internal validity of the lab experiment. These possible confounding threats pose a threat to internal validity. The seven major threats to internal validity are: 1.

HISTORY EFFECTS 2.

MATURATION EFFECTS 3.

TESTING EFFECTS 4.

INSTRUMENTATION EFFECTS 5.

SELECTION EFFECTS 6.

STATISTICAL REGRESSION EFFECTS 7.

MORTALITY EFFECTS

History Effects

Certain events or factors that would have an impact on the independent variable - dependent variable relationship might unexpectedly occur while the experiment is in progress, and this history of events would confound the cause and effect relationship between the two variables, thus affecting the internal validity.

Example

Independent variable Dependent variable Sales Sales Promotion Dairy Farmer’s Advertisement Uncontrolled variable

Maturation Effects

Cause and effect inferences can also be contaminated by the effects of the passage of time another uncontrollable variable.

Such contamination is called maturation effects. Examples of maturation effects processes could include growing older, getting tired, feeling hungry, and getting bored.

Maturation effects

Example Independent variable

Enhanced Technology Gaining experience and doing the job faster

Dependent variable

Efficiency Increase Maturation effects

Testing Effects

Frequently, to test the effects of a treatment, subjects are given

pretest

(e.g., a short questionnaire eliciting their feelings and attitudes). That is, first a measure of the dependent variable is taken (the pretest), then the treatment is given, and after that a second test, called the

posttest

, is administered. The difference between the posttest and pretest is then attributed to the treatment. However, the very fact that respondents were exposed to the pretest might influence their response on the posttest, which would adversely impact on internal validity.

Example of Testing Effect: Testing Effect = 27-24 = 3 We have to minus this score from posttest values of Group A and B to adjust for testing effects.

Instrumentation Effects:

Instrumentation effects are another source of threat to internal validity. These effects might arise because of a change in the measurement instrument between pretest and posttest, and not because of the treatment’s differential impact at the end.

Selection Bias Effects:

The threat to internal validity could also come from improper or unmatched selection of subjects for the experimental and control groups.

Statistical Regression Effects:

Statistical regression occurs when members chosen for the experimental group have extreme scores on the dependent variable to begin with.

Mortality

Another confounding factor on the cause and effect relationship is the mortality or attrition of the members in the experimental or control group or both, as the experiment progresses. When the group composition changes overtime across the groups, comparison between the groups become difficult, because those who dropped out of the experiment may confound the result.

How to Avoid the Confounding Factors

The

shorter the time span

of the experiments, the less the chances of encountering the history, maturation, and mortality effects.

Experiments lasting

an hour or two

do not usually encounter many of these problems. It is only when experiments take place over an extended period (e.g.,

several months

) that the possibility of encountering more of the confounding factors increases.

Summary

•

Factors Effecting the Internal Validity of Experiments