7:14 PM
Gee Marie Binag
Affiliation:
Title of the article: The Effectiveness of Interactive Distance Education
Technologies in K-12 Learning:
A Meta-Analysis
Journal: International Jl. of Educational Telecommunications (2001) 7(1), 73-88
Date of publication:
Review Article
Gee Marie S. Binag
INTRODUCTION
This article summarizes a quantitative synthesis of studies of the effectiveness of interactive distance education using videoconferencing and telecommunications for K-12 academic achievement. Effect sizes for 19 experimental and quasi-experimental studies including 929 student participants were analyzed across sample characteristics, study methods, learning environment, learner attributes, and technological characteristics.
The overall mean effect size was 0.147, a small positive effect in favor of distance education. Effect sizes were more positive for interactive distance education programs that combine an individualized approach with traditional
classroom instruction. Programs including instruction delivered through telecommunications, enhancement of
classroom learning, short duration, and small groups yielded larger effect sizes than programs using videoconferencing, primary instruction through distance, long duration, and large groups. Studies of distance education for all academic content areas except foreign language resulted in positive effect
sizes. This synthesis supports the use of interactive distance education to complement, enhance, and expand education options because distance education can be expected to result in achievement at least comparable to traditional instruction in most academic circumstances.
METHODS
Studies involving student academic achievement as a result of distance education at the K-12 level were assembled. Quasi-experimental studies were included with experimental studies because true experiments are done in artificial situations, while quasi-experiments usually occur in more realistic conditions. Quasi-experiments, although having lower internal validity, produce more realistic results and increased external validity (Carlberg, Johnson, Johnson, Maruyama, Kavale, Kulik, Kulik, Lysakowski, Pflaum, & Wlaberg, 1984). The setting for the studies was K-12 learning environments, with preference for studies using random sampling. Both published
and unpublished studies were included to avoid publishing bias. The dependent variable in this synthesis was student achievement in the academic content area measured by instruments appropriate to the individual study. The measure of student performance in the studies was indicated on achievement measures given at the end of the distance education period. Such measures included standardized tests and teacher/researcher
designed instruments. Traditional measures may not be effective in assessing the effectiveness of distance education systems that strengthen higher order skills. The treatment variables related to interactive distance education were categorized by duration, frequency, instructional design, and delivery system. Student characteristics including ability level and grade in school were also coded. Comparisons were made between distance education programs in which students used the technology for less than a half-year of instruction. The achievement data reported in each study contributed to the calculation of effect size for the meta-analysis. Cohen.s effect size is a standardized estimate of the difference in achievement between students learning
with interactive distance education and students learning with traditional methods. Effect sizes were computed as the difference between the control and experimental posttest mean scores divided by the average standard deviation. For studies in which a group of students was evaluated using multiple achievement measures, effect sizes were found for each measure, and then averaged. The average effect size was used in the overall effect size estimate of the meta-analysis. For studies in which more than one independent group of students was evaluated, an independent effect size was found
for each group and used in the meta-analysis separately. While different groups tested in one study were dependent with respect to study features and characteristics, the samples were independent. Based on the sample mentioned above it that’s not represent to general because the sample is below 50 %. The method used is purely survey method. The variable was identified and the assumption of having only one dependent variable is not violated. It is good that the author conducted the validity and reliability test. The observation of independence was not violated but the sample was not mentioned if the populations from which the samples were taken are normally distributed and as well as the homogeneity of variance.
RESULTS
The 19 independent effect sizes calculated in this synthesis compared students learning with interactive distance education technology with students learning with traditional classroom instruction only. The weighted mean effect size across all studies was 0.147 (SD = .69). The mean sample size for the studies was 48.9 (SD = 13.2). The mean grade level of the students was 8.3, with a standard deviation of 0.59. The average study measured student achievement after 14.7 weeks of distance education (SD =
2.6).In terms of the experimental design, two-thirds of the studies used a quasi-experimental design, and one-third were experimental. Only 10% of the studies made use of random assignment to groups, while 53% used convenience groups such as school classes. The remaining studies grouped students based on other criteria such as volunteering or parent selection. A full 74% of studies employed a pretest-posttest design, and 36% used posttest only. The most common form of measurement instrument in the studies was researcher-developed tests (32%), followed by teacher-developed and standardized (26% each), and publisher-developed tests (16%).
For analysis, the effect sizes were classified into 12 categories related to features of the studies. Table 1 displays the mean unbiased effect sizes for these categories. Effect sizes above Cohen.s threshold for .marginal. magnitude (below 0.45) occurred for studies of telecommunication, weekly use of distance education, use of distance education to supplement classroom instruction, mathematics and other subject areas (computer science and general elementary), and intermediate grade levels. Effect sizes above Cohen.s 0.80 cutoff of .large. effect size were found only for the other subjects, and for foreign language in a negative direction. Seven studies in this synthesis had heterogeneity values above 5.3, indicating that they were strikingly deviant.. To further investigate the significant heterogeneity, analysis of variance of effect sizes was used. An ANOVA was computed for levels of variables shown in Table 1. Each ANOVA resulted in no significant difference between levels of the variables at the alpha level of 0.01.
Even after the removal of the foreign language studies as outliers, the ANOVAs resulted in non-significant findings. The alpha values for the modified ANOVAs are presented in Table 2. The heterogeneity in the studies is unexplained. The optimal method
for combining effect size estimates with unexplained heterogeneity is the random effects model (Hedges et al., 1989). Random effects procedures treat between-study variations in effect sizes as random. The procedures begin with the estimation of between-study variance, used to determine the difference between the observed variance among effect sizes and the within study sampling variance. The sample variance for the 19 studies in the meta-analysis is 0.481. The between-study variance was found to be 0.413.
This variance was used in calculating confidence limits and in combining the effect size estimates in the random effects model. The effect size formula was weighted with both the within-study sampling error variance (standard error) and the variance.
Discussion
The meta-analysis presented here examined a sample of studies that met established inclusion criteria, including data from 929 learners. The analysis encompassed a broad view of K-12 interactive distance education by focusing on the range of subject areas, grade levels, and applications of distance education in use today. The questions of the overall effect on K-12 academic achievement of interactive distance education, and of the features of effective distance education systems were addressed through the use of
random effects effect size estimation and analysis of variance. The study design permitted review of a wide sample of research done on academic achievement of students learning with distance education. No significant differences were detected between grade levels, subject areas, ability levels, distance education technology, duration and frequency of use of distance education, or instructional design in relation to learning. However, the three foreign language studies made a close approach to significance with strong negative effect sizes.
Consequently, distance education can be expected to result in achievement at least comparable to traditional instruction in most academic circumstances. Educators planning implementations of distance education programs should expect no difference in academic performance as a result of the use of distance education. More importantly, when implemented with the same care as effective face-to-face instruction, distance education programs can be used to complement, enhance, and expand education options for students, at least at intermediate, middle, and upper grades levels.
Conclusion
In the area of foreign language instruction, great potential exists theoretically for linking students with native speakers and writers, but the results of the studies reviewed here indicate that distance education in foreign language should be studied closely. The three foreign language studies in this meta-analysis reported that students learning with distance education systems performed demonstrably lower than students learning in traditional classrooms. While foreign language options are needed in high schools, distance education courses for foreign language instruction should be evaluated
very carefully. No conclusion in the study only the implications
REFERENCES
Reference were clearly included.
0 comments:
Post a Comment