IT_Efficacy

octype html public "-//w3c//dtd html 4.0 transitional//en" IT_Efficacy[Instructivism] [ Constructivism][Information Processing] [Problem-Based Learning] [Critical Thinking] [Instructional Technology]
[History of Instructional Technology] [Effectiveness of IT] [Evaluation][Development] [Student Evaluation] [Objectives] [Systems] [Asynchronous] [Activities]
[Home]

AIL 601
The Effectiveness of Instructional Technology
George E. Marsh II, Professor

A great deal of money has been spent on computers and software for schools and there is considerable worry about the lack of training for teachers to use these tools effectively. Hundreds of evaluations of computers and software have been conducted but the ones that get the most notice are those that are well funded, include many subjects, and are said to be well controlled. It also helps if the sponsor or the research group has a good pedigree. With great fanfare and press releases, results of these studies are issued in encapsulated form and "experts" are asked to make comments. Since computers were first used for instruction in the 1960s, people have been asking, "Are computers good for learning?"
Although Socrates opposed books, it does not occur to people today to ask, "Are books good for learning?" Most people accept that people learn from books, and some groups and organizations go to great lengths to keep certain kinds of textbooks out of classrooms and certain kinds of literature out of libraries for fear of what people may learn from them. Research about the effectiveness of books and other materials is not likely to rise to the level of public attention in the media. The exception to this has been the "look say versus phonics" in the 1950s and the current iteration known as "whole language versus phonics," but these have been about methods of teaching reading because of the assumption that books are important in learning. Computers are coming under great scrutiny because many opponents believe they are expensive toys that do not contribute to learning, such as Neil Postman, Cliff Stoll, and Oppenhiemer.
There is indisputable proof that computers are highly effective for instruction. Research over the last 30 years has been preoccupied with comparisons of computerized instruction to conventional teaching, but people keep asking, "Can computers teach?" While many schools own computers, they are often in labs or the media center, extraneous to daily classroom routines. Until recently very few schools actually used computers extensively, and most still do not. But as we approach the time when most classrooms may have computers and access to the Internet, the doubters and faultfinders are intensifying their criticisms of expenditures for computers and questioning the effectiveness of computers in education based on the concept that computers must stand apart and compete with teachers.
In The End of Education,Neil Postman implies that the most enthusiastic idolaters seeking the "god of Technology" are educators, which is a curious notion if one examines the data on teacher training and attitudes toward technology. He fears that children may lose something they now have if we rely upon computer technology as the primary authority. There are many worries. Will students be dazzled by calculation rather than human judgment? Will speed define intelligence? What will be displaced by the computer that will be forever lost? What will be made impossible? Will there be time for reflection? The computer is here to stay, like it or not. Of course, similar fears were expressed in the past, most notably when books were first made and again when the printing press made books available to everyone. There are echoes in concerns expressed about the arrival of the cinema, radio, and television. Children are not impressed with calculation and speed, but older professors are, having slogged away on mainframes and electric (not electronic) calculators. What once took days or weeks now takes but a minute. No. Children are not impressed by speed, because they have never known anything else. The young cannot lament what they have never known.
Rather than seeing technology as a false God, perhaps we should see it for what it is and try to understand how we may use it more effectively. Research in instructional technology falls into the following general categories (Clark & Sugrue, (1995):

Behavioral: effects of media on performance
Cognitive: effect of media or method on cognitive processing and performance and/or interactions of individual differences and method on cognitive processes and performance
Attitudinal: interactions of individual differences and method or medium on cognitive processing and performance
Economic: effects of medium on cost of/time for instruction.

Instructional technology used to mean film, slide and overhead projectors, and then computerized instruction. The recent emergence of the WWW as a delivery system over the Internet has revolutionized instructional technology. Electronic mail, videoconferencing, multimedia, and the ability to share files easily makes it possible for augmentation of traditional classroom instruction and for extending the boundaries of the classroom, even reconceptualizing what instruction and the classroom really mean.

CAI was descended from "programmed instruction" used by B.F.Skinner. In programmed instruction the content to be learned is broken down into small pieces for learning. There are two methods, linear and branching. In the linear methods, the student is given a problem and provided with immediate corrective feedback and must try again. In branching the student may receive several alternative paths with different examples that ultimately return to the main path. The key is that the student is not permitted to advance until passing each step. CAI programs have received significant criticism from the beginning, including questions about quality and assertions that they are boring and are often referred to as "drill and practice" as a demeaning description. While CAI has been found to be highly effective (Niemiec, Blackwell, and Walberg, 1986), its demise may be attributed, in part, to the rise of constructivism. In fact, CAI requires the student to follow precisely the steps laid out by the programmer, which is contrary to constructivist precepts.

Until very recently the technology imposed significant restrictions on what could be accomplished, technically and economically, except for organizations with virtually unlimited budgets such as the military. The first serious research done with computers and CAI was in the funded projects of TICCIT and PLATO. Virtually all of the early work was done on mainframe computers and accessed through terminals. This was a very expensive way to do business. A slightly different approach that preceded the computers was the "Talking Typewriter" or the "Edison Responsive Environment," developed by a man named Omar Khyam Moore. In 1964 this was advertised as the world's first and most sophisticated complete learning system. Even before the current level of inflation, these machines were outrageously expensive at about $30K each. An electric typewriter, a slide projector, and tape recorder, and a primitive form of floppy disk (containing a sheet with programming instructions) were contained in a booth. There was virtually no software, so those who used it also had to make their own materials and program it, which included the need to shoot photographs for illustrations. (Your professor had four of these in 1967, where he learned his first lessons in software development and IT).

The persistent comments from those who question the effectiveness of computer-based instruction deal with (a) achievement, (b) quality, (c) time and (d) cost. Quality is not often clearly defined, although it seems to be related to achievement.

Achievement

A considerable amount of work has been done with meta-analysis, or comparing the effect sizes of different studies. An effect size is essentially the mean of the experimental group minus the control group mean, divided by the standard deviation of the control group or, more often, the estimate of the standard deviation derived from a table:

The effect size, also called Cohen's d', is becoming more important in research reports today. An effect size is equivalent to a Z score of a standard normal distribution. Thus, an effect size of 0 is equal to the 50th percentile, an effect size of 1.0 is equal to the 84th percentile, and an effect size of 2.0 is at the 98th percentile. To illustrate, if one group of subjects has a score of 550 and another a score of 500, and the standard deviation is 100, the effect size is (550-500/100) .5, which is equivalent to the 69th percentile. The greater the effect size, the farther apart the means. The effect size can be converted so that overlap between samples can be compared in terms of percentiles or areas under the normal curve, which can be visually displayed with thisapplet by David Lane. The greater the differences between the group means, the less the overlap among members of two groups.

Kulik (1994) examined over 500 studies and reported an overall average effect size of .35, suggesting an overall increase of 50th to 64th percentile performance after introduction of the CBI. Results for videodisc are higher, the effect size of .69 showing an improvement from 50th to 75th percentile performance (Fletcher, 1996). Meta-analyses of studies at the elementary school and secondary school show a significant advantage for computer-assisted instruction. For a review of research covering the 1980s, see the report of the Northwest Regional Educational Laboratory.

The videodisc was the first comprehensive multimedia format in the new electronic options, but it is now obsolete. It seems likely the Java and Flash technologies, along with streaming video and greater bandwidth will make it possible for the same kind of functionality on the WWW.

Quality

In meta-analyses of 200 research reports that have compared computerized instruction with traditional classroom lecture at all levels, learning was higher with computer-based instruction. (Bosco, 1986; Fletcher, 1989, 1990; Khalili & Shashaani, 1994; Kulik, Bangert, & Williams, 1983; Kulik, Kulik, & Bangert-Drowns, 1985; Kulik, Kulik, & Cohen, 1980; Kulik, Kulik, & Schwalb, 1986; Schmidt, Weinstein, Niemic, & Walberg, 1985)

One promising application of multimedia is the effect of multisensory learning or multisource stimulation. Students who get verbal and graphical/visual information achieve more than students who receive only verbal presentation, as in the classroom lecture. In fact, any combination of multimedia (text and audio, text and picture) is superior to any single channel presentation (Nugent, 1982; Mayer & Anderson, 1991, 1992). Even static materials, such as text with pictures, is more effective than text without pictures, and unrelated graphics do not provide a learning advantage (Levie & Lentz, 1982). The major problem with text is that the more pictures and graphics, the greater the cost and volume. The ability of multimedia to deliver virtually any kind and number of graphics is a decided advantage for technology.

Time

Kulik, Bangert, and Williams (1983) reported in one study there was an 88% savings in learning time. Fletcher (1990) reported an average time reduction of 31% in studies of interactive videodisc instruction applied in higher education; Johnston and Fletcher (1995) found time reductions of 28% across 23 studies of CBI applied in military training. Orlansky and String (1977) reported reductions in time to reach instructional objectives averaged about 30% for CBI in the military; Kulik, 1994) reported reductions of 34% in higher education and 24% in adult education (Kulik, 1994). Fletcher (1990) reported an average effect size of 0.39 for 29 studies with interactive videodisc to achieve knowledge outcomes, and an average effect size of 0.40 for 21 studies to achieve skilled performance outcomes. Use of simulated equipment is compared to use of actual equipment, with training time held constant and success in maintaining or operating actual equipment used as the final performance measure. Average effect size across all studies of this sort has been found to be about .40 (an increase from 50th to 66th percentile performance), for both knowledge outcomes and skilled performance. It is clear that research has shown that students at all levels and across many subject fields can reach objectives in about a third less time. In K-12 education, and most college programs for that matter, time is not a serious consideration because the semester calendar dictates the sequence.

Cost

There have been very few studies of costs in K-12 education. Fletcher, Hawley, and Piele (1990) found the costs to achieve a month of grade placement gain in total mathematics scores, using computer-based instruction, to be $20 compared to $33 for conventional instruction for third graders, and $17 compared to $27 using conventional instruction for fifth graders. These costs include both initial investment and operating and support costs bundled together. They involved the placement of 4-5 microcomputers in classrooms -- rather than in computer laboratories -- and used commercially available, off the shelf courseware.

Summing up the Research

While new concepts about using technology in classroom are emerging as quickly as there are new innovations, much of the debate about technology in classrooms is based on the former view of computers in classrooms for drill. For all types of research in all fields, computerized instruction is effective for many kinds of learning tasks, cognitive and perceptual motor. The cost-benefit ratio in training, corporate and military is impressive. Schools have unique problems determining their costs, so cost studies in K-12 are confusing.

Computer-based training (CBT), interactive video disk (IVD), and simulation save time and money. Many universities, based on the semester, are not concerned with saving time. As long as people drive to campus, they are not interested in saving money in the same way that a corporation might. Corporations have to bear the costs of transportation and per diem for their workers, universities do not take such costs of their students into consideration.

Kulik (1994) analyzed over 500 separate research studies of computer-based instruction, including drill,
tutorial, and integrated learning systems. He reported:

Students who perform at the 64th percentile on achievement tests compared to the 50th percentile for control groups.
Students reach criterion faster.
Students have positive attitudes toward subjects or classes where computers are used.

The Software & Information Industry Association (SPA) supported a review of 176 studies conducted between 1990-95. The results were similar to the early research, students experienced positive achievement in all major subject areas.
The SPA extended the report for 1990-1997 covering 219 studies. According to this study, with essentially the same results as before.

Sivin-Kachala (1998) reviewed 219 research studies from 1990 to 1997 and reported that students in technology-rich classrooms:

Demonstrated high achievement effects in all major subject areas.
There was increased achievement in from preschool through higher education for both special education and non-disabled students.

Mann (1999) examined 950 fifth-grade students' achievement from 18 elementary schools in West Virginia:

Test scores increase proportionately with the use of computers.
Greater access to technology was correlated with positive attitudes towards the technology by teachers and students.
Teacher training correlated with greater achievement gains.
Test scores rose the most for lower achieving students.
There was no difference on achievement or computer use by gender.

Harold Wenglinsky (1998) reported:

Simulation and higher order thinking software increased math scores up to 15 weeks above grade level.
Teachers who had professional development on computers showed gains in math scores of up to 13 weeks above grade level.
Training of teachers and use of computers correlated with academic achievement in mathematics for both fourth-and eighth-grade students.

Writing to Read. The Educational Testing Service evaluated results with 10,000 kindergarten and first grade students at 21 different locations and reported that students progressed faster than the national norm and increased by approximately 15 percentile points.

Higher Order Thinking Skills (HOTS). Used with Title I programs, the program uses software said to be based on the Socratic method and claims that average gains between fall and spring exceeded 15 percentile points on standardized reading and math tests. The results for spring-to-spring showed reading gains 67% higher than national averages, and math gains 123% higher. The program also claims that many students have been reclassified from "remedial" to "gifted."

Educational Testing Service. The recent "Does it Compute" study that attracted attention in the press actually reports that, among eighth graders, when students used computers for more complex mathematics like simulations, which measure changing variables, or applications, where data are manipulated and analyzed, gains of more than a third of an academic year were registered. For fourth graders using computers for mathematical games, the gain was a tenth of an academic year. Curiously, because of some cautions about training of teachers and concerns about higher order thinking skills for 4th graders (How much can you expect of 4th graders?), this study has been misinterpreted in the press to indicate that computers should not be used and may be a waste of resources!

The SPA makes good points, two of which are summarized here:

The specific student population, student access, software design, role of the educator, grouping of students, and training of the educator influence the level of effectiveness of educational technology.
Effectiveness of educational technology depends on a match between the goals of instruction, characteristics of the learners, the design of the software and technology implementation decisions made by educators.

Technology is making a significant positive impact on education. As indicated, computers do not have positive effects in all areas, but should this be expected? Results can vary from one class to another, one school to another, and from one classroom to another. Recent research by the SPA makes the claim that there are positive effects in all areas, if students are immersed in "technology-rich" environments. So far, most schools have not immersed students with computers, especially schools that corral their computers in labs.

Corley, (1997) makes these points:

Technology is particularly valuable in improving student writing.
There is are improvements in attendance and dropout rates.
Critical thinking or cooperative learning--are more difficult to evaluate.
There is preliminary evidence that the research skills of students improve with online access and this results in significantly higher learning measures.
Some studies show increased motivation and fewer disciplinary incidents.
Students engaged in long-term technology classrooms show greater independence and self-pacing.
Standardized achievement tests might not measure the types of changes in students that educational technology reformers are looking for.

Citing Roy Pea, now the director of SRI's Center for Technology in Learning in Menlo Park, California, Corley notes that the social contexts "of how technology is used are crucial to understanding how technology might influence teaching and learning. Educational technologies cannot be effective by themselves." Finally, Corley notes:

...we are confronted by a number of methodological and practical issues. First, we need to remember that technology is only one component of an instructional activity. Assessments of the impact of technology are really assessments of instruction enabled by technology, and the outcomes are highly dependent on the quality of the implementation of the instructional design.

Continuing Research

In a recent study by the Educational Testing Service, called "Does it Compute," what has been described as the first large-scale examination of how computers affect the learning of mathematics in American classrooms, shows that when used by trained teachers in middle schools they can significantly enhance academic performance. Their value in elementary school for math is said to be more limited or when used primarily for drills and practice at either level. If students used computers for long periods of time, computers may be counterproductive. Some have interpreted this to mean that computers are ineffectual, which was not the finding of the study.

In fact, the results of the study said school districts should spend more money on computers in middle schools than elementary schools, but they should focus attention on professional development for teachers to make sure they know how to use the computers effectively. In this regard, they may have gone beyond their data by making recommendations based on facts not in evidence.

It took the scores of thousands of fourth and eighth graders on the 1996 math portion of the National Assessment of Educational Progress (N.A.E.P).
Among eighth graders, it found that when students used computers for more complex mathematics like simulations, which measure changing variables, or applications, where data are manipulated and analyzed, gains of more than a third of an academic year were registered.
For fourth graders using computers for mathematical games, the gain was a tenth of an academic year.
The study only controlled for socioeconomic background, teacher preparedness, class size and school environment.

Christopher J. Dede said:

It seems that policies to promote computer access in schools have succeeded in eliminating inequities yet inequities in teacher preparedness and what is taught using computers remains. The report measured teacher preparedness by whether teachers had received training in computer use but it did not distinguish between a weekend seminar and a semester-long course (Bronner, 1998).

A great many other questions could be asked. Why is it reported as one of the first large-scale studies? It is not. Was there different content, especially at the elementary level? Was there more or less study time after school or in different parts of the school day? Were there different instructors involved? Much CAI software is based on behavioral principles and does not employ instruction that attempts to stimulate higher cognitive processes, which may have been a significant factor in the 4th grade findings. Is there a link between a simple drill-and-practice arithmetic program and a complex, higher order thinking skills that can be shown at the 4th grade level? Are computer games used at the 4th grade level, as indicated in the ETS study, a proper use of computers? It is illogical to assume that this single study will answer all the questions about what is right and good to do with computers in classrooms, even for mathematics instruction?

We need studies that avoid confounding variables, but the ETS study has not achieved this purpose. As Hagler and Knowlton (1987) said over a decade ago:

Those interested in teaching and learning cannot confuse the message with the medium. Any attempt to compare media, such as comparing traditional instructional methods with CAI, would have as a dependent variable some measure of learning. All message variables would have to remain constant, with medium being the only variable allowed to change (p. 86).

Elementary teachers do not have the same training and certification in math, nor use of computers for math instruction. Secondary teachers often have a credential in math. What was the emphasis in the elementary curriculum? Even assuming that with the best software and best teacher preparation, if gain scores in math were not significantly different, would this be a reason for not purchasing and using computers in elementary schools? Computers are used for many things, not just math instruction--reading, writing, research, and many other applications. Niemiec, Blackwell, and Walberg (1986) studied this problem and reported that---compared to peer tutoring, adult tutoring, increasing the length of the school day, and decreasing class size---an average CAI program produces the greatest gains per $100 of instructional expenditure.

Many authors have published reviews of research about educational computing spanning the last three decades (Kulik, Bangert, & Williams, 1983; Kulik, Kulik, & Bangert-Drowns, 1984; Kulik, 1994). It is curious that one study, like the well funded one just conducted by ETS, should catch such widespread media attention. Perhaps we really do have a problem in critical thinking among the media who review research they may not fully understand and the consumers of such information who seem to be willing to base policy decisions on one study. For example, Bronner (1998) refers to "Does it Compute?" as the first large-scale study in computers. Nothing could be further from the truth.

Despite the curious interpretations of some writers in the popular press, the effectiveness of educational technology is clear--the technology works. Over and over again, research has been conducted that tries to find differences in computer and regular class instruction expressed as traditional measures of student achievement or gain scores on tests. If one were to break down the studies, rarely does the computer lose; it either ties or beats the competition, a record to be envied by most football coaches! Fletcher sums up the current status this way:

The results discussed above are neither comprehensive nor conclusive, but they suggest that applications of technology in education and training are more effective and less costly than our current practice. . .What specifically appears to be next is the very difficult work of learning how to integrate our new technological opportunities into classroom practice, with all the implications for changing student and instructor roles that must be assessed and instituted. . . The point of these comments, has simply been to suggest that the return on this investment will be worthwhile.

Changing the Focus of Research

Of more interest and probably greater significance are studies that focus on the context of instruction rather than methods of delivery. In other words, the "horse race" type of research looking for winners in achievement gains may give way to a new dynamic. Teamwork, online cooperative projects, and other social factors can be employed along with the technology. The combination of technology with collaboration results in good outcomes (Means, 1993). The true revolution in technology may be the ability to get students to assume personal responsibility for their own learning. While there is no question that computers can achieve better achievement gains across a wide range of subject matter and at all levels through graduate school, we have been asking the same questions for 30 years and keep getting the same answers. We need to ask different questions.

As instruction has been transferred to the Internet there are research investigators concentrating on very specific aspects rather than outcomes. The amount of print on a screen, the font size, optimization of reading of continuous text, effects of small display windows, skimming from computer screens, and even "performance" assessments of the few video streams used in instruction. Whether or not these elements will be important factors in web-based instruction is hard to predict, but it is curious that such instruction is held to different standards than traditional instruction.

There have been few studies comparing conventional instruction and web-based asynchronous instruction. Thus far the results of such studies are similar to other distance education comparisons with equal learning outcomes for DE and comparison groups (Dobrin, 1999; Navarro & Shoemaker, 1999; Smeaton & Keogh, 1999; and Trinkle, 1999; Wright, 1999). As Russell (1999) has concluded, equal instructional outcomes does not mean inferior outcomes, only equal. While it may be desirable for some to hope that that distance education is superior to conventional instruction, it must be remembered that distance education is, for the most part, a form of duplication of the conventional classroom (Marsh, McFadden, and Price).

A practical and theoretical problem facing the instructional designer is deciding which instructional design theory to use. The most common are based on objectivism (behavioral and information processing models) or constructivist principles. Selecting one or another procedure implies that we know how people learn. When computer instruction was difficult and costly to develop, it was easy to use objectives, branching and feedback loops with various types of reinforcement. The Internet and new technology tools make multimedia formats easy to develop with either philosophy, so it is possible to abandon the objectivist approach. While it is safe to say that much of instructional design for the classroom and the computer have used behavioral principles, there is considerable confusion about how to use constructivism to adapt instructional materials. How to employ constructivist principles in design is problematic. Some writers maintain that the instructional design reveals the theories and philosophies of the designer, giving insight to how the designer sees the world, because it is impossible to know how the learner models reality.

While new concepts about using technology in classroom are emerging as quickly as there are new innovations, much of the debate about technology in classrooms is based on the former view of computers in classrooms for drill. In an Atlantic Monthly article entitled The Computer Delusion, Oppenheimer asserts there is no good evidence that most uses of computers significantly improve teaching and learning. There are many studies to show that Oppenheimer is wrong, but this is no longer the most important concern. Wright and Marsh (2000) have expanded on this concept:

While it is well known that teachers, as a group, are ill-prepared to use computers effectively, this will not deter the evaluations that are sure to come. Critics, as in the past, are likely to ignore the research that has already been done and to say that technology is unproven. What they will look for correlations between standardized test scores and technology use, and comparisons of computerized classrooms with those that have no computers. And where positive results are found to the advantage of technology, fault may be found in the design or the results will be questioned on the grounds that they cannot be applied to different populations. Instructional technology is at a turning point because for the first time in history it will be fully installed in most educational institutions and will be held accountable in terms of the Skinnerian traditions. If technology were to continue being used for drill in classrooms, then the status of technology would be secure. But it is not. The prevailing philosophy in education is now constructivism while the evaluation procedures remain based on behavioral outcomes. It is no longer possible, given the way computers are distributed, to consider computers in isolation, to measure narrow effects, such as gain scores in reading or math, or to consider the frequency of computer use as an important variable.

There is ample evidence that computer technology can improve achievement in very specific programs directed at procedural skills, something we have known since the 1980s. Nonetheless, the comparative studies continue, although the computer and related devices and its connection to the Internet provide a way for broader applications beyond drill programs. Najjar reports that multimedia can produce unique effects for specific kinds of learning. We need to know more about this rather than continuing the global comparisons based on gain scores. The emerging WWW and new technologies make many kinds of options available for teaching and learning. It would be interesting for research in the future to consider other questions instead of "achievement-gain horse races" that have been the primary research focus:

How much can multimedia reduce the need for processing textual information?
When are graphics most useful for illustrating knowledge structures and constructing mental models?
Are there differential effects for various learning styles under different conditions and different displays?
Are different presentations more effective at different points in learning, such as video first or last?
Is animation superior to static displays, and textual or tabular data representations better for retention and decisions?
Can technology improve rather than impair socialization?
Can technology aid in a shift of emphasis from faculty productivity to student productivity, from faculty teaching styles to student learning styles?

References

Aitken, G. (1997). Music technology for the 21^st century. In L. Enghagen (Ed.) Technology and higher education (pp.57-74). Washington, DC: National Education Association.

Allen, W. H. (1974). Media stimulus and types of learning. In H. Hitchens (Ed.), Audiovisual instruction (pp. 7-12). Washington, DC: Association for Educational Communications and Technology.

Anderson, J. R. (1983a). The architecture of cognition. Cambridge, Massachusetts: Harvard University Press.

Anderson, J. R. (1983b). Retrieval of information from long-term memory. Science, 220, 25-30.

Anderson, J. R. (1993). Rules of the mind. Hillsdale, NJ: Lawrence Erlbaum Associates.

Anderson, J. R., & Reder, L. M. (1979). An elaborative processing explanation of depth of processing. In L. S.

Arens, Y. (1992). Multimedia presentation planning as an extension of text planning. In R. Dale, E. Hovy, D.

Arens, Y., Miller, L., Shapiro, S. C., & Sondheimer, N. K. (1988). Automatic construction of user-interfacedisplays. In Proceedings of the 7th AAAI (American Association for Artificial Intelligence) conference (pp. 808-813). San Mateo, CA: Morgan Kaufman.

Baddeley, A. D. (1983). Working memory. Philosophical Transactions of the Royal Society of London B, 302, 311-324.

Baddeley, A. D. (1988). The uses of working memory. In P. R. Solomon, G. R. Goethals, C. M. Kelley, & B. R. Stephens (Eds.), Memory: Interdisciplinary approaches. New York: Springer-Verlag.

Baggett, P. (1979). Structurally equivalent stories in movie and text and the effect of the medium on recall. Journal of Verbal Learning and Verbal Behavior, 18, 333-356.

Bahrick, H. P., & Gharrity, K. (1976). Interaction among pictorial components in the recall of picture captions. Journal of Experimental Psychology: Human Learning and Memory, 2, 103-111.

Baker, E. L., Gearhart, M., & Herman, J. L. (1994). Evaluating the apple classrooms of tomorrow.

Bangert-Drowns, R.L., Kulik, J.A. and Kulik, C-L.C. (1985). Effectiveness of Computer-Based Education in Secondary Schools. Journal of Computer-Based Instruction. 12(3). 59-68.

Barrow, L. C., & Westley, B. H. (1959). Comparative teaching effectiveness of radio and television. Audio Visual Communication Review, 7, 14-23.

Barry, M., & Runyan, G. (1995). A review of distance-learning studies in the U. S. military.The American Journal of Distance Education, 9 (3), 37-47.

Bartram, D. J. (1980). Comprehending spatial information: The relative efficiency of different methods of presenting information about bus routes. Journal of Applied Psychology, 65, 103-110.

Beagles-Roos, J., & Gat, I. (1983). Specific impact of radio and television on children's story comprehension. Journal of Educational Psychology, 75, 128-137.

Bell, D., & Johnson, P. (1992). Support for the authors of multimedia tutorials. In L. Kjelldahl (Ed.), Multimedia: Systems, interaction and applications (pp. 307-323). New York: Springer-Verlag.

Bieger, G. R., & Glock, M. D. (1986). Comprehending spatial and contextual information in picture-text instructions. Journal of Experimental Education, 54, 181-188.

Blake, T. (1977). Motion in instructional media: Some subject-display mode interactions. Perceptual and Motor Skills, 44, 975-985.

Bloom, B. S., Engelhart, M. D., Furst, E. J., Hill, W. H., & Krathwohl, D. R. (1956). Taxonomy of educational objectives: The classification of educational goals. New York: David McKay.

Bosco, J. (1986). An analysis of evaluations of interactive video. Educational Technology, 25, 7-16.

Bransford, J. D., & Johnson, M. K. (1972). Contextual prerequisites for understanding: Some investigations of comprehension and recall. Journal of Verbal Learning and Verbal Behavior, 11, 717-726.

Bronner, E. (1998, September 30). Computers Help Math Learning, Study Finds. The New York Times on the Web.

Bryant, J., Brown, D., Silberberg, A., & Elliot, S. M. (1980). Humorous illustrations in textbooks: Effects on information
acquisition, appeal, persuasability and motivation. Paper presented at the Speech Communication Association
Convention, New York. (ERIC Document Reproduction Service No. ED 196 071)

Campeau, P.L. (1966). Selective Review of Literature on Audiovisual Media of Instruction. In Briggs, L.J., Campeau, P.L., Gagn , R.M. and May, M.A. (Eds.), Instructional Media: A Procedure for the Design of Multi-Media Instruction and Suggestions for Future Research. Monograph 2. Pittsburgh, PA: American Institutes for Research.

Cermak & F. I. M. Craik (Eds.), Levels of processing in human memory (pp. 385-403). Hillsdale, NJ: Erlbaum.

Chu, G.C. and Schramm, W. (1968). Learning from Television: What the Research Says. Stanford, CA: Stanford University. Institute for Communication Research

Clark, J. M., & Paivio, A. (1991). Dual coding theory and education. Educational Psychology Review, 3, 149-210.

Clark, R. (1994). Media will never influence learning. Educational Technology Research and Development, 42(2), 21-29.

Clark, R. E. (1983). Reconsidering research on learning from media. Review of Educational Research, 53, 445-459.

Clark, R. E. (1985). Evidence for confounding in computer-based instruction studies: Analyzing the meta-analyses. Educational Communication and Technology Journal, 33, 249-263.

Clark, R. E., & Craig, T. G. (1992). Research and theory on multi-media learning effects. In M. Giardina (Ed.),

Clark, R., & Sugrue, B. (1995). Research in instructional media, 1978-1988. In G. Anglin (Ed.), Instructional technology (pp 348-364). Englewood, CO: Libraries Unlimited.

Clark, R.E. (1983). Reconsidering Research on Learning from Media. Review of Educational Research. 53. 445-459.

Coburn, P., Kelmer, P., Roberts, N., Snyder, T., Watt, D., & Weiner, C. (1985). Practical guide to computers in education (2nd ed.). Reading, MA: Addison-Wesley.

Cohen, P.A., Ebeling, B. and Kulik, J.A. (1981). A Meta-Analysis of the Outcome Studies of Visual-Based Instruction. Educational Communication and Technology Journal. 29(1). 26-36

Cohen, P.A., Kulik, J.A. and Kulik, C.C. (1982). Educational Outcomes of Tutoring: A Meta-Analysis of Findings. American Educational Research Journal. 19. 237-248.

Conrad, R. (1964). Acoustic confusion in immediate memory. British Journal of Psychology, 55, 75-84.

Cooper, H.M. (1982). Scientific Guidelines for Conducting Integrative Research Reviews. Review of Educational Research. 52(2). 291-302.

Cooper, H.M. (1984). The Integrative Research Review: A Social Science Approach. Beverly Hills, CA: Sage.

Cooper, H.M. and Hedges, L.V. (Eds.) (1994). The Handbook of Research Synthesis. Russell Sage Foundation, NY, NY.
Department of Education. ERIC DIGEST - ED364220. ERIC Clearinghouse on Information Resources, Syracuse, N.Y.

Dobrin, J. (1999). Who's teaching online? ITPE News 2 (12), pp. 6-7.

Dwyer, F. M. (1967). Adapting visual illustrations for effective learning. Harvard Educational Review, 37, 250-263.

Dwyer, F. M. (1968). The effectiveness of visual illustrations used to complement programmed instruction. Journal of Psychology, 70, 157-162.

Ehrmann, S. (1995). Asking the right questions: What does research tell us about technology and higher learning? Change, 27 (2), 20-27.

Elhadad, M., Seligmann, D. D., Feiner, S., & McKeown, K. R. (1989). A common intention description language for interactive multi-media systems. In A new generation of intelligent interfaces (proceedings of the IJCAI89 workshop on intelligent interfaces) (pp. 46-51). Available from Steven Feiner, Department of Computer Science, Columbia University, New York, NY 10027 or feiner@cs.columbia.edu.

Ellsworth, J. (1997). Technology and change for the information age. Microsoft in Higher Education,[Newspaper, selected stories on-line]. Retrieved October 7, 1997 from the World Wide Web: http://www.microsoft.com/ education/hed/vision.htm

Evans, T., & Denney, M. R. (1978). Emotionality of pictures and the retention of related and unrelated phrases. Bulletin of the Psychonomic Society, 11, 149-152.

Feiner, S. K., & McKeown, K. R. (1990). Coordinating text and graphics in explanation generation. In AAAI-90 proceedings (pp. 442-449). Menlo Park, CA: AAAI (American Association for Artificial Intelligence) Press.

Feiner, S. K., & McKeown, K. R. (1991a). Automating the generation of coordinated multimedia explanations. Computer, 24(10), 33-41.

Feiner, S. K., & McKeown, K. R. (1991b). COMET: Generating coordinated multimedia explanations. In S. P.

Fletcher, D. (1989). The effectiveness and cost of interactive videodisc instruction. Machine-Mediated Learning, 3, 361-385.

Fletcher, D. (1990). The effectiveness and cost of interactive videodisc instruction in defense training and education (IDA Paper P-2372). Alexandria, VA: Institute for Defense Analyses.

Fletcher, J. D. (1990). Computer-based instruction: Costs and effectiveness. In A. P. Sage (Ed.), Concise Encyclopedia of Information Processing in Systems and Organizations. Elmsford, NY: Pergamon Press.

Fletcher, J. D. (1990). Effectiveness and cost of interactive videodisc instruction in defense training and education. Alexandria, VA: Institute for Defense Analyses.

Fletcher, J. D. (1992). Cost-effectiveness of interactive courseware. UTP-2. The Technical Cooperation Program. Sub Group U. Human Resources and Performance.

Fletcher, J. D. (1996). Does this Stuff Work ? Some Findings from Applications of technology to Education and training. Proceedings of Conference on Teacher Education and the Use of Technology Based Learning Systems. Warrenton VA: Society for Applied Learning.

Fletcher, J.D. (1990). Computer-Based Instruction: Costs and Effectiveness. In A.P. Sage (Ed.), Concise Encyclopedia of Information Processing in Systems and Organizations. Elmsford, NY: Pergamon Press.

Fletcher, J.D. (1990). Effectiveness and Cost of Interactive Videodisc Instruction in Defense Training and Education. (IDA P-2372). Alexandria, VA: Institute for Defense Analyses.

Fletcher, J.D. (1992). Cost-Effectiveness of Interactive Courseware. UTP-2. The Technical Cooperation Program. Sub Group U. Human Resources and Performance.

Fletcher, J.D. and Orlansky, J. (1986). Recent Studies on the Cost-Effectiveness of Military Training in TTCP Countries (IDA Paper P-1896) Alexandria, VA: Institute for Defense Analyses, March 1986.

Fletcher, J.D., Hawley, D.E., and Piele, P.K. (1990) Costs, effects, and utility of microcomputer assisted instruction in the classroom. American Educational Research Journal, 27, 783-806.

Fowler, B. T. (1980). The effectiveness of computer-controlled videodisc-based training. Unpublished doctoral dissertation, University of Iowa, Iowa City.

Gagne, R. (1962). Introduction. In R. M. Gagne (Ed.), Psychological principles in system development. (pp.1-10). New York: Holt, Rinehart, & Winston.

Garrison, W. T. (1978). The context bound effects of picture-text amalgams: Two studies (Doctoral dissertation,Cornell University). Dissertation Abstracts International, 39, 4137A.

Ghilardi, F. (1997). Getting to real-time knowledge management: From knowledge management to knowledge generation. Online, 21(5), 99-102.

Gifford, B. (1992). Where is the knowledge? Knowledge management, research and pedagogy in the electronic age. Education Libraries, 16(3), 14-22.

Gilbert, S., & Green, K. (1997). Moving information technology into the classroom. In L. Enghagen (Ed.) Technology and higher education(pp. 25-56). Washington, DC: National Education Association.

Glaser, R. (1962). Psychology and instructional technology. In R. Glaser (Ed.), Training research and education (pp.1-30). Pittsburgh: University of Pittsburgh Press.

Glass, G.V. (1976). Primary, Secondary, and Meta-Analysis of Research. Educational Researcher. 5. 3-8.

Glass, G.V. (1977). Integrating Findings: The Meta-Analysis of Research. Review of Research in Education. 5. 351-379.

Glass, G.V. and McGaw, B. (1980). Choice of the Metric for Effect Size in Meta- Analysis. American Educational Research Journal. 17. 325-327.

Glass, G.V., McGaw, B. and Smith, M.L. (1981). Meta-Analysis in Social Research. Sage Publications. NY, NY.

Grimes, P. W., Neilsen, J. E. & Niss, J. F. (1988). The performance of nonresident students in the "Economics USA" telecourse. The American Journal of Distance Education 2(2), 36-41.

Hagler, P., & Knowlton, J. (1988). Invalid implicit assumption in CBI comparison research. Journal of Computer-Based Instruction, 14(3), 84-88.

Hancock, V. (1997). Creating the information age school. Educational Leadership, 55 (3), 60-61.

Haynes, K. and C. Dillon. 1992. Distance education: Learning outcomes,interaction, and attitudes.Journal of Education for Library and Information Science 33(1): 35-45.

Hays, R.T., Jacobs, J.W., Prince, C. and Salas, E. (1992). Flight Simulator Training Effectiveness: A Meta-Analysis. Military Psychology. 4 (2). 63-74

Hays, R.T., Jacobs, J.W., Prince, C. and Salas, E. (1992). Requirements fir Future Research in Flight Simulation Training: Guidance Based on a Meta-Analytic Review. The International Journal of Aviation Psychology. 2 (2) 143-158. Hedges, L.V. and

Olkin, I. (1985). Statistical Methods for Meta-Analysis. Academic Press, Inc. NY, NY.

Hedges, L.V. (1981). Distribution Theory for Glass's Estimator of Effect Size and Related Estimators. Journal of Educational Statistics. 6. 107-128.

Hedges, L.V. (1982). Estimation of Effect Size from a Series of Independent Experiments. Psychological Bulletin. 7. 119-137.

Hedges, L.V. and Olkin, I. (1980). Vote-Counting Methods in Research Synthesis. Psychological Bulletin. 88. 359-369.

Hobbs, V. M. (1990). Distance learning in North Dakota: A cross-technology study of the schools, administrators, coordinators, instructors, and students. Two-way interactive television, audiographic tele-learning, and instruction by satellite. Denver, CO: Mid-Continent Regional Educational Lab., Inc. (ERIC Document Reproduction Service No. ED 328 225).

Hofstetter, F. T. (1994). Is multimedia the next literacy? Educators' Tech Exchange, 2(3), 6-13.

Holliday, W. G., Brunner, L. L., & Donais, E. L. (1977). Differential cognitive and affective responses to flow diagrams in science. Journal of Research in Science Teaching, 14, 129-138. In E. L. Baker, and H. F. O'Neil, Jr. (Eds.). Technology assessment in education and training. Hillsdale, NJ: Lawrence Erlbaum.

Jacoby, L. L., Craik, F. I. M., & Begg, I. (1979). Effects of decision difficulty on recognition and recall. Journal of Verbal Learning and Verbal Behavior, 18, 585-600.

Johnston, B.R. and Fletcher, J.D. (1996) Effectiveness of Computer Based Instruction in Military Training. Alexandria, VA: Institute for Defense Analyses.

Khalili, A., & Shashaani, L. (1994). The effectiveness of computer applications: A meta-analysis. Journal of Research on Computing in Education, 27, 48-61.

Kieras, D. E. (1984). A simulation model for procedural inference from a mental model for a simple device(Technical Report No. 15, UARZ/DP/TR-84/ONR-15). Tucson, AZ: University of Arizona, Department of Psychology.

Kieras, D. E., & Bovair, S. (1983). The role of a mental model in learning how to operate a device (Technical Report No. 13, UARZ/DP/TR-83/ONR-13). Tucson, AZ: University of Arizona, Department of Psychology.

Kieras, D. E., & Bovair, S. (1984). The role of a mental model in learning to operate a device. Cognitive Science, 8, 255-273.

Kosslyn, S. M. (1973). Scanning visual images: Some structural implications. Perception & Psychophysics, 14, 90-94.

Kosslyn, S. M. (1975). Information representation in visual images. Cognitive Psychology, 7, 341-370.

Kosslyn, S. M. (1976). Can imagery be distinguished from other forms of internal representation? Evidence from studies of information retrieval times. Memory & Cognition, 4, 291-297.

Kosslyn, S. M., Ball, T. M., & Reiser, B. J. (1978). Visual images preserve metric spatial information: Evidence from studies of image scanning. Journal of Experimental Psychology: Human Perception and Performance, 4, 47-60.

Kozma, R. B. (1991). Learning with media. Review of Educational Research, 61, 179-211.

Kulik J.A.& Kulik C. Computer based instruction: What 200 evaluations say? Centre for Research and Teaching, University of Michigan, 1988.

Kulik, C. and Kulik, J.A. (1987). Computer-Based Instruction: What 200 Evaluations Say. Paper Presented at the Annual Convention of the Association for Educational Communications and Technology.

Kulik, C. and Kulik, J.A. (Winter/Spring 1986). Effectiveness of Computer-Based Education in Colleges. AEDS Journal. 81-108.

Kulik, C., Chen-Lin C., & Kulik, J. (1991). Effectiveness of computer-based instruction: An updated analysis. Computers in Human Behavior, 7 (1-2), 75-94.

Kulik, C., Kulik, J. A., & Bangert-Drowns, R. L. (1990).Effectiveness of mastery learning programs: A meta- analysis. Review of Educational Research, 60(2), pp. 265-299.

Kulik, C., Kulik, J., & Cohen, P. (1980). Instructional technology and college teaching. Teaching of Psychology, 7 (4), 199-205.

Kulik, C., Kulik, J.A. and Shwalb, B.J. (1986). Effectiveness of Computer-Based Adult Education: A Meta-Analysis. Journal of Educational Computing Research. 2(2). 235-252.

Kulik, C., Kulik, J.A., and Shwalb, B. (1986) Effectiveness of computer-based instruction in adult education. Journal of Educational Computing Research, 2, 235-252.

Kulik, J. (1985). The importance of outcome studies: A reply to Clark. Educational Communications and Technology Journal, 34 (1), 381-386.

Kulik, J. (1994). Meta-analytic studies of findings on computer-based instruction. In Baker, E. L. & O'Neil, H. F. (Eds.) Technology Assessment in Education and Training. Hillsdale, NJ: LEA Publishers.

Kulik, J. A. (1994). Meta-analytic studies of findings on computer-based instruction. In E. L. Baker,and H. F. O'Neil, Jr. (Eds.). Technology assessment in education and training. Hillsdale, NJ: Lawrence Erlbaum.

Kulik, J., Bangert, R., & Williams, G. (1983). Effects of computer-based teaching on secondary school students. Journal of Educational Psychology, 75, 19-26.

Kulik, J.A. (1994) Meta-Analytic Studies of Findings on Computer-Based Instruction. In E.L. Baker and H.F. O'Neil, Jr. (Eds.) Technology Assessment in Education and Training. Hillsdale, NJ: Lawrence Erlbaum Associates.

Kulik, J.A. (1994). Meta-Analytic Studies of Findings on Computer-Based Instruction. In Baker, E.L. and O'Neil, H.F. (Eds.) Technology Assessment in Education and Training. Hillsdale, NJ: LEA Publishers.

Kulik, J.A. , Bangert, R.L., & Williams, G.W. (1983). Effects of computer based teaching on secondary school students. Journal of Educational Psychology, 75, 19-26.

Kulik, J.A., & Kulik, C. (1986) Effectiveness of computer-based education in colleges. AEDS Journal, 19, 81-108.

Kulik, J.A., Kulik, C. and Bangert-Drowns, R.L. (1985). Effectiveness of Computer-Based Education in Elementary schools. Computers in Human Behavior. 1. 59-74.

Kulik, J.A., Kulik, C. and Cohen, P.A. (1979). Research on Audio-Tutorial Instruction: A Meta-Analysis of Comparative Studies. Research in Higher Education. 11(4). 321-341.

Kulik, J.A., Kulik, C., & Bangert-Drowns, R.L. (1985) Effectiveness of computer-based education in elementary schools. Computers in Human Behavior, 1, 59-74.

Levie, W. H., & Lentz, R. (1982). Effects of text illustrations: A review of research. Educational Communication and Technology Journal, 30, 195-232.

Levin, J. R., & Lesgold, A. M. (1978). On pictures in prose. Educational Communication and Technology Journal, 26, 233-243.

Levin, J. R., Bender, B. G., & Lesgold, A. M. (1976). Pictures, repetition, and young children's oral prose learning. AV Communication Review, 24, 367-380.

Mann, D., Shakeshaft, C., Becker, J., & Kottkamp, R. (1999). West Virginia's Basic Skills/Computer Education Program: An Analysis of Student Achievement. Santa Monica, CA: Milken Family Foundation.

Martin, E. D., & Rainey, L. (1993). Student achievement and attitude in a satellite-delivered high school science course. The American Journal of Distance Education, 7 (1).

Mayer, R. E. (1993). Illustrations that instruct. In R. Glaser (Ed.), Advances in instructional psychology. Hillsdale, NJ: Lawrence Erlbaum Associates.

Mayer, R. E., & Anderson, R. B. (1991). Animations need narrations: An experimental test of a dual-coding hypothesis. Journal of Educational Psychology, 83, 484-490.

Mayer, R. E., & Anderson, R. B. (1992). The instructive animation: Helping students build connections between words and pictures in multimedia learning. Journal of Educational Psychology, 84, 444-452.

McCleary, I. D., & Egan, M. W. (1995). Program design and evaluation: Two-way interactive television.Readings in Distance Education (4).

McNeil, D. R. 1991. Computer conferencing project. Final report. Washington, D.C.: Academy for Educational Development, Inc.

Means, Barbara et al. (1993). Using technology to support education reform. Washington, DC: Office of Educational Research and Improvement, U.S.

Navarro, P. & Shoemaker, J. (1999). The power of cyberlearning: An empirical test. Journal of Computing in Higher Education.

Neal, E. (1998). Does Using Technology in Instruction Enhance Learning? or, The Artless State of Comparative Research Technology Source, http://horizon.unc.edu/ts/commentary/1998-06.asp

Niemiec, R., Blackwell, M., and Walberg, H.J. (1986). CAI can be doubly effective, Phi Delta Kappan, June, pp. 750-751.

Nugent, G. (1982). Pictures, audio, and print: Symbolic representation and effect on learning. Educational Communication and Technology Journal, 30, 163-174.

Orlansky, J. & String, J. (1977). Cost effectiveness of computer-based instruction in military training (IDA Paper P-1375). Arlington, VA: Institute for Defense Analyses.

Orlansky, J. & String, J. (1979). Cost-effectiveness of computer-based instruction in military training. Alexandria, VA: Institute for Defense Analyses.

Orlansky, J. (1983). Effectiveness of CBI: A Different Finding. Electronic Learning. 6. 59-60.

Orlansky, J. (1985). The Cost-Effectiveness of Military Training. Proceedings of the Symposium on the Military Value and Cost-Effectiveness of Training. Brussels, Belgium: NATO.

Orlansky, J. and String, J. (1979). Cost-Effectiveness of Computer-Based Instruction in Military Training. (IDA P-1375). Alexandria, VA: Institute for Defense Analyses. Phelps, R.H., Wells, R.A., Ashworth, R.L. and Hahn, H.A. (1991). Effectiveness and Costs of Distance Education Using Computer-Mediated Communication. The American Journal of Distance Education. 5 (3). 7-19.

Orlansky, J., & String, J. (1979). Cost-Effectiveness of Computer Based Instruction in Military Training. (IDA Paper P-1375). Institute for Defense Analyses, Alexandria, Virginia.

Orlansky, J., Dahlman, C.J., Hammon, C.P., Metzo, J., Taylor, H.L. and Youngblut, C. (1994). The Value of Simulation for Training. (IDA P-2982). Alexandria, VA: Institute for Defense Analyses.

Owston, R. (1997). The World Wide Web: A technology to enhance teaching and learning? Educational Researcher, 26 (2), 27-33.

Panel on Educational Technology for the President's Committee of Advisors on Science and Technology (1997). Report to the President on the use of Technology. Retrieved January 14, 1998 from the World Wide Web: http://www.whitehouse.gov/WH/EOP/OSTP/NSTC/PCAST/k-12ed.html

Pett, D., & Grabinger, S. (1995). Instructional media production. In G. J. Anglin (Ed.), Instructional technology: Past, present, and future (pp. 305-313). Englewood, CO: Libraries Unlimited., Inc..

Plowman, T. (1997). The information superhighway: Democracy or capitalism? Techtrends, 42 (5), 23-27.

Quiñones, S. & Kirshstein, R. (1998). An Educator's Guide to Evaluating The Use of Technology in Schools and Classrooms. Washington, D.C.: U.S. Department of Education, Office of Educational Research and Improvement.

Riel, M. (1994). Educational change in a technology-rich environment. Journal of Research on Computing in Education, 26, 452-474.

Robertson, G. M. Olson, & J. S. Olson (Eds.), CHI'91 conference proceedings (pp. 449-450). Reading, MA: Addison-Wesley.

Rosner, & O. Stock (Eds.), Aspects of automated natural language generation: Proceedings of sixth international workshop on natural language generation (pp. 278-280). New York: Springer-Verlag.

Russell, T. (1999). The "no significant difference" phenomenon. Retrieved August 25, 1999 from the World Wide Web: http://cuda.teleeducation.nb.ca/nosignificantdifference/

Scardamalia, M., & Bereiter, C. (1996). Computer support for knowledge-building communities.In T. Koschmann, (Ed.). CSCL: Theory and practice of an emerging paradigm. Mahwah, NJ: Erlbaum.

Sivin-Kachala, J. (1998). Report on the effectiveness of technology in schools, 1990-1997.Software Publisher's Association.

Skinner, B. F. 1953). Science and human behavior. New York: Macmillan.

Slee, E. (1989, February). A review of the research on interactive video. Paper presented at the meeting of the Association for Educational Communications and Technology, Dallas, TX.

Smeaton, A. & Keogh, G. (1999). An analysis of the use of virtual delivery of undergraduate lectures. Computers and Education 32, pp. 83-94.

Spencer, K. (1991). Modes, Media and Methods: The Search for Educational Effectiveness. British Journal of Educational Technology. 22, (1). 12-22.

Trinkle, D. A. (1999, August). Distance education: A means to an end, no more, no less. The Chronicle of Higher Education, A60.

Wenglinsky, H. (1998). Does it compute? The relationship between educational technology and student achievement in mathematics. Educational Testing Service Policy Information Center.

Wilson, E., & Marsh, G. (1995). Social studies and the Internet revolution. Social Education, 59, 198-202.

Wright, V. H. (1999). A comparison of the achievement and perceived satisfaction of graduate students in synchronous and asynchronous courses. Unpublished doctoral dissertation, University of Alabama, Tuscaloosa, AL.Anderson, J. R. (1976). Language, memory, and thought. Hillsdale, NJ: Lawrence Erlbaum Associates.