An Overview of Research
methods in ICTs
It can be argued that
all it takes to make a successful learning environment with or
without ICTs is a good teacher or a good learner. Theory is the
attempt to discover what it takes to produce such teachers and
learners, to identify successful practice and then reproduce it
in the widest variety of circumstances.
Educational theory
has suffered from being in the shadow of scientific theory. Interestingly
the scientific method has been deeply questioned within the sciences
while often being taken as given outside the discipline. For a
short history of the evolution of the scientific method visit
PHILOSOPHY OF SCIENCE Science vs. Religion, from the Renaissance
to the Enlightenment http://www.roga.demon.co.uk/sci.htm
In order to add weight to works
on education, early experiments were traditionally carried out
in much the same way that chemical reactions were tested. .
A new paradigm in studies
of IT in education began to evolve and Maddux (1993) has identified
'three waves' of educational research in information technology
in education. A condensed form of his description of the evolution
of research in IT is presented here.
1. Exposure to computers
in general will produce global educational benefits.
Maddux (op. cit.) states
that this first stage occurs when research is concentrated on
comparing the effectiveness of the microcomputer to other instructional
media. Methods for studying the introduction of the microcomputer
include 'distinct evaluation exercises' which involve looking
for improvements in financial cost, student time, staff time or
student learning (Kulik, Kulik, & Cohen, 1980). Pre- and post-test
experiments were carried out in order to test the effectiveness
of the computer compared to that of other instructional media.
Such short, controlled experiments were designed to measure behavioural
objectives and were based on Skinner's ideas of teaching machines.
Improvements in learning outcomes, it was believed, could be identified
by comparing the results. Reliance on such methods meant, however,
that more subtle points were lost and, thus, there was a call
for research to focus on more specific applications of computer
practice.
2. Exposure to some
particular computer application will produce general educational
benefits.
The second wave or
method of testing the effectiveness of particular computer applications
is described by Maddux (op. cit.) as being: IF LEARNERS (AT
ANY LEVEL, ANY AGE, ANY GENDER, ANY GRADE ANY IQ, ETC.) ARE
TAUGHT [some computer application] (FOR ANY LENGTH OF TIME,
USING ANY METHOD, BY ANY TEACHER, ETC.), THEY WILL IMPROVE MORE
IN [some cognitive or performance variable] THAN AN EXPERIMENTAL
GROUP WHO ARE TAUGHT TRADITIONALLY [whatever that is] (Typography
exactly as in source - p. 16).
The reason that most
of the comparisons between traditional instruction and computer-based
instruction are flawed, Willis (1993) argues, is that they fail
to control potentially powerful confounding variables. In the
research-to-support-theory (RTST) model this is a fatal flaw because
the goal is to support a general theory (such as 'that CBI is
superior to other methods'), "and all alternative explanations
to the results (for example, that students might come from different
cultural backgrounds) must be eliminated" (Willis, 1993; 37).
However, when all potential confounding variables are held constant,
Clark (1985) believes that there will be no differences (e.g.,
when students read some information on a screen and other information
from a text). He states that this type of task reduces understanding
and support of the very thing that makes IT so special - its great
flexibility and adaptability to learner variables. There is a
need for an alternative paradigm as researchers begin to question
the inherent assumptions of: 'for any length of time, using any
method, by any teacher'.
3. Which and how
learner and learning variables interact with variables of instruction
and instructional technology?
This is Maddux's third
and latest wave. It focuses mainly on computers used in natural
settings, within school classrooms or computer labs. Although
Maddux (op. cit.) states that research on IT has evolved, he also
argues that methodologies have not been quick to change and old
methods are being used to explore these new directions.
Collaboration and
interaction
Lewis (1990) stated
that, although pupils could work in groups for prolonged periods,
the role of the teacher is still very important and cautioned
that teachers must not interpret all interaction as relating to
the task at hand. Cox (1989) believes that researchers need to
observe just how children actually do interact when working in
small groups. She further recommends that researchers examine
the quality of pupil interaction and thinking in order to test
the claim that some microcomputer programs have the power to generate
purposeful discussions among groups of children.
Howe, Tolmie and Anderson
(1991) contend that computer work has come to be seen as group
work, which they argue can benefit students, but like Lewis (op.
cit.) they add certain cautions. Teacher intervention and specially
designed software were needed to overcome the gender-related patterns
of behaviour that interfered with student successes in problem-solving.
They found that joint decision making by male pairs drew out preconceptions
about the explanations of object paths. Female pairs did not discuss
the physics problems in the same ways but this did not impede
their progress and the girls matched male pairs in overall performance.
The mixed pairs, however, had significantly lower levels of interaction
that reflected the tendency for one person to have sole control.
The Spoken Language
and New Technology (SLANT) project was set up by a joint Open
University/ University of East Anglia research team to research
the ways in which children construct knowledge together through
discourse, and how (if at all) this process depends on the negotiation
of that knowledge through its formulation in the course of computer-based
activity (Mercer, Phillips & Somekh, 1991; Mercer, 1994).
When observing primary school children in 10 different classrooms,
the base segment that they sought to analyse was not single words
or sentences but rather the full discourse of negotiated meaning.
The researchers concluded that computer-based activities stimulated
talk but that the talk varied in terms of its educational content
and overall quality.
As part of an ESRC
project on group work with computers, Hoyles, Healy and Pozzi
(1994) attempted to identify factors, such as background or process,
which influenced successful collaboration. They identified as
main points: • the structuring of the task environment;
• conflict and negotiation; • role of computers in
stimulating formal mathematical expressions; • the influence
of personal variables such as gender and attainment. They concluded
that, for successful group work to take place, pupils must have:
computer access, software that supports group work; and a task
that allows them to discuss and evaluate away from the computer.
These findings have been echoed by a number of researchers (Crook,
1987; Hall & Rhodes, 1988; Hughes, MacLeod & Potts, 1985).
Although some studies
have found that children work well in pairs, taking turns with
the keyboard, others recorded a tendency, in large groups or with
a more advanced students, where one child would dominate the activity
(Crook, 1987; Hall & Rhodes, 1988). Shooter, Lovering and
Bellamy, (1993) found that, although the group had initially one
dominant member, other group members gained confidence over time
and more equal roles evolved. In terms of research design, some
studies were based on anecdotal evidence and teacher recall whereas
others used microphones to record classroom discourse. Often descriptions
and interpretations recorded by the researchers fleshed out the
classroom picture.
A fairly typical example
of the type of research method designed to review group work is
a study by Cox (1989) who observed and tape-recorded the discussions
of small groups of seven-year-old children working on the software
program Granny's Garden. Children were wired with microphones
and each child's voice was recorded on a separate machine. Cox
did not systematically analyse the content of the materials: instead,
like many researchers, he scanned the transcripts for sequences,
such as those involving reasoning or predicting, and collaborative
behaviour, such as expressing agreement or challenging other children's
ideas. In addition, discussions were judged on balance of contributions,
willingness to listen, and regard for collaborative rules such
as taking turns. The desire to involve teachers in the rapid changes
associated with IT in Education has led to a number of action
research projects (Candy, 1988; Somekh, 1986; Somekh, 1989). Candy
(op. cit.).
Such reseaerchers point
out that curriculum research which does not involve teachers directly
is unlikely to influence principles and practice. She defines
action research as a method that aims to identify existing assumptions
in practice and then develop new methods in the light of the findings.
It seeks to improve teaching by means of a more reflective professional
practice in IT. Recently, more studies have begun to use video,
partly as a result of an increased interest in the physical aspects
of classroom interaction. Hoyles, Healy and Sutherland (1991),
for example, summarise the work of a two-year project that sought
to discover how pupils generalise and formulate their thoughts
in mathematics when working in pairs. Research on IT in Education
in the UK has covered a range of topics including, history of
the use of computers in schools, their present use in classrooms,
and the evaluation of pupil access to computers. There are also
many different research designs and methodologies, ranging from
large-scale questionnaires to in-depth case studies. The literature
has been more focused on theory building and theory testing than
has research in Japan. Also, there is much more concern with social
and gender aspects. As the UK has had computers in its classroom
much longer than Japan has there has been a development of a body
of literature that points to specific findings. Overall, studies
found that programs can produce thinking and collaboration under
the right conditions. These include teacher intervention and monitoring
which is aimed at reducing inequality in access based on ability,
gender or race.
Computer-hardware
and software and the effects on the learner
The computer programmer
... is a creator of universes for which he alone is the law
giver. ... No playwright, no stage director, no emperor, however
powerful, has exercised such absolute authority to arrange a
stage or field of battle and to command such unswerving dutiful
actors or troops (Weizenbaum, 1984; 102 ).
The underlying 'intelligence' of any computer is its program and
underlying any program are the minds of intelligent programmers.
Computer-taught children might, therefore, have intellectual partnerships
with the best and brightest of computer experts. The new relationship
may be no longer one of the tutor and the tutored but instead
that of a 'cognitive partnership'; the quality of which depends
upon the mindful engagement of the learner and the open-endedness
of the tool (Salomon, Perkins & Globerson, 1991). As technology
becomes more powerful, learning environments are created (such
as virtual reality) which may re-structure the learner’s
view of the world. Although viewed positively on the whole, such
intellectual partnerships might have negative consequences if
they were unequal. Learners who do not share the same cultural
backgrounds as the computer programmers (who have been predominantly
male, white and from the West) might be 'culturally dominated'
by such machines.
There are several ways
in which the computer can have inherent cultural biases and one
of the most obvious examples is bias found in software. Chandler
(1992) argues that the computer projects an ideology and that
this ideology is found in a number of areas of the school including
simulations, word-processors and data-handling software. Inflexible
software may lead the child to be programmed by a microcomputer
rather than exploring and developing other routes through the
curriculum (Robinson, 1989). Robinson (1993a) argues that software
presentation 'manifest(s) the compiler's particular way of structuring
the world' (p. 123).
Databases, which were
previously considered a content-free type of software (Moss, 1992)
may, in fact, be contributing to a re-structuring of the learner's
world view. Johnson (1991) believes that databases are not empty
as they accept only certain types of factual data. Robinson, (1993)
argues that word processors and databases may reveal "a particular
cultural dependent way of categorising experience, presenting
it and making use of it" (Robinson, 1993; 123). These capacities
take the form of distinct linguistic, cognitive and organisational
orientations which serve particular ways of linear and analytical
thinking (Beynon, 1993b). Beynon (1993b) labels this a 'technicist
mindset' and states that it devaluates metaphorical and analogical
ways of thinking in favour of linear, 'digital' ways of manipulating
data. Other ways of categorising experience might not be so easily
to program into software databases. Consider the following example
of categorising data before the advent of Microsoft Excel:
...in an ancient
Chinese encyclopaedia animals were categorised as (a) belonging
to the Emperor, (b) embalmed, (c) tame, (d) sucking pigs, (e)
sirens, (f) fabulous and (g) stray dogs. Other diverse categories
included: (k) drawn with a very fine camel hair brush, and (n)
that from a long way off look like flies (Cornuéjols & Murphy-Judy,
1992; 1248).
Researchers have also studied the effects of using content-oriented
software in the class (Kurtz dos Santos & Ogborn, 1994; Spavgold,
1989; Strack, 1995). Strack (op. cit.) argues that software used
in schools needs to be considered in terms of issues of race,
and gender as well as previous experience so as to support tasks
that create equal opportunities for students. A survey reporting
on the use of software in Wales found that more teachers are aware
of the potential of content-free software than are using
it in their lessons (Moss, 1992).
This interest in culture
is not limited to software packages but instead researchers consider
it to be reflective of all aspects of the machine. Robinson (1993)
argues that the design of both hardware and software reflects
a range of considerations, including the economic, political,
social, ideological, epistemological and pedagogic:
In looking at
any machine it is easy to forget that it was designed under
certain circumstances for certain purposes under certain constraints
and that it was probably the product of a series of compromises.
Its very material being in front of us hides the series of choices
and decisions the authors and designers of these or mechanical
'texts' chose or were forced to make (Beynon, 1993b; 14).
For Bowers (1988) microcomputers are best regarded as an extension
of print technology and this amplifies the sense of individualism
and analytical thought and shapes communication into an asymmetrical
power relationship. Although Chandler (1985) argues that there
is the danger that certain types of language or ways of thinking
may be accorded more importance than others in computer literate
societies, he supports the use of computers to extend the potential
of all learners.
Many writings in this
area of research are theoretical. They are thought-provoking and
offer material for reflection but at the same time may prove difficult
to test. They are important since schools are searching for software
and hardware to purchase for use in the classroom:
… too often we
have seen the purchase and deployment of both hardware and software
run ahead of any clearly researched and articulated justification
for its educational use (Beynon, 1993b; 209).
The commercial nature of computer
design may, in fact, not be in line with educational needs. Educators
are often restricted either by financial or technological considerations
and quite simply a lack of any appropriate software.
Research focus
1. Exposure to computers
in general will produce global educational benefits.
Maddux (op. cit.) states that this first stage occurs when research
is concentrated on comparing the effectiveness of the microcomputer
to other instructional media. Methods for studying the introduction
of the microcomputer include 'distinct evaluation exercises' which
involve looking for improvements in financial cost, student time,
staff time or student learning (Kulik, Kulik, & Cohen, 1980).
Pre- and post-test experiments were carried out in order to test
the effectiveness of the computer compared to that of other instructional
media. Such short, controlled experiments were designed to measure
behavioural objectives and were based on Skinner's ideas of teaching
machines. Improvements in learning outcomes, it was believed,
could be identified by comparing the results. Reliance on such
methods meant, however, that more subtle points were lost and,
thus, there was a call for research to focus on more specific
applications of computer practice.