Researching IT in
Education
The methods you use will make or break the potential
impact of your findings and although the wide variety of qualitative
and quantitative methods used in education can be confusing they do
allow you to chose a method appropriate to your area.
Beynon (1993b) has argued that computers have
capabilities and cultural values built into them. He states that
teachers must begin to 'question technology' and suggests several
important questions including:
How do microcomputers 'frame' aspects of past and
present cultures? How do they mediate the pupils' encounters with
knowledge, values, concepts and ideas? What, thereby, is the
cultural impact on pupils' thought patterns, the nature of their
knowledge and the shaping of their perspectives? (p.
18).
These questions reflect the need for a new paradigm
in studies of IT in education. They are the final stage in what
Maddux (1993) identifies as 'three waves' of research. A condensed
form of his description of the evolution of research in IT is
presented here.
1. Exposure to computers in general will produce
global educational benefits.
Maddux (op. cit.) states that this first stage
occurs when research is concentrated on comparing the effectiveness
of the microcomputer to other instructional media. Methods for
studying the introduction of the microcomputer include 'distinct
evaluation exercises' which involve looking for improvements in
financial cost, student time, staff time or student learning (Kulik,
Kulik, & Cohen, 1980). Pre- and post-test experiments were
carried out in order to test the effectiveness of the computer
compared to that of other instructional media. Such short,
controlled experiments were designed to measure behavioural
objectives and were based on Skinner's ideas of teaching machines.
Improvements in learning outcomes, it was believed, could be
identified by comparing the results. Reliance on such methods meant,
however, that more subtle points were lost and, thus, there was a
call for research to focus on more specific applications of computer
practice.
2. Exposure to some particular computer application
will produce general educational benefits.
The second wave or method of testing the effectiveness
of particular computer applications is described by Maddux (op.
cit.) as being:
IF
LEARNERS (AT ANY LEVEL, ANY AGE, ANY GENDER, ANY GRADE
ANY IQ, ETC.) ARE TAUGHT [some computer application] (FOR ANY LENGTH
OF TIME, USING ANY METHOD, BY ANY TEACHER, ETC.), THEY WILL IMPROVE
MORE IN [some cognitive or performance variable] THAN AN
EXPERIMENTAL GROUP WHO ARE TAUGHT TRADITIONALLY [whatever that is]
(Typography exactly as in source - p. 16).
The reason that most of the
comparisons between traditional instruction and computer-based
instruction are flawed, Willis (1993) argues, is that they fail to
control potentially powerful confounding variables. In the
research-to-support-theory (RTST) model this is a fatal flaw because
the goal is to support a general theory (such as 'that CBI is
superior to other methods'), "and all alternative explanations to
the results (for example, that students might come from different
cultural backgrounds) must be eliminated" (Willis, 1993; 37).
However, when all potential confounding variables are held constant,
Clark (1985) believes that there will be no differences (e.g., when
students read some information on a screen and other information
from a text). He states that this type of task reduces understanding
and support of the very thing that makes IT so special - its great
flexibility and adaptability to learner variables. There is a need
for an alternative paradigm as researchers begin to question the
inherent assumptions of: 'for any length of time, using any method,
by any teacher'. A new focus is emerging.
3. Which and how learner and
learning variables interact with variables of instruction and
instructional technology?
This is Maddux's third and latest
wave. It focuses mainly on computers used in natural settings,
within school classrooms or computer labs. Although Maddux (op.
cit.) states that research on IT has evolved, he also argues
that methodologies have not been quick to change and old methods are
being used to explore these new directions.
Ethnographic research
- education and anthropological research
Ethnographic research is now
providing a new direction for studies of IT in Education. It has
been termed 'qualitative', 'post-positive', 'naturalistic' or
'subjective' (Borg & Gall, 1989) and in this dissertation the
term 'ethnographic research' will be distinguished from 'qualitative
methods' in that ethnography will apply to the naturalistic setting
of the experiment, whereas qualitative methods will be used to
describe the type of data analysis. Ethnography is a method of
research that stresses subjectivity and relies more on the
investigator's skills of observation and interpretation to provide
information than do more traditional scientific methods used in
quantitative research. Borg and Gall (op. cit.) list ten
characteristics of ethnographic studies including: purposeful rather
than random sampling, inductive data analysis, and the larger role
the subject has to play in the outcome. In general, they conclude
that such research involves holistic inquiry carried out in natural
settings. Bogdan and Biklen (1982) state that ethnographic field
research methods are framed, not by their operationalising
variables, but instead are 'formulated to investigate subjects in
all their complexity in context' (p. 2).
Ethnographic researchers are
interested in the aims and motives, not just the behaviour of those
being studied (Sherman & Webb, 1990). The technique has evolved
because of the belief that the representation of human learning by
results gained through laboratory environments missed the real
question (Borg & Gall, 1989). Research designs have taken the
form of in-depth studies in which researchers can preserve the flow
of data collection and chronological information. Ericsson and Smith
(1991) focus their research on tasks capturing life experience.
There is a lack at the present time, however, of standards for
collecting data in naturalistic settings. Eisner and Peshkin (1990)
in speaking of what constitutes good research state that: 'In
quantitative research, the good may be found in fidelity to design,
whereas in qualitative research, relatively lacking in canons and
conventions, the good is more elusive because its procedures are
more idiosyncratic' (p. 2).
Thus, while general agreement exists
as to the versatility of the ethnographic research method, it is a
difficult technique to assess and replicate. It is well documented
(Borg & Gall, op. cit.) that such techniques as
observational data collection are open to observer effects such as:
the effect of the observer on the observed; observer bias;
contamination; and rating errors. Researchers are seeking a standard
way of ensuring the validity and reliability of data collected in
field settings. Triangulation of data gathering, involving a layered
approach is one solution and is used in this dissertation. Multiple
methods of data collection, such as interview, observation, video
and audio recordings, and a variety of methods of analysis provide
ways to effectively study complex learning environments. By
providing for a variety of ways of collecting and presenting data we
begin to '...investigate experience as we live it rather than as we
conceptualise it' (van Manen, 1990; 30).
Wolcott (1988) states that: "The
ethnographer walks a fine line". He elaborates, stating that: "With
too much distance and perspective, one is labelled aloof, remote,
insensitive, superficial; with too much familiarity, empathy, and
identification, one is suspect of having 'gone native'" (p. 189).
For Schultz 'the stranger' is the one who has the clearest view of
the complex nature of society and so detachment and involvement are
equally important to the process. Goodman (1993) argues that "the
anthropologist should not present himself as a mere data-collecting
and analysing machine but as an aware and sensitive human being;
moreover, rather than that this approach should be seen as
'unscientific, on the contrary the more the anthropologist is
subjectively aware, the more he might, in effect, be called
'scientific" (p. 159).
Ethnography in education has emerged
from a combination of other disciplines such as anthropology.
Following World War II, the work of George and Louise Spindler, two
prominent American anthropologists, started the movement in the USA
for the appreciation and understanding of multicultural education.
The Spindlers, and researchers who followed them such as Wolcott
(1973), have influenced the fields of educational anthropology,
psychological anthropology, American Indian studies and European
cultural studies (Spindler, 1974; Suz-Orozco, 1991). While North
American cross-cultural research grew out of the study of
educational anthropology, in Britain there has been a tradition of
sociological case studies (Vulliamy, 1990) starting with a working
group in Manchester under Gluckman in the late 1960s and the 1960s.
It was here that the idea of comparative studies other than
descriptive ones became prominent.
A new method of teaching maths, for
example, might be implemented using a IT- rich environment.
Educational resarchers could explore a wide variety of issues in
such a project ranging from: national and school policies on
mathematics, the evolution of the curriculum, the role of ICTs in
instruction, teaching practice in a changing world, teacher-pupil
interaction, student-student interactions and combine any of these
topics with overall issues of concerning gender, disavantaged youth,
and multinational classroom environments etc.
Increasing the role of information and
communication technologies are moving social science fields towards
those of computer science and science in general. One traditional
research model has been to remove subjects from their environment
and test them by focusing on one or two variables in isolation.
Limitations in applying results to the real world have created a
search for more tenable research paradigms that seak to undercover
rather then eliminate underlying factors. Although there have always
been experimental learning methods that have been designed and
tested on new audiences it has been difficult for any researcher or
research centre to implement any project on a wide enough scale to
get significant results. New technologies on the otherhand are
allowing for the creation of learning environments and their
potential dissemination world wide.
Computer Science
A common model of research in computer
science is that of project implementation. Usually a problem drives
the work and the resulting program explores solutions. If possible,
data may be collected on the project once it is completed. In
education one might look at the general issues of teaching and
learning, or classroom activity, as projects that have already been
implemented. The underlying question for most of educational
research is whether or not these projects are successful and to
examine and evaluate the outcomes.
Dissertations on the
technical report site are 1st and can give you a good indication of
high quality work by TCD computer science Masters and PhD
students.
http://www.cs.tcd.ie/publications/tech-reports/tr-index.99.html
Categories of research in
computer science
The break down of research into the
following categories by Dawson (pp.15-17) may aid in your understanding of the
nature of research in the area of computer science.
Research-based project –
theoretical orientation
-
-
Identifying strengths &
weakness
-
Acknowledging areas for
further development and investigation
-
Usually involves some type of
literature search or review
-
Compare and contrasting
programming languages
-
Judge different user
interfaces