Introduction

Approaches to teaching using computers

Over the years, the use of computers has developed, enabling them to take an increasingly complex role in education. At the simplest level, computers may be used in teaching as a source of material, managing large amounts of readily accessible text and images for use in face-to-face teaching. As an example, computer systems are in common use in formal lecturing where they provide a means of producing and displaying visual aids as an alternative to slides or overhead projector acetates. Increasingly sophisticated technology allows the incorporation of, for example, sound and moving images.

Packages of suitably designed material of this type may be gathered together into units as a teaching resource. Some of these packages are designed primarily to provide information, although interactive teaching resources have also been developed that go some way towards being adaptive, supporting discussion, engaging activities, and providing timely feedback to learners. These are qualities that are found in the best of traditional teaching, but which are often lost when courses make poorly considered use of technology.1 In many cases, however, interaction remains limited to that between learner and computer, with teachers offering little or no feedback either during or after use of the package.2

More recently, sophisticated teaching packages that allow direct interaction between learner and teacher (and between learners) have become available, particularly through the internet. The potential of this type of resource to support effective learning has been recognised in disciplines such as computer science for over a decade,3 and has recently led to the development of successful courses in medical education.4,5

The use of computers in education is seen as offering advantages in terms of the diversity and scope of material that can be offered, and in terms of accessibility. This is particularly useful for practitioners learning at a distance from academic centres or who need to fit part-time study around work and family commitments.2 From the learner's viewpoint, computer-based packages may better allow for differences in learning styles and speeds than face-to-face methods do. However, although they have been seen as a cheap option, a number of hidden costs (such as replacing hardware, overheads and the expense in staff time of preparing materials) may well make online courses considerably more expensive than anticipated.6,7

Computer-based teaching in dentistry

Computer-based learning packages have been used as a part of undergraduate teaching in dentistry but are perhaps more often seen as a resource suitable for postgraduate training. In particular, web-based packages are now being introduced for CPD purposes – an area that is growing in importance.6

Whilst there has been some evidence of the effectiveness of teaching using computer-based methods, one recent study suggested that a computer-assisted learning programme was of little value in teaching clinical decision-making skills.8 This might be viewed as an example of the 'no significant difference' phenomenon often encountered in experimental educational research.9 However, it is important to recognise that experimental methods are rarely viable in an educational context. The reasons for this include the ethical and pragmatic problems that call into question the control of extraneous factors, the likelihood of compensatory behaviour skewing performance, cross-group contamination and the validity of assessment.10

Thus, conclusions must be drawn with extreme caution from such studies. Fundamentally, it is important to recognise that teaching and learning using computer-based packages typically differs from that seen in face-to-face learning. Because such situations typically replace the face-to-face interaction, different aspects of behaviour and personality come into play in the teaching/learning process, different skills in design are needed for the learning material and different things are learnt.1 In the study reported by Kay et al8 the learning package had clear educational objectives but there was no mention of the educational methods used, whether or not there was interaction or assessment as a part of the package, or how far the programme was integrated into other teaching and learning that was being undertaken by students at the time. Full integration has been seen as fundamental to the development of packages as a part of undergraduate education.11 The lack of apparent effect in such contexts seems much more likely to result from the inappropriate use of methodology, or to relate to the educational purpose to which the package was put, rather than to any inherent features of the package itself, or indeed to the fact that this represents an example of computer-assisted learning. In short, the report appears to have been over-simplistic in its understanding of educational processes.

Importantly, however, this particular study does attempt to grapple with the complex problem of educational impact –something that is omitted from many such studies, involving computers or otherwise. For example, Welbury et al.12 describe a study that uses self-reported perceptions as the sole basis for its conclusions (which are, appropriately, about acceptability and attainability); similarly, Holt et al,13 using a before and after questionnaire study, reported that a teaching video had been well received and had resulted in changes in intention (although they were unable to demonstrate substantive changes in behaviour). Such evaluations are valuable as a first step, in that an unacceptable educational approach is unlikely to have much actual impact. However, they remain essentially superficial, failing to engage with the more complex problems of assessing learning and teaching.14

Methodology

The present paper will illustrate the complexity of the educational process and highlight some methodological concerns by describing the pilot development of two modules in a dental public health course and their use by three postgraduate students as part of a full time programme of education. The case was designed to allow an open-ended investigation of whether web-based learning is good or effective, paying close attention to issues such as individual differences.

A qualitative approach has been adopted, following recognised guidelines for good practice in such methods.15 Specifically, the study involved three complementary qualitative approaches which were triangulated in order to re-inforce the reliability of the study's conclusions:

  • Direct observations of the course (including viewing online activities),

  • A focus group with all three course participants,

  • An interview with one participant in order to expand upon particular topics of concern.

All discussions were audio recorded and transcribed in full. These qualitative data will be summarised in this paper in the form of cases; a study of excerpts from the transcripts can be found in Holt et al.16

The case context

The programme of dental public health for which the modules were designed is a one year full time education programme on dental public health provided by a postgraduate dental institute. The programme prepares postgraduates for the MSc examination in Dental Public Health of the University of London and is also approved for those wishing to take the Diploma in Dental Public Health at the Royal College of Surgeons. The aim of the programme is therefore to provide an introduction to concepts and practice in dental public health.

The two modules were developed using WebCT, a commercially available package designed for development of programmes in higher education and used previously at UCL for the programme in primary care.17 The contents of the package are shown in Figure 1. WebCT provides a toolbox of items from which methods can be chosen to deliver different styles of educational package. These include, for example, web pages, bulletin boards, assessment facilities (multiple choice, answer matching, etc), essay submission and administration facilities, reference management resources, and so on.

Figure 1
figure 1

The 'home page' of the WebCT course for Dental Public Health.

Full size image

The material developed for use of the learners in this particular course included the following:

Introductory material

An introductory page explained the nature and purpose of the modules and how the package should be used. The page included simple interactive exercises for learners to gain confidence using the programme toolbox, providing hands-on experience of all of the features they would be required to use. This page remained available for reference throughout the course. Each student worked through this page as part of a face-to-face workshop, at which they were also registered with the software.

Module I: Introduction to epidemiology

The first module used a traditional text-based approach to introduce the basis and practice of epidemiology. The material was structured in a series of five text-based units and a practical exercise. Each unit was divided into a series of 'pages'. The text for a unit was built around the information that might have been delivered during one teaching session in the subject. Selected lists of preliminary, core and supplementary reading were included for each unit. Within the text, learners were given pointers for discussion and encouraged to reply to questions given in the online discussion area ('bulletin board'), on which questions and replies were posted. Two self-assessment tests were included for those wishing to use them and a series of formative assignments were included in the assessment area of the package. Each assignment required students to write a short or long answer and to submit this through the package for marking by a course tutor. The assignments had clear deadlines related to the course timetable, after which they ceased to be available.

The practical exercise included a series of bitewing radiographs which learners were asked to examine for caries and then assign a score. They were then asked to formally compare scores with those of a second learner and with those made by a gold standard examiner. A series of Excel files were developed which incorporated calculations of kappa values once data had been correctly entered. The exercise was carried out through the assignment area of the package. Grades and feedback on all assignments were returned confidentially through the package, usually within one to two days.

Module II: Evidence based dentistry

The first module was developed, piloted and evaluated; following the feedback gained from this process, the second module adopted a problem-centred approach. Following one unit consisting of an introduction to the subject, three units were based on a series of assignments for which learners were asked to carry out searches, critical reading and a review of items. These were more extensive and required more self-directed effort than the assignments in the first module.

A particular development within this module was the introduction of self-assessment as an educational technique. One face-to-face session was built into the module. This introduced the self-assessment element of the course and required the students to develop a set of self-assessment criteria that they would then apply to a piece of their own work.

The modules were delivered as part of an internal programme; other modules were delivered alongside these using traditional teaching techniques. It was stressed that the modules were a part of the programme but that, because they were new and experimental, these were not compulsory.

Three cases

Three postgraduates enrolled on the programme in dental public health were introduced to WebCT in the first term of their programme. All had basic computer skills but none had prior experience of computer-based learning.

Student one

This postgraduate student came from an armed services background. He was articulate and had a good analytical approach to work, both in the computer-based modules and in other parts of the programme. From the earliest stages, this student contributed little or nothing to web based discussion areas, clearly preferring the verbal discussions that formed part of face-to-face teaching sessions elsewhere in the programme and which he most often led. During development of the self-assessment exercise, this student valued outputs and objectivity above other elements and felt unable to compromise on this position. Subsequently he did not complete/present the exercise and may not have seen it as particularly helpful in the context of the programme.

From the focus group discussion, it became clear that this student preferred the more traditional 'sage on the stage' approach most often used in dental education and was not comfortable with having a greater responsibility for his own learning and assessment. This method of teaching and learning seemed unlikely to be his preferred option, at least within an internal programme.

Student two

The second student was female, with a meticulous, methodical approach to learning. She was quiet and reflective in her approach and developed this further whilst she was completing the two modules. English was her second language and her approach was well suited to the asynchronous, reflective style of online discussion. She contributed thoughtfully to this part of the modules. Self-assessment was very unfamiliar to her and although she gave good comments on her work she did not have sufficient confidence to score this. This student valued qualitative, reflective elements of assessment but compromised on her criteria to work effectively with the others. Because her style of learning was ideally suited to the asynchronous format of web-based learning, she would probably cope well with web-based distance learning too. She found self-assessment difficult at the first attempt but was likely to achieve it with further practice and support.

Student three

This male student came from a society with a strict social hierarchy. English was not his first language and his educational background was very dissimilar to that provided in the UK. He had a disorganised approach and tended towards rote learning to try to make up for deficiencies in knowledge. This student used the two modules frequently, but mainly as a resource, possibly regarding them as a source of authoritative information. He contributed a little to the discussion areas.

He engaged with all aspects of the course but was not always successful in returning items through the programme software. He genuinely struggled to complete the self-assessment and recognised his own weaknesses, effectively defusing this situation by humour. The student found the course modules difficult but managed well with direct face-to-face support. Because of the amount of support needed, he seemed less likely than student two to thrive with this learning method.

Discussion

Several factors were identified as being of primary importance in the study described above. It was clear that the variation in the students' approach to learning, and in their individual disposition, led the same course to be used and valued differently by each. These factors included their articulacy in spoken and written English, their predisposition towards reflection or engagement, and the extent to which they were able to cope with changes in the rôle of the teacher that resulted from the mediation of communication by WebCT and the adoption of student-centred learning approaches. The extent to which the students felt able to contribute and the extent to which the course 'empowered' them was determined by a complex interaction between these factors.

The student who gained the most from the course, both in terms of personal development and improvement in marks, felt most comfortable with the asynchronous exchange of ideas that the web requires. The ability to stop and reflect on questions, to marshal and analyse ideas, is especially encouraged by this form of learning, and this closely aligned with her preferred style of learning. The format may also have been helpful because it provided a means of carefully constructing and editing responses, since English was the student's second language.

However, the student who was initially the most able proved to be uneasy working in a context where his traditional strengths (such as his ability to think quickly and to lead group discussions) could not be used. The mismatch between his strengths and the characteristics of web-based learning left him understandably uncomfortable and he benefited much less from this part of the course.

The third student illustrated that simple substitution of a teacher by a computer is wholly inappropriate, because he would have needed a relatively high degree of support. Irrespective of the course format, it became particularly important to provide him with additional face-to-face meetings alongside the web-based delivery of the programme. Simple substitution of the computer-based materials for face-to-face teaching would have left him unsupported and floundering. However, with the support, he was able to make good progress; he also used the materials very effectively as a learning resource and a point of reference.

This final case illustrates a further important factor that must be accounted for: whilst teacher-student contact is still an essential part of this course, its nature has changed. In line with developments in many other disciplines, the role is mediated, and has altered to become less directive and more supportive – a shift 'from the sage on the stage to the guide on the side'.18

The case as a whole provides a critique of the widely held belief that web-based learning is good and effective; however, it does this by careful attention to the complexities of the educational process, recognising the fact that generalisations are hard to draw in an educational context. Adopting qualitative methods allowed unanticipated factors to be recognised and explained – for example, the importance of the differences in personalities between the three that would have been lost by a simple quantitative approach that only considered factors such as age and gender.

Significantly, because they are harder to identify or quantify, these more subtle factors are precisely the kind of things that are likely to make generalisations difficult. The account provided above illustrates both their impact on learning and, through this, the inappropriateness of drawing inferences from controlled studies in an uncritical way.

It is important to recognise that the impact of this is a shift in terms of the subtlety of the research question – away from a simplistic yes/no decisions about the 'goodness' or 'effectiveness' of web-based learning, and towards a consideration of what it means to be 'good', who a particular programme is good for, and how it achieves its effect.

Future developments might include applying this altered understanding to research in both undergraduate and postgraduate education. With the launch of major CPD initiatives (including web-based schemes) appropriate evaluation may be especially critical if these large resources are to be used to maximum effect.

Conclusions

This paper presents a novel part of a course in dental public health. It is one of the few examples currently available of a hybrid course in this area, combining both web-based and traditional methods of teaching and learning. Importantly, the case study highlights the fact that such approaches are not equally suitable for all learners.

Whilst there are several direct implications of this study for dental education, perhaps the most significant issues concern the methodologies used to carry out dental education research. Simplistic evaluations risk inappropriate conclusions about the value of different educational approaches. Controlled experiments must be used in educational studies with great caution. Whilst they clearly have value, inferences must be made carefully, since factors that have a considerable impact on learning may not have been considered, and may be difficult to quantify or control for. In order to deal with this complexity, irrespective of the research method used, qualitative criteria for rigour (such as triangulation) must also be applied when designing and interpreting studies. With the possible exception of very large controlled trials, the complex factors present in any educational context make generalisation highly problematic, and potentially misleading since the specific characteristics of individual learners (many of which may remain unidentified until after the study) may be a crucial determinant of educational impact. Fundamentally, this illustrates that it is not the outcome of a quantitative study but the interpretation that is put on it that is of critical importance – this calls into question whether the hierarchy currently used in evidence-based dentistry is appropriate for educational research.

It may be possible that, with a fuller understanding of the influence of difference factors on learning, controlled experiments will become more meaningful and helpful in the future. However, since few dental academics receive formal training in education, few may have a clear understanding of educational principles in the same way that they might have an understanding of, say, dental disease. This means that an understanding of education is being redeveloped from first principles within the discipline. In general terms, assessments of teaching quality are likely to drive improvements in dental education and there have been initiatives both in the past and more recently, such as the ILT, which carry potential to be helpful in this context. However, until dental academics begin to engage with these issues in a systematic way, it seems unlikely that the field will be able to progress to a more rigorous study of dental education.