Towards a Theoretical Base for Educational Multimedia Design

Tom Boyle

Learning Technology Research Institute (LTRI)
University of North London
166-220 Holloway Road
London, N7 8DB
UK

t.boyle@unl.ac.uk

Abstract: The aim of this paper is to contribute to the construction of a systematic theoretical base for educational multimedia design. The paper delineates different layers of explanation. It then argues for the interactional layer as the most appropriate for multimedia learning environment design. It proposes 'context' as the central construct at this layer. The relationships between multimedia contexts are explored, especially the concept of different levels of contexts corresponding to different educational demands. Further meta-theoretical clarification on the difference between procedural and declarative modes of explanation precedes the final section of the paper. This section explores how the internal structure, the morphology, of contexts might best be delineated for capture in a systematic knowledge base. The paper argues strongly that this type of theoretical clarification is required if we are to move towards a more systematic, 'scientific' base for the construction of educational multimedia systems.

Keywords: educational multimedia, design, theory.

Commentaries: All JIME articles are published with links to a commentaries area, which includes part of the article's original review debate. Readers are invited to make use of this resource, and to add their own commentaries. The authors, reviewers, and anyone else who has 'subscribed' to this article via the website will receive e-mail copies of your postings.

Introduction

There has been a very rapid expansion of educational multimedia since the early 1990s. The driving factors have been the advent of widely available multimedia computers, the impact of hypertext and hypermedia, and the explosive growth of the World Wide Web. The rapid expansion in activity has led to an increasingly urgent need for the development of a sound theoretical base. Despite the influence of constructivism, and to a lesser extent traditional instructional systems design (ISD), there is no clear formal theory of educational multimedia design. The aim of this article is provide a coherent contribution to the development of this theoretical structure.

There are a number of requirements that such a theory should satisfy. These include the following attributes. It should be:

Universalistic: it should not be proposing a particular ideological position, however attractive this may seem. It should be able to assimilate valid knowledge generated from quite different research traditions (e.g. constructivism and ISD).

Expandable and Open. This area is expanding rapidly. The conceptual base ought to be able to capture new developments in a form that relates them in a clear, structured way to the established body of theoretical knowledge.

Formalisable: the framework should support increasing precision in the representation of concepts and their relationships

Useful: the systematic representation of knowledge cannot be just about educational design, but ought to be also for educational design. It should provide a deep knowledge base to guide the design process.

The main themes of this paper are expounded in a series of sections:

Section 2 deals with the important meta-theoretical issue of layers of explanation and the selection of the appropriate layer of explanation for designing IMLEs (Interactive Multimedia Learning Environments).

Section 3 proposes 'context' as the central explanatory construct at the appropriate layer of explanation

Section 4 expands on the possible structural relationships between IMLEs

Section 5 returns to the meta-theoretical issues - in this case whether knowledge of learning context design is best captured with procedural or declarative modes of knowledge representation.

Section 6 looks at how the morphology of contexts might be delineated in a way that supports formal descriptions.

Layers of Explanation

• The aim of this section is to clarify the issue of 'layers of explanation', and to identify that layer that provides the most secure foundation for a theory of educational multimedia design. There is considerable fuzziness in the literature over which layer of explanation to employ in constructing theoretical understanding. This fuzziness obstructs the construction of a clear, formal theoretical framework to guide design.
• After reviewing a range of psychological theories Anderson (1990) outline three major layers of explanation. These layers are:

the physiological layer,

the cognitive layer and

the 'rational' layer

One might hope that research on the fundamentals of learning would provide guidance for the instructional designer. However, it is also a fact of life that we know very little about the detailed mechanisms of learning. (p. 53).

...the more we are concerned with the practical aspects of instructional design the less importance this ignorance becomes. What matters is that we have some understanding of the situations and conditions that promote effective learning even if we don't really understand what is going on in the learner's head. It is the engineering rather than the science of learning that is important (p53).

Anderson's third layer of explanation is the 'rational' layer. This layer of explanation focuses on the functional adaptation of the person to the environment. It might thus better be described as the interactional layer. Anderson argues that considerable explanatory power can be generated by explanations expressed at this level. Explanations can be expressed at this interactional layer given fairly minimal assumptions about the nature of the underlying cognitive processes. This expresses more formally the practical point made by Hammond. We know a lot about the conditions that promote effective learning even if we do not understand the precise cognitive underpinnings. The job of the designer of IMLEs is to create effective environments for learning. Insights expressed at this interactional layer should thus provide a more direct mapping onto the task of designing multimedia learning environments.

The interactional approach simplifies the relationship between learning theory and educational design. It argues that we often have good robust knowledge of the contextual factors which affect learning. We often know how to vary these to improve learning. The 'problems' created by this knowledge for cognitive psychology - what are the precise underlying processes - are not the problems of the IMLE designer. The task of the designer is to incorporate these contextual factors appropriately in the design. It should be noted that the interactional layer is not inconsistent with the cognitive layer, any more than the cognitive layer is inconsistent with the physiological layer. There is no ban on insights from the physiological or cognitive layers. The approach simply states that explanations at the interactional layer are the simplest and most productive. It also states that all insights from the other layers will have to be translated into changes at the interactional layer if they are to be applied.

The simplicity and clarity of the transfer go beyond the provision of heuristics for changing situational factors. What also carries across is the method of validation of the knowledge. These principles were established in psychology because experimentally (i.e. in controlled situations) they were shown to improve learning. The IMLE designer continues that process by testing the design changes in the system against user behaviour. The designer can then optimise the particular learning situation by adapting the situational variable to achieve an improved effect. These processes are normally called formative or integrative evaluation (Draper, Brown, Henderson and McAteer 1996). There is thus a natural affinity between the technology of IMLE design and psychological insights expressed at the interactional layer.

The proposal is thus that the interactional layer provides the most secure and productive base to build a theory for IMLE design. This proposal leads directly to the question of elucidating the central explanatory constructs to be applied at this layer.

The central explanatory construct

The central explanatory concept proposed at the interactional layer is that of 'context'. The importance of context is echoed in a number of disciplines: psychology (e.g. Donaldson 1978; Bruner 1990), ethnomethodology and situated action research (e.g. Suchman 1987), linguistics (e.g. Halliday 1975; Coulthard 1985) and to a lesser extent film theory (Hodges and Sasnett 1993). Halliday (1975) points out that context must be treated as more than a vague social backdrop to action. It is rather, 'an abstract representation of the relevant environment' (Halliday 1975, p.11). Context is a construction that makes selective, holistic sense of the environment of interaction. This construct then guides adaptive action in that environment, e.g. what type of learning actions to undertake. The central challenge for educational multimedia designers is to create contexts that promote effective learning.

Specific guidance on how to achieve this goal is given in several contributory disciplines to multimedia design, especially linguistics, situated action theory, film theory and psychology. The contributions from film theory, linguistics and psychology are developed in some detail in Boyle (1997). It is not the intention here to rehearse these arguments here in detail. However, a brief r?sum? of these arguments is provided to illustrate how they fit within the overarching theme of this paper.

Hodges and Sasnett (1993) point out that certain traditional concepts from film theory can be carried over into multimedia design by using context as the central organising concept. In film theory the tasks of creating and linking scenes are treated through the use of concepts such as mis-en-scene and montage. Mis-en-scene is concerned with the selection and framing of the content in a scene. Montage concerns the linking of these scenes to create a coherent overall artefact. A scene, however, is essentially a one-way mode of communication. Hodges and Sasnett (1993) argue that the transformation of the central explanatory concept from 'scene' to 'context' entails the incorporation of a major extra design concern: interactivity. They argue that the concept of 'context' can both capture concepts from traditional film theory and enhance this understanding through the addition of the concept of interactivity. From this perspective a context might be visualized as an interactive scene. As designers we engage learners in interactive scenarios. We impact on the learner through our skill in building these contexts.

The key point captured in the quote from Hammond given earlier is that the study of psychology has revealed considerably more about the contextual factors that influence learning than about the underlying cognitive processes involved. There is a central pivot where the primary concerns of psychology and learning technology both join and diverge. Psychologists attempt to create theories to explain the cognitive processes involved in learning (although the behaviourists eschewed this endeavour). However, the role of the learning technologist or educational multimedia designer is quite different. Their role is to construct contexts that promote effective learning. Context is the natural base concept for the learning technologist.

Two major challenges arise in the design of contexts:

• the creation of the internal structure of the context;
• the structuring of contexts in relationship to each other.

The second question will be discussed before discussing possible data-structures for capturing, in a declarative form, the design knowledge involved in constructing contexts for learning.

'Montage': the structural relationships between learning contexts.

A context consists of the framing of content along with associated interactivity. Figure 1 provides an example from a system called DOVE (Boyle, Stevens-Wood, Zhu and Tikka, 1996). The DOVE system provides a kind of virtual field trip in Biology. The topic is the observation and recording of animal behaviour. Figure 1 is a screen from the first learning block. The student observes the video and selects a description of the animal behaviour. The aim of this early section is to help the student distinguish between anthropomorphic and non- anthropomorphic descriptions. In Figure 1 the learner has just made a choice, by clicking on the panel on the right of the screen, and received feedback. The DOVE system frames the main context and associated interactivity in a book-like form. The students engage in the activity on the screen and then page forwards/back, or in later pages jump to a video glossary section. Contexts, however, may have other contexts embedded within them. For example, the video of the animal behaviour along with the appropriate interactivity is presented with the familiar contextual frame of a VCR player. This sub-context has a clear functional role in the main context and the two operate harmoniously together.

Figure 1 Screen from the DOVE system

Figures 2a and 2b provide another illustration of a significant 'montage' relationship between contexts. The screen dumps are from the DFML Web based system (Boyle and Payne 1999). This was developed to complement the book 'Design for Multimedia Learning' (Boyle 1997). Figure 2a is a screen from the site. The site was constructed to support very rapid navigation from a section in the book to the equivalent section on the Web site. The site does not attempt to duplicate the book. It gives access to multimedia resources that expand and illustrate the abstract points made in the book. The panel on the left permits rapid drill down navigation to any section in the site (Figure 2a). The main panel on the right then displays the key points from that section in the book. Opposite the paragraph there may be a link to multimedia resources that illustrate/expand on the key point made in the paragraph. The main context thus provides controlled access to other contexts which have their own framing of content and interactivity. Figure 2 shows one of these contexts activated.

The DFML site is based on a very explicit 'layering' of contexts. The main site operates at the courseware level Ð it covers a substantial curriculum area for a module on interactive multimedia design. The 'micro-contexts' act at the next level down Ð 'resources'. These learning contexts deal with specific themes or issues, e.g. using illusions to illustrate the active nature of perception. The interface between the two levels is kept very clean. It is managed through specific links kept on a separate part of the courseware context screens. This greatly aids portability and re-use, and ongoing development. The resource contexts can all be used independently from this particular courseware context.

Figure 2a: Screen showing a main DFML page

The courseware context in turn 'plugs in to' the (higher level) classware level context, as exemplified in a VLE such as WebCT. This permits the multimedia courseware context to be incorporated in a wider virtual classroom with organised discussion groups etc. The construction of learning contexts can thus be structured on a series of levels. If the interfaces between these levels are kept as simple as possible this greatly facilitates re-use and re-combination.

In the first system (Figure 1) the layering is implicit. The 'VCR' context was authored to be embedded within the wider context; it was not designed to be used as a separate, independent context. The second system (Figures 2a and 2b), however, was designed explicitly on a series of levels. The interfaces between entities at different levels (e.g. courseware and resources) are kept as simple and clean as possible. This greatly facilitates the re-use of the resources in different contexts. It also supports the dynamic, ongoing development of the courseware through the addition of extra multimedia resources.

The manipulation of learning contexts goes beyond the provision of 'static' contexts for action. It may involve changes over time in the context to suit the evolving needs of the learner, e.g. the manipulation of scaffolding. Scaffolding involves the provision and gradual removal of extra contextual support for the learner. In scaffolding the designer first finds a level of contextual support which the learner can handle. This level of extra support in the context is then gradually removed to enable the learner to become a more independent problem solver (for examples see Jackson, Stratford, Krajcik, and Soloway 1996; Linn 1996; Guzdial, Kolodner, Hmelo, Narayanan, Carlson, Rappin, Hubscher, Turns, and Newsletter 1996). It may be noted that the fact that the precise cognitive base for scaffolding is problematic is unimportant for the designer. It provides a powerful tool for the engineering of learning contexts. The goal of a design theory is to articulate, evaluate and capture in a communicable form these 'engineering' techniques.

Figure 2b: Screen with a resource context loaded

Mode of explanation: declarative versus procedural

The next task in exploring 'context' is to delineate the internal structure of contexts in a more analytic and formal way. Before tackling this issue, however, it is useful to clarify the mode of explanation in which we should attempt to capture this knowledge.

How should knowledge of the design options in structuring learning contexts be captured? Strategic approaches as varied as constructivism and ISD have shared the implicit assumption that knowledge should be captured in the procedural mode i.e. as guidelines or instructions on how to do things. It seems natural to express knowledge in this way, given that the purpose is to construct effective learning environments. A 'scientific' approach to a subject aims to produce a body of knowledge that is increasingly comprehensive, systematic and precise. However, the more precise we make a procedural representation, the more prescriptive it becomes. Since there are many rival procedural approaches increasing prescription also leads to increasing fragmentation. A fragmented set of prescriptions does not provide a good theoretical base for design. It is noticeable that knowledge expressed in the constructivist tradition tends to remain at a high level. Concepts like 'authentic contexts', 'rich interaction' and 'collaborative learning' provide powerful heuristics. But they do not provide precise guidance, nor can they be formalized to do so without undermining the spirit of the constructivist approach. The format of representing knowledge is self-limiting: it cannot become too precise for fear of undermining its central ethos.

What might be an appropriate alternative form for knowledge capture and representation? Many disciplines (for example, linguistics, computer science) make a distinction between procedural and declarative representations of knowledge. A declarative approach sets out not how to do something but what can be done. A grammar in linguistics, for example, describes the abstract structure underlying what can be produced in a language. The grammar does not specify what will be done. The creative activation of this knowledge base is at the disposal of the user/speaker. A theory in linguistics is thus quite different from a theory in physics. It provides both for systematic formal description while retaining creativity in the activation of that knowledge. The computer language Prolog in a similar way describes the structure of a knowledge base rather than specifying a set of instructions for action.

This systematic, integrated declaration of knowledge turns out to have some very powerful features. A declarative knowledge base can be activated in many different ways. This might be best illustrated by a common form of declarative representation Ð a map. A map provides a systematic, integrated description of a particular geographic domain. This description, it should be noted, is both abstract and conventionalized. This knowledge base can be used to construct a route from any location on the map, through a series of 'valid' steps, to any other location on the map. This is far more powerful than any set of procedural descriptions. The language Prolog operates in the same way, although in this case the knowledge is captured in textual rather than graphic form.

The argument in this paper is that the systematic capture of knowledge for IMLE design is best based on a declarative approach. This has a number of advantages over a procedural approach:

The declarative approach allows for a systematic, integrated representation of knowledge.

Increasing levels of precision in this knowledge (e.g. as in a map) empower the creativity of the user rather than constrain it as in a procedural approach.

A declarative approach can capture different approaches to design as alternative options for action. It does not reject knowledge because it comes from a rival ideological prescription for action; a declarative approach judges solely on whether a proposal provide a valid option for action. It then seeks to relate that to other options to clarify the choices available

One of the criteria for a good theory is that it ought to be able to assimilate rather than dismiss its predecessors or rivals. The key point here is that the declarative approach does not negate procedural approaches. It aims to provide a descriptive, conceptual representation of the 'action potential' open to the designer. Procedural activation across this declarative landscape may be guided by specific procedural heuristics or guidelines. Creative designers may choose their own paths. The important point is that procedural and declarative representations of knowledge are not alternatives. They operate at different levels of abstraction. It is the deeper declarative level of abstraction that offers the potential for a systematic, unified theoretical base for the discipline.

In order for a declarative approach to succeed it requires an appropriate data structure which will capture the analysis and representation of knowledge in the domain. An important question is thus: what is the appropriate concept or construct for the capture and representation of knowledge of learning environment design? Given the space limitations of this article, and the focus in the paper on fundamentals, it is impossible to go into a detailed exposition. However, the paper will point to one research direction and to reading where a more detailed exposition of these ideas is given

The morphology of contexts: towards the analysis and synthesis of contexts

The paper has already argued that 'context' should be the central explanatory construct. This provides a starting point. A major research task is then how to delineate and analyse the formal internal structure of contexts. This analytic knowledge, captured in an appropriate descriptive form, should provide a knowledge base for IMLE design.

One approach that would capture the key aspects of context in a formal way seems to be provided by context-based approaches in formal linguistics. Systemic Linguistics argues that language has evolved to provide communication in context, and the deep structure of language reflects this fundamental influence. It argues that there are three abstract macro-functions that underpin the production of all linguistic communication (Halliday 1973a, Halliday 1973b). These macro-functions provide the architectural base for the deep structure of grammar. Choices are made in parallel from the options available on these three macro-functions in the creation of all communicative utterances. These macro-functions concern:

the construction of the content of the message - the coherent linking of agents, actions states and objects to convey a message (called the ideational function);

the management of the interpersonal roles and relationships in the communication - whether the message is embedded the form of a statement, question, order etc. (called the interpersonal function);

the integration of all the other elements to create of a coherent overall communicative 'text', e.g. a coherent description stretching over several sentences (called the textual function).

the content structuring macro-function: the selection and structuring of the learning content in the multimedia context;

the interactivity macro-function: designing for user interaction with this content;

the compositional macro-function: the creation of a coherent overall composition, both within and across contexts.

There are strong correspondences between the first two macro-functions and the traditional educational concerns of curriculum (the structuring of the content to be learned) and pedagogy (the structuring of learning interactions). The macro-functions thus synthesise contributions from a number of significant contributory disciplines. These contributions tend to complement one another, and provide a richer picture for the multimedia designer. The third macro-function has no marked parallel in educational theory, but the contributions from linguistics and film theory help to fill out this concept.

It is beyond the scope of this paper to discuss these issues in depth. A detailed discussion of the analysis of learning contexts using this approach, with several worked examples, is provided in Boyle (1997). However, it does point to a significant research challenge: how to capture in a systematic, unified knowledge base the sophisticated options in constructing educational multimedia contexts. Such knowledge might be attached to re-usable learning objects to mark the choices of content structuring, interaction and composition they embody. This would greatly enhance the educational use of learning objects which are at present pedagogically limited (Cowley and Wesson 2000, Hepburn and Place 2000). There are thus considerable practical as well as theoretical benefits from engaging in this task of the systematic capture of design choices and their relationships in the construction of IMLE contexts.

Summary

The paper is concerned with how to construct a systematic theoretical base for educational multimedia design. It has discussed two meta-theoretical issues (examination of the basis on which a theory may be built) and three main theoretical issues.

The meta-theoretical issues concern layer and mode of explanation Ð

There are distinct layers at which a theoretical framework may be constructed. In particular there is a distinction between cognitive and interactional layers of explanation. The paper proposes that the interactional layer is the most appropriate for multimedia learning environment design.

There is a clear and important distinction between procedural and declarative modes of knowledge representation. Explanatory knowledge may be expressed in either mode. However, the declarative mode provides a basis for knowledge representation that is more systematic, unified and precise.

The theoretical issues concern the central explanatory construct, and the internal structure and external structural relationships of this construct -

Context is proposed as the central explanatory construct at the interactional layer.

The paper pointed to the importance of finding data structures for capturing the basic abstract structure of contexts conceived as a trinity of macro-functions involving content structuring, interactivity and the composition.

Multimedia learning environments may consist of several interrelated contexts. The concept of layering of contexts from 'resources' through courseware to classware is particularly important.

A basic proposition of the paper is that this type of theoretical clarification is necessary to support a move from conflicting prescriptions towards a more systematic, 'scientific' basis for computer-based multimedia learning environments.

References

Anderson J. R. (1990) The Adaptive Character of Thought. Lawrence Erlbaum. [cited]

Boyle T. (1997) Design for Multimedia Learning. Prentice Hall. [cited] [cited] [cited] [cited] [cited]

Boyle T., Stevens-Wood B., Zhu F., and Tikka, A. (1996) Structured Learning in Virtual Environments, Computers and Education, 26, 1/3, 41-49. [cited]

Boyle T. and Payne L. (1999) The Design and Evaluation of Web Based Multimedia Learning Resources. Procs. of the Seventh Annual Conference on the Teaching of Computing at the University of Ulster, August 1999 [cited]

Bruner J. S. (1990) Acts of Meaning. Harvard University Press. [cited] [cited]

Coulthard M. (1985) An Introduction to Discourse Analysis. Longman. [cited]

Cowley L. and Wesson J. (2000) Design Patterns for Web-Based Instruction. In J.Bordeau and R. Heller (eds) Procs. of Ed Media 2000, World Conference on Educational Multimedia, Hypermedia and Telecommunications. AACE [cited]

Draper S.W., Brown M.I., Henderson F. and McAteer E. (1996) Integrative Evaluation: An Emerging Role for Classroom Studies of CAL, Computers and Education, 26, 17-32. [cited]

Donaldson M. (1978) Children's Minds. Fontana. [cited]

Gagné R. M. and Briggs L. J. (1979) Principles of Instructional Design. Holt Rinehart and Winston. [cited]

Grabinger R. S. and Dunlap J. C. (1995) Rich Environments for Active Learning: A Definition, ALT-J Association for Learning Technology Journal, 3, No. 2, 5-34. [cited]

Guzdial M., Kolodner J., Hmelo C., Narayanan H., Carlson D., Rappin N., Hubscher R., Turns J. and Newsletter W. (1996) Computer Support for Learning Through Complex Problem Solving, Communications of the ACM, 39, No. 4, 43-45. [cited]

Halliday M. A. K. (1973a) The Functional Basis of Language. Appendix in B. Bernstein, Class Codes and Control, Vol. 3. Routledge and Kegan Paul. [cited]

Halliday M. A. K. (1973b) Explorations in the Functions of Language. Edward Arnold. [cited]

Halliday M. A. K. (1975) Talking One's Way In: A Sociolinguistic Perspective on Language and Learning. In A. Davies (ed.) Problems of language and learning. Heinemann. [cited] [cited] [cited]

Hammond N. (1993) Learning with Hypertext: Problems, Principles and Prospects. In McKnight C., Dillon A. and Richardson J. (Eds) (1993) Hypertext: A Psychological Perspective. Ellis Horwood. [cited]

Hepburn G. and Place G (2000) Learning Objects: Communicating the Pedagogical Potential. In L. Bourdeau and R. Heller (eds) Procs of Ed Media 2000, World Conference on Educational Multimedia, Hypermedia and Telecommunications, AACE. [cited]

Hodges M. E and Sasnett R. M. (1993) Multimedia Computing: Case Studies from MIT Project Athena. Addison-Wesley. [cited] [cited] [cited]

Jackson S. L., Stratford S. J., Krajcik J. and Soloway E. (1996) A Learner-Centred Tool for Students Building Models, Communications of the ACM, 39, No. 4, 48-49. [cited]

Jonassen D. Mayes T. and McAleese R. (1993) A Manifesto for a Constructivist Approach to Uses of Technology in Higher Education. In T. M. Duffy, J. Lowyck, D. H. Jonassen and T. M. Welsh (Eds), Designing Environments for Constructive Learning. Springer-Verlag. [cited]

Laurillard D. (1993) Rethinking University Teaching: A Framework for the Effective Use of Educational Technology. Routledge. [cited]

Linn M. C. (1996) Key to the Information Highway, Communications of the ACM, 39, No. 4, 34-35. [cited]

Perkins D. N. (1991) Technology Meets Constructivism: Do They Make a Marriage?, Educational Technology, 31, No. 5, 18-23. [cited]

Price R.V. (1991) Computer Aided Instruction: A Guide for Authors. Brooks/Cole. [cited]

Suchman L. (1987) Plans and Situated Action. Cambridge University Press. [cited]