Category Archives: Chapter 2

Why is University/LMS e-learning so ugly?

Yesterday, Derek Moore tweeted

University webs require eye candy + brain fare. Puget Sound’s site does both with colour palate & info architecture http://bit.ly/9U1kBN

In a later he also pointed to the fact that even Puget Sounds instance of Moodle looked good. I agreed.

This resonated strongly with me because I and a few colleagues have recently been talking about how most e-learning within Universities and LMS is ugly. Depressing corporate undesign seeking to achieve quality through consistency and instead sinking to being the lowest common denominator. Sorry, I’m starting to mix two of my bete noires:

  1. Most LMS/University e-learning is ugly.
  2. Most of it is based on the assumption that everything must be the same.

Let’s just focus on #1.

I’m using ugly/pretty in the following in the broadest possible sense. Pretty, at its extreme end, is something that resonates postively in the soul as your using it effectively to achieve something useful. It helps you achieve the goal, but you feel good while your doing it, even when you fail and even without knowing why. There’s a thesis or three in this particular topic alone – so I won’t have captured it.

Why might it be ugly? An absence of skill?

Let me be the first to admit that the majority of e-learning that I’ve been responsible for is ugly. This design (used in 100s of course sites) is mostly mine, but has thankfully improved (as much as possible) by other folk. At best you might call that functional. But it doesn’t excite the eyes or resonate. And sadly, it’s probably all downhill from there as you go further back in history.

Even my most recent contribution – BIM – is ugly. If you wish to inflict more pain on your aesthetic sensibility look at this video. BIM rears its ugly head from about 1 minute 22 seconds in.

In my case, these are ugly because of an absence of skill. I’m not a graphic designer, I don’t have training in visual principles. At best I pick up a bit, mostly from what I steal, and then proceed to destroy those principles through my own ineptitude.

But what about organisations? What about the LMS projects like Moodle?

Why might it be ugly? Trying to be too many things to too many?

An LMS is traditionally intended to be a single, integrated system that provides all the functionality required for institutional e-learning. It is trying to be a jack of all trades. To make something so all encompassing look good in its entirety is very difficult. For me, part of looking good is responding to the specifics of a situation in an appropriate way.

It’s also not much use being pretty if you don’t do anything. At some level the developers of an LMS have to focus on making it easy to get the LMS to do things, and that will limit the ability to make it look pretty. The complexity of the LMS development, places limits on making it look pretty.

At some level, the complexity required to implement a system as complex as a LMS also reduces the field of designers who can effectively work with to improve the design of the system.

But what about organisations adopting the LMS, why don’t they have the people to make it look good?

Why might it be ugly? Politics?

The rise of marketing and the “importance of brand” comes with it the idea of everything looking the same. It brings out the “look and feel” police, those folk responsible for ensuring that all visual representations of the organisation capture the brand in accepted ways.

In many ways this is an even worse example of “trying to be too many things”. As the “brand” must capture a full range of print, online and other media. Which can be a bridge too far for many. The complexity kills the ability for the brand to capture and make complete use of the specific media. Worse, often the “brand police” don’t really understand the media and thus can’t see the benefits of the media that could be used to improve the brand.

The brand and the brand police create inertia around the appearance of e-learning. They help enshrine the ugliness.

Then we get into the realm of politics and irrationality. It no longer becomes about aesthetic arguments (difficult at the best of times) it becomes about who plays the game the best, who has the best connection to leadership, who has the established inertia, who can spin the best line.

The call to arms

I think there is some significant value in making e-learning look “pretty”. I think there’s some interesting work to be done in testing that claim and finding out how you make LMS and university e-learning “pretty”.

Some questions for you:

  • Is there already, or can we set up, a gallery of “pretty” LMS/institutional e-learning?
    Perhaps something for Moodle (my immediate interest) but other examples would be fun.
  • What bodies of literature can inform this aim?
    Surely some folk have already done stuff in this area.
  • What might be some interesting ways forward i.e. specific projects to get started?

Embedding behaviour modification – paper summary

A growing interest of mine is an investigation of how the design of the environment and information systems to support university learning and teaching can be improved with a greater consideration given to factors which can help encourage improvement and change. i.e. not just building systems that do a task (e.g. manage a discussion forum) but design a discussion forum that encourages and enables an academic to adopt strategies and tactics that are known to be good. If they choose to.

One aspect of the thinking around this is the idea of behaviour modification. The assumption is that to some extent improving the teaching of academics is about changing their behaviour. The following is a summary of a paper (Nawyn et al, 2006) available here.

The abstract

Ubiquitous computing technologies create new opportunities for preventive healthcare researchers to deploy behavior modification strategies outside of clinical settings. In this paper, we describe how strategies for motivating behavior change might be embedded within usage patterns of a typical electronic device. This interaction model differs substantially from prior approaches to behavioral modification such as CD-ROMs: sensor-enabled technology can drive interventions that are timelier, tailored, subtle, and even fun. To explore these ideas, we developed a prototype system named ViTo. On one level, ViTo functions as a universal remote control for a home entertainment system. The interface of this device, however, is designed in such a way that it may unobtrusively promote a reduction in the user’s television viewing while encouraging an increase in the frequency and quantity of non-sedentary activities. The design of ViTo demonstrates how a variety of behavioral science strategies for motivating behavior change can be carefully woven into the operation of a common consumer electronic device. Results of an exploratory evaluation of a single participant using the system in an instrumented home facility are presented

Summary

Tell’s how a PDA + additional technology was used to embed behaviour modification strategies aimed at decreasing the amount of television watching. Describes a successful test with a single person.

Has some links/references to strategies and research giving principles for how to guide this type of design.

Introduction

Set the scene. Too many Americans watch too much TV, are overweight and don’t get exercise. Reducing TV watching should improve health, if replaced with activities that aren’t sedentary. But difficult because TV watching is addictive and exercise is seen to have high costs and initial experience not so good.

The idea is that “successful behavior modification depends on delivery of motivational strategies at the precise place and time the behavior occurs”. The idea is that “sense-enabled mobile computing technologies” can help achieve this. This work aims to:

  • use technology to disrupt the stimulus-reward cycle of TV watching;
  • decrease the costs of physical activity.

Technology-enabled behavioral modification strategies

Prior work has included knowledge campaigns and clinical interventions – the two most common approaches. Technology used to reduce television usually gatekeepers used to limit student access – not likely to be used by adults. There are exercise-contingent TV activation systems.

More work aimed at increasing physical activity independent of television. Approaches use include measuring activity and providing open loop feedback. i.e. simple, non-intrusive aids to increase activity. The more interactive, just in time feedback may help short-term motiviation – e.g. video games. Also technology interventions that mimic a human trainer.

For those not already exercising small increases in physical activity may be better than intense regimens.

The opportunity: just-in-time interactions

Technological intervention based on the value of: that people respond best to information that is timely, tailored to their situation, often subtle, and easy to process. This intervention uses a PDA device intended to replace the television remote control and adds a graphical interface, built-in program listings, access to a media library, integrated activity management, and interactive games.

It tries to determine the goals of the user and suggest alternatives to watching TV in a timely manner. The addition of wearable acceleration sensors it can also function as a personal trainer.

Challenges

Provide a user experience rewarding enough to be used over time.

Grabbing attention without grabbing time

Prior work on behavior change interventions reveals them to be:

  • resource-intensive, requiring extensive support staff;
  • time-intensive, requiring the user to stop everyday activity to focus on relevant tasks.

This is why the remote is seen as a perfect device. It’s part of the normal experience. Doesn’t need separate time to use.

Sustaining the interaction over time

Behavior change needs to be sustained over years to have a meaningful impact.
Extended use of a device might run the risk of annoyance, so avoided paternalistic or authoritarian strategies. Focus instead on strategies that promote intrinsic motivation and self-reflection. Elements of fun, reward and novelty are used to induce positive affect rather than feelings of guilt.

Avoiding the pitfall of coercion

Temptation of using coercion for motiviation. The likelihood that users will tolerate coercive devices for long is questionable.

Avoiding reliance on extrinsic justification

Optimal outcome of any behavioural intervention is change that persists. Heavy reliance on extrinsic justification – rewards or incentives – may result in dependency that can hurt persistence if removed. Also problems if the undesirable behaviour – watching TV – is the reward for exercise.

Case study

Low cost remote produced from consumer hardware. Laptop provided to manage media library. GUI with finger input.

Provides puzzles that use the TV for display and physical activity for input.

Behavior modification strategies

Most derived from basic research on learning and decision-making – suggestibility, goal-setting and operant conditioning). Examples include:

  • value integration – having persuasive strategies embedded within an application that otherwise provides value to the user increases the likelihood of adoption.
  • reduction – reducing the complexity of a task increases the likelihood that it will be performed.
  • convenience – embedding within something used regularly, increases opportunities for delivery of behaviour change strategies.
  • ease of use – easier to use = more likely to be adopted over long term.
  • intrinsic motivation – incorproating elements of challenge, curiosity and control into an activity can sustain interest.
  • suggestion – you can bias people toward a course of action through even very subtle prompts and cues.
  • encouraging incompatible behaviour – encouragement can be effective
  • disrupting habitual behaviour – eliminate bad habits by the conditions that create them are removed or avoided.
  • goal setting – concrete, achievable goals promote behaviour change by orienting the individual toward a definable outcome.
  • self-monitoring – motivated people can be more effective when able to evaluate progress toward outcome goals.
  • proximal feedback – feedback that occurs during or immediately after an activity has the greatest impact on behaviour change.
  • operant conditioning – increase frequency of desirable behaviour by pairing with rewarding stimuli.
  • shaping – transform an existing behaviour into more desirable one by rewarding successive approximations of the end goal.
  • consistency – draw on the desire of people to have a consistency between what they say and do to help them adhere to stated goals.

Exploratory evaluation

Use it with a single (real life) person to find out what happens.

Done in a specially instrumented apartment, including 3 phases: baseline with normal remote, 12 days at home, 7 days in lab with special remote. Participant not told that this was aimed at changing behaviour around watching TV and physical activity.

Results

Television watching reduced from 133 minutes a day during baseline to 41 minutes during intervention.

Evaluation against the adopted strategies were positive.

Conclusions

Substantial improvement important. Phase strategies in over time. Strategies are initially seen as novel – can use this curiosity. Not all users will react well.

References

Nawyn, J., S. Intille, et al. (2006). Embedding behavior modification strategies into a consumer electronic device: A case study. 8th International Conference on Ubiquitous Computing: 297-314.

Different perspectives on the purpose of the LMS

Antonio Vantaggiato gives one response to a post from Donald Clark titled “Moodle: e-learning’s frankenstein”. Clark’s post is getting a bit of traction because it is being seen as a negative critique of Moodle.

I think part of this problem is the failure to recognise the importance of the perceived purpose to which Moodle (or any LMS) is meant to serve. Just in my local institution, I can see a number of very different perceptions of the purpose behind the adoption of Moodle.

In the following I’m stealing bits of writing I’ve done for the thesis, some of it has appeared in previous posts. This probably makes the following sound somewhat pretentious, but I’ve gotta got some use out of the &%*#$ thesis.

The importance of purpose

Historically and increasingly, at least in my experience, the implementation of e-learning within universities has been done somewhat uncritical with the information technology taken for granted and assumed to be unproblematic. This is somewhat surprising given the nature of the universities and the role academics are meant to take. However, in my experience the selection of institutional LMSs is driven by IT and management with little room for critical thought or theory informed decision making.

Instead they rely on a very techno-rational approach that takes a very narrow perspective of what technology is, how it has effects and how and why it is implicated in social change (Orlikowski and Iacono 2001). A different perspective is that technology serves the goals of those who guide its design and use (Lian 2000).

This is important because many, if not most, universities follow, or at least profess to follow, a purpose driven approach to setting strategic directions (McConachie, Danaher et al. 2005). The implementation of an LMS is being done to achieve specific institutional purposes. The very definition of a teleological design process is to set and achieve objectives, to be purpose driven (Introna, 1996). When an institution engages in selecting an LMS, the purpose is typically set by a small group, usually organisational leaders, who draw on expert knowledge to perform a diagnosis of the current situation in order to identify some ideal future state and how to get there.

Once that purpose is decided, everything the organisation does from then on is about achieving that purpose with maximum efficiency. By definition, any activity or idea that does not move the organisation closer to achieving its stated purpose is seen as inefficient (Jones and Muldoon, 2007).

Differences of purpose

Many of the folk responding to Clark’s post who are defending Moodle have their own notion of the purpose of Moodle, usually how they have used it. Others draw on the purposes espoused by the designer(s) of Moodle. There is little recognition that there exists a diversity of opinions about the purposes of Moodle.

A little of this diversity is represented in discussions about how Moodle is used in individual courses. For example, this comment mentions that Moodle does teacher centered very well. i.e. if a teacher sees the purpose of a course site to distribute information, Moodle can do that. This comment makes the point that Moodle is a tool, the pedagogy is not about the tool, it is about the approach.

Now, while to some extent that is true, I also agree with Kallinikos (2004) that systems can have profound effects on the structuring of work and the forms of human action they enable or constrain.

While Moodle’s designers may have all sorts of wonderful intents with the purpose of Moodle. Within a university the purpose assigned to Moodle by the people implementing it and supporting play a significant part. The processes, structures etc that they put around Moodle within an institutional setting can enable or constrain the purpose seen by the Moodle designers and the purpose seen by the staff and students who will use it.

Moodle/LMS as an integrated enterprise system

Due to the complexity of implementing Moodle for a largish organisation the people driving Moodle implementations within universities are usually IT folk. It is my suggestion that the purpose they perceive of Moodle is that of an integrated, enterprise system. A university’s LMS forms the academic system equivalent of enterprise resource planning (ERP) systems in terms of pedagogical impact and institutional resource consumption (Morgan 2003).

To slightly paraphrase Siemens (2006), the purpose of an LMS for the institution is to provide the organisation with the ability to produce and disseminate information by centralising and controlling services. The LMS model with its nature as an integrated, enterprise system fits the long-term culture of institutional information technology and its primary concern with centralizing and controlling information technology services with a view to reducing costs (Beer and Jones 2008).

An LMS is designed to provide an organisation with all the tools it will need for e-learning. Weller, Pegler and Mason (2005) identify two approaches to the design of an LMS: monolithic or integrated approach, and the best-of-breed approach. The monolithic approach is the predominant approach and seeks to provide all common online learning tools in a single off-the-shelf package (Weller, Pegler et al. 2005).

The evidence of this purpose can be seen when you go to your LMS folk and ask them “I’d like to do X”. The response to this question will generally be not what is the best marriage of pedagogy and technology (the best blended learning) to achieve your goal. The response to this question will generally be “How to do X in the LMS”. Regardless of how much extra work, complexity and just plain inappropriateness doing X in the LMS requires.

If all you have is an LMS, every pedagogical problem is solved by the application of the LMS.

What should the purpose of an LMS be?

BIM is a representation of what I think the purpose of an LMS should be. i.e. the LMS should provide the services that are necessary/fundamental to the university/institution, and only those. Increasingly, most of the services should be fulfilled by services and resources that students and staff already use and control.

BIM provides academics teaching a course a way to aggregate blog posts from students and, if they want to, mark them. Those marks are integrated into the Moodle gradebook. The assumption is that marking/accreditation is one of the main tasks a university performs and that there aren’t external services that currently provide that service.

There are, however, a great many very good and free blog services. So students use their choice of blog provider (or something else that generates RSS/Atom) to create and manage their contributions.

The purpose of the LMS isn’t to provide all services, just those that are required for the institution’s tasks.

Eventually, the term LMS becomes a misnomer. The system isn’t about managing learning. It’s about providing the glue between what the institution has to provide and what the learners are using. The purpose is about achieving the best mix of pedagogy and technology, rather than on how to use the LMS.

This perspective obviously has connections with Jon Mott’s (2010) article and the various folk who have written about this previously.

References

Beer, C. and D. Jones (2008). Learning networks: harnessing the power of online communities for discipline and lifelong learning. Lifelong Learning: reflecting on successes and framing futures. Keynote and refereed papers from the 5th International Lifelong Learning Conference, Rockhampton, Central Queensland University Press.

Jones, D. and N. Muldoon (2007). The teleological reason why ICTs limit choice for university learners and learning. ICT: Providing choices for learners and learning. Proceedings ASCILITE Singapore 2007, Singapore.

Kallinikos, J. (2004). “Deconstructing information packages: Organizational and behavioural implications of ERP systems.” Information Technology & People 17(1): 8-30.

Introna, L. (1996). “Notes on ateleological information systems development.” Information Technology & People 9(4): 20-39.

Lian, A. (2000). “Knowledge transfer and technology in education: Toward a complete learning environment.” Educational Technology & Society 3(3): 13-26.

McConachie, J., P. Danaher, et al. (2005). “Central Queensland University’s Course Management Systems: Accelerator or brake in engaging change?” International Review of Research in Open and Distance Learning 6(1).

Morgan, G. (2003). Faculty use of course management systems, Educause Centre for Applied Research: 97.

Mott, J. (2010). “Envisioning the Post-LMS Era: The Open Learning Network.” EDUCAUSE Quarterly 33(1).

Orlikowski, W. and C. S. Iacono (2001). “Research commentary: desperately seeking the IT in IT research a call to theorizing the IT artifact.” Information Systems Research 12(2): 121-134.

Siemens, G. (2006). “Learning or Management System? A Review of Learning Management System Reviews.” from http://ltc.umanitoba.ca/wordpress/wpcontent/uploads/2006/10/learning-ormanagement-system-with-reference-list.doc.

Weller, M., C. Pegler, et al. (2005). “Students’ experience of component versus integrated virtual learning environments.” Journal of Computer Assisted Learning 21(4): 253-259.

The Ps Framework and the mismatch created by the product and process of industrial e-learning

Last year I gave a a couple of presentations titled “Alternatives for the institutional implementation of e-learning”. In those presentations I essentially argued that what passes for current practice of e-learning in universities – what I now call “industrial e-learning” – suffers from a significant mismatch which, I believe, contributes to most of its limitations. My argument is that the nature of the product (the LMS) and the process (teleological) adopted in industrial e-learning to be completely unsuited to the nature of the people involved in e-learning, the nature of learning and teaching and the nature of universities and the context in which they operate.

Anyone who has been following this blog for some time will recognise that sentiment in any number of posts. But this one is a little different.

This thinking has formed the basis for and arisen out of my PhD thesis which I’m trying to rapidly finish. Over the last couple of weeks I’ve been cutting down the enormous Chapter 2 (literature review) that I wrote last year into something approaching an acceptable size. The basis for this cutting has become the argument used in those presentations.

It is with some relief that I release into the world Chapter 2 of my thesis. It’s an attempt to make the above argument in a formal way and set up the space for my contribution in the form of an information systems design theory (i.e. a better way) for e-learning.

The current state of this draft is that it basically fits together. I’m walking away from it for a couple of weeks to get some head space, before I come back to it and polish it up. Any and all feedback is welcome.

Design processes for teaching

The following is a first draft of a section from my thesis. It will form part of the newly cut down section on Process within chapter 2 (170 pages down to 50). The following tries to say something about the design processes used for teaching within universities. It starts with a characterisation of instructional design, looks at the limitations (referring to some earlier work about teleological and ateleological processes) and seeks to describe what the literature has to say about how teaching academics actually design/plan their courses.

As with previous thesis drafts, this is an early draft, I’ll re-edit and improve later, but thought I’d get this out there.

Design processes for teaching

Having introduced a framework for understanding different types of processes and examining the institutional strategic and institutional learning and teaching processes used by universities, this section examines the types of processes used to plan, develop and run individual university courses or units. It starts with a one description of what this type of process embodies, examines the ideal instructional design process before describing what is known about the processes used by individual university academics. The same tendency toward teleological processes is seen. As can the same problems and limitations that arise from such processes within a context that show significant diversity, levels of uncertainty and human agency.

This section and its topic, while somewhat related to the discussion on Pedagogy (Section cross reference), has a different focus. Pedagogy focuses on what is known about learning and how to improve the learning that occurs within universities. This section examines the processes used to design learning as embodied in individual university courses or units. Without question this design process should be informed by knowledge of pedagogy, but the process itself is worthy of description as there are differing options and perspectives.

Reigeluth (1983) defines instructional design as a set of decision-marking procedures that, given a set of outcomes for student to achieve and knowledge of the context within which they will achieve them, guides the choice and development of effective instructional strategies. Reiser (2001) describes how the field of instructional design arose out of the need for large groups of psychologists and educators to develop training materials for the military services. The backgrounds and skills of these people the materials were developed through instructional principles derived from theory on instructiona, learning and human behaviour (Reiser 2001). After the war this work continued and increasingly training was viewed as a system to be designed and developed using specialised procedures (Reiser 2001).

Models of instructional design still have strong connection to the models developed in the 1950s based on the ADDIE (Analyse, Design, Develop, Implement, Evaluate) process (Irlbeck, Kays et al. 2006). ADDIE is a framework designed for objectivist epistemologies where front-end analysis precedes the development of curricular content (Der-Thanq, Hung et al. 2007). The learning theory used to inform instructional design has moved on from its behaviourist origins, moving through cognitivism, constructivism and slowly into connectivism. However, Winn (1990) identifies three areas where behaviourism still exerts power over the processes used by instructional design: the reductionist premise that you can identify the parts, then you can teach the whole; separation of design from implementation; and the assumption that following good procedures, applied correctly results in good instruction.

Visscher-Voerman and Gustafson (2004) identified four different paradigms for instructional design – instrumental, communicative, pragmatic and artistic – with ADDIE situated within the instrumental paradigm. They found, in confirmation of other studies, that the instrumental paradigm as dominated instructional design and that there are questions about its relevance given recent epistemological and technical developments (Visscher-Voerman and Gustafson 2004). Evidence of this dominance can be found in more recent conceptions of instructional design such as constructive alignment (Biggs 1999). Constructive alignment is based on constructivist theories of learning (Entwistle 2003) and focusing on a shift from teacher-centered to student-centered teaching, (Harvey and Kamvounias 2008) and is an example of outcomes-centred design. Outcomes-centred design is a four step process: definition of learning outcomes; design assessment tasks for students to demonstrate achievement; design learning activities for students to develop the appropriate skills; and identify the content that will underpin the learning activities (Phillips 2005).

This teleological view of the instructional design process has a number of flaws. Some of these flaws arise from the teleological nature of the process. Table 2.1 draws on literature around learning, teaching and instructional design to illustrate that it could be argued that Introna’s (1996) three necessary conditions for a teleological process do not exist in the instructional design context. The following seeks to describe other criticisms of this teleological approach to instructional design that have arisen from the literature, including the observation that it does not match what is known about how teaching academics plan their courses

Table 2.1 – Suggestions that instructional design does not satisfy Introna’s (1996) 3 necessary conditions for teleological processes
Necessary conditions Reality
Stable and predictable system Discipline categories bring differences (Becher and Trowler 2001) and are social constructions, subject to change from within and between disciplines.
If a student finds a learning strategy troubling, the student can switch to another at will. The designer could not have predicted which strategy the student would actually use (Winn 1990).
Traditional instruction design is not responsive enough for a society characterised by rapid change (Gustafson 1995).
Manipulate behaviour Change in student strategy can circumvent the intent of the design, unless the design is extremely adaptable (Winn 1990)
Human behaviour is unpredictable, if not indeterminate, which suggests that attempts to predict and control educational outcomes cannot be successful (Cziko 1989)
Academic freedom in teaching refers to the right to teach a course in a way the academic feels reasonable (Geirsdottir 2009)
Most teachers believe they have considerable autonomy in course planning (Stark 2000)
Accurately determine goals curriculum decision making is characterised by conflict and contradictions and by attempts to guard the interest and power relations within the disciplinary community (Henkel and Kogan 1999).
As the student learns, their mental models change and hence decisions about instructional strategies made now, would be different than those made initially (Winn 1990).
Influences on the choice of teaching approaches adopted are clearly more complex than any simple analytic model can convey (Entwistle 2003)
It cannot be assumed that everything is planned in advance (Levander and Mikkola 2009)
In the real world, no-one is sure what the instructional goals should be (Dick 1995).

The above description of instructional design as a teleological process represents the dominant paradigm of instructional design, but not the only paradigm. This is representative of the more homogeneous view of instructional design built around the ADDIE framework (Visscher-Voerman and Gustafson 2004). While instructional designers do apparently use process-based instructional design models (e.g. ADDIE), a majority of their time is not spent working within such processes nor do they follow them in a rigid fashion (Kenny, Zhang et al. 2005). The design processes used by instructional designers are much more heterogeneous and diverse (Rowland 1992). Dick (1995) suggestions that models, such as ADDIE, are ultimately judged on their usefuleness, not on whether they are good or bad.

Models, such as ADDIE, are most useful in the systematic planning of major revisions of an existing course or the creation of a new course. However, traditional university academics spend relatively little time in systematic planning activities prior to teaching an existing course (Lattuca and Stark 2009). A significant reason for this is that academics are not often required to engage in the development of new courses or major overhauls of existing courses (Stark and Lowther 1988). The pre-dominant practice is teaching an existing course, often a course the academic has taught previously. When this happens, academics spend most of their time fine tuning a course or making minor modifications to material or content (Stark 2000).

It is also known that academics practice: is not described by a rational planning model; generally starts with content and not explicit course objectives; and does not separate planning from implementation (Lattuca and Stark 2009). Since academics have traditionally not been required to document their teaching goals for a course ahead of time it is possible that the actual teaching and learning that occurs is more in line with the teacher’s implicit internalised knowledge and not that described in published course descriptions (Levander and Mikkola 2009). Formal description of the curriculum do not necessarily provide much understanding about how teachers put their curriculum ideas into action (Argyris and Schon 1974).

As stated earlier, the instructional design process can be seen as drawing on the knowledge of learning and instructional design to identify appropriate instructional strategies to achieve required outcomes within a given context. Most university academics do not have this knowledge of learning and instructional design. In addition, these staff rarely read educational literature or call upon any available expert assistance when planning a course (Stark 2000). In the absence of formal qualifications of knowledge in this area, most academics teach in ways they have been taught (Phillips 2005) and/or which fit with disciplinary norms and their recent teaching experience (Entwistle 2003). These in turn influence the conceptions of teaching and learning held by academics, which in turn influences their approaches to teaching as described in a significant body of literature discussed in more detail in (cross reference to just after Figure 2.2 and section 2.7.1).

In seeking to describe what is known about the approaches to teaching used by academics, Richardson (2005) developed the integrated model shown in Figure 2.1. While useful, Entwistle (Entwistle 2003) suggests that the simply analytic models are too simple to capture the full complexity of the decision making that occurs when choosing teaching approaches. Stark (2000) suggests that instructional design is not only a science, but also a creative act, linked to teacher thinking that must be examined contextually, meaning that it is not amenable to a single formula or prescription. Or perhaps to a teleological process.

Figure 2.2 - An integrated model of teachers' approaches to teaching, conceptions of teaching and perceptions of the teaching environment (Richardson 2005)

References

Argyris, C. and D. Schon (1974). Theory in practice: Increasing professional effectiveness. Oxford, England, Jossey-Bass.

Becher, T. and P. Trowler (2001). Academic tribes and territories: Intellectual enquiry and the culture of disciplines. Buckingham, SRHE and Open University Press.

Biggs, J. (1999). Teaching for quality learning at university. Buckingham, Open University Press.

Cziko, G. A. (1989). "Unpredictability and indeterminism in human behavior: arguments and implications for educational research." Educational Researcher 18(3): 17-25.

Der-Thanq, C., D. Hung, et al. (2007). "Educational design as a quest for congruence: The need for alternative learning design tools." British Journal of Educational Technology 38(5): 876-884.

Dick, W. (1995). Enhanced ISD: A response to changing environments for learning and performance. Instructional design fundamentals: a reconsideration. B. Seels. Englewood Cliffs, Educational Technology: 13-20.

Entwistle, N. (2003). Concepts and conceptual frameworks underpinning the ETL Project. Occasional Report 3. Edinburgh, University of Edinburgh.

Geirsdottir, G. (2009). We are caught up in our own world : conceptions of curriculum within three different disciplines at the University of Iceland, Iceland University of Education. PhD: 326.

Gustafson, K. (1995). Instructional design fundamentals: clouds on the horizon. Instructional design fundamentals: a reconsideration. B. Seels. Englewood Cliffs, Educational Technology: 21-30.

Harvey, A. and P. Kamvounias (2008). "Bridging the implementation gap: a teacher-as-learner approach to teaching and learning policy." Higher Education Research & Development 27(1): 31-41.

Henkel, M. and M. Kogan (1999). Changes in Curriculum and Institutional Structures: Responses to Outside Influences in Higher Education Institutions. Innovation and adaptation in higher education. C. Gellert. London, Jessica Kingsley Publishers: 66-92.

Introna, L. (1996). "Notes on ateleological information systems development." Information Technology & People 9(4): 20-39.

Irlbeck, S., E. Kays, et al. (2006). "The Phoenix Rising: Emergent models of instructional design." Distance Education 27(2): 171-185.

Kenny, R., Z. Zhang, et al. (2005). "A review of what instructional designers do: Questions answered and questions not asked." Canadian Journal of Learning and Technology 31(1): 9-26.

Lattuca, L. and J. Stark (2009). Shaping the college curriculum: Academic plans in context. San Francisco, John Wiley & Sons.

Levander, L. and M. Mikkola (2009). "Core curriculum analysis: A tool for educational design." The Journal of Agricultural Education and Extension 15(3): 275-286.

Phillips, R. (2005). "Challenging the primacy of lectures: The dissonance between theory and practice in university teaching." Journal of University Teaching and Learning Practice 2(1): 1-12.

Reigeluth, C. M. (1983). Instructional design: what is it and why is it? Instructional design theories and models. C. M. Reigeluth. Hillsdale, NJ, Lawrence Erlbaum Associates.

Reiser, R. (2001). "A history of instructional design and technology: Part II: A history of instructional design." Educational Technology Research and Development 49(2): 57-67.

Richardson, J. (2005). "Students’ approaches to learning and teachers’ approaches to teaching in higher education." Educational Psychology 25(6): 673-680.

Rowland, G. (1992). "What do instructional designers actually do? An initial investigation of expert practice." Performance Improvement Quarterly 5(2): 65-86.

Stark, J. (2000). "Planning introductory college courses: Content, context and form." Instructional Science 28(5): 413-438.

Stark, J. and M. Lowther (1988). Strengthening the Ties That Bind: Integrating Undergraduate Liberal and Professional Study. Ann Arbor, MI, Professional Preparation Project.

Visscher-Voerman, I. and K. Gustafson (2004). "Paradigms in the theory and practice of education and training design." Educational Technology Research and Development 52(2): 69-89.

Winn, W. (1990). "Some implications of cognitive theory for instructional design." Instructional Science 19(1): 53-69.

Implications arising from the absence of the “sameness of meaning”

A few days ago Stephen Downes – a little unusually – made a sequence of comments/tweets on Twitter around the “sameness of meaning” and its impossibility. Since then I’ve had a number of experiences and discussions that suggest some of the problem associated with learning and teaching policy, process and structure within universities arises because too many people assume that there is sameness of meaning.

Communication and the commonality of meaning

Let’s start with this tweet

Communication isn’t about commonality of meaning. That’s impossible. It’s about being able to at least approximately predict the response.

Further explanation flows from this tweet

What happens is, the word we transmit (and maybe gestures, etc) forms only a small part of the other person’s understanding of what you said

and this one

Your word is only a stimulus; Most of the person’s understanding is based on his prior knowledge, and that is what produces the response

And I particularly like this one as a guideline for how to move forward

“Get me a gazelle” would work just as well if your listener understood that he should deliver a Heineken; meaning doesn’t matter, results do

Implications for design

This talks to me because much of what I do could be broadly called “design”. Mostly it’s around the design of information systems. This means much more than technology. Information systems (in the meaning I am using) also embodies all the other “wetware” (i.e. people and organisational) stuff required for the technology to be used and used effectively.

This definition means that I include the following as design:

  • The design and implementation of training and support for the system.
  • The creation of the policies and procedures around the system.
  • The design of the organisational structures and positions within those structures that will impact on the system.
  • How people are encouraged to make decisions about the system.

As I wrote previously (Jones, 2004) – and really just repeating what others had already said – about the impact of representation/meaning

The formulation of the initial state into an effective representation is crucial to finding an effective design solution (Weber, 2003). Representation has a profound impact on design work (Hevner et al., 2004), particularly on the way in which tasks and problems are conceived (Boland, 2002). How an organisation conceptualises the e-learning problem will significantly influence how it answers the questions of how, why and with what outcomes

The answers that a university arrives at in terms of the how, why and with what outcomes end up embodying a collection of meanings. When the organisation and its members implement e-learning they too often assume that there is a commonality of meaning. Commonality of meaning is a key part of how they represent the system. Consequently, their design is fundamentally based on the idea of commonality of meaning. I think that this is a fatal flaw for much of what is designed.

What follows are some examples of where it doesn’t hold.

Software design

My main current task is the design of BIM (code should be out by Monday at the latest) and today was a day to watch a “clueless user” (she’s actually quite intelligent she just knows little about computers and BIM) interact with BIM. BIM is designed by me. It embodies the meaning that I have formed about BIM and its task over the 3 years or so I’ve been working on it. It also embodies meanings/ideas/understandings that have formed over the last 12/15 years of doing e-learning and developing e-learning systems. That same meaning is informed by my experiences in social media (e.g. this blog)

The “clueless user” is a sessional teaching academic in management/human resources. She’s done a bit of e-learning and used BAM. She doesn’t have very much in the way of detailed mental models about how her computer works, how the Internet works or how Moodle, BIM, blogs and feeds work (or even mean).

Needless to say, having observed the user and the meanings she has demonstrated of BIM, I have a long list of improvements for the interface and operation of BIM. If some of them aren’t made, the other academics going to be using BIM are going to struggle. Understanding her meaning and responding to it has been helpful. It has forced me to reconsider and hopefully improve BIM to better fit with other meanings. It should improve BIM.

Of course, 1 person does not make a universe. But that 1 person being very different from me will help a bit.

Downes tweeted

… and my take-away is that we should be careful not to assume that people see things the same way we do, because invariably they don’t

If I’d assumed the same meaning and left BIM as, there would have been trouble for someone. The academics using the unmodified BIM would have suffered increased levels of frustration of dealing with a new system for which they did not understand the embodied meaning. There will still be some of that, but hopefully not as much.

Downes, on the implications of this

there’s so much room for error in communication we don’t notice that we mean different things, usually, and then it surprises us when we do

I’m a little bit surprised by the level of changes needed in BIM. It’s based on a system that’s been used for 3 years, that has been used by this same “clueless user”.

But by engaging in what I’ve done I’ve opened myself up to that surprise at a stage much earlier where it is simpler for me to respond. Too much of how e-learning is implemented in universities does not allow itself to be exposed to “good” surprise, instead they get “bad” – often hugely problematic – surprise.

Minimum service standards

I know of an institution that has implemented minimum service standards for course websites. The standards have been approved at all the right committees, the designers of the standards have written a paper about it, there has been mention of it some of the training sessions for staff and it is now a couple of weeks out from the start of the first term using these standards.

The meaning being heard from the designers of these standards, at least until very recently, has been “it’s all good”. The meaning being heard from the academic staff now being required to fulfill the standards and complete the accompanying checklist includes: “Where did this come from?” and “How do we comply with it?”

Even some of the designers and promulgators of the standards have different meanings. Perhaps the two extremes of those meanings are:

  • The standards are a stick with which to identify the bad teachers.
  • The standards provide a scaffold within which to have discussions about the design of learning experiences within the LMS.

Now, will this difference of meaning result in a “bad” surprise. I’m not so sure. I think organisations and how they choose to perceive the world has a lot in common with what Downes says about communication

In fact, we mostly don’t detect the errors, there’s a huge tolerance for error in communication, that’s why it works

LMS training

I would characterise the standard approach used to “train” academics how to use a new LMS – or any new system – as:

  • In the months before the release of the system hold numerous training sessions in places and at times that suit the academics.
  • Have the supervisors of the academics, and especially the senior management of the institution, reinforce how important it is to attend these sessions.
  • Within the session seek to get the academics to understand the meaning embodied into the system so they can interact with it.
  • Provide these sessions at a time and place removed from the normal context within which the academics will use the system.
  • Employ a range of technical folk who can easily understand the meaning of the system to explain it to the academics in a way that is very similar to how the technical folk learned it.
  • Assume that at the completion of the training they only need a much lower level of support and training. Generally limited to repeating the original training for new staff and providing front-line helpdesk staff to explain how any problems are due to the academic misunderstanding the meaning embodied in the system.

Can you see how the lack of a commonality of meaning is going to cause problems here. To me it’s obvious that the academics will not get the meaning embodied in the system.

The “clueless user” I mentioned above expressed this understanding of the training she experience.

I did the training in the first batch. Over 6 months ago. I haven’t touched the LMS since. How much do you think I remember?

If there is no commonality of meaning, then what?

Downes suggestion is (remember he’s thinking/tweeting in a different context, but I think it applies)

You need to experiment- Wittgenstein called it a game – to test and feel to see what word evokes what response- there is no common ‘meaning’

Given the impossibility of any commonality of meaning and the huge complexity and diversity of the meaning associated with e-learning, learning, teaching, universities, people and technology, the processes within universities and e-learning should be aimed much more at experimentation, at sharing of meaning, at encouraging surprise and enabling effective response and interaction.

What if the assumption of commonality of meaning remains? You keep operating as if there was commonality of meaning? Downes

if ‘sameness of meaning’ were required, communication would grind to a halt.

Is there value in strategic plans for educational technology

Dave Cormier has recently published a blog post titled Dave’s wildly unscientific survey of technology use in Higher Education. There’s a bunch of interesting stuff there. I especially like Dave’s note on e-portfolios

eportfolios are a vast hidden overhead. They really only make sense if they are portable and accessible to the user. Transferring vast quantities of student held data out of the university every spring seems complicated. Better, maybe, to instruct students to use external services.

Mainly because it aligns with some of my views.

But that’s not the point of this post. This morning Dave tweeted for folk to respond to a comment on the post by Diego Leal on strategic plans for educational technology in universities.

Strategic plans in educational technology are a bugbear of mine. I’ve been writing and thinking about them a lot recently. So I’ve bitten.

Summary

My starting position is that I’m strongly against strategic plans for educational technology in organisations. However, I’m enough of a pragmatist to recognise that – for various reasons (mostly political) – organisations have to have them. If they must have them, they must be very light on specifics and focus on enabling learning and improvement.

My main reason for this is a belief that strategic plans generally embody an assumption about organisations and planning that simply doesn’t exist within universities, especially in the context of educational technology. This mismatch results in strategic plans generally creating or enabling problems.

Important: I don’t believe that the problems with strategic plans (for edtech in higher education) arise because they are implemented badly. I believe problems with strategic plans arise because they are completely inappropriate for edtech in higher education. Strategic plans might work for other purposes, but not this one.

This mismatch leads to the following (amongst others) common problems:

  • Model 1 behaviour (Argyris et al, 1985);
  • Fads, fashions and band wagons (Birnbaum, 2000; Swanson and Ramiller, 2004)
  • Purpose proxies (Introna, 1996);
    i.e. rather than measure good learning and teaching, an institution measures how many people are using the LMS or have a graduate certificate in learning and teaching.
  • Suboptimal stable equilibria (March, 1991)
  • Technology gravity (McDonald & Gibbons, 2009)

Rationale

Introna (1996) identified three necessary conditions for the type of process embedded in a strategic plan to be possible. They are:

  • The behaviour of the system is relatively stable and predictable.
  • The planners are able to manipulate system behaviour.
  • The planners are able to accurately determine goals or criteria for success.

In a recent talk I argued that none of those conditions exist within the practice of learning and teaching in higher education. It’s a point I also argue in a section of my thesis

The alternative?

The talk includes some discussion of some principles of a different approach to the same problem. That alternative is based on the idea of ateleological design suggested by Introna (1996). An idea that is very similar to broader debates in various other areas of research. This section of my thesis describes the two ends of the process spectrum.

It is my position that educational technology in higher education – due to its diversity and rapid pace of change – has to be much further towards the ateleological, emergent, naturalistic or exploitation end of the spectrum.

Statement of biases

I’ve only ever worked at the one institution (for coming up to 20 years) and have been significantly influenced by that experience. Experience which has included spending 6 months developing a strategic plan for Information Technology in Learning and Teaching that was approved by the Academic Board of the institution, used by the IT Division to justify a range of budget claims, thrown out/forgotten, and now, about 5 years later, many of the recommendations are being actioned. The experience also includes spending 7 or so years developing an e-learning system from the bottom up, in spite of the organisational hierarchy.

So I am perhaps not the most objective voice.

References

Argyris, C., R. Putnam, et al. (1985). Action science: Concepts, methods and skills for research and intervention. San Francisco, Jossey-Bass.

Birnbaum, R. (2000). Management Fads in Higher Education: Where They Come From, What They Do, Why They Fail. San Francisco, Jossey-Bass.

Introna, L. (1996). “Notes on ateleological information systems development.” Information Technology & People 9(4): 20-39.

March, J. (1991). “Exploration and exploitation in organizational learning.” Organization Science 2(1): 71-87.

McDonald, J. and A. Gibbons (2009). “Technology I, II, and III: criteria for understanding and improving the practice of instructional technology ” Educational Technology Research and Development 57(3): 377-392.

Swanson, E. B. and N. C. Ramiller (2004). “Innovating mindfully with information technology.” MIS Quarterly 28(4): 553-583.