The life and death of Webfuse: What’s wrong with industrial e-learning and how to fix it

The following is a collection of presentation resources (i.e. the slides) for an ASCILITE’2012 of this paper. The paper and presentation are a summary of the outcomes my PhD work. The thesis goes into much more detail.

Abstract

Drawing on the 14-year life and death of an integrated online learning environment used by tens of thousands of people, this paper argues that many of the principles and practices underpinning industrial e-learning – the current dominant institutional model – are inappropriate. The paper illustrates how industrial e-learning can limit outcomes of tertiary e-learning and limits the abilities of universities to respond to uncertainty and effectively explore the future of learning. It limits their ability to learn. The paper proposes one alternate set of successfully implemented principles and practices as being more appropriate for institutions seeking to learn for the future and lead in a climate of change.

Slides

The slides are available on Slideshare and should show up below. These slides are the extended version, prior to the cutting required to fit within the 20 minute time limit.

References

Arnott, D. (2006). Cognitive biases and decision support systems development: a design science approach. Information Systems Journal, 16, 55–78.

Behrens, S., Jamieson, K., Jones, D., & Cranston, M. (2005). Predicting system success using the Technology Acceptance Model: A case study. 16th Australasian Conference on Information Systems. Sydney.

Brews, P., & Hunt, M. (1999). Learning to plan and planning to learn: Resolving the planning school/learning school debate. Strategic Management, 20(10), 889–913.

Cecez-Kecmanovic, D., Janson, M., & Brown, A. (2002). The rationality framework for a critical study of information systems. Journal of Information Technology, 17, 215–227.

Central Queensland University. (2004). Faculty teaching and learning report. Rockhampton, Australia.

Davenport, T. (1998). Putting the Enterprise into the Enterprise System. Harvard Business Review, 76(4), 121–131.

Dede, C. (2008). Theoretical perspectives influencing the use of information technology in teaching and learning. In J. Voogt & G. Knezek (Eds.), (pp. 43–59). New York: Springer.

Dillard, J., & Yuthas, K. (2006). Enterprise resource planning systems and communicative action. Critical Perspectives on Accounting, 17(2-3), 202–223.

Fleming, P., & Spicer, A. (2003). Working at a cynical distance: Implications for power, subjectivity and resistance. Organization, 10(1), 157–179.

Haywood, T. (2002). Defining moments: Tension between richness and reach. In W. Dutton & B. Loader (Eds.), (pp. 39–49). London: Routledge.

Hutchins, E. (1991). Organizing work by adaptation. Organization Science, 2(1), 14–39.

Introna, L. (1996). Notes on ateleological information systems development. Information Technology & People, 9(4), 20–39.

Jamieson, K., & Hyland, P. (2006). Factors that influence Information Systems decisions and outcomes: A summary of key themes from four case studies. Adelaide, Australia.

Jones, D. (1996). Solving Some Problems of University Education: A Case Study. In R. Debreceny & A. Ellis (Eds.), Proceedings of AusWebÕ96 (pp. 243–252). Gold Coast, QLD: Southern Cross University Press.

Jones, D. (2002). Student Feedback, Anonymity, Observable Change and Course Barometers. In P. Barker & S. Rebelsky (Eds.), World Conference on Educational Multimedia, Hypermedia and Telecommunications 2002 (pp. 884–889). Denver, Colorado: AACE.

Jones, D. (2003). Course Barometers: Lessons gained from the widespread use of anonymous online formative evaluation. QUT, Brisbane.

Jones, D., & Buchanan, R. (1996). The design of an integrated online learning environment. In A. Christie, B. Vaughan, & P. James (Eds.), Making New Connections, asciliteÕ1996 (pp. 331–345). Adelaide.

Jones, D., & Luck, J. (2009). Blog Aggregation Management: Reducing the Aggravation of Managing Student Blogging. In G. Siemns & C. Fulford (Eds.), World Conference on Educational Multimedia, Hypermedia and Telecommunications 2009 (pp. 398–406). Chesapeake, VA: AACE.

Jones, N., & OÕShea, J. (2004). Challenging hierarchies: The impact of e-learning. Higher Education, 48, 379–395.

Katz, R. (2003). Balancing Technology and Tradition: The Example of Course Management Systems. EDUCAUSE Review, 38(4), 48–59.

Kurtz, C., & Snowden, D. (2007). Bramble Bushes in a Thicket: Narrative and the intangiables of learning networks. In M. Gibbert & T. Durand (Eds.), . Blackwell.

Laurillard, D. (2002). Rethinking University Teaching: A Conversational Framework for the Effective Use of Learning Technologies. London: Routledge.

Light, B., Holland, C. P., & Wills, K. (2001). ERP and best of breed: a comparative analysis. Business Process Management Journal, 7(3), 216–224.

March, J. (1991). Exploration and exploitation in organizational learning. Organization Science, 2(1), 71–87.

Mintzberg, H. (1989). Mintzberg on Management, Inside our Strange World of Organisations. New York: Free Press.

Morgan, Glenda. (2003). Faculty use of course management systems. Educause Centre for Applied Research.

Morgan, Glenn. (1992). Marketing discourse and practice: Towards a critical analysis. In M. Alvesson & H. Willmott (Eds.), (pp. 136–158). London: SAGE.

Pozzebon, M., Titah, R., & Pinsonneault, A. (2006). Combining social shaping of technology and communicative action theory for understanding rhetorical closuer in IT. Information Technology & People, 19(3), 244–271.

Robey, D., Ross, W., & Boudreau, M.-C. (2002). Learning to implement enterprise systems: An exploratory study of the dialectics of change. Journal of Management Information Systems, 19(1), 17–46.

Rossi, D., & Luck, J. (2011). Wrestling, wrangling and reaping: An exploration of educational practice and the transference of academic knowledge and skill in online learning contexts. Studies in Learning, Evaluation, Innovation and Development, 8(1), 60–75.

Seely Brown, J., Collins, A., & Duguid, P. (1989). Situated cognition and the culture of learning. Educational Researcher, 18(1), 32–42.

Seely-Brown, J., & Hagel, J. (2005). From push to pull: The next frontier of innovation. The McKinsey Quarterly. McKinsey & Company.

Simon, H. (1991). Bounded rationality and organizational learning. Organization Science, 2(1), 125–134.

Sturgess, P., & Nouwens, F. (2004). Evaluation of online learning management systems. Turkish Online Journal of Distance Education, 5(3).

Thomas, J. (2012). Universities canÕt all be the same – it’s time we embraced diversity. The Conversation. Retrieved June 28, 2012, from http://theconversation.edu.au/universities-cant-all-be-the-same-its-time-we-embraced-diversity-7379

Truex, Duane, Baskerville, R., & Travis, J. (2000). Amethodical systems development: the deferred meaning of systems development methods. Accounting Management and Information Technologies, 10, 53–79.

Truex, Duanne, Baskerville, R., & Klein, H. (1999). Growing systems in emergent organizations. Communications of the ACM, 42(8), 117–123.

Tversky, A., & Kahneman, D. (1974). Judgment under uncertainty: Heuristics and biases. Science, 185(4157), 1124–1131.

Underwood, J., & Dillon, G. (2011). Chasing dreams and recognising realities: teachersÕ responses to ICT. Technology, Pedagogy and Education, 20(3), 317–330. doi:10.1080/1475939X.2011.610932

Wagner, E., Scott, S., & Galliers, R. (2006). The creation of Òbest practiceÓ software: Myth, reality and ethics. Information and Organization, 16(3), 251–275.

Weick, K., & Quinn, R. (1999). Organizational change and development. Annual Review of Psychology, 50, 361–386.

Ateleological travels in a teleological world: Past and future journeys around ICTs in education

In my previous academic life, I never really saw the point of book chapters as a publication form. For a variety of reasons, however, my next phase in academia appears likely to involve an increasing number of book chapters. The need for the first such chapter has arisen this week and the first draft is due by February next year, which is a timeline to give me just a little pause for thought. (There is a chance that this book might end up as a special edition of a journal)

What’s you perception of book chapters as a form of academic publication? Am particularly interested in the view from the education field.

What follows is a first stab at an abstract for the book chapter. The title for the book/special edition is “Meanings for in and of education research”. The current working title for my contribution is the title to this post: “Ateleological travels in a teleological world: Past and future journeys around ICTs in education”.

Abstract

The Australian Federal Government are just one of a gaggle of global stakeholders suggesting that Information and Communication Technologies are contributing to the creation a brave, new, digital world. Such a digital world is seen as being radically different to what has gone before and consequently demanding a radically different education system to prepare the next generation of learners. A task that is easier said than done. This chapter argues that the difficulties associated with this task arise because the meanings underpinning the design of education systems for the digital world are decidedly inappropriate and ill-suited for the nature of the digital world. The chapter draws upon 15+ years of research formulating an Information Systems Design Theory for emergent e-learning systems for universities to critically examine these commonly accepted meanings, suggest alternate and more appropriate meanings, and discuss the potential implications that these alternate meanings hold for the practice of education and education research.

The plan

The plan is that this chapter/paper will reflect on the primary focus of my research over recent years and encourage me to think of future research directions and approaches. Obviously it will draw on the PhD research and in particular the Ps Framework and the presentation I gave at EdMedia a couple of years ago. It will also draw on the presentation I gave analysing the Digital Education Revolution as part of my GDLT studies this year.

The insanity of changing LMSes/VLEs

There is a definition of insanity that I’ve seen seen attributed to either Einstein or Benjamin Franklin,

“The definition of insanity is doing the same thing over and over again and expecting different results”

That quote, at least for me, has connections with one of more certain origins.

Those who cannot remember the past are condemned to repeat it.

which comes from George Santayana

The connection with LMSes and e-learning

There is an orthodoxy in e-learning at universities. Implement a learning management system like Blackboard, Moodle, Sakai…. Different names, slightly different features but essentially the same type of tool. A big integrated “ring to rule them all”.

At least going by the literature I read and the experience I have the success of LMSes has been far from good. Either the LMS is rarely used or what use it is put to is at a very low level in terms of quality learning and teaching.

Given this is known, then why are many universities up to their second, third and even fourth learning management system? Why are they doing the same thing over and over again and expecting different results?

My answer

In the following presentation I give my answer, which is essentially

  • Implementation of e-learning is really complex and requires a mix of skills and knowledge.
  • It’s easier to adopt a fad – the LMS – than engage with the complexity.

To some extent, this might have some connection with the idea of task corruption.

Those who disagree with the definition of insanity

There is not universal agreement on the validity or source of the “Einstein” quote. George Sanger has a post titled “The definition of Insanity is, perhaps, using that quote”. Of course, I and a number of the folk commenting on the post disagree.

One of the most credible seeming points made against this quote is

It contradicts the notions of experimentation and practice.

Which, on reflection, doesn’t apply. For me at least, experimentation and practice, means that you will not be doing the same thing again and again. You will be trying slightly different things. Each time you practice you will be working to improve what you are doing, to learn from your mistakes.

The source of the insanity quote

Wikiquote attributes the insanity quote to Rita Mae Brown and her book “Sudden Death”

Task corruption in teaching @ university – negative impact of Place?

Busy being a good boy working on the thesis, currently reading a collection of literature to flesh out Chapter 2 which is drawing on the Ps Framework to illustrate the current state of e-learning within Universities. As the last post illustrates, the most recent paper I’m reading is White (2006).

The Ps Framework: a messy version

In her concluding remarks, White draws on the idea of task corruption suggested by Jan Chapman (1996) to describe some of the negative impacts of broader societal issues on learning and teaching at Universities. I’m attracted to this idea for two reasons:

  1. Increasingly I’ve thought most learning and teaching at universities is increasingly of less than stellar quality and “task corruption” provides an interesting (and at current glance, appropriate) perspective on why.
  2. It reinforces the potentially negative impacts that “Place” (one of the components of the Ps Framework) can have on the practice of learning and teaching (again one of many).

In the following, I’m trying to explain what task corruption is and explore what impact it might have on learning and teaching, and particularly e-learning (topic of the thesis), within universities.

What is “task corruption”

Task corruption is where either an institution or individual, conciously or unconsciously, adopts a process or an approach to a primary task that either avoids or destroys the task. Yesterday’s Dilbert cartoon – see below – is a great example.

Dilbert.com

White (2006) identifies two types of task corruption:

  1. amputation; and
    Where parts of the task are no longer performed or are ‘starved’ of attention at the expense of other parts of the task. White (2006) uses the following quote from one of the students she talked with as an example

    I personally believe that the way universities are run today is not necessarily in the best interests of students, but rather in securing numbers to generate a wealthy university and to establish research programs and post graduate programs rather than focusing on the majority of student who come to study in undergraduate degrees.

    I don’t think it would be too hard at some institutions to find a similar student quote in relation to full-fee paying overseas students at commercial campuses.

  2. simulation.
    Where the system or the individual is seen to comply with the task. i.e. they adopt the appearance of task engagement with the aim of avoiding real engagement.

    Perhaps an example of this is what happens in response to a rule at one organisation that states a course shall have no more than 2 assignments, if there is an exam worth more than a certain percentage. If you check course profiles this rule has essentially been followed. But scratch the surface and you find multi-part assignments, including sub-parts that have different due dates than other sub-parts of the same assignment.

Drawing further on Chapman, there is the observation that task corruption occurs most frequently with tasks where it is difficult to define or measure the quality of service (learning and teaching anyone?). Consequently, incentives (or punishiments) are based on quantity rather than quality. This certainly has resonances with personal experience and the almost exclusionary concern about end of term on failure rates, rather than on actual quality of learning.

Sadly, the most recent discussion of this work (Chapman, 2003) is in a journal to which I don’t currently have access. But it is interesting enough to follow up.

Implications for universities

My interest is in how you improve learning and teaching at universities, and one in particular. What implications do these ideas have for that?

Perhaps the most important one I can think of at the moment is to increase awareness of task corruption.

My feel is that task corruption is the dirty little secret of learning and teaching. Most people are aware it goes on, can probably point to some examples but professional pride (and perhaps other reasons) will prevent them from admitting this in a broader sense. In my experience management, especially those at a senior level, have developed the ability to ignore task corruption.

A certain sense of abstraction at the senior management level is good, otherwise you’d never get anything done. But perhaps, it’s been taken too far. Looking for and talking about the forms of task corruption within a university around learning and teaching could be a good first step in identifying those factors within the organisational and the social setting that are contributing to the task corruption. Hopefully as a first step in addressing these problems (and who says I can’t be wildly optimistic).

The problem isn’t limited to senior management. In an organisation that places emphasis on top-down, teleological design procesess the problem is (I believe) likely to occur within instructional design groups, information technology “support” groups etc.

References

Chapman, J. (1996). Hatred and corruption of task (Australian Institute of Social Analysis Working Paper No. 3). Carlton: AISA.

Chapman, J. (2003). Hatred and corruption of task. Organisational and Social Dynamics, 3(1):40-60

White, N. (2006). “Tertiary education in the Noughties: the student perspective.” Higher Education Research & Development 25(3): 231-246.

An information systems design theory for e-learning

Yesterday I gave a presentation at the Australian National University on my PhD. I’m doing it through ANU and this 30 minute presentation is a standard requirement of study. The slides are up on slideshare (embedded below). I recorded the audio and will be trying to put that online later on today and make the slides into a slidecast.

The presentation

It appears embedding the presentation in this post isn’t working at the moment. The slides can be found here on slideshare. — seems the embedding is working now.

The description

It’s been a while since I worked directly on the PhD and creating this presentation was a way to become deeply familiar with the thesis again, in preparation for writing it up. So the presentation is structured in line with the thesis and provides a high level overview of the whole thing.

While the information systems design theory (ISDT) that is the main product of the thesis gets a mention, explaining the design theory is not the primary goal of the presentation. Such descriptions have been given in other papers (Jones and Gregor, 2002; Jones and Gregor, 2004). Instead the emphasis of the presentation is on the other components of the thesis that are in need of some extra work.

Most of the content of the presentation is focused on chapter 2 and the Ps Framework. In fact, must of it is related around the content of a paper I’ve proposed for later in the year.

Essentially the idea is that the practice of e-learning within universities has a definite orthodoxy (which LMS will we adopt). I suggest that for a number of reasons the understandings that underpin that orthodoxy are entirely inappropriate and this is why most university e-learning implementations are plagued by less than widespread use by academics, low quality learning by those that do use it and some concerns around return on investment.

There’s also some early work on the structure of chapter 3 – the research method. But still early days there.

References

Jones, D. and S. Gregor (2004). An information systems design theory for e-learning. Managing New Wave Information Systems: Enterprise, Government and Society, Proceedings of the 15th Australasian Conference on Information Systems, Hobart, Tasmania.

Jones, D. and S. Gregor (2006). The formulation of an Information Systems Design Theory for E-Learning. First International Conference on Design Science Research in Information Systems and Technology, Claremont, CA.

Frameworks and representation – tidy versus messy

I’m a fan of frameworks and taxonomies. Also known as theories for understanding (Gregor, 2006). It’s the understanding part that I like. They provide, or at least good ones do, a leg up in understanding difficult concepts. As Mischra and Koehler (2006, p 1019) say

Having a framework goes beyond merely identifying problems with current approaches; it offers new ways of looking at and perceiving phenomena and offers information on which to base sound, pragmatic decision making.

As it happens, I’m currently doing a lot of work around one framework and its application and the following arises out of that work.

Two of my current most used frameworks include Dave Snowden’s cynefin framework (Snowden and Boone, 2007) and Mischra and Koehler’s TPACK (2006). Representation is important to frameworks. The cynefin framework, in particular, has a very specific representation that has very specific meaning and purpose.

TPACK framework

The TPACK crew have just released an updated representation of their framework (see the image to the left). I particularly like the addition of ‘contexts’ around the outside. The use of ven diagrams is important, one of the contributions of TPACK is the overlaps.

Tidy versus messy

One of the things I don’t like about frameworks is that they have (for very good reasons) to be tidy. This certainly helps understanding, a key purpose of frameworks, but it also can give the false impression of tidiness, of simplicity of a tame problem. My interest is currently in e-learning within universities, which I consider to be extremely messy. To me it is an example of a wicked problem.

A message version of TPACK

Last week I ran a session on course analysis and design for some CQUniversity academic staff. I used TPACK as one of the major themes. However, at one point I really wanted to emphasise to the participants that none of our discussions should be taken to assume that this is a neat and simple problem. The image to the left is the one I used to reinforce this (they’d already seen the tidy version of TPACK).

In doing this, I sacrificed much of the representational value of TPACK to highlight the messiness involved.

The Ps Framework – Tidy versus messy

For about 3 years (this presentation is the first public evidence) I’ve been working on what is now known as the Ps Framework as part of my PhD.

The first representation of the Ps Framework, taken from the first presentation is included below. A photo of some frozen peas used as a “pun”. The arrows are included, but don’t really mean anything. Still very early days.

Version 1 of the Ps Framework

The next public iteration of a graphical representation of the Ps Framework was the following one for a more recent presentation (you can even watch the video of this one). In this “Place” becomes the underlying context for all the other Ps. Much like the addition of context in TPACK. The frozen peas disappear for nice tidy circles (to some extent each one is meant to be a pea) and the arrows are still there. The arrows are meant to indicate that each of the Ps impacts upon the other in some unspecified way.

Version 2 of the Ps Framework

I had to prepare the above images to deadlines for presentations. I never liked them. Too tidy and they appeared to indicate linear or simple connections between the individual Ps. I don’t believe that. The relationships between these Ps when talking about e-learning implementation within universities is messy, complex and unpredictable – at least beforehand.

So I had to come up with something else. For the last few months of last year Jocene, Nathaniel and I spent a lot of time discussing and arguing about how to represent the Ps Framework. The following is my current best effort – it’s the effort I’ll be using this week at ANU.

a messy version

I have a range of problems with this representation including:

  • It’s still a little too structured.
    i.e. People only overlaps with Past Experience, Purpose and Process. Those overlaps aren’t intentional. They aren’t meant to represent some specific connection. I’m not sure what the connections are, I have an inkling that each and everyone is connected/overlaps with the other but I am stuck with this current conceptualisation.
  • It’s too static.
    The relationships between these components is forever changing. Universities and the place they inhabit are continually changing, each of the other components are changing and each change has some, unpredictable impact on the other components. In my mind I see this dynamic representation of this image where each component is seething and roiling and impacting upon each other.
  • It doesn’t capture perspective.
    Still not certain if this should be another P added to the framework or whether different instantiations of the Ps Framework represent different perspectives. I tend to prefer the latter, but then that leaves unsaid the important point about the perspectives of different groups being very diverse and that this is one of the fundamental problems with e-learning within universities.

Any suggestions?

References

Gregor, S. (2006). “The nature of theory in information systems.” MIS Quarterly 30(3): 611-642.

Mishra, P. and M. Koehler (2006). “Technological pedagogical content knowledge: A framework for teacher knowledge.” Teachers College Record 108(6): 1017-1054.

Snowden, D.J. Boone, M. “A Leader’s Framework for Decision Making”. Harvard Business Review, November 2007, pp. 69-76.

Barriers to innovation in organisations: teleological processes, organisational structures and stepwise refinement

This video speaks to me on so many levels. It summarises many of the problems I have faced and encountered trying to implement innovative approaches to e-learning at universities over the last 15 plus year. I’m sure I am not alone.

Today, I’ve spent a lot of time not directly related to what I wanted to achieve. Consequently, I had planned not to do or look at anything else until I’d finished. But this video resonates so strongly that I couldn’t resist watching, downloading it and blogging it.

I came across the video from a post by Punya Mishra. Some more on this after the video. I should also link to the blog post on the OpenNASA site. Would your University/organisation produce something similar?

If Nona ever gets around to watching this video, I am sure she will see me in a slightly different role in the video. Until recently I had the misfortune to be in the naysayer role. That’s no longer the case. Who said no good could come of organisational restructures?

Barriers to innovation and inclusion

The benefits of being open

Coming across this video, provides further evidence to support an earlier post I made today on the values of being open. I became aware of Punya’s post because of the following process:

  • Almost a year ago Punya published this post on his blog that openly shares the video of a keynote he and Mat Koehler gave.
  • I came across it not long afterwards through my interest in TPACK (formerly known as TPCK).
  • About two weeks ago I decided to use part of the video in some sessions I was running on course analysis and design.
  • A couple of days ago I blogged on an important part of the presentation (not used in the sessions I ran) that resonated with my PhD work.
  • My blog software told Punya’s blog software about my post and added it as a comment to his blog.
  • This afternoon Google Alerts sent me an email that this page on Punya’s blog was linking to my blog (because of the comment – see the comments section in the right hand menu).
  • Out of interest (some might say in the interest of procrastination) I followed the link and saw the video.

I plan to use parts of this video in future presentations around my PhD research. I believe it will resonate with people so much better than me simply describing the abstract principles.

So while not directly contributing to what I wanted to do today. It’s provided with a great advantage in the future.

of a Google Alert I have set on my site. Google emailed me to say that Punya had made this post because his blog software includes a list of the

I’ve spent a lot of time today doing stuff not necessarily directly related to what I wanted to achieve today. To such an extent I’d decided not to blog anymore.