Evaluating an LMS by understanding the underpinning “model”

Currently, CQUni is undertaking an evaluation of Sakai and Moodle as a replacement for Blackboard as the organisation’s Learning Management System. The evaluation process includes many of the standard activities including

  • Developing a long list of criteria/requirements and comparing each LMS against that criteria.
  • Getting groups of staff and students together to examine/port courses to each system and compare/contrast.

Personally, I feel that while both approaches are worthwhile they fail to be sufficient to provide the organisation with enough detail to inform the decision. The main limitation is that neither approach tends to develop a deep understanding of the affordances and limitations of the systems. They always lead to the situation that after a few months/years of use people can sit back and say, “We didn’t know it would do X”.

A few months, at least, of using these systems in real life courses would provide better insight but is also prohibitively expensive. This post is the start of an attempt to try another approach, which might improve a bit on the existing approaches.

What is the approach?

The approach is based somewhat on some previous ramblings and is based on the assumption that an LMS is a piece of information technology. Consequently, it has a set of data structures, algorithms and interfaces that either make it hard or easy to perform tasks. The idea is that if you engage with and understand the model, you can get some idea about what tasks are hard or difficult.

Now there is an awful lot of distance between saying that and actually doing it. I’m claiming that the following posts are going to achieve anywhere near what is possible to make this work effectively. My existing current context doesn’t allow it.

At best this approach is going to start developing some ideas of what needs to be done and which I didn’t do. Hopefully it might “light” the way, a bit.

Using the concept elsewhere

We’ve actually been working on this approach as a basis for staff development in using an LMS. Based on the assumption that understanding the basics of the model will make things work somewhat easier for folk to use. The first attempt at this is the slidecast prepared by Col Beer and shown below.

Blackboard@CQ Uni

View SlideShare presentation or Upload your own.

What will be done?

Given time constraints I can only work with a single course, from a single designer. More courses, especially those that are different would be better. But I have to live with one. I’ve tried to choose one that is likely to test a broader range of features of the LMSes to minise this. But the approach is still inherently limited by this small sample set.

The chosen course is the T2, 2008 offering of ACCT19064, Auditing and Professional Practice. For 2008 this course underwent a complete re-design driven by two talented and committed members of staff – Nona Muldoon (an instructional designer) and Jenny Kofoed (an accounting academic). As part of this re-design they made use of machinima produced in Second Life. The re-design was found to be particularly successful and has been written about.

The basic steps will be

  1. Explain the model underpinning Blackboard and how it is used within the course.
  2. Seek to understand and explain the model underpinning Moodle and then Sakai.
  3. Identify and differences between the models and how that might impact course design.

Hopefully, all things being equal, you should see a list of posts describing these steps linked below.

3 thoughts on “Evaluating an LMS by understanding the underpinning “model”

  1. Pingback: The model underpinning blackboard and how ACCT19064 uses it « The Weblog of (a) David Jones

  2. joelcfisler

    From what I understand you are only evaluation Sakai and Moodle. Is there a particular reason why you are looking at these two LMS? I would consider also talking a look at OLAT, a Swiss open source LMS that we use here at the University of Zurich. Check http://www.olat.org for more information or demo.olat.org for a demo LMS server.
    Greetings from Zurich
    Joël

  3. G’day Joel,

    Thanks for the comment.

    The choice of Sakai and Moodle is out of my hands. It was an organisational decision which I have to live with.

    Frankly, I think the whole LMS model is broken and is one of the contributing factors to the current woeful state of most e-learning within universities. If I had my choice I would not have adopted any LMS.

    So what’s the alternative, essentially something based on the information systems design theory I’ve been developing over recent years.

    https://davidtjones.wordpress.com/publications/the-formulation-of-an-isdt-for-e-learning/

    But that’s not surprising given my prejudices.

    Some other related posts on my blog

    https://davidtjones.wordpress.com/2007/09/30/its-all-over-no-need-to-select-an-lms/

    https://davidtjones.wordpress.com/2007/06/12/cqus-first-web-20-course-site/

    and this one

    https://davidtjones.wordpress.com/2007/03/13/ateleological-development-as-a-better-way-to-develop-university-e-learning-systems/

    David.

Leave a Reply

Please log in using one of these methods to post your comment:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out /  Change )

Google photo

You are commenting using your Google account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s