Which Moodle (or other LMS) tool is best at support and training?

A question for those with experience of Moodle or other LMS tools/plugins/services etc

Which tools do a really good job of embedded support/training?

i.e. assume the tools is used to support a fairly complex task, but if I’m a first time user of the tool I can access all that I need to know about using the tool effectively from within the tool itself. I don’t need to visit the institutional e-learning support site, contact help desk or Google resources placed online by others.

A “really good job” might be judged on a variety of criteria, including, but not limited to,

• Clear instructions that get you underway quickly.
• Well structured, scaffolded support that minimises the need for calling helpdesk.
• Instructions that move beyond simple technical “how-tos” into offering pedagogical insights.
• Some aspect of how it supports the people using it is different from the run-of-the-mill.

Also, feel free to throw in suggestions of any software in general (most modern, quality computer games are probably good examples) or literature.

Context

I’m finally getting started on my Moodlemoot’AU 2013 presentation and am thinking about how BIM can be improved.

Improving the support BIM provides out of the box has long been an aim. Mostly because I think most of the tools I use don’t do a particularly good job and most of the additional supports (e.g. separate training sessions or separate websites) provided by institutions are also not as good as they could be.

This idea has been percolating for more than a few years originally proposed back in 2010 as “Making the LMS more like the Globe Theatre”.

So, I’m after ideas of where it’s been done well.

The absence of a search function – my current big problem with a Moodle installation

Consider this a plea for suggestions. In particular, consider it a plea for workarounds that I can implement quickly (and painlessly).

The problem

I have a Moodle course site. It has a range of activities, many with a page or two of text that sets the context and explains the task. The image below shows what the activities for one week look like.

Now this works fine if a student works sequentially through the activities. It tracks what they’ve completed etc.

It fails miserably when they want to revisit the page about “X”. They have to remember in which week “X” was talked about, under which activity “X” was addressed.

I have problems doing this and I wrote the stuff.

The “web way” solution

If this was any other website, we’d follow the advice of Jakob Nielsen

Search is one of the most important user interface elements in any large website. As a rule of thumb, sites with more than about 200 pages should offer search.

The “web way” solution would be to have a search engine. But the Moodle installation of the University I teach the course for doesn’t appear to provide this functionality. I believe the only way this can occur is to allow Google to have access to all courses on the site. While there may be reasons for this, it’s not a solution I’m pushing just to solve my problem.

How can I provide my students with a search function? How can I make my course site “of the web” and not “on the web”?

I have heard mention made of being saved by repositories. i.e. Moodle is not a content hosting platform and doesn’t try to be. If you want searchable content, place it in a repository. The trouble is we’re not talking here about large documentation. Just a lot of small pages that are closely wrapped around specific learning activities in Moodle. I’m yet to see an information repository integration that works as seamlessly as I’d expect.

My interim solution

In the absence of any brilliant ideas, it appears that the only way to do this is to create a duplicate website that is actually “of the web”. i.e. one that is indexed by Google. I’m thinking probably a blog with pages set up to match the weeks and other components.

Some have suggested providing the pages as a PDF document (or three). The problem with this is that there is web content (videos, animations etc) embedded throughout. Producing a print document would allow folk to search, but then they wouldn’t have access to the web content (unless they clicked on a link etc).

Producing a second website is by no means a perfect solution, some of its limitations include

• Extra workload for me.
• Large potential to create confusion amongst the students
  e.g. which website do I visit? Which website has the correct content? Do I need to check both websites?
• Loss of some Moodle functionality.
  The course currently uses the Moodle activity completion functionality to allow students to track their completion, but also as part of the assessment. If students start working through the blog version of the website it will lead to “But I already did that activity!” problems.

Surely there has to be a better solution?

Implementing a course barometer in Moodle: A kludge

It’s the start of the second week of the course I’m teaching. I’m directly responsible for 60 odd on-campus students and 130 or so online/distance students. That split reminds me a lot of my teaching at CQU in the mid-1990s. The deja vu continues in terms of getting a feel for how the students are going, how are they responding to the course, its model and content? Back at CQU the solution was inspired by course barometer idea from some University folk in Sweeden.

The original course barometer was a purpose-built application in Webfuse, an “LMS” used at CQU from 1996 through 2009. This post records an initial attempt to recreate something simliar using standard Moodle 1.9 modules.

What?

The barometer is meant to be a simple form that allows the students to

• Indicate whether how they are feeling about the course at the moment: good, bad, or indifferent.
• Provide some free-text comments to supplement the feeling.

Preferably this is done anonymously – previous research has shown that anonymity isn’t as important as doing something with the feedback – and would allow us to break up the students by campus/mode of study.

Some form of report should be generated to allow teaching staff to analyse student responses. One the nice list is a method for staff to respond.

How?

Thanks to @markdrechsler and @mguhlin the Moodle tool possibilities (with links to Moodle 2.2 docs) are :

• Choice,
  Appears that the choice module is limited to MCQs, but I do want the free text response.
• Feedback,
  Looks like this could be the one.
• Questionnaire (though apparently deprecated), and
  Doesn’t appear to be included in the USQ Moodle instance.
• my original idea Quiz.
  As Mark suggested, having the concept of a “right” answer built into the quiz means it’s not great for the purpose of a barometer.

Place with feedback

Time to get familiar with what the Feedback module can do. Add a new Feedback activity and the form provides (which seem the same as those documented here)

• Name and description.
• Timed release of the activity.
• An anonymous option – FTW.
• Allow the students to see the analysis.
  There are two sides to this. Yes is good, allows students to get a sense for how others are going. No it is bad, because of the possibility of “bad” responses. I’ll go with yes.
• Email notification of submissions.
  Will turn this one, will help mitigate the risk of “bad” responses.
• Multiple submit – no.
• It does allow separate groups.
  Wondering if this will provide the separation of students into the different modes.

Creating it’s a fairly simple process. Add the questions. Create a template (allow use of these same questions in other feedback activities). Away we go.

I do wonder if USQ automatically create student groups based on mode of study? And yes they do. And the Feedback module allows separation of students into groups.

Done

Fairly simple to set up and even before I’d formally announced it, one student has submitted their first bit of feedback.

Moodle, blogs, feeds and the Google feed API

Time to tweak the course site again. I attempting to encourage the students to engage with technology, to become digital residents. The assumption is that they will really only be able to design great teaching with ICTs, if the use of ICTs is part of their everyday life. One aspect I’m attempting to encourage is blogging.

To make the blogging process a bit more obvious, I wanted to include some aggregated view of the students’ blogs on the course site to increase the visibility and hopefully the prevalence of blogging. Here’s how I did it with the Google feed API.

What does it look like?

The following image (click on it to see a bigger version) shows what the site looks like now. The new bit is labelled EDC3100 blogs. Every five seconds the link (e.g. “My Animoto video”) scrolls onto the next one. The links are chosen from the 8 most recent blog posts aggregated by this Yahoo pipe.

If you move the mouse over the scrolling blog links, the scrolling pauses. Click on the blog title and you will be taken to the original blog.

How does it work?

The process goes something like this

1. I created this Yahoo pipe to aggregate the feeds.
   Currently the pipe is hard-coded with the feeds of the student blogs. In the future I need to connect this with diigo bookmarks the student blogs so I (or anyone in the group) can add their blog.
2. Eventually found this explanation of Google’s feed API.
   It transforms the RSS feed into some nice HTML that can be placed on a web page.
3. Stuck an iframe in a Moodle label.
   It appears that Google feed API wants to change the head of the HTML, something you can’t easily do in Moodle. So I had to upload a separate web page onto the Moodle service and then use an iframe to include it on the site page.

Reflections and work to do

Time to stop playing with the tech and design some prompts to encourage the students to participate.

Should probably look at putting a “help” or “about” link near the object so students can scratch their itch about what it is.

Need to get the Yahoo pipe interacting with the Diigo group bookmarks.

This was a useful respite from some other work, but in the end the technical aspect won’t be enough with additional work. The work around with the iframe was a bit kludgy. Including the object at the top of the page, does increase scrolling. So I wonder about the value.

I’m also wondering how much of this should be talked about with the students? If feel that an understanding that this form of manipulation of existing systems is important to teachers if they are looking to integrate ICTs into teaching. A bit of the whole Rushkoff, Program or be Programmed ethos.

On the potential flexibility of open source LMS and its limits

Today a mate posted to his blog about a small project he’s involved with. The context of this project seems to be a good opportunity to comment on the potential flexibility of open source LMS and the limits of that flexibility within an institutional context. It’s also an attempt to link it back to the design theory described in my thesis (if you want more of the theory behind the following, look at the thesis).

The following uses Moodle as an example, but I believe that similar limitations exist regardless of the open source LMS. This is in part because a significant limit on the flexibility is not the LMS, but instead the institutional governance processes and associated factors..

The need

In this case, the need is to send students an email with a link to a survey. The link has been personalised based on the students’ membership of Moodle groups. They survey asks them to answer questions based on their experience of a group task.

My initial thought would be that this sounds like something Moodle should be able to do. Given the increasing emphasis on group related work I doubt that this is a novel requirement. So, there might be something in Moodle that can do this, however, based on my limited knowledge of Moodle I can’t think of anything off the top of my head.

I believe that there might be the functionality within Moodle to do each of the components of this task. There is probably a way to send emails to members of a group. There might even be a way to customise that email to some extent (there is a bulk email facility in Moodle 1.9, but, from memory, it seems somewhat limited). There is also probably a way to do a group-based survey (a MCQ might be the obvious solution).

But I doubt that there is an easy way to combine these separate functions so that the group email can automatically include the link to the group’s MCQ/survey.

Doing it outside of Moodle

There is another interesting and related comment in post describing this project

hile not ideal in that it is a separate system from the LMS, it is hoped that this trial will help inform the development of a Moodle module that will perform the same function albeit in a more integrated and seamless way

Over 6 months ago, I used to work at the institution being described. Based on that out-of-date experience, my initial guess is that “doing it outside of Moodle” is deemed to be easier than engaging with Moodle and the institutional IT department.

Two limits of open source LMS flexibility

Drawing on the above examples, I’d like to propose at least two, somewhat related limits on the flexibility of open source LMS:

1. Inflexible institutional structures and processes.
2. The difficulty of producing/the absence of scaffolding conglomerations.

Inflexible institutional structures and processes

Modifying an enterprise implementation of Moodle effectively and efficiently is hard. You don’t want your institution’s Moodle instance to be unavailable to students and staff because a code change has broken something drastically. The Moodle code-base is itself quite difficult to get a handle on. Not overly difficult, but a non-Moodle developer can’t simply front-up and start making changes quickly. They need to be enculturated into the Moodle way, to learn what works and what doesn’t. Such a requirements means that someone who is able to modify Moodle is often a scarce and expensive resources. Especially within most universities who often don’t have someone dedicated as a Moodle developer.

To address this difficulty and also to CYA (some might argue that CYA is the major reason) institution’s spend a lot of time and effort setting up appropriate governance structures. The theory being that these are objective and rational ways to manage the difficult process and the expensive and scarce resource.

The trouble is that the difficulty and expense involved means that it becomes difficult for such processes to effectively engage in “small” problems like this one. i.e. problems that don’t actually require development of any significantly new functionality or large-scale modules. It just needs a few minor changes or wrappers around existing functionality. For example, the requirement above could possibly be solved (the following is an example description given off the top of my head without any investigation as to whether this would work) by

• Modifying the Moodle quiz function to populate a database table linking groups to URLs for group specific quizzes.
• Modifying the existing Moodle bulk-email facility (or perhaps adding a wrapper around it like I did with bim) to use this database table to send personalised emails to group members.
• Perhaps add a new quiz report that allows viewing/comparing within/across groups.

For a variety of reasons traditional institutional LMS policies and processes are too heavy-weight to respond to this sort of need. Instead, in order for something like this to be considered, it has to be blown up into some institutional priority. e.g. a system to support peer and group-based assessment for the entire institution. A project that will require a significant amount of time doing a needs analysis,……..

A big project that requires lots of resources is expensive enough to be efficiently considered by the governance and related processes. Small projects are too cheap to be efficiently considered by the expensive institutional processes.

In the hardware/operating systems field, this is a problem known as starvation or indefinite postponement. The situation where a task is forever ignored because of a flaw in the priority mechanism.

So, I’m proposing that the institutional implementation of open source LMS end up suffer from the “starvation limit” on flexibility.

The need for rapid development of scaffolding conglomerations

The need in this case, at least to me, sounds like an example of what I termed scaffolding, context-sensitive conglomerations. Rather than necessarily requiring a brand new Moodle module or block, this problem sounds like something that actually needs to combine the functionality from a number of existing Moodle services. Something that conglomerates the lower-level functionality provided by Moodle into something that better meets this higher-level need.

A large part of the popularity of Moodle arises from its modularity. A feature that allows for the easy development of lots of new functionality. Something that increases the flexibility of Moodle.

The problem is that this flexibility arises, in part, from keeping these different modules separate. It’s the separation that makes it easy to add a new function without (theoretically) worrying about how it will effect the other modules. They are meant to be independent. The current problems moving to Moodle 2.0 is an example of the problems that arise from dependency. All the third-party modules depend on the Moodle core, so when the Moodle core changes all the third-party modules have to change.

A strict separation between modules makes it more difficult to combine parts of these different modules into a scaffolding conglomeration.

So, I’m proposing that open source LMS have an “over reliance on module independence” that limits their flexibility.

It’s really all about balance

I can already here proponents of traditional institutional IT governance processes or strict software engineers bemoaning the problems of not having institutional governance or of module dependence. And I do agree. There are dangers and problems. I’m not suggesting that they should necessarily be done away with.

I do, however, think that too often the balance has gone too far one way. There needs to be more recognition of a need for balance the other way. A bit less of a focus on the objective, best ways of technical implementation, and a bit more on the subjective, best ways to improve learning and teaching.

Making the LMS more like the Globe Theatre: distributed cognition, the extended mind and Moodle

The following draws together some work on distributed cognition in the Globe Theatre and some early thoughts on how that might be useful in improving the design of learning management systems such as Moodle (there’s also an argument to be made about university teaching and learning, but I’ll mostly leave that alone). While primarily focusing on helping teachers teach and thus improve the student learning experience, the ideas could equally be applied to learners (and probably have been).

At the core of the following seems to be the question of whether o not the design of LMS, and e-learning tools in general, have been limited due to the following observation made by Tribble (2005)

consistently distorted by a tendency to view cognition as individual rather than social, which has caused us to imagine the workings of complex group structures in mechanistic terms. In other words, we have mistakenly assumed that properties of the system as a whole must be possessed by each individual within it.

The post continues to build some recent thoughts about this issue and was sparked by listening to this episode from ABC Radio National show – The Philosophers Zone. This led me to the article Distributing Cognition in the Globe by Tribble (2005).

After finishing the post, the idea of adding “plots” to LMS tools might be the most interesting outcome. Will need to do more reading and thinking. Must be other folk looking at this, pointers?

Cognitive demands of the Elizabethan repertory system

Imagine you are a member of an Elizabethan repertory theatre company in the late 16th century. Just how cognitively demanding is your work? Tribble (2005) draws on a number of sources to give the following examples:

• As a leading player you would have to remember 71 different roles, around 52 of which were new.
• Your company would be putting on 6 plays a week, there would be infrequent repetition, and would have a new play every fortnight.

This appears to be quite different, and cognitively a lot more demanding, than more modern theatre companies.

So how did they do it? Tribble (2005) summarises some literature that suggests it was achieved through “mechanistic” approaches such as: actors performing the same type of role, learning roles by imitation, and having little or not interest in creative interpretation.

How did the Globe and distributed cognition help?

Tribble’s (2005) argument is that the design of theatre system – embedded in the theatres, the plots, actors’ roles, the plays’ verbal structures, the apprentice system, and the organisational practices of the companies – provided elements of cognitive structure that helped individuals and the company to fulfill their cognitive demands. In particular, she argues that the design of the theatres played a part

The more thinking that can be off-loaded onto the environment, the more mental energy remains available for those tasks that are primarily internal (memory for the spoken lines, for instance).

For example, it is argued that “plots” – summaries/directions of when, who enters the scene – hanging back stage from pegs by stage entrances provided players with reminders of the order of scenes. Especially helpful for actors playing multiple small roles. Another is that rather than all players having the full script, they instead have their lines in detail and various cues to indicate when they are to be performed. Similarly, it’s argued that some of the physical spaces within the theatre (e.g. a balcony) provided cognitive hints as to what type of scene you were meant to perform.

The podcast that brought me to Tribble and her work includes a better description starting at about 13m48s. Tribble’s summary

The productive constraint of the stripped down part reduces the need to filter signal (one’s own part) from noise (everyone else’s); the plot provides a schematic diagram of the shape of the play as a whole to supplement the part; the physical space of the theater and the conventions of movement it supports enable the transition from the two-dimensional maps of plot and part to its three-dimensional embodiment onstage; and the structures and protocols of the theatrical company pass on its practices to new members. Such a theater can best be understood, in other words, through a framework that takes group practices seriously, that assumes that systems can work well, and that sees individual agency as constrained but not contained by these practices.

Tribble (2005) draws inspiration from Hutchins (1995) and describes on of his contentions as

that the lines between “inside” and “outside” are frequently misdrawn or misidentified, “creat[ing] the impression that individual minds operate in isolation and encourag[ing] us to mistake the properties of complex sociocultural systems for the properties of individual minds.”

and that when the design of these systems are done well

“Novice[s]” can thus be “embed[ded]” in the system and perform at a level far above that which they could attain by means of their individual cognitive powers alone.

The design of the LMS and the tyranny of the individual mind

It is my suggestion that the design of the LMS (and probably the assumptions underpinning much of the actions around improving university teaching) is limited by the tyranny of the individual mind. Sturgess and Nouwens (2004) describe one universities rationale for an LMS as being

to enable teaching staff to develop and
manage online courses with little professional support

. This fits with the broader practice of university teaching being essentially a solo act. It’s my course, I’ll teach it my way. Give me control of the course on the LMS and let me do my thing.

It is assumed that the academic (or their proxy who has been employed to create the course) has the skills and knowledge to effectively use the features of the LMS to prepare the course site. When adopting a new LMS, a focus for most institutions is on training sessions to provide these skills and knowledge to staff.

But what additional scaffolding or “distributed cognition” does the design of most LMS provide? In my experience it is generally limited to either some canned “help” document or the contact details of a support group. The “help” resources are usually fairly limited and it’s not unusual for them to be out of date. The support group are generally keen and willing to help (often above and beyond the call of duty) but are often limited to knowing where the academic might find a query, or about how to use the LMS features as they exist.

I’m not sure that most LMS are designed in a way such that

“Novice[s]” can thus be “embed[ded]” in the system and perform at a level far above that which they could attain by means of their individual cognitive powers alone.

. The evidence from past experience with LMS seems to indicate that most e-learning is not all that good, perhaps indicative of how well the LMS support distributed cognition.

Some ideas for BIM

BIM is a Moodle module that I’ve written. It’s probably going to be the first place I experiment with some of these ideas. Some initial thoughts:

• Improve the help documentation provided with BIM to include the videos that have been produced and also provide links and summaries of some of the literature around the use of blogs in higher education.
  This is a simple first step that arguably should have been done already.
• Link the help documentation with some the BIM community.
  Rather than just static help documentation, provide mechanisms for asking questions of and seeing answers/resources from a broader BIM community.
• Design a scaffolded “BIM configuration” process informed by common problems and linked to the help documentation, especially the literature.
  As it stands, the BIM configuration process is your typical IT artifact. It is a better representation of the data requirements of BIM than a process that helps academics create and design a BIM activity that is effective.
• Figure out a way to develop standard BIM “plots”.
  There are common events in the use of a BIM activity (e.g. not all students have registered a blog, not all the students within the course have been allocated to groups and markers, etc.). These can be picked up from the BIM interface, but the academic has to apply a fair bit of cognition in understanding how to do this.

  A better approach might be to implement some “plots” (in the sense described above) that suggest when and how to handle a particular event. For example, when BIM sees that not all students in a course are allocated to groups, generate a “plot” entry that includes a list of the unallocated students and links to the Moodle group allocation process and the help documentation associated with groups explaining this problem.

Ideas for Moodle

Some of the ideas for BIM could potentially be applied to any complex process within the use of an LMS/Moodle such as the design of a course site or the management of a discussion forum. This links to some extent to the earlier suggestion of combining analytics and discussion forums.

Misc reflections

Beginning to wonder how much of this is related to the design of computer systems being done by programmers more interested in representing the models underpinning the system, rather than providing cognitive support to the folk using the system?

Wondering how much the type of support given to academics using an LMS within a University context in terms of the skills of the support staff and the organisational structures within which they work limits the provision of appropriate cognitive support?

Wondering if an increasingly hierarchical organisational structure can effectively provide the type of distributed cognition that might actually help improve the LMS and university teaching?

Wondering if the hierarchical organisational structure can effectively grok the need for this type of approach? Can it provide the environment in which this type of work can grow?

References

Hutchins, E. (1995). Cognition in the Wild. Cambridge, MA, MIT Press.

Sturgess, P. and F. Nouwens (2004). “Evaluation of online learning management systems.” Turkish Online Journal of Distance Education 5(3).

Tribble, E. B. (2005). “Distributing cognition in the Globe.” Shakespeare Quarterly 56(2): 135-155.

Trip report – Moodlemoot’AU 2010

The following is a report of my attendance at Moodlemoot’AU 2010 during the first half of this week. The aim is to engage in a touch of reflection, outline tasks to do, and inform colleagues back at CQUniversity about the conference.

My contribution

I was mainly responsible for two presentations at the conference. THe following presentation links include a range of resources, including slides. However, the planned audio/video wasn’t generated. The two talks were:

1. A short show and tell of the idea for adding and harnessing curriculum mapping and alignment within Moodle.
   The 3 minute limit on this presentation was interesting, but was kept to (sort of). Some interest expressed by folk and a couple of links to follow up.
2. A presentation showing off BIM and talking very briefly about limitations in developing innovative pedagogy.
   I decided to focus mostly on showing off BIM and how it worked. That was, I think, a good move. Though as it turns out a bit more though on the limitations side might have gone down well. Some good feedback on this presentation via twitter, more on that below.

I was also somewhat associated with two presentations from the Indicators project. Almost all the work for these presentations was done by Colin Beer and Ken Clark, and a great job they did. Seems there is growing interest in the indicators project, the next year or so looks like being very interesting.

Reflections

For me personally, the conference was – for a variety or reasons – perhaps the most valuable I’ve been to in recent times. Most of the reasons had nothing to do with the actual presentations. There were some interesting presentations, however, it was the connections made and the possibility of future work (in a range of sense) that made it incredibly worthwhile.

This is especially important given that it wass obvious that adoption of Moodle is rapidly expanding with UNE, LaTrobe and Monash announcing moves to Moodle in the months leading up to the conference. Each of these institutions had groups of staff at the conference. In addition, a number of non-Moodle universities also had representatives at the conference. Checking out the competition was the reason given, though I do suspect there are likely to be a few more Australian universities adopting Moodle in the coming years.

This suggests that the institutions that get into Moodle early and effectively have the opportunity to make useful contributions. It also suggests the potential for a critical mass of institutions sharing and collaborating around Moodle and its use for learning, teaching and beyond.

The Gold Rush

The other side to this is the observation that to some extent, there was a feel of a “Moodle” (gold) rush about the conference. Perhaps the settling of the wild west is a better metaphor. Lots of excitement from new settlers exploring a new land, trying to establish how it all works and plan what the future might bring. There were also a few old hands there to help and occasionally shake their head at the new arrivals. There was certainly a sense of excitement amongst the new settlers about the possibilities. I have to admit, that at times and at least for my somewhat cynical tastes, this fervour went a bit too far and on occasion started to take on the air of a gathering of evangelical Christians.

At the same time there was also a sense of there not being any collective history. Without a connection to the land, the settlers were making some fundamental mistakes, implementing practices that don’t make sense in the new land. Reporting and discussing these mistakes at the Moot is a step toward developing a collective history, however, it was somewhat disappointing that some of the insights developed around e-learning, educational technology, distance education and many other “groupings” from past literature weren’t widely known about.

Even more scary is the observation that at times, this lack of awareness, wasn’t limited to individuals new to a field. It was also evident in some of the large scale, “strategic” organisational projects implementing Moodle.

Presentation feedback and twitter

In the past I’ve belonged to academic units where it was compulsory to product a “trip report” on return to the host institution. An almost obligatory or compulsory part of these trip reports was the

My session was received positively by those there.

statement. Given this was a case of self-reporting by the person who gave the presentation, it always felt a bit self-serving. Not to mention that the use of “self-reporting” had to bring into question the validity of the statement.

Of course, I was as guilty as anyone else of using this comment, and in the context of my presentations at this conference,

My sessions was received positively by those there.

However, we’re much more modern now, we have Twitter a tool that’s increasingly being used at conferences to make explicit what was generally implicit. At this conference there was a healthy twitter stream and due to that stream, I have some hard evidence of good comments. See the following image, click it to make it bigger.

Of course, most of the folk shown in that image are people who follow me on twitter. So, the validity of such comments might remain questionable.

Use of innovations by other institutions

One of the nicer aspects of moving to Moodle (from an institution specific system) has been the adoption of the tools I’ve developed (e.g. BIM) by other people at other institutions. Alan Arnold used BIM as an example in his presentation with James Strong about how University of Canberra worked with Netspot to maintain the balance between “staying vanilla” and innovation with UC’s Moodle.

Though I wasn’t too sure about the naming scheme Alan adopted – “CQU BIM”. According to the re-branding, it probably should’ve been (at the least) “CQUni BIM”.

The lack of TPACK

At least for me, there seemed to be a fairly visible division between the teaching academics, the teaching support folk and the information technology folk. There didn’t seem to be a lot of really strong cross-fertilization between the different groups. And that’s before we start talking about management.

The TPACK folk argue that it’s the effective combination of the knowledge that each of these groups hold that is needed to make really innovative and high quality use of IT for learning and teaching. And at least for me, the moot experience bears this out. Most of the really interesting presentations were those that drew on effective combinations of the various different types of knowledge. An example is given below of Michael de Raadt’s presentation.

Marnie Hughes-Warrington

The conference was opened by the Monash PVC for Learning and Teaching – Professor Marnie Hughes-Warrington – who linked Monash’s plans for Moodle and their broader VLE. The comment that stuck with me, probably because it mirrors my own thoughts was

learning with technology is about the connections and relationships

In describing some of the directions they are taking she mentioned that one of the first steps was listening to teaching academic staff and actually fixing the problems they’ve been reporting for the last five years.

There are lessons here for other institutions. At least currently, it appears that Monash are going to be amongst the most interesting to watch of all the institutional adoptions of Moodle. It shall be interesting to learn more about their migration to Moodle, the strategies and thinking underpinning that migration, and the resulting outcomes. In particular, would be interesting to hear from a collection of Monash teaching staff to see how/if their perspective differs from those driving the migration.

Peer review, progress bar, distributive “leadership” and behaviour change

Michael de Raadt – an information technology academic from USQ – gave a presentation on two plugins he’s developed for Moodle: a peer review assignment type and a progress bar block. As an IT academic with an interest in educational research, Michael had first hand experience of a teaching and learning problem, insights into educational solutions, and the technical ability to implement those solutions within Moodle. Both of Michael’s Moodle plugins could be useful for CQU staff and students.

I found the progress bar block particularly interesting. Michael described how students could become almost compulsive about ensuring that the bar was “all green”. The on-going presence of “red” in the bar was visible every time they used Moodle and acted as an encouragement to complete all the tasks, to be more active.

It is this sort of modifications to the Moodle and broader learning and teaching environments within universities that I am most interested in. The progress block appears to be particular effective examples of a “nudge theory” approach to improving learning and teaching.

To some extent, this type of approach is related to a presentation from some folk at ANU titled “Translating Learning Outcomes in Moodle” designed to aid teaching staff make the connection between constructive alignment and the activities available in Moodle. The approach described in this presentation offers some interesting ideas about how the Moodle environment can be extended to improve the capacity of staff to design more aligned activities. In particular, the approach has some potential to compliment the ideas behind the alignment project.

Bridging the gap between Moodle, institutional practice and academic requirements

A number of the presentations at the conference were examining the question of how to automate and/or ease the workload associated with creating Moodle course sites or integrate it with other related organisational processes (e.g. linking it with course outlines/profiles). Though none really seem to have moved beyond fairly limited “administrative automation”. The ANU outcomes approach in the last section looked at this task from another perspective.

No-one seems to be yet moving beyond these fairly limited forms of course site creation. To improving the level of abstraction.

Online assignment submission and management

Perhaps the most obvious collection of presentations at the conference were associated with various questions/issues around online assessment. For example:

• de Raadt’s presentation that discussed his peer review assignment type;
  Essentially a Moodle assignment type that provides a higher level of abstraction to help with managing student peer marking of assignments with a reasonable level of staff oversight.
• a presentation on the Lightwork tool for managing/marking online assignments;
  I didn’t attend this presentation, but Lightwork is a project that’s been going for a while now.
• a presentation around a fair bit, including a Word template/document approach to marking.
  I didn’t go to this either, but know of some folk who did.

The supporting page for the last presentation does make the point that assignment marking and management remains a difficult, time consuming and expensive consideration within universities. Not something that is always done well. A practice within which there is significant capacity for innovation and improvement.

Tasks to do

Throughout a conference there are generally long lists of interesting stuff to follow up upon. After some less than perfect recollection and some reflection, the following is the list of important tasks I need to follow through upon:

• Follow up with Michael de Raadt around getting more insight into how to make BIM “more moodle like”.
• Prepare a video version of the BIM presentation that can be uploaded for folk to view.
  Done: video is available here
• Give more thought to when/how I’ll start moving BIM to Moodle 2.0.
• Talk with Col, Ken and Damien about how and when we continue the development of the Indicators project’s Moodle block.
• Turn the idea behind the BIM presentation into a conference paper and subsequently a journal paper critiquing conceptions and attempts to implemented blended learning.

Whether or not I work on the following list of tasks depends more on the outcome of imminent job applications and interviews (not to mention work on the thesis):

• Follow up with the ANU crew about how and what we might do in partnership around alignment.
• Follow up with Jonathan Moore from Remote-Learner about connections between the alignment project and the work they are doing around K-12 standards in Moodle.
• Try and figure out how the alignment project can be progressed beyond a thought experiment into something concrete.
• Think about how the increasing number of Australian universities adopting Moodle can most appropriately harness this new community and the open source nature of Moodle.
• Think how the insights from OASIS can be combined with other work around online assignment submission and management to develop innovations and improvements, especially given issues at my current institution.
• See how and if both the alignment and indicators projects can be turned into successful cross-institutional projects and also successful ALTC grant applications.
• Would love to see how different Australian universities might review each others Moodle migrations in an attempt to make reporting on these projects more independent and hopefully useful for future action.

There is much, much more to think about and do arising from presentations and conversations at Moodlemoot. I’ve only captured a small sample.

Integrating alignment into Moodle and academic practice: A proposal and a RFI

I’m off to the 2001 Australian MoodleMoot next week. The conference program includes a collection of 3 minute show and tell sessions on the Tuesday afternoon. The following is a summary of what I think I’m going to talk about and a call for suggestions.

I’m starting to add all the associated resources with the presentation to this post.

More information

Other resources/information around this idea include:

• A blog post introducing how curriculum mapping might work in Moodle.
• A detailed, draft grant proposal for a broader project around embedding mapping/alignment into a university.
  This proposal includes a fairly long reference list which points to some of the literature that informed this idea.

Video

The following video is a slightly extended version of the talk, using the same slides, recorded after the Moodlemoot.

Slides

The purpose

The title of this post is probably going to be the title of the talk. From that you can assume that this is not a show and tell of something that is working, but instead a proposal of an idea. The aim is to find out if there are other people interested in this project or already working on something similar. The aim is to start a conversation. The talk is also request for interest (an RFI). I’m keen to hear from folk interested in working on this idea, especially in terms of a potential ALTC grant for next year.

The proposal is based on previous ideas posted here. At the core is the idea of how curriculum mapping might work in Moodle. However, the intent is to do much more than simply modify Moodle. The broader aim is to modify the environment and processes within which teaching academics work in order that consideration of alignment (be it constructive, instructional, curriculum or graduate attributes) is part of every day practice.

A more detailed description of this idea is available here. The rest of this is a written summary of what I think the 3 minute show and tell will cover next week at the Moot.

The problem

Within Australian Universities, the alignment of what happens within a course (sometimes known as a unit) against some outcomes or graduate attributes is becoming widespread, even standard practice. For example, there’s a presentation at the Moot with the title “Translating Learning Outcomes in Moodle”. This presentation draws on Bigg’s (1996) idea of constructive alignment, which is probably the most common, currently used concept of alignment. The push toward graduate attributes for everything is perhaps the other common application of alignment within Australian higher ed.

The Moot presentation identifies as a problem the difficulty of translating learning outcomes into an effective course design within an LMS. The problem which I’m interested is connected to this, but is also a little different. The problem I’m interested in is that the every day, regularly experience of an academic doesn’t require them to think about alignment. More broadly, the everyday experience of teaching academics doesn’t encourage nor enable them to think about learning and teaching from an educational perspective. Instead the focus on low level tasks like uploading documents because of the low-level of abstraction in most LMS.

Experience is important

What people experience is important. There’s a growing body of literature from neuroscience (e.g. Zull, 2002) and psychology (e.g Bartunek and Moch, 1987) that suggests your experiences shape who you are, what you think and how you see the world. Which in turn is related to insights like Kolb’s learning cycle.

If alignment is not something academics experience regularly, and experience within a context that encourages and enables them to reflect and experiment with alignment, then how are they expected really to learn and adopt alignment?

The proposal

The proposal aims to modify the environment in which academics operate such that they are encouraged and enabled to consider alignment as a regular component of their everyday teaching experience. To provide an environment in which they can move through all of the stages of Kolb’s learning cycle. The proposal is based on the following assumptions and propositions:

• The most common teaching experience for university academics is teaching and slightly tweaking a course that has been taught before.
• It is fairly simple to modify Moodle to enable the mapping of alignment relationships between Moodle activities and resources and outcomes or graduate attributes.
• Once this alignment information is being maintained, an ecosystem of services can be added to Moodle that enable reflection, abstraction, and active testing of ideas around alignment in a collaborative and open way.
• If such an ecosystem enabled and encouraged effective, on-going use, then the quality of learning and teaching would improve.
• On-going use of such an ecosystem would raise interesting questions about the design and operation of Moodle.

Disclaimer: I have some reservations about alignment, however, it’s almost become a requirement within Australian higher education and I do believe that consideration of alignment could provide a useful McGuffin for learning and teaching.

The 3 minute show and tell will focus on showing some proposed screen shots of how curriculum mapping might work within Moodle and some initial ideas of how the resulting alignment information could be used to create an ecosystem of services.

Request for interest

Effectively implementing something like this is not easy. It would be improved by having a good combination of skills and perspectives. I’m keen to work with people who are interested in trying to further develop and eventually implement this idea.

I’m especially interested in hearing about projects that are related to, or already implementing something like this.

References

Bartunek, J., & Moch, M. (1987). First-order, second-order and third-order change and organization development interventions: A cognitive approach. The Journal of Applied Behavoral Science, 23(4), 483-500.

Biggs, J. (1996). Enhancing teaching through constructive alignment. Higher Education, 32(3), 347-364.

Zull, J. (2002). The art of changing the brain. Stirling, Virginia: Stylus Publishing.

The VLE model and the wrong level of abstraction

I’m currently trying to formalise the information systems design theory for e-learning that is meant to be the contribution of my thesis. i.e. what is the model, set of principles etc that I think is important.

As it happens, I recently came across this post on models around the use of VLEs/LMS from Barking College. It’s part of a discussion involving a group of folk (James Clay, David Sugden, and Louise Jakobsen) talking about models for getting academics to use the complete functionality of the VLE/LMS.

This is interesting for two reasons. First, it helps me see what others are thinking in this sphere. Second, it provides a spark for me to think about my “model”. As an example there is an interesting point made on the Barking College post that I want to pick up on in the following.

The basic idea is that the functionality of a bog-standard VLE/LMS – like Moodle – embodies the wrong level of abstraction. At least in terms of encouraging and enabling effective use of the VLE/LMS within a university by academics. A traditional VLE model is at a very low level of abstraction which means lots of flexibility, but also lots of problems. I think there is some value (also some dangers) in moving the level of abstraction up a few notches.

Moodle document management and the wrong level of abstraction

The post from Barking College makes the point that uploading and maintaining document content on Moodle “is one of the most round-about and time consuming things anyone can do”. I agree. But it doesn’t end there. Even when academics understand and engage in the uploading process, their are more problems.

Then we teach staff to upload files to the VLE … and in no time they have transferred their heap and bad habits onto the file areas of VLE courses. If, in addition to their own unique ways of filing things, they are not the only editing teacher on a course then total chaos is almost guaranteed.

There are some exceptions. There are folk who are well organised and anally retentive enough that they have their files organised into well thought out directory structures with meaningful file names. But most academics don’t. As pointed out, this causes enough problems when it is just that originating academic having to deal with the resulting chaos, but when a course is taught by multiple academics….

This highlights one of the major flaws I see in most VLEs/LMS. They are design at the wrong level of abstraction. The file upload capability is very basic, it simply provides the ability to manage files. It provides no additional cognitive support for structuring content and certainly none that connects with the act of learning and teaching. The file management capability can’t tell the difference between a course synopsis, a tutorial sheet, a trial exam, a set of solutions or a set of lectures. Most can’t even tell you if it’s a PDF file, a Powerpoint file or a Word document.

This low level of abstraction is necessary to enable the broadest possible flexibility in use. The more abstraction you build into a system, the more specific you make it.

The CQU eStudyGuides

One example of what I mean here is the CQU eStudyGuide Moodle block I played with last year. Some brief context:

• CQU has a long history of being a print-based distance education provider.
• A print-based study guide was a standard component of the package of materials traditionally provided to distance education students.
• The study guide was written by CQU academics and intended to guide the distance education student through the learning materials and activities they were meant to complete.
• The production of the study guides was a formal process involving a central “desktop publishing unit” which would produce “professional” quality documents that were then printed.
• The “desktop publishing” group is part of the unit I work with.
• Back in 2007/2008 the unit modernised the study guide production process so that it improved the quality of the final document and produced both a print and electronic version.

The electronic version was long over due. For a long time academic staff wanted to provide electronic versions of the course study guide on the course website. Some had done this via an ad hoc, manual process, but it was time for something better.

Producing the electronic version of the study guide was only the first step. We also had to produce an automated process that would allow academics place the eStudyGuide on their course website. Requiring academics to do this manually was inefficient and likely to result in less than professional outcomes. This connects with the observation about file management made in the Barking College post about file management.

So, we implemented the automatic generation of an eStudyGuide web page for every course. The following image is one example, click on it to see it in a bigger size.

The eStudyGuide page for a course was produced by a script. The script was aimed at a much higher level of abstraction. The script knew about CQU courses, it knew about the format we used for the eStudyGuide and it was able to use that knowledge to produce the file. e.g. it pulled the title of each part of the eStudyGuide from the eStudyGuide.

The CQU eStudyGuide Moodle block

By 2010 CQU had moved to Moodle as its LMS. As part of learning Moodle development I played with the creation of an Moodle eStudyGuide block. A block that would embody the same greater level of knowledge about CQU’s eStudyGuide than a more general Moodle block. Consequently, significantly simplifying the process of uploading an eStudyGuide to a Moodle course site. The following table compares and contrasts the eStudyGuide block approach with the more traditional manual appproach.

eStudyGuide block

Moodle file upload

Login to Moodle and go to course site. Turn editing on. Choose “eStudyGuide” block from “Add a block menu” Position it where you want.

Get a zip file containing the eStudyGuide from somewhere. Login to Moodle and go to course site. Go to the “Files” link under administration. Create a directory for the eStudyGuide. Upload the zip file containing the eStudyGuide. Unzip it. Return to the course site. Turn editing on. For each of the chapters (usually 10 to 12) of the eStudyGuide Manually add a link to the chapter To make sure each link is meaningful you may have to open the chapter PDF to remember what the title of the chapter was. Add another link to the PDF containing the entire eStudyGuide. Add the blurb about how to use PDF files and where to get a copy of Acrobat or other PDF viewer.

The trade-off

There isn’t a perfect solution. Both low and high levels of abstraction involve a tradeoff between different strengths and weaknesses.

A low level of abstraction means the solution is more flexible, can be used in more institutions and for unexpected uses (good and bad). It also means that the user need to have greater knowledge. If they do, then good things happen. If they don’t, it’s not so good. It also means that the user has to spend a lot more time doing manual activities, which increases the likelihood of human error.

A high level of abstraction, especially one that connects with practices at a specific institution reduces workload on the part of the users, reduces the chance of errors and perhaps allows users to focus on other more important tasks. But, it also limits portability of the practice. i.e. the CQU eStudyGuide process probably wouldn’t work elsewhere. Which means it requires additional resources to implement and maintain.

A greater level of abstraction also removes some flexibility from what the user can do. The simple solution to this is not to mandate the higher level of abstraction. e.g. at CQU we provided the automated eStudyGuide page, but academics weren’t required to use it. They could do their own thing if they wanted to. Most didn’t.

Other examples

Providing a high level of abstraction to the VLE is almost certainly going to be a component of my ISDT, of my model. This is exactly what the default course site approach attempted to do. Provide a much higher level of abstraction on top of the VLE.

Into the future, it’s also a key part of what I’m interested in investigating. I think the addition of a “default course site” approach to Moodle, especially one that increases the level of abstraction but at the same time can be used at multiple institutions, is especially interesting.

Academics, course websites and power laws

Last week I was thinking that academics shouldn’t manually create course sites. That arose out of the process of writing up the why/what behind what we did with Webfuse from 1999 through 2004. Today, I’ve been continuing that and looking at the usage statistics from that period.

The following focuses on statistics about how often an academic modified a course site. The Webfuse model was to automatically create a default course site for every course offered by the faculty. Academics could then modify that site as much, or as little as they wanted. The following two graphs (first in 1999 and then in 2005) shows how many staff were editing Webfuse course sites and how many times they made page updates. Staff are ordered on increasing number of updates. What is striking to me is the similarity of the curves – both look like a “power law”. Some rambling on implications below.

1999

2005

In terms of the 2005, the top 22 academic staff performed 21,298 updates on course sites. That’s over 8,000 more updates than the remaining 127 academic staff (13,472 updates in total). 17% of the academic staff performed 61% of the updates of course websites.

Implications

With the Indicators Project we’ve been using three questions to frame investigations of LMS usage:

1. What?
   What is actually going on within LMS usage? What are the patterns that can be identified?
2. Why?
   Why do these patterns exist? Can we identify why this pattern has arisen?
3. How?
   How can this insight be used to improve practice?

What?

The Wikipedia page on “power law” states “Power-law relations characterize a staggering number of naturally occurring phenomenon”. With the Webfuse data shown above, which is spread over a 6 year time period during which there were significant changes, there is a fair indication that this might be leaning towards a “natural phenomenon”. It would be interesting to perform a similar analysis on more recent data and more “traditional” LMS to see how “natural” this if this might represent a more widespread, “natural” phenomenon.

Based on my experience, and without looking at the data, I suspect that this type of pattern is likely to exist in most universities around use of their LMS.

Why?

So, why do you think this pattern exists? Suggestions?

My suspicion is based on Geoghegan’s (1994) identification of a chasm – a la Moore (2002) – in the adoption of instructional technology. i.e. that there is a chasm/difference between two groups of academics – innovators/early adopters and the pragmatists. It’s the innovators/early adopters that are the big users of instructional technology.

So, one interpretation of the above figures is that the majority of academics are pragmatists. This is not necessarily a negative. They want to do a reasonable job of teaching (as measured by the institution, themselves and their students) but aren’t going to allow other work (mostly research) to suffer. My suspicion is this “pragmatic” perspective is the dominant perspective amongst academics. It’s the type of perspective that environment within universities encourages.

How?

So, if you found support for this perspective, how might it be used to improve learning and teaching?

If it is the university teaching environment that creates this “pragmatic” approach, perhaps it needs to be changed.

If a majority of academics aren’t editing course sites, this suggests that these course sites aren’t that great. Perhaps it also suggests that the quality of the student learning experience isn’t all that great. If this is the case, then continuing the practice of academics having to create course sites within an LMS may not be the way to go. Perhaps, it is time to investigate alternatives ranging from the evolutionary – provided a default course site for academics to build upon – to the revolutionary – such as PLEs etc.

Postscript – Implications for LAMS?

LAMS – Learning Activity Management System has for quite some time been positioned as a better, alternative to the LMS model. From the “About” section on the LAMS site

LAMS is a revolutionary new tool for designing, managing and delivering online collaborative learning activities. It provides teachers with a highly intuitive visual authoring environment for creating sequences of learning activities. These activities can include a range of individual tasks, small group work and whole class activities based on both content and collaboration.

LAMS must be good, it has won a gold medal

Does LAMS usage – within institutions that have adopted it – follow the same “power law”?

The question of how to do an apples versus apples comparison between LAMS and a LMS would be interesting as they follow very different models.

If this could be done appropriately, then my prediction is that yes, in a university environment LAMS would follow this pattern, possibly even more pronounced because LAMS is that much more different to past practice for academics than the LMS.

Also, from the perspective of a typical teaching academic (and perhaps even students?), there’s a lot more to an “online course” than learning activity design. Most of which LAMS doesn’t support directly, hence the need to integrate it with LMSs.

References

Geoghegan, W. (1994). Whatever happened to instructional technology? 22nd Annual Conferences of the International Business Schools Computing Association, Baltimore, MD, IBM.

Moore, G. A. (2002). Crossing the Chasm. New York, Harper Collins.