Which Moodle (or other LMS) tool is best at support and training?

A question for those with experience of Moodle or other LMS tools/plugins/services etc

Which tools do a really good job of embedded support/training?

i.e. assume the tools is used to support a fairly complex task, but if I’m a first time user of the tool I can access all that I need to know about using the tool effectively from within the tool itself. I don’t need to visit the institutional e-learning support site, contact help desk or Google resources placed online by others.

A “really good job” might be judged on a variety of criteria, including, but not limited to,

• Clear instructions that get you underway quickly.
• Well structured, scaffolded support that minimises the need for calling helpdesk.
• Instructions that move beyond simple technical “how-tos” into offering pedagogical insights.
• Some aspect of how it supports the people using it is different from the run-of-the-mill.

Also, feel free to throw in suggestions of any software in general (most modern, quality computer games are probably good examples) or literature.

Context

I’m finally getting started on my Moodlemoot’AU 2013 presentation and am thinking about how BIM can be improved.

Improving the support BIM provides out of the box has long been an aim. Mostly because I think most of the tools I use don’t do a particularly good job and most of the additional supports (e.g. separate training sessions or separate websites) provided by institutions are also not as good as they could be.

This idea has been percolating for more than a few years originally proposed back in 2010 as “Making the LMS more like the Globe Theatre”.

So, I’m after ideas of where it’s been done well.

I’m helping to organise a conference – sort of

One of the courses I’m responsible for next semester is EDU8719 Contemporary Issues Conference. The synopsis of the course is

The course will be structured around an online conference that will include several themes or strands that reflect current trends and issues in education. Students will prepare a proposal for a paper in which they will draw upon their prior study and experience to respond to one of the conference themes and will participate in anonymous peer review of submitted proposals. Using their reviews for guidance, students will complete their papers, present them online using an appropriate medium, and engage in discussion of their own and other papers.

So, I’m helping organise a conference.

The following is meant to encourage me to get to know the course better by retelling it in my own words, to save any vague ideas I get while learning about it, and perhaps get some good ideas for the course. In particular, I’d appreciate any

• Pointers to similar courses that are more open than not.
• Suggestions about how to enhance the learning experience.
• Thoughts and experiences on online services for running conferences.
• Suggestions for literature and people that provide good insight on the four main topics.

The course has been run numerous times before and appears to be fairly well set up. So, at the moment the intent is not to make any radical changes. There is another course I’m running at the same time to which I’ll be making more radical changes, only want to do one course at a time. Just looking to avoid any major pitfalls and make any gradual refinements.

Summarising the course

The course has four main topics/modules. The content in these is minimal, the interests of the students should drive what they actually look at.

1. Engaging in professional networks.
2. Locating, accessing and critiquing current professional discourse.
3. Writing for professional audiences.
4. Online presentation techniques.

The aim is to encourage students to deepen their reading and discussion of contemporary issues in education. The main task is to engage with a simulated online conference in a range of roles. Students need to

• Prepare and submit a paper proposal.
• Review and comment on proposals from two other students.
• Review and comment on full papers produced from those proposals.
• Complete their paper based on comments from the marker and two other students.
• Present the paper using an online format of their choice.
• Host online discussion of their paper.
• Participate in discussion of serval papers during the conference period.

The idea is that as author they go into some depth and as reviewer they develop some breadth.

Assessment is summarised in the following table

Assignment	Due Date	Description
1 – 20%	14 Aug 2013 – Week 5	Abstract proposal Proposal reviews
2 – 60%	18 Sep 2013 – Week 10 Just before mid-semester break	Draft paper for review Paper review Final revised paper
3 – 20%	23 Oct 2013 – Last week	Recorded presentation Conference contributions Self-rating quiz

The weekly structure is something like the following table

Week	Module	Tasks
Weeks 1 & 2	1 – Connected professionals	Module 1 readings View sample papers and presentations. Begin reading for paper preparation Prepare and submit abstract with proposal
Weeks 3, 4 & 5	2 – Professional conversation	Module 2 readings Reading for paper Begin drafting paper
Weeks 6, 7, 8, 9, 10, 11, 12 includes the break.	3 – Writing as a professional	Wk 6 Module 3 readings Consider proposal reviews Wk 7 – Review draft and prepare for submission. Wk 8 – Complete paper & submit ** different in specification and ICE ** Wk 9 & 10 – Read assigned papers and submit reviews Wk 11 – Consider paper reviews, revise and submit final paper
Week 13-15	4 – Presenting online	Wk 12 Submit to conference site and EASE. Read Module 4 readings Work on conference presentation Wk 13 Complete presentation and load to study desk Wk 14 – Deliver conference presentation and host discussion. Participate in others. Wk 15 – Deliver and participate and finally assess conference participation of self and others.

Online structure/services

• EasyChair conferencing system used to run the conference.
• Forums
  • A tea forum for introductions.

    Not a lot in terms of replies and doesn’t seem to lead into further conversations.

  • Class bulletins.

    Announcements and responses. Seems a good place to find what issues there were with the course in prior offerings.

  • Course management and Broken links

    General forums for reporting issues problems, both empty. So, no problems or problems raised in other forums?

• Other Moodle resources
  • A suggested resources Database activity that is empty. A place to share resources. Would social bookmarking work better?
• Other resources
  • Link to prior conference papers – including 2010 hosted on an OJS site.
  • Links to prior conference presentations.
  • Large collection of ICE-based content.

Minor bugs, possibilities and questions

The following is an ad hoc collection of observations while looking through the course.

Bugs/issues

• The “ezproxy” link for “Canter, D.V., Fairbairn, G., & ebrary Inc 2006, Becoming an author advice for academics and other professionals” in the specification is broken. May point to similar problems elsewhere.
• Appears getting the reviews in is as much a problem in this course as in others.
• There’s a chance that the ICE content will not match some of the details of the course including: staffing, due dates etc.

Possibilities

• Short interviews/invited talks from external folk on various topics at appropriate times.
• Replace the “suggested resources” Moodle database with a Diigo group?
• A couple of simple screencasts to illustrate aspects of using EasyChair?
• Be a bit more specific with some of the readings focused around the act of thinking about and preparing a conference paper.

  e.g. types of research papers, fitting with a conference theme, mechanical tasks of reference management/writing etc.

Questions

1. Is there any formal or advisable pre-requisites for students entering the course and do all the enrolled students fulfil this?

   It would appear that the students would need to have a fairly good grasp of a particular issue and the some experience with writing. So this wouldn’t appear to be a good first course.

2. Can I get access to prior uses of the easychair conferencing system? What else is involved in setting this up
3. What’s the difference between a “real” conference and a “simulated” one?
4. Is there some value in having a fixed online format to allow more of a conference community to form and perhaps to invite participation from others?
5. EDU8117 also uses an online symposium idea, is there value in exploring connections between these two courses? At the very least using the same technology or approach to the conferences.
6. How to make the purpose and process of the course easier to understand?

   Some early forums suggest this was a problem for some students.

7. How to align some of the assessment detail between ICE and course specification?
8. How to deal with issues of anonymity of submissions within such a small class

Moodle, BIM, reflective journals and TPACK: Suggestions for moving beyond

The following is an abstract for a talk I’ll be giving at the 2013 Moodlemoot’AU in late June. The slides and other presentation resources will eventually be added here.

Abstract

The TPACK (Technological Pedagogical Content Knowledge) framework provides one way to conceptualise the knowledge required to leverage technologies to improve learning. In proposing the TPACK Framework, Mischra and Koehler (2006, p. 1029) argue that

“Quality teaching requires developing a nuanced understanding of the complex relationships between technology, content, and pedagogy, and using this understanding to develop appropriate, context-specific strategies and representations. Productive technology integration in teaching needs to consider all three issues not in isolation, but rather within the complex relationships in the system defined by the three key elements.”

This presentation will use the TPACK framework to explain and explore the use, existing features, and changes being made to the BIM activity module. Released in 2010, BIM supports the use of individual student blogs as reflective journals. It’s been used in a small number of institutions, written about (Jones & Luck, 2009; Reaburn, Muldoon, & Bookallil, 2009), ported to Moodle 2.x, and in 2013 is being used by the developer in his own teaching for the first time. These experiences have identified a number of possible areas for improvement. Beyond using TPACK to explore how and what changes to make, the presentation will address the following questions. How can BIM:

• reduce the workload associated with student reflective journals and make this practice more sustainable?
• better support different pedagogical approaches, especially connectivism?
• leverage learning analytics?

While BIM will be the concrete example used in the presentation, the presentation will raise questions of interest to the broader Moodle community. In particular, the presentation seeks to explore how the Moodle community might better support the integration of technology into teaching by examining the complex relationships between the three components of the TPACK framework.

Audience

While BIM is the example used in the presentation, the major aim of the presentation is to explore how and what insights TPACK might provide to the broader Moodle community. In particular, to explore how the development and support of Moodle can be enhanced to better develop the effective context-specific integration of technological, pedagogical and content knowledge required for effective technology integration into learning and teaching.

As a result, it is hoped that developers, teaching and technical support staff, teaching staff and management will benefit from the presentation.

How can an “enterprise” e-learning tool be agile?

I have a problem. If I’m really lucky, BIM will get added to my institution’s version of BIM for Semester 2 and I will be able to use it. Based on my experience this semester – where I’ve used an approach that depends on BIM – there has been limitations and workload issues. Having BIM installed in the “enterprise LMS” will help significantly reduce these problems. It will also severely limit my ability to learn.

That limitation will arise from the nature of being an “enterprise” LMS. i.e. not at all agile. Instead a lumbering behemoth that takes a while to turn around. Getting the “enterprise” installation of BIM changed in anyway will involve going through a governance process that will have numerous steps. During these steps the expense of changing BIM will have to compete for the scarce resources available to change the “enterprise” LMS with other requirements. Requirements that are likely to be significantly more important than the couple of hundred students in the 2 or 3 courses I teach.

This causes problems because while BIM has been used at other institutions. It’s typically been supported just like most “enterprise” LMS. i.e. if there are any problems or limitations with the tool it is learners and teachers who are aware of it first. These folk will either ignore/workaround the problem (and blame the &^%%## technology) or they will ask for help. The people they ask for help will be either IT helpdesk folk or L&T staff development/training folk. If they are lucky these folk will actually know how to use the particular tool that has the problem without having to quickly read the manual. In the worse case scenario, they’ll have to do a quick read of the manual/Google search (which theoretically the learner/teacher could have done in the first place). Either way the only options open to the support folk are

1. Here’s where you went wrong and how you fix the problem.
2. You have just discovered one of the known problems with that tool, there’s nothing we can do about it.

Either response often involves the learner/teacher engaging in a large laborious manual process to workaround the limitation of the tool.

A different situation

When I’m using BIM, I’ll be in a slightly different situation. I designed and wrote BIM. When there’s a problem or limitation with BIM, I can generally change BIM to fix it. For example, earlier this week I discovered that one of the main pages wasn’t displaying individual student posts in order of time published. Five minutes later it did.

The fact that BIM has been around for 3/4 years and this problem still existed in the code is a nice piece of evidence of the limitations of the “enterprise” approach, even if it is based on open source technology.

The trouble is, I was able to make and use this fix because I’m currently running BIM on my laptop. Next semester, when (or if) it is installed on the “enterprise” LMS it is very unlikely that a change like this would ever get installed on the “enterprise” LMS in any reasonable time frame. Perhaps ready for next semester, if I’m lucky.

This is a real problem because next semester I will have a real opportunity to do some really interesting experimentation and development with BIM. Activities that will be somewhat curtailed by the constraints of the enterprise process.

How can I work around this?

Some possibilities

Two short-term possibilities are

1. The backup/restore shuffle.

   This is where the students interact with the enterprise version of BIM. I then back that data up and restore it on my laptop. This is where I have the agile version of BIM that I can play with. If I make any change to the data, I then have to shuffle the data back the other way. In reality, the round trip of taking data from the agile version to the enterprise version probably isn’t going to work in any consistent and safe way.

   This approach also doesn’t help enable some of the ideas where the changes to BIM will enable students to do new and interesting things with BIM. Perhaps a version of BIM installed on an outside server the students could interact with might work. But it raises all sorts of other issues.

2. The client-side scripting workaround.

   This is where I create browser/client based scripts that modify how BIM works. Each student/staff member would need to install the scripts on their browser to get the functionality.

   Perhaps I could make changes to the BIM code to make this sort of workaround more effective and simpler?

The other possibility is to explore how the enterprise approach could be changed to be more agile. At the very least this would involve building a better relationship with the institutional IT folk, but even then there are limitations.

Are there other possibilities?

The grammar of enterprise IT

The grammar of school is an idea to explain why reforms of education have failed to take root. Especially the use of ICTs. The rationale is that any proposed reform is so different from the accepted mindsets of schooling (the grammar) that it is seen as nonsensical, as ungrammatical. i.e. it gets rejected or ignored in much the same way a nonsensical sentence.

I suggest that there is also a “grammar of enterprise IT”. Ideas such as

1. Wanting to make rapid, unplanned changes to a piece of software; or,
2. Trusting a member of the Education Faculty to make those changes.

would simply be seen as nonsensical and rejected. Changing that grammar is going take a lot longer.

Even in writing this post, I run the chance that someone in enterprise IT will see how this is an attempt to break the grammar of enterprise IT. A perception that could lead to additional constraints on the development and use of BIM. Shall be interesting to see how it develops.

Everything old is new again

I have this growing sense of deja vu. I’m beginning to think that my current experience with the institutional policies and processes around Australian university enterprise e-learning is essentially a repeat of my experience with the institutional policies and processes around Australian university print-based distance education systems of the mid-1990s. Almost twenty years on its deja vu all over again?

Some of this arises from an on-going conversation between @cj13, @timklapdor (I must start reading his blog more), and myself. A conversation about how innovation might arise within existing university structures (or more likely not arise).

As it happens, I’ve also been reading a bit about Morozov’s “To Save Everything, Click Here” book and its intellectual links. Through which I came to this in which Morozov has a short piece (linked to his book and other writings) where innovation and how we consider it is questioned. It included this on the history of innovation

According to historian Benoit Godin, for more than 2,500 years, the innovator was “a heretic, a revolutionary, a cheater.” Innovators brought little but trouble: They challenged the status quo and undermined the stability of the state. As late as the 1940s, innovation was seen as a form of deviant behavior — like crime or delinquency.

The idea of the notion of innovation being innovated and the view of the innovator as a heretic made some interesting connections for me. For example, back in 1994 I was asked to formally apologise to the entire staff of the Division of Distance and Continuing Education (DDCE) at a University because I gave a presentation about the World-Wide Web. The abstract for the presentation asked the question “Is this the death of DDCE?”. Most of my work with web-based learning over the following 5 years arose from battling against the limitations of the formal institutional mechanisms for distance education. Now, in 2013, I find myself battling against the limitations of the formal institutional mechanisms for e-learning. Perhaps the “innovator” – at least the one outside the formal institutional innovation framework – is still a role of a heretic?

All of which gives further weight/credence to the argument of @cj13 about the need for skunk works. It also connects to one of the central ideas of Morozov’s book about solutionism. i.e. that University learning and teaching has fallen for solutionism and the on-going search for “imperfections soon to be overcome by applying the right method” (Ven Den Eede, 2013) and ignoring broader problems. Suggesting that the current bandwagons of MOOCs, Learning Analytics etc. are likely to result in internally driven changes within Australian universities, but only the type of internally driven changes that have arisen from earlier bandwagons such as web-based learning, online learning, or blended learning. In 20 years time, I suspect it’s not entirely impossible that I will be struggling against the constraints put in place by a system relying on the local use of a MOOC from the USA and the support provided by IBM’s latest learning analytics with inbuilt learner and teacher nudge-based interventions?

How to capture the “full benefits of the creative, original and imaginative efforts of” teaching staff

What’s good for research, must surely be good for teaching?

An article on the Australian’s higher education page quotes the following advice from this policy note from the Group of 8 (an obviously non-self-serving document, of course)

If Australia is to capture the full benefits of the creative, original and imaginative efforts of its researchers, it will always need a means to support the ideas and challenges coming from individuals and small groups, even when these ideas fall outside formal priority setting mechanisms

Having engaged a bit in the formal priority setting mechanisms around institutional e-learning over the last month or so, I was struck by how this perspective could be moved across from research to institutional e-learning.

I don’t think anyone could claim that the institutional governance processes around e-learning – especially the LMS – could ever be described as “a means to support the ideas an challenges coming from individuals and small groups”.

This is not to suggest there isn’t some level of need for these processes to ensure the availability of institutional systems. It is to suggest that if you want “creative, original and imaginative” efforts then the processes need (I would argue) to be able to to support the ideas an challenges coming from individuals and small groups”.

For example, as mentioned previously as part of the case for getting BIM installed on the institutional version of Moodle I had to explain why others might use it. It seemed that the governance processes/bodies etc didn’t know that there were 30 odd courses this year that were using learning journals of one type or another that might have benefited from BIM. There appears to be a lack of knowledge of the ideas and challenges of teaching staff and students with institutional e-learning systems within the priority setting mechanisms that “govern” them.

The trouble with this type of argument is that it’s strange. Perhaps because of the lack of knowledge about the issues and challenges, it’s impossible for those responsible to see a problem with the priority setting mechanisms. Or perhaps it’s an example of the following.

From “Status Quo”

Or, of course, it’s not that big of a deal.

How Knowledge Workers like to learn and implications for BIM and LMS design

I found out last week that the abstract I submitted to Moodlemoot AU 2013 had been accepted. The talk will attempt to outline what I’m hoping will be my primary line of research over the next couple of years, which is probably going to be something like

• How can the design of institutional e-learning tools be improved to support teachers and students to learn?
• If this is done effectively, what happens?

The focus on institutional e-learning tools is mainly one of self-interest. I have to work with these tools in my current position and I want better tools. If my research can help my teaching, then it’s two birds and one stone.

In an earlier post I gave an initial idea of the “knowledge” problem that is one area with potential for improvement. i.e. most of the existing e-learning tools do less than a stellar job of helping teachers and students develop/access the sort of knowledge needed to get the most out of e-learning.

This morning, first Stephen Downes and then Steve Wheeler took me over to Jane Hart’s 5 characteristics of how Knowledge Workers like to learn at work. A post that describes findings from a 2013 “Learning in the Workplace” survey with 600 respondents from 46 countries. The image below summarises the 5 characteristics.

These characteristics may offer suggestions about how e-learning tools can be better designed to help teachers and students.

In the flow of work and other characteristics

The first characteristic is “In the flow of work” which Hart describes as

Workers don’t want to leave the workflow unless it is absolutely necessary for them to do. This means EITHER physically to go to a classroom OR virtually to work on an online course for an extended period of time (i.e more than about 10-15 mins) and/or which is more than a couple of mouseclicks away. (Taking a course at your desk, doesn’t mean it’s in the workflow!) Workers prefer to learn as an integral (NOT an extra) part of their daily job and not separately from it, either.

This resonates with me. For better or for worse, when I’m currently teaching “in the flow of work” means within my Moodle course site. The Moodle course site is the learning environment I work most in. If I want to learn – be it something about the students in my course or some new pedagogical strategy or technological technique – I would prefer it to be in the flow of work. i.e. in the Moodle environment. When I’m using BIM (or any other tool) I want it to be able to help me learn about my students, their learning and how I can improve it. I don’t want to book a session with an instructional designer or attend a scheduled training session. Raising the questions of what shape might this take and how might you do it?

The remaining characteristics offer similar suggestions. In particular, Immediately

Workers want to be able to find answers to their learning and performance problems as soon as they encounter them

which has some overlap with the above.

Socially suggests knowledge workers want to learn from others, as they work both internally and with external networks and communities. Are there any e-learning tools (within an LMS) that allow teaching staff to connect with a network of other people using the tool? To compare and contrast how others are using the tool and learn new ideas about how the tool might be used.

Making e-learning tools that are more supportive – BIM, TPACK and truncated feeds

The following is a mini-argument for and example of how the e-learning tools should be made more supportive. i.e. actually help the staff and students using them actively address common problems in a pro-active way. It continues some more thinking about an earlier question I asked, Does institutional e-learning have a TPACK problem?” and hopefully will inform the on-going research and development around BIM (now officially released to the Moodle community) and also inform the Moodlemoot’AU 2013 presentation I proposed.

The problem

The last post mentioned the recent research around the increasing workload faced by academics dealing with the current practice of e-learning in Australian Universities. One of the premises of my thinking is that a contributing factor to this workload pressure is that the tools aren’t provided sufficient help.

Just recently @masmithers re-tweeted an “oldie but goodie” blog post of his from 2011 – eLearning at Universities: A Quality Assurance Free Zone? – in which he reports on a range of poorly designed online courses he had seen. He talks about the “secret communion with students in the classroom” as one of the contributing factors for this. The comments on the post point to a range of other factors: “no requirement to have any teaching experience or qualifications” and limited (if any) funding for the move from on-campus to online (or dual mode). There are also a couple of comments along the lines of “we are at the mercy of crappy tools….If we’re going to build decent sites, we need decent tools” and connecting back to the workload question “spend extra hours and time away from my family trying to remediate what is essentially a bad system”.

The questions I’m keen to explore are along the lines of

• Would the provision of better tools help reduce workload and increase the quality of the learning and teaching experience?
• What does it mean for these tools to be better? What types of problems need to be removed? What positives built in?
• How can this be type of improvement be carried out within the current institutional processes?

A solution

In the previous post and my current line of thinking is that the TPACK (Technological Pedagogical And Content Knowledge) framework can provide a useful lens for thinking about this problem. The basic idea is that

• TPACK proposes that it “identifies the knowledge teachers need to teach effectively with technology” (Koehler, n.d.).
• This knowledge does not need to reside in each individual academic, the tools they use can help provide this knowledge (either directly or by providing in context connections to others).
• Can we use TPACK to identify the type of knowledge that we need to design into these tools.

An example

BIM is the tool that I plan to use to explore these ideas. It’s a tool I wrote and hope to increasingly use in my own teaching. The following illustrates one idea for how BIM could be re-designed to better contribute to the overal TPACK required to “teach effectively with technology” (Koehler, n.d.).

BIM is all about aggregating the posts students make to their own individual blogs. These blogs are hosted on whatever blogging platform they decide. BIM provides ways to mark (both manually and eventually automatically) the student posts.

Today, one of my students has reported a problem with the marking of her blog. After a bit of email tag between both of us, I have identified that the student has configured her blog so that the RSS feed generated has summaries of her posts, rather than the full text. This means BIM cannot see the full post, it’s marking a portion. No surprise it got it wrong.

This problem has caused confusion and disappointment on the part of the student. It has required her to expend more effort on chasing this up and required me to do more work to diagnose and remedy the situation.

Wouldn’t it have been so much better if BIM was capable of identifying this problem as soon as it happened and informed both the student and myself about the problem? Technically, it would be fairly easy to implement this.

Doing this requires that the tool have embedded into it a lot more technical knowledge (e.g. that feeds might be summarised and how that looks) and the ability to make use of that knowledge.

Being aware of this need requires that the people capable of designing and changing BIM, are close enough to its operations that this type of problem becomes recognised. I’m not sure that in all situations this is the case. How can a tool like BIM be designed to make this possible?

And they don’t even know enough to expect better

The title for this post is (probably a slight re-phrasing) of something @palbion mentioned last week during a conversation about the low quality of information systems within higher education (or at least our experience thereof). The comment was in relation to the professional and academic staff who are struggling with the various information systems universities are increasingly using to support tasks such as managing research higher degree students form application through to graduation, managing the process of sending students out into industry for practicums, and of course the more general LMS and student records system.

All of the staff involved are having to bumble along with systems with inherent limitations and attempt to develop what workarounds they can. For example, the system that sends out an email to students saying that their application is incomplete and to try again. The problem is that it doesn’t tell the students what is missing (even though it must know to have identified the incomplete state) and doesn’t tell them how to try again. Or, for example, the online assignment submission and management system that requires the staff involved with marking to repeat the same manual process for each assignment they are marking. Repeating processes is what computers are good at, not human beings. This inevitably leads to mistakes which need to be fixed. Leading to less than stellar efficiencies (which last weekend took on a much higher profile within Australian higher education)

The point of the title and Peter’s point was that many of the staff struggling with these systems think this is how information systems work. In their experience there has always been problems like this, it’s something you live with. They don’t know it can be better.

This particular discussion also arose out of some of my earlier discussions about the limits of our students’ technical knowledge. A reason, perhaps a significant reason, that these limitations became obvious was the poor design of the technical (perhaps socio-technical systems) within which the students had to operate.

Experience with the online assignment submission system suggests that it’s not just the students that are struggling with these problems

Application or targeted learning analytics – another scope?

Was skimming through Mark Drechsler’s slide deck from THETA 2013 when I came across the following slide.

It’s part 3 of a model of learning analytics (Target, Consumer, Scope, Automation) Mark used in his talk and got me thinking and hence the following. Still early days on this.

The slide above describes two of the extremes of the data being used in learning analytics. Just the stuff from the LMS at one end through to the entire learning ecosystem. As the slide points out, the former is increasingly becoming limited and the latter is just a bit pie in the sky.

I’m wondering if there is another part of the scope that might be a bit more fruitful, or at least of more interest to me. i.e. learning analytics at the application – or to use Moodle speak, the module/plugin level.

Tweeting from the Learning Analytics and Knowledge 2013 Conference @shaned07 suggests

https://twitter.com/shaned07/status/321161897573498880

This resonates a bit with the point @beerc made in Beer et al (2012)

The inherent unpredictability of agents within a CAS (complex adaptive system) suggest that the most appropriate place to situate learning analytics tools and resources designed to inform and improve online learning and teaching, would be within the micro-level context.

One interpretation of this “micro-level context” could be the individual application. i.e. the scope is a specific application being used for learning. Perhaps the best example of this is the discussion forum and the SNAPP tool (as it happens an output from @shaned07 and friends).

It’s within the application as it is applied within a particular learning context where the deepest knowledge about what is happening may reside. Analytics that help the student or the teacher understand what is (or isn’t) happening would seem to have the best potential of improving outcomes.

It may be helpful to have information from other sources, but I wonder if more specific analytics about the specific learning process as embodied in a particular tool can be more useful.

Of course, this raises the question about how such tools are used. For example, I’m pretty sure that many discussion forums aren’t used in a “pedagogical” way.

I’m wondering whether the origins/connections of learning analytics with data warehouses and business intelligence units (not to mention the nature of big data toward collating information into “big data”) drives this interest in the broader scope at the expense of the specific? Wondering just how useful “application analytics” might be as it would likely lose the “big” in big data?

Much more to think about here, but I have to get back to other work.

References

Colin Beer, David Jones, Damien Clark (2012) Analytics and complexity: Learning and leading for the future. In M. Brown, M. Hartnett, & T. Stewart (Eds.), Future Challenges, Sustainable Futures. Proceedings of ascilite Wellington 2012 (pp. 78–87). Wellington, NZ.