Examining diffusion and sustainability of e-learning strategies thorugh weblog data

The following is a summary and some thoughts on Lam et al (2010). It’s a paper from the same authors/research from which I summarised an earlier paper.

The abstract for Lam et al (2010) is

The study focuses on ‘horizontal’ and ‘vertical ‘adoption of e-learning strategies at The Chinese University of Hong Kong as revealed through computer log records in the centrally supported learning management systems. Horizontal diffusion refers to whether e-learning has spread to influence the practice of more teachers and students. in vertical diffusion, the authors examined whether or not teachers tend to adopt more varied online learning activities in successive years. The overall findings are that, while adoption of simple strategies is increasing. There is little evidence of horizontal and vertical diffusion of more complex strategies. Indeed, the use of some of the more complex strategies, which may relate to greater potential learning benefits, decreased. Results have led to discussions about new focuses and strategies/or our institutional eLearning Service

My thoughts

The same data is used to drive another examination of the use of the LMS at a Hong Kong University. Data suggests that e-learning usage is fairly typical. Emphasis on content distribution. But more surprisingly is that usage is dropping. Which suggests e-learning is in more trouble. Would be interesting to see how its evolved since then.

Using just two years worth of data is limiting.

There is less contextual information provided to explain some of these trends than in the prior paper.

The recommendations for e-learning support can’t be supported from the log analysis and seem to rely more on existing conceptions held by the authors. Log analysis couldn’t give you these insights, would have to use other means or use log analysis to measure changes after interventions.

The recommendations are all fairly typical change management recommendations, though the “comfort zone” recommendation is a little unusual.

Introduction

Adoption of innovations is hard. Links to the need for teachers’ to undergo a conceptual change required as finding new ways of working. Draws on Lewin (1952) and then some educational researchers to have a 3 stage conceptual change process

  1. Diagnosing existing conceptual frameworks and revealing them
  2. A period of conflict which creates dissatifaction with existing conceptions
  3. A reforming/reconstruction phase of the new conceptual framework.

Raises the question of how and when these 3 stages aren’t followed or are corrupted. Or is that simply where the new conceptual framework is the old slightly modified to fit the new ways of working under old concepts. e.g. placing lecture slides and tutorial sheets into the LMS.

Brings in the J curve to explain that “things often get worse before they get better because of the expenses and challenges that occur early on in the innovation cycle”.

Not sure how this works as an introduction, but good to see this idea of that things get worse early in the innovation cycle entering into the discussion of the LMS. Will be interesting to see how the authors build on this.

I wonder how this plays in the context of regular upgrades of Moodle every year. Is there sufficient change between versions to lead to a continual J curve? Is broader change within a university setting enough to contribute to this continual J curve?

Does the adoption of a new LMS really class as innovation? Conceptual change perhaps, but innovation?

Adoption of e-learning innovations

Talks about two possible ways of examining adoption of e-learning innovations

  1. Horizontal – see as diffusion of the innovation.

    Important if e-learning will have significant impact. Mentions own research showing that innovative teachers don’t effectively disseminate their practice (Why should they? Do they have to be innovative and disseminators? Also ignores the very personal/contextual nature of innovation. What works for innovators probably won’t work for others. Even other innovators.).

    Mentions reluctance of staff to spend the time required on e-learning and the removal of senior professors from innovative work of more junior staff as barriers.

    Now picks up the idea of others perhaps not likely to pick up innovative practice of others. But mentions UK research about lack of time, confidence, competence and comfort for new skills. What about the inherent difference between people and where they are?

    Brings up one of the authors prior research reporting on e-learning developments being behind schedule so that evaluation misses out. Of 26 projects, less than half had evaluations completed as planned.

    Mentions that evaluations need to be specific to courses which limits their usefulness in aggregation of findings to a higher level.

  2. Vertical – linked to the concept of sustainability where functionality is used in subsequent offerings of a course to benefit (benefit may be a strong word) multiple cohorts.

    The costs of moving online can provide benefit through reuse and the smoothing out of procedures. The idea of “threshold obstacles”.

    Sustainability focuses on the reuse of new teaching and learning designs and strategies. Mentions no change in approach given an LMS.

Factors impinging on diffusion and sustainability

Starts with Rogers 5 characteristics of an innovation. But again – like so many others – fails to mention the other parts of of Rogers theory though rate of adoption gets a mention. Of course doesn’t mention critiques of Rogers and diffusion.

But argues that the adoption of different e-learning strategies can be explained by difference in the above characteristics.

What this misses, however, is that the innovation characteristics are meant to be personal perceptions. Suggesting that someone from a constructivst background (e.g. a faculty of education instructor) would see great advantage in a discussion forum or the use of blogs and hence should be more likely to adopt that practice.

Mentions the over focus of LMS use as a content distribution mechanism.

Proposes the idea of a “mutual comfort zone” for all stakeholders (teachers, students, technical and pedagogical support staff) as a requirement for an e-learnign project to be successful and thus reused. The idea is that this zone is quite small and that explains why you don’t see many complex, successful and sustainable e-learning projects.

Ahh, now goes onto to cite one of the author’s earlier research about contextual factors and a model of drivers that influence the growth of blended learning and identifies the following of most relevance as

  • senior management committment

    I always wonder how much of this is “compliance”. i.e if senior management think something is important, is it possible for it to be a failure? And other logical flaws.

  • allocatioon of time
  • positive cost-benefit decision by teachers of a pay off from investment

LMS logs

Talks about logs and that they don’t reveal everything. Some similarities with similar section in prior paper.

No totally online courses in this work.

Data

  • two academic years 2007/2008 and 2008/2009.
  • Term 1 and 2 data included
  • Course was unit of analysis, if at least one session had an active website
  • Both under and post graduate included
  • Excludes classes with less than 10 students
  • One student had to access the course site during term
  • Same gain to other features
  • Only online services on central servers counted

Findings

Diffusion

Compared and contrasted active course sites between years

  • 48.8% to 53.3% with active websites
  • Independent t-tests reveal statistically significant increases
  • Active content – 97.3 down to 96.6
  • Active discussion – 20.8 to 18.1 (numbers are hard to read on scanned document, so may be slightly out).
  • Active assignment – 21 to 19
  • Active quiz – 39? to 8

Sustainability

Process:

  1. Identify courses run in two consecutive years
  2. If a site in first year, was their a site in 2nd year?
  3. If yes, compare e-learning strategies

Findings

  • 790 courses with a website in both
  • 82 courses ceased using the LMS in 2nd year.
  • 158 courses started in 2nd year.
  • 31.1% of the 790 courses had fewer active features in the second year – suggested teachers had stopped using some strategies.
  • Chi square tests reveal stat significance of p less than/equal to 0.001.
  • Mostly it was discussion forums that were dropped.

Discussion

Talks about multiple J curves to explain. With content, most are comfortable, but others are “still near the bottom of their J-curves”.

Suggests that e-learning support should focus on the areas where there is difficulty.

With the factors that influence adoption in mind to suggest

  • Acknowledge the need for appropriate motivation for teachers.

    Online can be both draining (for most) but also rewarding.

  • Support the departmental context.

    A collection of approaches – promotion, technical staff, peer groups, money.

  • Use an evidence-based and pragmatic approach

    If you hit the comfort zone, then features are more likely to be diffused and sustained. And a willingness to evaluate and respond to the evaluation No real mention made here of the institution perhaps having to evaluate it’s move to e-learning and change what it’s doing

  • Support projects that are most likely to succeed

    This should be the focus at the institutional level. Characteristics of such projects are identified as

    • Start with the intention to sustain and diffuse
    • Projects headed by the “right” people
    • In the “right” context (collegial, supportive)
    • In the comfort zone
    • Commitment to evaluation

References

Lam, P., Lo, J., Yeung, A., & McNaught, C. (2010). Examining diffusion and sustainability of e-learning strategies through weblog data. International Journal of E-Adoption, 2(3), 39–52.

3 thoughts on “Examining diffusion and sustainability of e-learning strategies thorugh weblog data

  1. Just after posting this, I read this from George Veletsianos. The section on hammers and problems in particular resonated

    If educational technology companies (and Centers for Teaching and Learning) are eager to improve education, rather than searching for problems to apply their solutions, they should focus on identifying problems and designing solutions to those problems.

    The connection I see between this and discussion of the LMS is that the LMS and the associated institutional e-learning ecosystem becomes the solution looking or a problem. Due to the cost involved, the central L&T and IT folk are increasingly employed and tasked with enhancing and expanding the sustainability and diffusion of that ecosystem and its fixed capabilities. Rather than discovering and responding to the problems being faced by teachers and learners.

  2. Even though the data is limiting, it is in line with my own experience and anecdotal reading, especially in The Chronicle of Higher Education. Instructors will try something and find it’s work or outside their comfort zone so they drop it. In my own experience, instructors begin with a set of givens – lectures, assignments, and quizzes – then add another element or two, adding to their workload. At this point in time, very few can see beyond this “given” mindset.

    Kevin

    1. G’day Kevin, Thanks for the comment. Especially since it expands upon my experience. I’ve always worked and taught in universities with a significant distance/online cohort (e.g. just finished teaching a course with 100 online only students) so my practice – and what I’ve observed in others – has been a slow growth in what is used. Your experience also breaks the Malikowski et al proposition that more features are explored. Hoping to do some weblog analysis shortly to explore what is actually happening at a couple of institutions. David.

Leave a Reply

Please log in using one of these methods to post your comment:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out /  Change )

Google photo

You are commenting using your Google account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s