The research “foundation” for the missing Ps

The work on the missing Ps is intended to contribute some research publications within the information systems field and also form the structure for chapter 2 of my PhD thesis. Consequently, it is important that the work have a good research foundation. This post is intended to provide that foundation, or at least document my development of the foundation.

The attempt is to develop a descriptive type of theory, a framework or taxonomy. Literature that talks about this type of research, appropriate processes and content include

The missing Ps – Process

The Missing Ps framework is my attempt to generate a way of identifying the flaws in the methods used by Universities to select an LMS.

In this section I’m expanding out my thoughts associated with process. It will include

  • The plan-driven assumption
    The almost automatic assumption that a plan-driven process will be adopted. The almost complete ignorance of the adaptive alternative.
  • IT governance model
  • Process and tool alignment
  • The importance of process change

The Plan-driven assumption

Over emphasis on plan-driven development at the cost of adaptive approaches
This is the one I’ve banged on about in previous papers (e.g. an early one, and the most recent one). The on-going acceptance of agile development methodologies, the Enterprise 2.0 meme incorporating emergence and the idea of rapid incrementalism from the two Johns make pushing this view a bit easier.

There are a number of reasons why this plan-driven approach is not appropriate to elearning

  • elearning, what it is and how best to do it are two very open questions.
    It is impossible to properly plan because what we know now is not going to be the best practice into the future. Examples include the arrival of blogs and social software onto the scene. Also LAMS and the learning design folks.
  • A large portion of any system cost is enhancement.
    This is a finding from software engineering research.

There are problems with the plan driven approach

  • The blind men and an elephant story
    The type of top-down design characteristic of plan driven development leads to the loss of the whole.

Some Peter Drucker quotes

from “The Effective Executive”

“Most discussions of the knowledge worker’s task start with the advice to plan one’s work. This sounds eminently plausible. The only thing wrong with it is that it rarely works. The plans always remain on paper, always remain good intentions. They seldom turn into achievement.”

“Innovation and Entrepreneurship”

“‘Planning’ as the term is commonly understood is actually incompatible with an entrepreneurial society and economy….innovation , almost by definition, has to be decentralized, ad hoc, autonomous, specific and microeconomic.”

IT Governance

The Wikipedia defintion

Corporate governance aims to:

  • align the actions of the individual parts of an organisation toward aggregate mutual benefit
  • provide the means by which each individual part of the organisation can trust that the other parts each make their contribution to the mutual benefit of the organisation and that none gain unfairly at the expense of others
  • provide a means by which information can quickly flow between the various stakeholders to ensure that the changing nature of both the stakeholder needs and desires and the environment in which the organisation operates get effectively factored into decision processes

Need to look more into the idea of IT governance and get some nice definitions.

How this is typically implemented in organisations is through a hierarchical committee structure. i.e. there is some central committee that meets to decide what should happen with IT. That committee has representatives from each part of the organisation. If someone at the coal-face identifies a way to make a potential large improvement their suggestion has to climb up the hierarchy, either within their part of the organisation or up the IT division, until it makes it to the committee. Along the way some small suggestions may get implemented, but anything that is large, crosses organisational boundaries or is a significant change from current practice will generally have to wend its way to the committee.

This is a rather negative description of the problem. On the surface this approach does appear to be somewhat logical if you buy into the rational model of people, organisations and decision making.

The problems with this model include

  • Even if it works it is incredibly slow.
    Often this committees meet no more than 4 times a year. Not exactly agile.
  • At any stage the idea can get knocked on the head.
    Any one of the people on the ladder up to the committee can kill off the idea, especially if it is perceived as significantly different, difficult or threatening.
  • It assumes the committee itself always acts rationally in the best interests of the organisation.
    e.g. the killing off of the AIS idea.
  • It assumes that the decisions of this committee will actually be followed and acted upon.
  • It assumes that people at the coal-face will even bother to start the ball rolling
    This working paper from the Harvard Business School reports on research that indicates that people don’t speak up.

    Qualitative data collected in 190 interviews with employees from all levels and functions suggest that fear of speaking up, even with pro-organizational suggestions, is pervasive and, for many, a source of intense negative affect.

Process and tool alignment

When you’ve been told to use a particular tool, you can’t use a process that doesn’t fit the tool. Or if you do there are going to problems of inefficiency or poor quality.

The implication for higher education when adopting an LMS (or any information system) is that how things are currently done and the system being adopted have to achieve some sort of alignment.

The problem with the implementation of many information systems is that it is assumed that it is the tool that cannot be changed and instead how things are done in the organisation must be changed to meet the tool. With enterprise resource planning systems this is seen as a good thing because these systems are meant to encapsulate “best practice”. But this assumption is highly questionable.

It also ignores the difficulty of forcing process change on an organisation. Especially when the organisation is full of knowledge workers, like a University. This difficulty often means that process change is often ignored or not completed and consequently leads to inefficiencies and poor quality.

Need to mention the idea that the organisation now becomes captive to the system. When the system changes, often due to the vendors needs, not the organisations, the organisation must go through yet another round of difficult process change (or simply ignore it).

There is an alternative. Modern information technology can be implemented in a way when it becomes significantly more malleable than previously thought. It is possible to mould the technology to fit the organisation. This makes it possible to enable a conversation between the organisation, it’s members and the technology where both are modified to provide significantly improved processes. .. this needs to be expanded.

The importance of process change

One way to build a comprehensive model is to place IT in a historical context. Economists and business historians agree that IT is the latest in a series of general-purpose technologies (GPTs), innovations so important that they cause jumps in an economy’s normal march of progress. Electric power, the transistor, and the laser are examples of GPTs that came about in the nineteenth and twentieth centuries. Companies can incorporate some general purpose technologies, like transistors, into products, and others, like electricity, into processes, but all of them share specific characteristics. The performance of such technologies improves dramatically over time. As people become more familiar with GPTs and let go of their old ways of thinking, they find a great many uses for these innovations. Crucially, general purpose technologies deliver greater benefits as people invent or develop complements that multiply the power, impact, and uses of GPTs. For instance, in 1970, fiber-optic cables enabled companies to employ lasers, which had already been in use for a decade, for data transmission. (McAfee 2006)

What should the institution provide in a Web 2.0 world?

Clarence Fisher has an interesting experiential post about the difficulties that start to arise when using a lot of cool, new “Web 2.0” apps in a teaching/learning context. Free(ish) services he lists are blogs, wikis, podcasts/vlogs, superglu, flickr etc.

But it’s not all free.

But we do pay, having to access different accounts across the web. The kids need to remember URLs, passwords, and how to navigate through different interfaces. They need to remember how to run WordPress, make a photostream in Flickr, add blogs to their aggregator, and format a wiki.

His reason for not using an integrated system?

I have consciously not worked with a single piece of software such as Moodle because I wanted kids to see the power of collecting resources across the web into a single environment

He sees the need for an interface that would combine the pieces.

When I thought/dribbled about Web 2.0 course sites one initial negative thought was, “why keep the whole course structure idea?”. Isn’t this an example of the horseless carriage problem. That’s how we did it the old way, so we must do it that way in the new Web 2.0 world?

Sort of, yes. Most people, students and staff, aren’t going to make that leap straight away. They need the comfort of the familiar to ease the pain.

As the above post points out. Integrating these disparate applications together is going to require some glue to reduce the difficulty of using it. Otherwise it won’t work.

Starting my “blogs for learning” article about BAM

blogs for learning is an “online resource about instructional blogging being set up by Michigan State University. I’ve volunteered/been asked to write an article about the use of blogs in COIS20025 and BAM.

This post where I’m going to start my writing.

Thinking about the article

Would be nice to link in with some of the previous articles on the site. Here’s a summary of articles I might link to and why

The site also contains various other posts, some of the ones to refer to include

  • Nicole Elison’s post on Non-Pseudonymous Student Blogging: Ethical and Legal Concerns
    The advantage of BAM is that students public blogs can be anonymous. The identification of which student owns which blog is only visible to CQU staff. Of course, in the absence of any specific advice students have tended to name their blogs using their student identifier.

What will I write about?

  • Why and what we did
  • How it worked out
  • What’s next

Additional improvements to BAM

It’s almost the end of term. We’ve started marking the blogs using BAM and not surprisingly some rough edges are showing. Things to look at

  • Merging feeds
    Currently BAM simply copies over the local mirror of the feed with the new one. This causes problems because WordPress, and likely other providers, only provide a feed with the last 10 posts. The solution would be to merge the new feeds with the local mirrored copy. XML::Feed, the Perl module used in BAM, offers the splice method to do this. So should be easy.
  • Making copy detection, pro-active
    I find it hard to believe but we have had students who have copied the blog entries of other students. At least half a dozen. Now that it is written, need to have this run as a more regular event and notify students as soon as it happens.
  • Increase visibility to students
    The students don’t really see anything from CQU’s end. This causes problems in terms of whether they’ve completed the questions etc. Implement a view and email notification system that gives the student a visible view of CQU’s summary of where they are up to. Notify students when the miss certain events.
  • Remove the special characters
    Students tend to cut and paste their posts from Word. This introduces special characters which, after the blog has finished with them, can cause problems.

Who is to blame for plagiarism: technology, lecturers or context?

At the Plagiarism Conference 2006 Baroness Ruth Deech, the first Independent Adjudicator for Higher Education gave a presentation which has been widely reported on the Web and in the press. It was interesting to note that the conference was co-sponsored by Turnitin’s UK arm. Rather than sound entirely negative the conference proceedings are online and, from the titles, there looks to be some good stuff.

Some of the comments online, especially from Digg and ArsTechnia which both include a fair number of student comments.

It is being widely reported, because of the title of the Times article, that plagiarism is the fault of indulgent lecturers. The Chronicle of Higher Education also carries the story with some interesting comments.

It suprises me that such views are getting such airplay as they miss the point entirely. My problems with these views include

  • The idea that there is a single cause of academic misconduct
  • That the primary cause is rooted in either the students or the teachers
  • The idea of placing blame is not exactly a helpful way to frame the conversation
  • That a change or introduction of technology will reduce academic misconduct

What was said

First, in the press Baroness Deech’s views are often reported along with a comment from Gill Clarke of the UK “education watchdog the Quality Assurance Agency”. From the Times article

Gill Clarke, of the education watchdog the Quality Assurance Agency, said that students should be made aware that most universities were using detection software, such as Turnitin, to spot plagiarism.

Baroness Deech’s views include the following, taken from an article in the Australian newspaper

“There is a culture of expectation among today’s students,” she says. “They just take whatever is put in their hands, be it a handout or a PowerPoint presentation. That way you end up boiling down complex things to three bullet points.

Taking down notes in longhand from a book is better than cutting and pasting from the internet, she says, because it requires students to digest material.

Taking notes is the solution

This point basically ignores most of what the educational literature has to say about learning. Learning isn’t enabled by copying content by hand. Information distribution is not learning.

The limited influence of copy detection software

In the course I’m currently teaching we have used copy detection on all assessment items. Part of the assessment is designed using strategies suggested as best practice for helping minimise plagiarism.

Teaching staff have repeatedly mentioned that copy detection software will be used on all assignments. It’s mentioned in the course profile and in the course discussion forum.

Currently (term not quite finished) the copy detection reports on the 3 assignments are showing about 40 students (about 14% of the class) have something that might be academic misconduct. The course staff already have 17 offences entered into the institutions system for managing academic misconduct. That 17 only includes assignment 1 and a small part of assignment 2.

For assignment 2 the students were provided with 3 references (including direct links to online versions) to help them with a report. The copy detection report shows students with reports where 20%, 23%, 49%, 58% and 70% of the report is a direct copy from one of the references we provided.

What should change?

Changing the technology doesn’t appear to make any difference.

Baroness Deech comes close to a possible solution when she says

Students need to be told that their own thoughts about a subject are very important. They need to be challenged to respond in their own way instead of downloading, cutting and pasting. The weighing up of views, the encouraging of nonconformity, the imbuing of intellectual tradition of inquiry are getting lost. If lecturers can imbue students with the view that they are searching rather than copying, then we might go some way towards tackling plagiarism.

The only way I see that we can “imdbue” our students with this view is if our primary form of teaching moves away from the information distribution model that emphasizes traditional lectures and textbooks and move towards more modern, effective pedagogies.

The Program in Course Redesign was a project that used technology as the basis for 30 different US Universities to redesign their instructional approaches to achieve cost savings and quality improvements.

The results from the last round included

  • Average savings of 39%
  • 8 of the 10 projects reporting improved learning outcomes, the other two reported no significant difference
  • 5 of the 10 projects showed improved course completion, 1 reported no change, 1 showed reduced completion (the other 3 did not measure course completion)

From the lessons page

All ten projects have effected significant shifts in the teaching-learning enterprise, making it more active and learner-centered. The primary goal is to move students from a passive, note-taking role to an active, learning orientation. Lectures are replaced with a wide variety of learning resources, all of which involve more active forms of student learning or more individualized assistance. In moving from an entirely lecture-based to a student-engagement approach, learning is less dependent on words uttered by instructors and more dependent on reading, exploring, and problem-solving undertaken actively by students.

Why isn’t it being done – who is to “blame”?

The type of change described by the Program for Course Redesign is large scale innovation within existing organisational settings. This type of thing is extremely difficult and time consuming and involves a great deal of battling against organisational norms. How is a poor academic expected to do something like this without organisational support at the highest levels?

For example, think of yourself as an academic who is thinking of implementing this sort of change. But your organisational context has the following characteristics

  • Increasing emphasis on research.
    The rhetoric from senior management is that research, in the form of top journal publications, is important. You hear from selection committees for new jobs at your institution using the publication/research record of new applicants as the main criteria.
  • On-going restructure and review
    For the last 4 or 5 years the academic unit and the institution as a whole has been either on-hold waiting for, or involved in organisational restructures and the introduction of new senior management. You’ve just heard that there are also going to a number of other reviews in the coming months.
  • Limitations of current models and processes
    The complexity of your institutions teaching means that there are a range of support units and other commercial organisations which manage aspects of teaching delivery. Those units have adopted specific models for the delivery of teaching that enshrine the traditional lecture/tutorial approach for face-to-face delivery. For a range of reasons they are very reluctant to change those models.

In this context, is it really the fault of the academic if he/she decides that it’s better to concentrate on the research than trying to actively engage with the large scale innovation around teaching and learning that is necessary to seriously address this issue?

Even if the issue is made to engage, how is it suggested that necessary organisational changes be made?

The missing Ps – People

LMS adoption decisions are made by people. LMSes are used by people. The nature of those people who they are, how they think and what they believe, amongst other characteristics, have a significant impact.

They myth that people are rational

Extensive research shows that our brains have certain hardwired propensities that might be exploited. For example, our brains tend to register frequently heard facts as true, even if they are patently false. As a result, our memories and beliefs are highly malleable and unreliable. We also tend, if unchecked by the conscious reasoning mind, to focus overly on risk, inconvenience, hassles—anything negative. And researchers have found that we all carry around an innate hostility toward “otherness,” which means anyone not like us. (Herbert, 2006)

An alternative is bounded rationality which suggests that people employ heuristics in decision making rather than strict optimization. This is in response to the complexity of the situation is to great and the individual is unable to process and understand all of the alternatives.

Various fields including cognitive science and social psychology have identified a range of cognitive biases that distort how people perceive reality. Wikipedia provides a list of cognitive biases. The following list includes some related to the LMS adoption decision – they are from the decision-making and behavioural biases section of the list

  • Bandwagon effect – tendency to do things because many others are doing it.
  • Bias blind spot – tendnecy not to compensate for one’s own cognitive biases
  • Choice-supportive bias – the tendency to remember one’s choices as better than they actually were
  • Confirmation bias – the tendency to search/interpret information to confirm your pre-conceptions
  • Congruence testing – the tendency to use tests that only confirm your hypotheses rather than disprove it, or prove another
  • Deformation professionnelle – the tendency to look at things through the conventions of your own profession with no thought to the broader view
  • Disconfirmation bias – tendency to critically examine information that contradicts your prior beliefs but accept uncritically information congruent with your beliefs
  • Endowment effect – the tendency for people to value something more as soon as they own it
  • Focusing effect – prediction bias when you place too much importance on one aspect
  • Hyperbolic discounting – tendency to have a stronger preference for immediate payoffs relative to later payoffs
  • Illusion of control – tendency to believe you can influence outcomes over which you have no control
  • Impact bias – tendency to overestimate the intensity of impact on future states
  • Information bias – tendency to seek information even when it cannot affect attention
  • Loss aversion – tendency to strongly prefer avoiding losses over acquiring gains
  • Outcome bias – tendency to judge a decision but its eventual outcome instead of on the quality of the decision at the time it was made
  • Planning fallacy – tendency to under estimate task completion times
  • Post-purchase rationalisation – tendency to persuade oneself through rational argument that a purchase was a good one.
  • Pseudocertainty effect – tendency to make risk averse choices if the expected outcome is positive, but make risk-seeking choices to avoid negative outcomes
  • Status quo bias – the tendency for people to like things to stay relatively the same
  • Zero-risk bias – preference for reducing a small risk to zero over a greater reduction in a larger risk

But let’s not leave out the social biases

  • False consensus effect – tendency to overestimate the degree to which others agree with you
  • Fundamental attribution error – tendency to over-emphasize personality-based explanations for the behaviour of others while under-emphasizing the role of power and situation influences
  • Halo effect – the tendency for a person’s positive or negative traits in one area to influence your perception of them in other areas
  • Illusion of asymmetric insight – the perception that your knowledge of your peers is greater than their knowledge of you
  • Illusion of transparency – you overestimate your ability to know others, and they overestimate their ability to know you
  • Ingroup bias – preferential treatement to people who you percieve to be part of your group
  • Project bias – tendency to unconsciously assume that others share your thoughts, beliefs, values or propositions
  • Self-server bias – tendency to claim more responsibility for successes than failures.