Gaps, shadow systems and the VLE/LMS

One of my continuing “rants” that long-time readers of this blog will be familiar with is the lack of fit between enterprise systems and what people want to do with them. I’ve blogged about this with enterprise systems, learned to live and thrive in spite of that gap and drawn some lessons from it for enterprise systems.

It’s even become a bit of a family activity with my wife’s Masters research being aimed at attempting to explain the most common response to the lack of fit between people’s requirements and the enterprise systems put in place to fulfill them – shadow systems. The following image is of the model that arose out of Sandy’s work (Behrens and Sedera, 2004).

Sandy's Shadow System Model

One description of the model is that a gap arises (almost inevitably in my opinion) between the enterprise system and the needs of the users. It is created by a range of conditions and can be increased or reduced by two others. The existence of this gap leads to the development of shadow systems. These might simply be employing lots of other people to perform tasks manually that a system should provide. Or it might include developing additional systems to fill the gap.

The gap, shadow systems and the VLE/LMS

My current institution is in the process of adopting Moodle as its sole VLE/LMS. From one perspective Moodle will become another enterprise system supported by the IT folk to achieve business outcomes. That’s certainly one perspective of how Moodle is being rolled out at the institution. For me this raises some interesting questions:

  • Will Moodle suffer the same problems in terms of the gap and shadow system as other enterprise systems?
    I tend to think this is almost certain to happen given the diversity and contrary nature of academics, the diversity inherent in L&T, attempts at the institution to standardise L&T at the same minimum and on-going lack of institutional support for L&T.
  • What form will those shadow systems take?
    The rise of social media, web 2.0 etc tools and their broader use by academics – especially those in the younger generation – and students offers one likely source of the shadow systems.
  • How will the organisation respond?
    The traditional, almost reactionary, response from organisations is that shadow systems are evil and need to be stamped out. That is, if the organisation even becomes aware of them.
  • What are more appropriate ways for the organisation to respond?
    Some colleagues and I have made suggestions previously.
  • Where will the gaps arise?

Where will the gaps arise? An example.

Thomas Duggan another member of staff at my current institution has recently posted an outline of a paper he is working on which seems to detail one of the sources for the gap. It’s a source that seems to potentially fall within the “People” causal condition from Sandy’s model above.

Tom teaches at the institution’s indigenous learning centre Nulloo Yumbah. The paper Tom is working on is built on literature around indigenous learning styles and seeks to see how well Moodle can accommodate those styles. It will be interesting to see what he finds out.

My guess is that how well Moodle will fit these learning styles will depend on many of the factors covered in Sandy’s model. For example:

  • Technology/People;
    Moodle is meant to embody/support a specific learning theory. If you accept that (I still question it) there will be a good fit if that learning theory matches the indigenous learning styles as outlined in the literature Tom is drawing up. If the Moodle learning theory and the indigenous learning styles don’t match, then there will be trouble.
  • Organisation;
    Much is made of Moodle being open source and open source meaning flexible and able to be changed. This point was pushed quite hard in the various sessions promoting Moodle at the institution. However, such a point misses the significant role played by the policies adopted by an institution implementing Moodle. Open source might mean more flexibility, but if the organisation decides to implement as vanilla, that is meaningless. If the organisation doesn’t set up the resources and processes to support and inform that flexiblity, then open source is meaningless.
  • Business processes;
    The institution as adopted a minimum standard for online courses. If there is a mismatch between the indigenous learning styles and the minimum standards, then it might be interesting.
  • Organisation (again);
    The minimum standards are mostly (almost entirely) being driven by the two faculties at our institution and their management. Nulloo Yumbah and the courses it teachers do not, I believe, fit within a faculty. Perhaps the minimum standards don’t apply, or don’t apply as strictly.
  • People; and
    Tom has a background in technology. This means that even if there is a mismatch between Moodle and the indigenous learning styles he may be able to come up with kludges within Moodle that overcome that mismatch. In part, this will come from Tom really understanding the Moodle model and then being able to innovate around it.
  • People (again).
    Tom, through the blogosphere, twitter and general disposition has established social networks with a range of people that also have experience in L&T and technology, including e-learning. He’s also likely to be able to draw upon those people to come up with workarounds to any gaps.

Just one example

There will be anywhere from about 500 to 1000 courses at this institution that will have to go into Moodle over the next year or so. The above process/set of conditions is likely to apply in each of them. There will be a large number of people having to go through this process. My fear/belief is that most of them, because of a range of contextual and personal reasons, simply won’t bother. They will do the bare minimum of work necessary to meet the set minimal standard and won’t bother overcoming the gap that exists.

Another fear is that many of the people who want to overcome the gaps won’t have the knowledge, time or support to overcome the gap. Instead they will have to make do with what they have and over time get increasingly dispirited.

Lastly, those people that have the knowledge and time (a very small minority) will spend a large amount of time experimenting and will end up developing workarounds. The knowledge that goes into those workarounds will never be captured and disseminated, which makes it likely that many people will repeat the exploration process over and over again. Make the same mistakes and reinvent the wheel. Worse, few or none of these people will be recognised.

Solutions?

Well, there’s probably quite a few interesting little research projects and publications in keeping a close eye on how this rolls out. What are the gaps that people face?

Even more interesting would be putting in processes and resources that would enable people to effectively respond to these gaps. And this doesn’t mean using the traditional process for requirements gathering with enterprise systems.

References

Sandy Behrens, Wasana Sedera, (2004) Why do Shadow Systems Exist after an ERP Implementation? Lessons from a Case Study, Proceedings of PACIS’2004, Shanghai, China

David Jones, How to live with ERP systems and thrive, Presented at the Tertiary Education Management Conference’2003, Adelaide

Jones, D., Behrens, S., Jamieson, K., & Tansley, E. (2004). The rise and fall of a shadow system: Lessons for enterprise system implementation. Paper presented at the Managing New Wave Information Systems: Enterprise, Government and Society, Proceedings of the 15th Australasian Conference on Information Systems, Hobart, Tasmania.

How silly can enterprise IT get? Tools should fit the people, not the other way around

I have a thing against large scale enterprise information systems (such as ERP systems like Peoplesoft or learning management systems – commercial or open source) and how they are generally implemented within organisations. This morning provides a wonderful example of why.

An organisation I’m aware of has, like many others, a human resources system that maintains information about staff. In the last couple of years it’s gotten really modern and added a web interface to it so that staff can change and view their details.

The example of the constraints of such systems and the limitations of how they are implemented in organisations is highlighted by an email today from the HR folk. It reminds staff how to keep their details up to date and includes some instructions on how to enter the data correctly.

Apparently, the system doesn’t like the use of characters such as “/” or “-” in address fields. As in

5/105 Fred Street

So the advice is to use a space instead of either offending character. i.e. the usage that would break the system

5/105 Fred Street

should become

5 105 Fred Street

i.e. a space instead of the “/”.

Tools should fit the people, not the other way around

Dave Snowden gave a keynote, which he described here and uses the following quote which I’ve used before

Technology is a tool and like all tools it should fit your hand when you pick it up, you shouldn’t have to bio-re-engineer your hand to fit the tool.

This is a prime example of how enterprise systems in organisations and how they are implemented generally don’t deliver this. How they expect people to “bio-re-engineer” what they do to fit the technology, rather than the other way around. The above advice requires people to change how they write an address to fit the limitations or a poorly designed system.

Not to mention that they expect you to remember to do this from some out of context advice. I’m sure folk will remember this advice in 3 months when they move. At the very least the web page that takes the address should/could have some validation that complains about the use of “-” or “/” and provides some in-context advice on how to fix it.

Better yet, the validation should automatically change it before submitting the data and/or the server-side code that updates the database should sanity check and correct this limitation.

The tool should fit the practice of the human beings. Not the other way around.

So what?

I can here some folk responsible for enterprise systems asking “So what? It’s only a small thing people have to remember on the odd occasion they change address. No big thing. Much harder to change the system. Cost/benefit isn’t there.”. What these folk forget is that this problem is symptomatic of all enterprise information systems. There are lots of “little things” like this that reinforce to users of these systems that the they are having to change to fit the tool, not the other way around.

It’s the IT folk, or at least their reputation, suffering death by a thousand cuts as all of these “small things” build up and create a general impression that IT don’t care about users and are more interested in their systems and keeping costs down. That the users have to carry the burden. The type of build up that can’t be solved by employing public relations folk and forums to “communicate” with the users. You have to walk the walk, not just talk the talk.

Let’s not forget the detail of this example. Entry of an address!!! It’s not rocket science. It’s not something that’s just been developed in the last six months. Computer systems have been doing address entry for decades. How can a system not handle this!!!

What sort of message do you think this sends to the users?

All this reminded me of the song From Little Things Big Things Grow

Organisational fit and success

Yesterday, as part of my thesis travels, I came across this paper (Hogarth and Dawson, 2008) that reminded me of the work around organisational fit and the use of configuration theory to explain/understand the success or failure of change through the application of information technology. This discussion seems tightly linked with this example.

Here’s some background from Hogarth and Dawson (2008)

The notion of configurational fit is based on a theory described in the OS literature (Miles & Snow, 1984), and encapsulates the extent to which the multivariate components of organisational life, such as strategy, structure, management processes, and technology, function in-tune with each other (Sauer, Southon, & Dampney, 1997). In the case of IT innovations, organisational fit is attained when the innovation functions in a way that is consistent with the way the organisation functions, and is managed in the way the organisation is managed.

So what happens if the IT innovation has a weak organisational fit? Hogarth and Dawson (2008) again

In configuration theory, weak fit is the underlying condition that promotes the existence of risk-related behaviours in organisations……Sauer et al. (p. 350) referred to the outcomes of these risk factors as failure modes, which can commonly include process failures (cost and schedule overruns) and interaction failure (non-use of the innovation).

i.e. if there is perceived to be poor fit, the users start to work around the system in ways that will increase the likelihood of system failure. Where system failure has a variety of outcomes.

References

Hogarth, K. and D. Dawson (2008). “Implementing e-learning in organisations: What e-learning research can learn from instructional technology (IT) and organisational studies (OS) innovation studies.” International Journal on E-Learning 7(1): 87-105.

Between the idea and the reality,…. falls the shadow

T. S. Elliot’s poem The Hollow Men has a section that goes

Between the idea
And the reality
Between the motion
And the act
Falls the Shadow

I find some connection with this section due to my interest in shadow systems. Applications of information technology that arise within organisations without the knowledge of, and often in spite of, the official, centralised information technology division.

The idea

The typical response of organisational officialdom is that shadow systems are horrible, beastly things that must be stamped out. These horribly inefficient, unscalable, incorrect toys cobbled together by amateurs create problems for the organisation with no corresponding benefit. All IT development must be done by the heroic, rigorous and knowledgeable folk within the central IT division. This is the idea that underpins much of the existing IT development practice within organisations.

The content of the current wikipedia page on shadow systems gives some insight into this view. The wikipedia page has three sections:

  1. Overview – general summary of shadow systems and what they are.
  2. Cause – some general points about what leads to the development of shadow systems.
  3. Problems – a larger list of the problems shadow systems exhibit. Including, poorly designed, not scalable, poorly documented, untested, may allow unauthorised access to information, easy to introduce errors, one hard disk failure away from disaster, and several versions of the truth.

There is no mention of any potential benefits that shadow systems may provide. One potential benefit that is somewhat mentioned gets twisted. Often shadow systems will enable the real work of the organisation to go ahead in spite of unresponsive IT systems. This gets a mention but in a negative way as a cause of shadow systems, not as a benefit. The description of this “benefit” on the page places shadow systems in a negative light and uncritically accentuate the positives of centralised IT divisions. For example, (emphasis added)

Quite properly, when a reporting system is put together by IT professionals, they need to consider all aspects of how the system will be used.

The various skills that are required to achieve all of this means that inevitably a number of different people will all be involved in the task of creating the new report. This increases the amount of time and effort it takes to put a rigorously engineered solution in place. Shadow Systems typically ignore this kind of rigor, making them much faster to implement, but less reliable and more difficult to maintain.

I find it particularly interesting that the current Wikipedia page on shadow systems seems to contain much of the same content that appears on a commercial website that is linked to from the Wikipedia page. I find it increasingly interesting that the purpose of the commercial website is promote the idea of business intelligence applications and if you dig a bit further is associated with a company selling “ERP” solutions.

The reality

Behrens and Sedera (2004) developed a model (shown below) in an attempt to explain why shadow systems exist.

Sandy's Shadow System Model

The model suggests that shadow systems are created by a gap between the needs of organisation and its use of information technology and the features provided by the organisational IT systems. This mismatch can arise due to characteristics of the people, business processes, organisation and technology. How the gap is overcome is mediated by the available resources and support.

The reality is that the organisational information technology resources are not providing the features required. They are failing.

The shadow

The shadow system becomes the method by which the people within the organisation overcome this gap. How they attempt to address the failing of the organisational information technology division to provide what is required by the organisation.

Overcoming the negative perception of shadow systems

Members of the central IT divisions really hate shadow systems because, at least sub-conciously, they are aware that shadow systems point out where they have failed. And no-one likes having their failures pointed out. Especially folk within central IT divisions who generally have a pretty tough time of it. Especially because most of them have this image of themselves as the supreme, rigorous rationalists. They believe they can and do spend sufficient time and intelligence analysing the situation and coming up with the best solution for the organisation.

What they don’t realise is that there is never going to be one best solution. The implementation and use of information technology within any sufficiently large and complex organisation is going to be a wicked problem and one of the defining characteristics of a wicked problem is that there is no one best solution.

There appears to be some value for IT and broader organisational management to overcome this automatic and negative reaction to shadow systems and learn that they can help identify, and perhaps, solve organisational failures by valuing and seeking to understand shadow systems and the gap between idea and reality.

Perhaps rather than seeking to squash shadow systems, IT divisions should seek to encourage the development of shadow systems and put in place practices and support which mitigate some of the flaws of shadow systems. Such a practice might leverage the increasing capability of the current generation of end-user tools that make “shadow system” development so easy and also seek to address the problem of reducing costs most IT divisions seek.

There is a great chance that shadow systems are almost certainly going to be the source of more innovation than central IT divisions. This is because there will be more diversity of perspective around the development of shadow systems than exist within the development arms of central IT divisions. And diversity and through it perspective shift is a necessary condition for innovation.

References

Sandy Behrens, Wasana Sedera, Why do Shadow Systems Exist after an ERP Implementation? Lessons from a Case Study, Proceedings of PACIS’2004, Shanghai, China