Teleological approach to organisational reviews and change projects

I know of a few organisations that are currently undergoing that favourite past-time of all new management and the greatest fear of people in their new organisations, the organisational restructure. As part of another thread of reading I came across a post that has a nice take on these events.

Too often, the top management hires a high-priced expert who claims to have the best knowledge while ignoring the numbskull advice of the front-line workers because they are deemed less educated or not as “sophisticated”. All expert knowledge is limited to the fact that it is knowledge gained from experience. It is the rearview-mirror perspective of the world which, although can be very useful, is limited. The knowledge that counts in a new project and/or change effort is the tacit knowledge gained by the participants as they go through the change and learn from it. During a crisis or change, people will generate many numbskull ideas but they will also gain valuable insights that can later become the expert knowledge that creates a great CBOK.

The missing Ps – Perception

The Missing Ps framework is my attempt to generate a way of identifying the flaws in the methods used by Universities to select an LMS.

This post looks at perceptions.

The sections will include

  • Perceptions drive adoption
  • IT perceived as a cost
  • Sub-cultures drive perceptions

Perceptions drive adoption

(The following is a bit of plagiarism from a previous post.)

If you draw on the information systems literature you come to TAM (I talked about this before) which posits two main factors influencing adoption of technology

  1. Perceived ease of use
  2. Perceived usefulness
    1. Perceived usefulness is defined as

      The degree to which a person believes that using a particular system would enhance his or her job performance.

      A recent EDUCAUSE Review article focused on finding out what faculty and students want from future LMSes contains reference to this.

      For the most part, faculty members expressed a desire for easier-to-use functions that would reduce time and effort on their part. As one noted: “Maybe it can’t be made easier, but it shouldn’t be made harder.â€? Grading papers, entering grades, and returning files were seen as barriers that do not in exist in the traditional classroom but that are enormously problematic in the L/CMS environment.

      Consequently

      Tools provided within an L/CMS are not, in general, utilized to their fullest capacity. The interviewees gave many reasons for this phenomenon. The amount of time needed to use many tools demands too much of both learner and instructor. The need to change teaching styles in order to incorporate collaborative tools is not rewarded by the institution.

      IT perceived as a cost

      IT continues to be seen by higher education as a cost to be minimised.

      Sub-cultures drive perceptions

The missing P – Product

The Missing Ps framework is my attempt to generate a way of identifying the flaws in the methods used by Universities to select an LMS.

This post is the first step in expanding out one of the Ps into more detail. This post covers “Product”. Like the other posts in this discussion, this is a work in progress. I’m following the open source model – release early and often – though I’ll probably succeed more at the early, than the often.

The sections to (eventually) be covered here include:

  • The product fascination – this is just about the only thing they concentrate on.
  • The LMS model – what it encapsulates
  • The limitations of this model
  • The alternatives – what other options are being considered/are available
  • The broader product picture

The Product Fascination

Almost all LMS evaluations become comparisons of product features.

The LMS model

Almost all current LMSes encapsulate the same basic model which Blackboard recently obtained a patent for.

The limitations of the LMS model

First generation e-learning systems tend to focus almost entirely on the management and measurement of training processes. They add little or no value to the learning process. Furthermore, they do not provide any means to support internal content production processes, relying instead on commercial courseware. These ‘‘learning management systems’’ (LMS) were seen to be nothing more than launch pads for third party content that the organization would purchase or outsource. (Ismail 2002)

The role and importance of technology in the development of e-learning systems is often overstated by technology providers. It is often stated that the deployment of an LMS alone is all it takes to implement e-learning. The problem is, in many cases the development of e-learning projects devolved into a purely technical process, resulting in expensive software implementations, essentially unused by uninformed, fearful, or resentful employees. (Ismail 2002)

Most LMS vendors deliberately distance themselves from pedagogical issues, often adopting an indifferent attitude or sometimes even trying to disguise it as a praiseworthy act of impartiality. This finding is coherent with Firdiyiyek’s (1999) argument that there is a serious mismatch between the abundance of features in LMS and the lack or total absence of explanation on the pedagogy underlying the inclusion of these tools. (Govindasamy, 2002)

Often many features and tools of LMS are left unused. This is a terrible waste of resources since these tools account for the cost of implementing e-Learning. In a worse case scenario, the tools may end up being used in a manner entirely opposed to pedagogical principles, and in turn, will hamper learning. In either case, the impact inevitably will be reflected in the return on the e-Learning investment. (Govindasamy, 2002)

The alternatives

A recent EDUCAUSE article suggests

Furthermore, we need to investigate the limitations and shortcomings of current L/CMS systems so as to determine whether the existing system design has the capacity to fulfill the needs and requirements of today’s learners and other stakeholders.

The current LMS model encapsulates three assumptions

  1. One application provides all the services
  2. That application is hosted by a single entity – usually the University
  3. Only members of the host institution can get access

The same EDUCAUSE Review article mentioned above includes the following

Interestingly, today all major L/CMS products, both commercial and open source, are using a system design and framework very similar to that of the original L/CMS system invented in the mid-to-late 1990s. This system uses a dedicated or isolated L/CMS computer server that services a campus community or several campuses within a university system. The L/CMS software may run within a computer server located on the campus, or it may use a hosting service, which in most cases is supplied via an ISP (Internet Service Provider) by the vendor marketing the L/CMS software. Under this model, L/CMS services and collaboration are characteristically limited to members of that institution.

The rise of Web 2.0 means that neither of these assumptions need, nor possibly should, hold. I’ve talked about the concept of Web 2.0 course sites elsewhere as a stepping stone from the current comfort zone to something more radical (dropping the course sites entirely).

The recent announcement of Google Apps for Education makes this even more likely. Other institutions are also aware of this idea. A report from Monash/Melbourne Unis (Wise and Quealy, 2006) suggests

Broader innovative opportunities exist in terms of the possibility of partnering with popular Web 2.0 service providers (eg Yahoo, Google, MySpace. Flickr) for interactive web services in the same way as universities currently provide academic versions of popular software. Instead of having to log in to a range of online services depending whether it is for personal use, work use, or for study, a student could login to their service of choice for each functionality, and link the service back to their university course or other activity. Online access to the university through partner service providers could act as a ‘virtual gateway’ to the university. This would be the online equivalent of coming on to the physical campus.

Ismail (2002) talks about three types of systems

  1. Learning design system (LDS)
    Alloow the quick anlysis and esign of instructionaly sound learning programs.

    Many e-learning projects do not realize their full potential because they fail to adequately meet basic instructional goals and objectives. In the worst cases, these goals and objectives are never even defined beyond a broad statement of direction. The key to developing effective material lies in combining clear learning goals with pedagogical models. Often this is not done because content developers are not trained in instructional design principles.

  2. Learning Content Management System (LCMS)
    Provide a collaborative authoring environment for the content. Enable workflow between SMEs, content developers and media developers.
  3. Learning Support System (LSS)
    Web environment for supporting the teaching and learning activities.

    While most LMSs tend to provide these capabilities, the features implement in the LMS has tended to support self-directed learning. While this is an important mode of online learning, e-learning should be first and foremost about creating a social space that must be managed for the teaching and learning needs of the particular group of people inhabiting that space. This requires a platform that can be easily modified to take into consideration the needs of the particular learners in the course.

Derek Wenmoth has a post/image covering the difference between PLEs (owned/managed by student) and MLEs (owned/managed by University).

The broader product picture

Instead, designers should seek to understand the basic components of what constitutes an e-learning ‘‘ecosystem.’’ This systems framework is crucial in guiding the decisions relating to the choice and development of each component in relation to the objectives outlined in the organizational e-learning strategy. The framework will specify a learning systems architecture for pedagogical development and systems integration. Learning and the needs associated with supporting learning evolve and change over time, and so should learning systems. The reference architecture provided by a systems framework will allow an organization to progressively select and construct systems depending on requirements and budget. (Ismail 2002)

Ismail (2002) then goes on to talk about the Learning Technologies Systems Architecture as a reference model. This will need to be mentioned but it will need to be translated into something CQU specific, something a little more concrete, to enable people to more easily grasp it.

References

Ismail, J. (2002). “The design of an e-learning system: Beyond the hype.” Internet and Higher Education 4(3-4): 329-336

The Missing Ps are a waste of time

The missing Ps framework is very much under construction attempt to identify the holes and flaws in the methods used by Universities to evaluate and select a learning management system (LMS).

In reading around about this topic, in particular via a document from George Siemens, I came across a report funded by the Melbourne-Monash Collaboration in Educational Technologies, LMS Governance Project Report (Wise and Quealy, 2006).

Skimming through this report I was struck by the following quote from the foreword

We were also very mindful of the number of internal and external reviews of LMS and educational technology (not all of which were made available to us) and the lack of impact these reports appear to have had in decision-making.

A great little quote to bring me back to ground. The best that can come from this idea of the missing Ps is that I get a few publications and hopefully a couple of trips to nice places. Assuming you ignore the personal value that will be generated by simply reading and thinking about this stuff.

Another review of LMS Reviews

George Siemens has posted a document entitled “Learning or Management System? A Review of Learning Management System Reviews” which, as the title suggests, reviews a recent collection of reviews of LMSes in an attempt to offer a more nuanced view.

The abstract states

The position offered in this report encourages an organizational definition of learning as the starting point for selecting a technology platform for creating and delivery learning content. A clear definition of learning vision and desired future states, created through input from stakeholders (administrators, faculty, students, and information services) should provide the foundation for decision making, and the boundaries of platform selection.

This is extremely timely given I’ve only just recently embarked publically on the development of my “missing Ps framework”. My attempt at organising all of the flaws and problems I have with how institutions of higher education are currently going about the process of selecting an LMS.

I find myself agreeing with much of what is in the document. However, I also think there is more here, a broader perspective to be considered.

For example, the quote from the abstract above strongly suggests that an “organisational definition of learning” should be the starting point for this process. In a perfect world I would agree but I have two arguments against this.

  1. Establishing an organisational definition is likely impossible and may in fact be damaging in the short and long term.
    Have you ever tried to get two academics to agree on anything? How about two academics from different disciplines? Most of the organisational definitions of anything created by Universities are often so general as to be meaningless. If it isn’t so high level the definition means that some part of the organisational community has be sidelined. Their understanding, their definition deemed not to be acceptable to the organisation. This loss of diversity is often problematic for the institution. Finally, I’m sure if you could find an organisational definition of learning from 5, 10 or 50 years ago, it would be considerably different from the definition in vogue/use today. Such definitions can be a drag.
  2. Is learning always the primary focus behind the selection of an LMS?
    Sure, most places pay lip service to the rhetoric that the decision is all about learning. But scratch the surface and there is often a very different reason. If you take the perspective that LMSes do little or nothing to support learning then perhaps it’s acceptable for an organisation to choose a different purpose.

The starting place I would suggest should be that the organisation state up-front what is the purpose the tool will be put to. Why does the organisation need it and what is it hoped you can achieve.

The purpose might, hopefully would, include a significant/primary focus on learning. But I think honesty is more valuable in this sort of decision than lip service.

This seems to connect with a quote in the paper from Wise and Quealy (2006)

“the educational significance of LMS is largely overemphasized and misunderstood …[suggesting it is critical for a university to] … understand itself—what it values, what it does well and how it does it, what it would like to do, and how it might do thisâ€?

Question allocation difficulties – BAM

A long time ago I identified “reducing meaningless freedom” as a key design principle for elearning type apps – basically anything web-based and possible broader. It ties in with ideas others have identified, it’s even discussed in the sad textbook we’re using in COIS20025 this term.

The idea is, that if a user shouldn’t be able to do something, don’t let them.

With BAM as used in COIS20025 this term students are meant to use their blog to respond to a fixed set of questions.

In doing this I had assumed that

  • Each post would use the question title as the subject of the post
  • The answer to each question would be included in one post
  • That students could identify the URL for their blog and register it with the CQU system

We’ve had a number of students break each of those. Especially identifying the URL of their blog. We’ve had two students who have registered the home URL of their blog host (e.g. http://wordpress.com/) as the URL for their blog!

This will be one of the hurdles in using real applications, ones we don’t design and control. It means we’re going to have build more smarts into the integration apps we do build.

The missing Ps in LMS adoption decisions

For, what seems the longest time, CQU has been talking about the need to evaluate and choose yet another LMS. We’ve sort of done this two times before, and from my perspective, both adoption decisions have been fatally flawed. Some discussion about those processes can be found, to a lesser or greater extent, in a variety of publications by CQU authors including: Danaher, Luck and McConachie (2005), Sturgess and Nouwens (2005), and Jones, Luck, McConachie and Danaher (2005)

I don’t think these flaws are unique to CQU. I’ve seen little evidence of other Universities having adoption procesess that are significantly better. They all seem, to a less or greater extent, seem to suffer from the same fatal flaws. The following is my attempt to identify and talk about what some of those flaws are and, because I’m an academic who has to publish, provide a “neat” framework to encapsulate them.

My working title is the “The missing/forgotten Ps of LMS adoption”

The aim is to give the people embarking on this sort of journey a bit of a pause for thought. To stop them from following all the other lemmings off the cliff. Or, at the very least, be a little more nuanced and informed about the implications of their decision.

To start the ball rolling I’m going to try and identify the Ps I want to talk about. The following is my working list. Hopefully, I will expand these out in separate posts.

The missing Ps include

  • Product
    The almost entire aim of LMS adoption processes is to choose a product. Typically they become simply nothing more than feature comparisons. So what’s missing?

    • A broader understanding of the limitations and future of the LMS product.
      The standard LMS product features are the result of very early experimentation followed by almost no innovation. They all follow the same model – now patented by Blackboard. A model that offers little or no support for learning.
    • Limited understanding of alternative models. e.g. You don’t need a single system that provides all services. You can have many software systems that are integrated. You don’t need to host all the software on University computers. You don’t need a system that is structured around courses, you can have one structured around learners. You can have multiple views of the system – course, learner, teacher, visitor – to represent competing requirements.
  • Process
    This also has many flaws

    • Over emphasis on plan-driven development at the cost of adaptive approaches
      This is the one I’ve banged on about in previous papers (e.g. an early one, and the most recent one). The on-going acceptance of agile development methodologies, the Enterprise 2.0 meme incorporating emergence and the idea of rapid incrementalism from the two Johns make pushing this view a bit easier.
    • Once implemented, a failure to recognise the need for process change
      Any information system encapsulates a certain way of doing things. A certain process that must be followed. If you purchase a system the current institutional approaches will have to change. Ever tried to get an academic to change how they teach?
  • Purpose and place
    If you accept the traditional IS development approach then the first step is to identify the purpose of the system you are trying to adopt. Many Universities don’t appear to do this. Then there is the problem of whether or not it is sensible to state that all academics that want to get involved in eLearning actually have the same purpose.

    Then there is the issue that there are competing sub-cultures involved in the use of an LMS and each will have a different purpose. e.g. Academic versus management versus technologist versus student.

    Place – you can’t generalise one approach to all organisations. Unversities are different, they have different purposes.

  • People
    LMS adoption decisions are made by people. LMSes are used by people. The nature of those people who they are, how they think and what they believe, amongst other characteristics, have a significant impact.

    • They myth that people are rational

      Extensive research shows that our brains have certain hardwired propensities that might be exploited. For example, our brains tend to register frequently heard facts as true, even if they are patently false. As a result, our memories and beliefs are highly malleable and unreliable. We also tend, if unchecked by the conscious reasoning mind, to focus overly on risk, inconvenience, hassles—anything negative. And researchers have found that we all carry around an innate hostility toward “otherness,” which means anyone not like us. (Herbert, 2006)

      An alternative is bounded rationality which suggests that people employ heuristics in decision making rather than strict optimization. This is in response to the complexity of the situation is to great and the individual is unable to process and understand all of the alternatives.

  • Perceptions
    • Perceptions drive adoption and use
      You purchase an information system in order for it to be used. The information systems research and literature identifies two factors as the most likely contributors to adoption of a system – perceived ease of use and perceived effectiveness.
    • Perceptions of IT
      Management of Universities, and most organisations, perceive IT to be a cost to be minimised rather than the value it provides.
  • Proof
    • Proof of assumptions driving selection of LMS e.g. LMS help learning, proprietary are cheaper, FLOSS is cheaper etc.
    • Proof that the claims made during previous adoption decisions have been achieved
  • Past Experience
    It’s important that lessons be learnt from past experience, both within the institution and at other institutions. Questions that arise include

    • What were the levels of use of various parts of the system? Why?
    • What problems did people face?
    • What is the literature saying about such systems and their implementation?

The missing ground rule for Enterprise 2.0

In his MIT Sloan Management Review article, “Enterprise 2.0: The Dawn of Emergent Collaboration”, Andrew McAfee cites two “intelligent ground rules” that people building Enterprise 2.0 technologies are following

  1. Making sure the applications are easy to use.
  2. Avoiding any preconceived notions about categories or structure by building tools that let these aspects emerge.

I see some connections with this ground rule and the concept of rapid incrementalism that has been talked about by John Seely Brown. In this John Hagel talks briefly about rapid incrementalism as one of the responses to the IT Doesn’t Matter discussion kicked off by Nick Carr.

The two John’s position is that economic impact from IT comes from incremental innovations. Rapid incrementalism “enhances learning potential and creates opportunties for further innovations”.

This sounds very much like the type of emergence.

The design theory I have formulated from Webfuse includes a heavy emphasis on emergence. But it also combines it with aspects of Roger’s diffusion theory to address one of the challenges to Enterprise 2.0 identified by Andrew McAfee

The first is that busy knowledge workers won’t use the new technologies, despite training and prodding. Most people who use the Internet today aren’t bloggers, wikipedians or taggers. They don’t help produce the platform — they just use it. Will the situation be any different on company intranets? It’s simply too soon to tell.

Actually, I don’t think it is too soon to tell. The “build it and they will come” aproach doesn’t work with knowledge workers. There’s got to be something in it for them. This is the 9X email problem which Andrew McAfee has talked about.

If you draw on the information systems literature you come to TAM (I talked about this before) which posits two main factors influencing adoption of technology

  1. Perceived ease of use
  2. Perceived usefulness
    1. Perceived usefulness is defined as

      The degree to which a person believes that using a particular system would enhance his or her job performance.

      So I think there’s a ground rule missing for Enterprise 2.0 applications. The must not only be easy to use, they must be useful.

      In an Enterprise 2.0 I believe the role for an organisation’s IT/IS people must change from supporting the technology or specific business processes to continually being on the look out for how to leverage the technology to increase perceived usefulness of the systems.

      This is how you encourage adoption and use by knowledge workers. By creating a trust that the information systems for an organisation are being developed to be useful to them.

IS as the initiator of new topics in business schools

ISWorld is a mailing list that acts as a major forum for discussion for information systems academics. Over the last few days the discussion has been about dropping enrolments and the search for bumper sticker phrases as one way to encourage enrolments.

In one post Neil Ramiller from Portland State University argues the point that IS has a history of starting off new topics that are then adopted/appropriated by other colleagues in the business school. His examples

  • Customer relationship management – marketing
  • Electronic commerce – marketing
  • Knowledge management – management
  • Enterprise systems, supply-chain systems – operation management

This is one potential argument for attracting students – IS isn’t plumbing. It isn’t the boring infrastructure – which is the point of Nick Carr’s IT doesn’t matter stuff – it’s where all the new and interesting ideas in business come from.

Not only does it provide the potential to distance IS from IT and CS. It also offers an argument for students interested in other business school areas.

Helping “NESB” students realise – there is no one answer

Stephen Downes has a paper titled “Learning Networks and Connective Knowledge” that is currently being discussed on the ITForum. He blogs about the paper and includes links to ITForum and the paper.

I’m currently about 7 pages into the paper. The paper aims to provide the background to Stephen’s thinking about eLearning 2.0 which in turn is based on connectivism and the characteristics of the next-gen students. The paper has a very strong theoretical base and has required me to exercise a bit more brain power than I have recently. The following is an attempt to reflect on those first 7 pages, but also to connect it to a recent experience.

What I’ve read so far is essentially trying to establish the foundations of connectivism and demonstrate its advantages/differences over the more widely held cognitivist theories.

Part of Stephen’s description of cognitivism based approaches is

The argument, in a nutshell, is that the claims of folk psychology are
literally true, that there is, for example, an entity in the mind corresponding to the belief
that ‘Paris is the capital of France’, and that this belief is, in fact, what might loosely be
called ‘brain writing’ – or, more precisely, there is a one-to-one correspondence between
a person’s brain states and the sentence itself.

So, in a cognitivist based approach learning/teaching (I’m limiting myself to learning within higher education, which is my context) involves the attempt to find the correct “brain state” required by the course and the ability to demonstrate that it has been discovered.

Stephen explains how connectivism views this entirely different. That there is no single brain state, that this is an incorrect characterisation of how it all works, on a number of levels.

The local connection

Yesterday I was involved in a de-brief session for some focus groups/interviews that were done with Sydney CQU students around the first use of the BAM (Blog Aggregation Management) system in COIS20025

In this course students use their own blog to maintain a reflective journal in which they respond to some set questions that are due at set times during the term.

During the focus group students commented on how they wouldn’t post their responses early because of the fear that other students would be searching the web looking for answers to copy.

The Sydney students are primarily from non-english speaking backgrounds, primarly Asia and currently mostly India. The perspective of the staff interviewed were that these students are culturally or because of previous school experience firmly believe that there is one correct answer, one brain state, and that their whole focus is on getting the right answer.

This characterisation is also used to explain why the students in the focus group, apparently amongst the best students, found the questions that required more reflective/personal responses difficult. They wanted to know, what is the correct answer for this question?

The poorer students, generally those with weaker english language skills, find this even harder because their language ability makes it difficult for them to understand/internalise the “one correct brain state” and then provide it back on queue when required to do so for assessment. Hence, their increasing reliance on what we deem to be plagiarism.

The question

It appears that these students, unknowingly, have a strong cognitivist stance – at least that’s my interpretation. There is one correct answer. My goal is to get that correct answer.

Could we help address this drive for the “one correct brain state” and consequently the issue of plagiarism, if we made them more aware of alternate views of knowledge like connectivism?

If so, how would we do this. I’m finding Stephen’s paper a difficult read, our NESB students would find it impossible.

What are the teaching/learning strategies to help change this view?

Stephen’s paper includes a large collection of references to papers arguing against various aspects of cognitivism. One is “Patterns of Discovery” by Hanson which is said to argue that causal explanations are context-sensitive, e.g.

‘What was the cause of the accident?’ It
depends on who you ask – the police officer will point to the speed, the urban planner will
point to the road design, the driver will point to the visibility.

An activity based around something like this might have some potential. It could also be used to demonstrate some very important aspects of developing information systems in organisations.