How do you implement PLEs into higher education courses?

Jocene reflects a bit upon a slidecast (titled “Personal Learning Environments: The future of education?”) by Graham Atwell.

I tend to sense a touch of frustration in the post. Which I don’t think is at all surprising since the question “How do you implement PLEs into higher education courses?” is extremely complex. Doing anything to change learning and teaching within higher education is extremely difficult. This is made almost impossible when it is something that potentially brings into question not only the pedagogical practice of individual academics, but also the assumptions underpinning the administrative and technological bureaucracies that have accreted within tertiary institutions.

Institutions of higher education have essentially failed to implement “enterprise e-learning” in a way that caters for and values the diversity inherent in university teaching. I’m somewhat pessimistic about its ability to implement e-learning that caters for the diversity of university students – an order of magnitude (or two) greater level of diversity.

A way forward

That diversity, is for me, a clue to a way forward. Implementing PLEs within higher education is about a focus on the potential adopters, both the teaching staff and the students. By answering questions like: “What do they want?”, “What do they do?”, “What is a problem you can solve for them that makes a difference?” with something that is related to, or at least moves them towards the ideals of a PLE.

However, answering these questions should not be done by asking them. When it comes to something brand new, something that challenges established ways of doing things simply asking people “what would you like to do with X” is a waste of time. If they tell you anything, the will tell you what they’ve always done.

I wonder if this explains the current suggestion that the next generation of students don’t want their university life mixed in with their social life. They don’t want universities getting into Facebook and other social spaces. If the students haven’t seen good examples of how this might work, you can’t really rely on their feedback, they don’t know yet.

I’ve talked about the 7 principles of knowledge management and in particular principle #2

We only know what we know when we need to know it. Human knowledge is deeply contextual and requires stimulus for recall.

Which is why I like this comment from Jocene

He talks about the need to contextualise the PLE. Well, yes. My colleague and I have decided to push ahead with our own contextualised understanding, so we can start to reflect upon rather that speculate about our PLE work.

Get stuck in, try a few things and then reflect upon what worked, what didn’t. What did the students like, what might be better. This sounds like a much more effective way than researchers and prognosticators extrapolating what they think people will need and how they should use it. However, I think Jocene’s next quote highlights the difficult in drawing a barrier between teleological and ateleological design.

But we still keep getting stuck, half way over the implementation hurdle! If we telelogically suggest a way forward for any group of learners, then we are not facilitating a PLE, we are imposing our values.

Traditionally e-learning within universities is teleological and because the nature of teleological design is a complete and utter mismatch with the requirements of e-learning problems arise. Some colleagues and I have pointed these problems out in two publications (Jones, Luck, McConachie and Danaher, 2005; Jones and Muldoon, 2007).

One defining characteristic of teleological design is that the major design decisions are made by a small group of experts and/or leaders. There decisions are meant to be accepted by the rest of the group and are meant to be the best decisions possible. I think Jocene’s worried about this type of “imperialism” within PLEs. If she and her colleague make decisions about what should be done, aren’t they being teleological?

They don’t have to be, but it does depend on how you go about it. To my mind you reduce the teleological nature of your decisions by doing the following

  • make small changes to existing practice;
  • ensure that the changes solve problems or provide new services that will be valued by the participants;
  • ensure that you will learn lessons/try new things/make different mistakes than you have before;

i.e. you are doing safe-fail probes rather than fail-safe design. This is a distinction from Dave Snowden and which is talked about more here.

What does that mean for PLEs in higher education

Some quick thoughts on what this might mean in concrete form for implementing PLEs in higher education:

  • Modify existing e-learning infrastructure to enable it to work with the Web 2.0/mashup/PLE technology and approaches.
    e.g. generate RSS feeds out of various course management system (CMS) features and make them available to students and staff.
  • Use the “web 2.0’ifying” of the CMS to build features that solve problems or provide better services for staff and students.
  • Build some examples using these services (of PLEs) in existing social media applications – the obvious is probably facebook – but this decision should be guided by some of the following.
  • Don’t be exclusionary, don’t focus all efforts on one type of PLE or social media application.
  • Implement strategies and techniques to really engage with the students and staff to learn about what they do. NOT what they say they do, but what they actually do and experience. Use this insight to guide the above.
  • Use the strategies and techniques in the previous point to observe what happens when staff and students do (or do not) use the PLE services and use that insight to identify the next step.
  • Ensure a tight connection with and awareness of what other interesting folk are doing in this area and use it to inform the design of the next safe-fail probes you are doing.
  • Try not to do too much for staff or students. The whole notion of the PLE is that you are empowering them to do things for themselves. If the “instructional assistant/designer” does too much for them it breaks this ideal and it also doesn’t scale.

References

David Jones, Jo Luck, Jeanne McConachie, P. A. Danaher, The teleological brake on ICTs in open and distance learning, To appear in Proceedings of ODLAA’2005

David Jones, Nona Muldoon, The teleological reason why ICTs limit choice for university learners and learning, In ICT: Providing choices for learners and learning. Proceedings ASCILITE Singapore. pp 450-459

Using a blog for course design foult sessions

I’ve bitten the bullet and have decided to use WordPress blog to support the 6 hour orientation to course analysis and design I’m supposed to run next week.

It’s probably going to be much more work than I should or planned to put in, but so far it’s been fairly easy. It may be worthwhile.

Good teaching is not innate, it can be “learned” – and what’s wrong with academic staff development

The title to this post is included in a quote from Kane, Sandretto and Heath (2004)

The research team, comprising two teacher educators and an academic staff developer, embarked upon this research confident in the belief that good teaching is not innate, it can be learned. With this in mind, the project sought to theorise the attributes of excellent tertiary teachers and the relationships among those attributes, with the long-term goal of assisting novice academics in their development as teachers.

This coincides nicely with my current task and also with an idea I came across on the week-end about deliberate practice and the work of Anders Ericsson.

The combination of these “discoveries” is also providing some intellectual structure and support for the REACT idea about how to improve learning and teaching. However, it’s also highlighting some flaws in that idea. Though the flaws aren’t anywhere near as large as what passes for the majority of academic staff development around learning and teaching.

The following introduces these ideas and how these ideas might be used to improve academic staff development.

Excellent tertiary teaching

Kane et al (2004) close the introduction of their paper with

We propose that purposeful reflection on their teaching plays a key role in assisting our participants to integrate the dimensions of subject knowledge, skill, interpersonal relations, research/teaching nexus and personality into recognised teaching excellence. We conclude with a discussion of the implications of our model for staff development efforts.

Their proposition about the role of reflection in contributing to excellent teaching matches with my long held belief and perception that all of the best university teachers I’ve seen have been those that engage in on-going reflection about their teaching, keep looking for new knowledge and keep trying (and evaluating) innovations based on that knowledge in the hope to improve upon their teaching.

The authors summarise a long history of research into excellent teaching that focused on identifying the attributes of excellent teachers (e.g. well prepared, stimulate interest, show high expectations etc.) but they then suggest a very important distinction.

While these, and other studies, contribute to understanding the perceived attributes of excellent teachers, they have had limited influence on improving the practice of less experienced university teachers. Identifying the elements of “good” university teaching has not shed light on how university teachers develop these attributes.

The model the develop is shown below. The suggest

Reflection lies at the hub of our model and we propose that it is the process through which our participants integrate the various dimensions

Attributes of excellent tertiary teaching

The authors don’t claim this model to have identified any novel sets of attributes. But they do suggest that

the
way in which the participants think about and understand their own practice through purposeful reflection, that has led to their development of excellence

What’s been said about reflection?

The authors have a few paragraphs summarising what’s been said about reflection in connection to tertiary teaching, for example

Day (1999) wrote “it is generally agreed that reflection in, on and about practice is essential to building, maintaining and further developing the capactities of teachers to think and act professionally over the span of their careers”

.

They trace reflection back to Dewey and his definition

“an active, persistent, and careful consideration of any belief or supposed form of knowledge in light of the grounds supporting it and future considerations to which it tends

The also mention a framework of reflection outlined by Hatton and Smith (1995) and use it to provide evidence of reflection from their sample of excellent teachers.

Expertise and deliberate practice

Among the many quotes Kane et al (2004) provide supporting the importance of reflection is this one from Stenberg and Horvath (1995)

in the minds of many, the disposition toward reflection is central to expert teaching

Another good quote (Common 1989, p. 385).

“Master teachers are not born; they become. They become primarily by developing a habit of mind, a way of looking critically at the work they do; by developing the courage to recognize faults, and by struggling to improve”

Related to this view is the question “Was Mozart, and other child prodigies, brilliant because of some innate talent?”. This is a question that this blog post takes up. The answer it gives is no. Instead, it’s the amount and quality of practice they engage in which makes the difference. Nurture wins the “nature versus nurture” battle.

The blog post builds on the work of Anders Ericsson and the concept of “deliberate practice”. The abstract for Ericsson et al (1993) is

The theoretical framework presented in this article explains expert performance as the end result of individuals’ prolonged efforts to improve performance while negotiating motivational and external constraints. In most domains of expertise, individuals begin in their childhood a regimen of effortful activities (deliberate practice) designed to optimize improvement. Individual differences, even among elite performers, are closely related to assessed amounts of deliberate practice. Many characteristics once believed to reflect innate talent are actually the result of intense practice extended for a minimum of 10 years. Analysis of expert performance provides unique evidence on the potential and limits of extreme environmental adaptation and learning.

Implications for academic staff development

If reflection or deliberate practice are key to developing mastery or expertise, then how do approaches to academic staff development and associated policies, processes and structures around university learning and teaching help encourage and enable this practice?

Seminars and presentations probably help those that are keen to become aware of new ideas that may aid their deliberate practice. However, attendance at such events are minimal. Much of existing practice seems to provide some level of support to those, the minority, already engaging in deliberate practice around learning and teaching.

The majority seem to be able to get away without engaging like this. Perhaps there’s something here?

References

Common, D.L. (1989). ‘Master teachers in higher education: A matter of settings’, The Review of Higher Education 12(4), 375–387.

Hatton, N. and Smith, D. (1995). ‘Reflection in teacher education: Towards definition and implementation’, Teaching & Teacher Education 11(1), 33–49.

Kane, R., S. Sandretto, et al. (2004). “An investigation into excellent tertiary teaching: Emphasising reflective practice.” Higher Education 47(3): 283-310.

Sternberg, R. and Horvath, J. (1995). ‘A prototype view of expert teaching’, Educational Researcher 24(6), 9–17.

The design of a 6 hour orientation to course analysis and design

It’s that time of year again, next week I get to run a session with 20 or so new CQU academics looking at course analysis and design. The session is part of a four day program entitled Foundations of University Learning and Teaching (FoULT). The session is run twice a year.

The following post gives an overview of some of my thinking behind the session this year. The sessions won’t really be finalised until the sessions are over, so if you have any feedback or suggestions, fire away.

Constraints

The following constraints apply

  • The session lasts 6 hours.
  • I’m told there will be 24 participants, I expect less than that.
  • I’ll be the only facilitator.
  • The participants are required to do this as part of the employment and some may be less than enthusiastic, though there are generally some very keen participants.
  • The sessions will be held in a computer lab. The computers are arranged around the walls of the room and there is a table without computers in the middle of the room.
  • 3 hours after lunch on one day and the 3 hours before lunch the following day.
  • The participants will be a day and a half into the four days by the time they get to this session (information overload kicking in).
  • Earlier on the first day they will have done sessions on “knowledge management” and assessment – moderation and marking.
  • The title of the sessions is “course analysis and design” so should probably do something close to that.
  • I don’t have the time to do a lot of work because of time constraints and other responsibilities.
  • Have done this session a few times before (slides from the last time are Introduction, Implementation, Analysis and design) so that experience will constrain my thinking.
  • Theoretically, I don’t believe that there is much chance of radically changing minds or developing expertise in new skills. The best I can hope for is sparking interest, raising awareness and pointing them in the right direction.

The plan

I’m thinking that the session should aim to

  • Make people aware of the tools and support currently available to help with their teaching.
  • Introduce them to some concepts or ideas that may lead them to re-think the assumptions on which they base their course design.
  • Introduce them to some resources and ideas that may help them design their courses.

Activities during the session will include

  • Some presentation of ideas using video and images.
  • Discussion and sharing of responses and their own ideas via in class discussion but also perhaps through the CDDU wiki and/or perhaps this blog.
  • A small amount of activity aimed at performing some design tasks.
  • A bit of playing around with various systems and resources.

There won’t be any assessment for this one.

The sessions

I’m planning on having 4 sessions over the 6 hours

  1. Introduction
    Set up who I am and what we’re going to be doing. Find out more about the participants – maybe get them to put this on the wiki or perhaps a WordPress blog — that sounds like an idea. Introduce the Trigwell (2001) model of university teaching that I’ll be using as a basic organising concept. Use it to introduce some of the ideas and explain the aim of the sessions. Introduce them to the technology we’ll be using and get them going.
  2. The T&L Context
    Talk about the details of the CQUni T&L context. What tools and resources are available? What do students see when they use various systems (something staff often don’t see)? Who to ask for help? etc. Also include mention of “Web 2.0” tools i.e. that the context and tools for T&L aren’t limited to what is provided by the institution. Provide an opportunity to play and ask questions about this. Aim is to be concrete, active and get folk aware of what tools they can use. Hopefully to keep them awake after lunch.
  3. Teachers’ thinking
    Introduce and “attack” some basic ideas that inform the way people think about learning and teaching. Some ideas about course design, learning and teaching and human cognition.
  4. Teachers’ planning
    Talk about the process of actually doing course design and some of the ideas, resources and tools that can be used during this process.

The plan is that the first two would be on the afternoon of the first day with the last two on the following day.

The Trigwell (2001) model of teaching is shown in the following image and is briefly described on the flickr page for the image. You should see the connection between the names of the sessions and the model

Trigwell's model of teaching

Actually, after posting this I’ve made some changes to expand the use of the Trigwell (2001) model including teachers’ strategies and in particular gathering some of their strategies.

What’s needed? What would be nice?

I want to provide pointers to additional resources and also make use of good resources during the sessions. The list of what I’ve got is available on del.icio.us.

If you know of any additional resources you’d recommend please either add them in the comments of this post or tag them in del.icio.us with foult

Feedback on the above ideas would also be welcome.

References

Trigwell, K. (2001). “Judging university teaching.” The International Journal for Academic Development 6(1): 65-73.

Some possible reasons why comparison of information systems are broken

All over the place there are people in organisations performing evaluations and comparisons of competing information systems products with a strong belief that they are being rational and objective. Since the late 1990s or so, most Universities seem to be doing this every 5 or so years around learning management systems. The problem is that these processes are never rational or objective because the nature of human beings is such that they never can be (perhaps very rarely – e.g. when I’m doing it ;) ).

Quoting Dave Snowden

Humans do not make rational, logical decisions based on information input, instead they pattern match with either own experience, or collective experience expressed as stories. It isn’t even a bit fit pattern match, but a first fit pattern match. The human brain is also subject to habituation, things that we do frequently create habitual patterns which both enable rapid decision making, but also entrain behaviour in such a manner that we literally do not see things that fail to match the patterns of our expectations”.

Dave also makes the claim that all the logical process, evaluations, documents and meetings we wrap around our pattern-matching decisions is an act of rationalisation. We need to appear to be rational so we dress it up. He equates the value of this “dressing up” with that of the ancient witch doctor claiming some insight from the spirit world leading him to the answer.

Via a Luke Daley tweet I came across a TED talk by Dan Gilbert that provides some evidence from psychology about why this is true. You can see it below or go to the TED page

As an aside the TED talks provide access to a lot of great presentations and even better they are freely available and can be downloaded. Putting them on my Apple TV is a great way to avoid bad television.

The “dominant” assumptions underlying university-based e-learning: an introduction

As part of working on my thesis I’m working on chapter 2. As the traditional literature review one purpose of the chapter is to demonstrate what I know about the topic and to highlight what I think are the flaws or holes in current research and practice that I believe my research will address. The following builds on some initial ideas from a previous blog post and serves as some practice in formulating my ideas. So it will still be rough. Feel free to suggest improvements, point out problems and disagree.

I was going to leap headlong in describing the flaws of university based e-learning that I perceive. However, before getting to those I thought I’d address what I see as the source of these problems – the “dominant assumptions”.

I was going to develop a list of those dominant assumptions to include in this post. Then it got too long and was getting to inflexible. So now I’m going to give an introduction in this post and have separate posts to develop the assumptions I associated with different components.

The argument

The basic argument is that much of the organisational practice of selecting, designing and supporting e-learning information systems (including both the technology and how the technology is harnessed through organisational practices) within universities is far less effective than it could be. The vast majority of this practice can be argued to be more closely aligned with the band-wagon effect than of being appropriate for the organisation.

This is because of the complexity of the factors to be considered in order to make informed decisions about this practice and the observation that most most of these decisions are flawed because they draw on a limited set of dominant assumptions of different components that contribute to e-learning. These dominant conceptualisations are entirely incompatible with the nature of e-learning within universities and/or their unquestioned acceptance limits consideration of alternate perspectives that might be useful.

Important: I’m thinking directly about the practice of e-learning individual courses, though this is directly impacted upon by what I am thinking about. My focus is at the organisational level. With how a university, or perhaps one university organisational unit, and its management decide how to implement and support e-learning in terms of technology, policy and processes.

Introducting the Ps Framework

The Ps Framework is defined in this paper (Jones, Vallack and Fitzgerald-Hood, 2008) as

As a descriptive theory, the Ps Framework is proposed as a tool to make some sense of the complex, uncertain and contradictory information surrounding the organisational adoption of educational technology. The seven components of the Ps Framework identify many (any claim to exhaustive coverage would require additional research) of the important factors to be considered in such decisions.

Earlier in the same paper the explanation of descriptive theory (aka taxonomy or framework) is given as

Frameworks offer new ways of looking at phenomena and provide information on which to base sound, pragmatic decisions (Mishra & Koehler, 2006). Gregor (2006) defines taxonomies, models, classification schema and frameworks as theories for analysing, understanding and describing the salient attributes of phenomena and the relationships therein.

The sheer complexity and scope of the factors to be considered around the organisational implementation of e-learning within universities is such that, I believe, there is a need for some sort of framework to structure the discussion. The Ps Framework may not be the final or best word frameworks to structure this discussion but it’s the most complete one that I’m currently aware of.

There are other frameworks that have been used in e-learning, including:

  • the 4Es conceptual model (Collis et al, 2001)
    Used to preduct the acceptance of ICT innovations by an individual within an educational context – environment, effectiveness, ease of use and engagement.
  • the ACTIONS model (Bates, 2005)
    Provides guidance on selecting a particular educational technology – Access, Costs, Teaching and learning, Interactivity and user-friendliness, Organisational issues, Novelty and Speed.
  • the P3 model (Khan 2004)
    An approach to course development, delivery and maintenance.

I’m sure there are many others. If you know of any, please share them.

The components of the Ps Framework

The seven components of the Ps Framework (I’m quoting from the paper) are

  1. Purpose.
    What is the purpose or reason for the organization in adopting e-learning or changing how it currently implements e-learning? What does the organization hope to achieve? How does the organization conceptualise its future and how e-learning fits within it?
  2. Place.
    What is the nature of the organization in which e-learning will be implemented? What is the social and political context within which it operates? How is the nature of the system in which e-learning will be implemented understood?
  3. People.
    What type of people and roles exist within the organization? What are their beliefs, biases and cultures?
  4. Pedagogy.
    What are the conceptualisations about learning and teaching, which the people within the place bring to e-learning? What practices are being used to learn and teach? What practices might the people like to adopt? What practices are most appropriate?
  5. Past experience.
    What has gone on before with e-learning, both within and outside of this particular place? What worked and what didn’t? What other aspects of previous experience at this particular institution will impact upon current plans?
  6. Product.
    What type of "systems" or products are being considered? What is the nature of these products? What are their features? What are their affordances and limitations?
  7. Process.
    What are the characteristics of the process used to choose how or what will be implemented? What process will be used to implement the chosen approach?

The relationship between the seven components can be explained as starting with purpose. Some event or reason will require an organization to change the way in which it supports e-learning. This becomes the purpose underlying a process used by the organization to determine how (process) and what it (product) will change. This change will be influenced by a range of factors including: characteristics of the organization and its context (place); the nature of the individuals and cultures within it (people); the conceptualisations of learning and teaching (pedagogy) held by the people and the organization; and the historical precedents both within and outside the organisation (past experience). This is not to suggest that there exists a simple linear, or even hierarchical, relationship between the components of the Ps Framework. The context of implementing educational technology within a university is too complex for such a simple reductionist view. It is also likely that different actors within a particular organization will have very different perspectives on the components of the Ps Frameworks in any given context.

The dominant assumptions

In the previous post that this one has grown out of I argued near the end that

E-learning in universities generally suffers from bandwagons because the decision makers draw upon a number of dominant assumptions the negative influence what they do.

Many of these assumptions are so fundamental that they are never questioned. In fact, they are never even thought about. The decision makers don’t even know that they don’t know. This limits the quality of their decisions and contributes to the bandwagon effect.

For the rest of this post I use the components of the Ps Framework to develop an early list of these dominant assumptions and the problems they create. Where possible I try to include some explanation of the problem with each assumption and some examples.

Disclaimer: “dominant” is a bit strong for some (perhaps all) of these assumptions. Each of the assumptions has a different “dominance level”, which is a factor of how many people believe it and to what level they are unaware that there are more appropriate alternatives.

References

Bates, A. W. (2005). Technology, E-learning and Distance Education, Routledge.

Collis, B., O. Peters, et al. (2001). “A model for predicting the educational use of information and communication technologies.” Instructional Science 29(2): 95-125.

Gregor, S. (2006). “The nature of theory in information systems.” MIS Quarterly 30(3): 611-642.

Jones, D. and S. Gregor (2004). An information systems design theory for e-learning. Managing New Wave Information Systems: Enterprise, Government and Society, Proceedings of the 15th Australasian Conference on Information Systems, Hobart, Tasmania.

Jones, D., J. Vallack, et al. (2008). The Ps Framework: Mapping the landscape for the PLEs@CQUni project. Hello! Where are you in the landscape of educational technology? ASCILITE’2008, Melbourne.

Khan, B. (2004). “The People-Process-Product Continuum in E-Learning: The e-learning P3 model.” Educational Technology 44(5): 33-40.

Mishra, P. and M. Koehler (2006). “Technological pedagogical content knowledge: A framework for teacher knowledge.” Teachers College Record 108(6): 1017-1054.

OECD. (2005, 17 January 2006). “Policy Brief: E-learning in Tertiary Education.” Retrieved 5 December, 2006, from http://www.oecd.org/dataoecd/55/25/35961132.pdf.

Walls, J., G. Widmeyer, et al. (1992). “Building an Information System Design Theory for Vigilant EIS.” Information Systems Research 3(1): 36-58.

Somethings that are broken with evaluation of university teaching

This article from a Training industry magazine raises a number of issues, well known in the research literature, about the significant limitations that exist with the evaluation of university teaching.

Essentially the only type of evaluation done at most universities is what the article refers to as “level 1 smile sheets”. That is student evaluation forms that ask them to rank what they felt they learn, what they felt about the course and the teacher. As Will Thalheimer describes

Smile sheets (the feedback forms we give learners after learning events) are an almost inevitable practice for training programs throughout the workplace learning industry. Residing at Donald Kirkpatrick’s 1st level—the Reaction level—smile sheets offer some benefits and some difficulties.

His post goes on to list some problems, benefits and a potential improvement. Geoff Parkin shares his negative view on them.

The highlight for me from the Training mag article was

In some instances, there is not only a low correlation between Level I and subsequent levels of evaluation, but a negative one.

The emphasis on level 1 evaluation – why

Most interestingly, the article then asks the question, “why do so many training organisations, including universities, continue to rely on level 1 smile sheets?”

The answer it provides is that they are too scared to do them in case of what they find. It’s the ostrich approach of sticking the head in the sand.

What else should be done?

This google book search result offers some background on “level 1” and talks about the other 3 levels. Another resource provides some insights and points to other resources. I’m sure if I dug further there would be a lot more information about alternatives.

Simply spreading the above findings amongst the folk at universities who rely and respond to findings of level 1 smile sheets might be a good start. Probably necessary to start moving beyond the status quo.