Moving the indicators Moodle block to a factory class

The following reports on some work on the indicators block to move it towards using some object-orientation and the factory design pattern.

Why?

One of the requirements we’ve talked about for the indicators block was the ability to show many different “indicators” (visualisations of something important about learning and teaching within a Moodle course). The idea is that the individual user could either scroll through the different visualisations or they could configure it to show a subset that they are interested in. Some examples of indicators that are already out there which might be included are:

  • An effort meter;
    This is what the block currently shows. It’s closely related to the Moodle meter idea of Lewis Carr. To the extent that both the meter and the current indicator both use Google’s chart API for the graphics. Though the meter appears to place users into four groups. The indicators block currently uses a straight numeric scale.
  • Traffic lights;
    Purdue University has used traffic lights to represent students’ standing in a class. This is a little bit like Michael de Raadt’s Moodle progress bar block, at least in terms of helping students visualise their progress, this time within a course.
  • Network visualisations of connections;
    The SNAPP project is the main example of this I know. Visualise the network of interactions between participants in a discussion forum.
  • Waterfall visualisation;
    This comes from work done by David Wiley and his students. It’s connected to the traffic lights/progress bar idea, but is focused more on showing student progress to the teacher. Allowing the teacher to see which students are struggling or not.

The last one is somewhat connected to a visualisation that Col has been working on. He describes the rationale and the approach in this blog post. More on that later.

Are there any other ideas for visualisations?

What?

Given the aim is to include multiple visualisations and we do want people to be able to add their own to the block, we need to have a clean way of separating the code for the different indicators. This is what the factory design pattern does.

The idea of the factory pattern is that you want the code for the different indicators to be separated out. These indicators are different, but they all do essentially the same task (look at the LMS data and generate some visualisation). The decision about which of the indicators you want to show to the user is a fairly complex decision. In terms of the indicators this decision will eventually need to consider:

  • What type of user is this?
    Students and staff will be able to see different indicators.
  • How has the user configured the indicators block?
  • Based on the above, which indicator should we show?

The initial version of the block had the decision about which indicator to show and the code for both indicators (staff and student) combined into the one function. A bit messy with two. Have 6 or 7 and it would’ve been a nightmare.

With the factory design pattern, the guts of the block’s main function looks like this.

// get details about the context
$context = get_context_instance(CONTEXT_COURSE,
                                $SESSION->cal_course_referer);

// create the correct indicator
$indicator = IndicatorFactory::create($context);
// use the indicator to generate the HTML to put in the block
$this->content->text = $indicator->generateText();

There’s a class called IndicatorFactory which when given the current context decides which indicator should be used. The factory then constructs the right indicator and returns it.

All indicators simply generate the “text” that is placed in the block. So we call the indicators generateText function and we’re done.

All the IndicatorFactory class does at the moment is look at the type of user. If the user is a teacher, then it creates an object of the class staffActivity. If it is a student, it creates an object of class studentActivity

if ( has_capability( 'moodle/legacy:teacher', $context ) ||
     has_capability( 'moodle/legacy:editingteacher', $context ) ) {
     require_once(
                    $CFG->dirroot.'/blocks/indicators/staff/staffActivity.php');
     return new staffActivity;
} else if ( has_capability( 'moodle/legacy:student', $context ) ) {
      require_once(
                    $CFG->dirroot.'/blocks/indicators/student/studentActivity.php');
      return new studentActivity;
}

Both the “activity” classes contain the SQL statements, a bit of maths and the call to the Google charts API that was necessary to generate the particular visualisation. Both of the “activity” classes extend the abstract class Indicator. The idea is that for each of the above indicators we implement, each one will have its own class that extends the Indicator class.

What’s next

This has laid the foundation for having multiple indicators for each user category. The next step might include:

  • How will users move between different indicators?
    One idea is that there are left and right arrows at the bottom of the indicators that allow the user to scroll through the available indicators. Have other Moodle blocks done this? How? Does this mean a refresh of the entire page or do we do some HTML trickery?

    Alternatively, do we randomly (or sequentially) show the indicators everytime the page is refreshed? Do we build some smarts into the Indicators block so that at certain times or based on certain events it shows specific indicators first? For example, in the day or two leading up to an assignment due date it might show the percentage of students who have submitted and those that haven’t.

  • How will users be able to configure which indicators they are interested in?
    Eventually (not right now) we may want to allow the users to configure which ones they are interested in.
  • Can we improve the current indicators?
    For example, some of the SQL for the student activity indicator users “roleid=5” to indicate a student. I think this is deprecated.
  • Do we need to and how will we “cache” the data required for the indicators?
    At the moment, both indicators query the database every time the block is shown. In a large installation this could lead to some performance problems. Eventually, we will need to look at an approach to “caching” to reduce the performance hit.
  • Do we need to move to a model/view pattern for the indicators?
    Are we going to want the one set of data around an indicator to be visualised in many different ways? SNAPP already does this with the different types of network visualisation it supports. If so, we may wish to split the indicators into a model (calculate the data) and view (visualise it) objects.
  • How do we support bigger indicators?
    SNAPP visualisations are probably not going to fit within a block. Especially if we’re talking about building on SNAPP by enabling visualisations of discussion forums across a range of courses, not just the current one. How do support indicators that need quite large areas? A popup? A new page? What’s the Moodle way? What’s the cool way?
  • Are there other visualisation tools?
    The Google chart API looks like a good, low impact way of doing visualisations. But it might not provide everything we need. Are there other alternatives?

Is there more to communities of practice?

Markus and I have been talking about behaviour change and lots more for a bit. He’s about to start a new job has been speculating about what he might do and how it connects with what we’ve been talking about. This post is an attempt to make explicit my initial gut reaction to the idea that

distributive leadership may not help, but communities of practice might

It’s main outcomes will likely be to encourage me to read and understand more about both conceptions.

A cynical, ill-informed view of communities of practice

My knowledge of communities practice is limited to my observations of local attempts to implement then and having purchased and skimmed one of Wegner’s books on the topic. Based on the vastly ill-informed perspective my current opinion is that communities of practice suck. I’ve previously outlined two of the reasons I don’t think they are working to improve L&T within universities. On reflection, my problems with CoP are the following, (most based on observations of local context, hence generalisability is somewhat limited)

  • If voluntary, they attract the folk who least need them.
    A voluntary CoP around assessment within a university is likely to attract people who are intrinsically interested in assessment. People who aren’t interested won’t join. I suggest that it’s the folk who won’t join that are more likely to be the folk who need to join.
  • If compulsory, they encourage compliance or corruption.
    Force an academic (knowledge worker? Dentist?) to do something they don’t want to do and they aim to be seen to comply, but won’t really participate. There’s a chance participation might lead to a change of mind, but that’s an if.
  • They are likely to suffer all the common collaboration problems.
    Get any group of people together and you are likely to find problems such as the herd mentality, echo chamber etc. Not to mention the problem that for some CoPs are becoming seen as yet another tool of management.
  • They find it hard to change the system.
    Most of the CoPs I have seen set up are not directly connected with the formal leadership of the institution or the formal processes/systems of the institution. Consequently, the discussions do not end up changing the institution’s systems or processes. The focus becomes sharing and talking, not making changes.

Obviously most of these can be addressed in setting up the CoP. But what I’m interested in is getting the whole system/context setup in a way that encourages on-going reflection and change. And that’s more than just people talking.

Distributive leadership as different/better

At the moment, the most interesting aspect of distributive leadership for me is its foundation on distributed cognition. Due to this foundation, leadership within distributive leadership is not a function of the formal leadership hierarchy within an organisation. It is also more than distributing leadership responsibility into the people within an organisation. It’s about distributing leadership responsibility into the people and the systems and the processes of the organisation.

This is important, if you limit leadership to just the formal leaders or the people you are missing the point. I also think this is the bit most people don’t get and one I’m still figuring out how to explain. Is this a point of departure from CoP, inclusion of systems/processes as well as people into consideration?

How about we consider what leadership is? What does it entail? Some previous reading and thinking identified two functions that are thought to be indispensable to leadership (Sourthwell and Morgan, 2009; Leithwood an Levin, 2005):

  • Direction-setting; and
    What is the direction or purpose considered valuable/appropriate?
  • Influence.
    Encouraging folk to move towards the direction, to achieve the purpose.

Let’s use Markus’ problem as the example. Thinking of this from a “distributive leadership” perspective (or at least what I currently think might be such a thing) might suggest the following perspective.

The direction or purpose is to reduce the sense of isolation of the dentists within Scottish prisons. In itself, this is a pretty big assumption. What’s the evidence that the dentists feel this and that it’s a problem? Is the lack of funding priority indicative that this is not important?

As I’m ignorant of this evidence, but could see that it could be important, Let’s assume that it is important. Why aren’t the dentists already moving in this direction? Why aren’t they talking more?

My interest/argument would be in finding out what is about the systems and processes within which they work that prevent them from reducing isolation and identifying the interesting ways you could reduce those barriers. The question is, if getting rid of isolation is an important direction, why aren’t the systems and processes used by the dentists in their job helping them get there?

For me this isn’t about setting up a community of practice, as that doesn’t immediately fundamentally address the barriers in the system that is creating the isolation. Participating in a CoP might creates a reason to overcome the isolation, but it doesn’t remove the barriers.

Some examples

The following is an attempt to provide some more concrete examples of what I mean by focusing on the prison dentists case. Rather than seriously suggests specific solutions, the idea here is to give a taste for how small, diverse and widespread the changes might be.

Markus describes a system that includes under-staffing, logistic difficulties, difficulties sourcing equipment and materials, an under-funded system. From that I get a picture of dentists that are overworked by a system that is continually showing that it doesn’t really value what they do.

Change? the prison hierarchy do something that (continually) illustrates to the dentists that there services are valued (e.g. all prison dentists be given the choice car park place).

Would folk in such a context have any sense of ownership or control over the work they do?

Change? Get prison hierarchy buy-in to engage in a social network stimulation type project involving the dentists and others within the system. Perhaps focused on oral health?

Would folk in such a context have the time, motivation and technology available to talk to each other?

Change? Identify some dentistry good practice that requires collaboration and is broadly accepted. Give them all iPads (sexy technology that they can collaborate with, but also do other things) as part of a project that seeks to embed the select practice into their everday live

This is definitely echoing the alignment project (internal echo chamber?).

Another thought, do they need to communicate with other prison dentists, what about other dentists, other prison workers?

Some tweaks to the indicators block

Yesterday’s post introduced Col’s initial work on the indicators block. This post reports on some minor tweaks I’ve been doing this afternoon, trying to find escape in something concrete.

Setting the title

As reported in the last post, the block ended up having a title [[Indicators]]. This was because get_string was being used to set the title but the necessary language file (from which to source the string) was not created.

First fix is to just hard code the title.

$this->title = "Indicators"; //get_string('Indicators','block_indicators');

That works. But the proper solution would be to figure out where the “lang” file should go for a block. According to this, it should be lang/en/block_BLOCKNAME.php. Small problem, that should be lang/en_utf8, not lang/en (as per here).

$this->title = get_string("indicators","block_indicators");

Will commit those changes.

How to distinguish user roles?

The block was using roleid=5 as a way to identify students. I believe this is a deprecated approach. So need to find a better way. In my wonderings, I came across an approach that users the has_capability function along with some “legacy” capabilities for student, staff, guest and admin. The following is an early example from the block

if ( has_capability( 'moodle/legacy:teacher', $context )) {
    print "This is a teacher<br />";
} else if ( has_capability( 'moodle/legacy:student', $context )) {
    print "This is a student<br />";
    $canview=0;
}

What’s next

At least in my head, the plan is to enable different groups of users to see different sets of “indicators”. Where an “indicator” is a single graphic. This means that we need an good way to:

  • distinguish between different users;
  • call different code to generate the indicators for the different users;
  • distinguish which indicator a user wishes to see;
  • call different code based on the indicator.

A nice structure to do that, might be next on the list. If I was in Perl, I’d be doing this with a factory class. Should we go OO in PHP?

Some other tasks:

  • I’m getting errors when running the block as a teacher.
    Table 'moodle.m_course' doesn't exist
    
    select (count(*)/count(distinct(userid))) from mdl_log where course='4' and userid='3' and action in ('add discussion','add post','update post') and course in (select id from m_course where idnumber like '%2010') and userid in ( select userid from m_role_assignments where roleid !='5' and contextid in (select id from m_context where contextlevel='50'))
    line 686 of lib/dmllib.php: call to debugging()
    line 379 of lib/dmllib.php: call to get_recordset_sql()
    line 71 of blocks/indicators/block_indicators.php: call to count_records_sql()
    line 317 of blocks/moodleblock.class.php: call to block_indicators->get_content()
    line 341 of blocks/moodleblock.class.php: call to block_base->is_empty()
    line 338 of lib/blocklib.php: call to block_base->_print_block()
    line 276 of course/format/weeks/format.php: call to blocks_print_group()
    line 229 of course/view.php: call to require()
    
    Warning: Division by zero in /Applications/XAMPP/xamppfiles/htdocs/moodle/blocks/indicators/block_indicators.php on line 80
    
    Warning: Division by zero in /Applications/XAMPP/xamppfiles/htdocs/moodle/blocks/indicators/block_indicators.php on line 86
    
  • What should an admin user see when they view the block?
  • Check the performance of the existing SQL code and think about how/what we might need to do to significantly reduce that.

Qualms about the alignment project

Yesterday, I posted a draft of an application for what is currently being called, the alignment project. Stephen Downes has commented on the alignment project in is OLDaily. Stephen’s comments are

This is totally not my approach, but a careful and detailed articulation of an alternative. It should be considered…..I would criticize the duality the approach presupposes, between ‘quality’ (alignment) and the disorganized ‘lone wolf’ approach to teaching.

Stephen is not alone in having qualms about the project. I have some as well. This is an attempt to make those qualms explicit and see if I can develop an argument/perspective that addresses at least some of them. After a week or two of developing the application draft, I’m a bit too close to the idea. I need to be more critical so that the idea can be improved.

I welcome suggestions and arguments, especially those targeting weaknesses or mistakes in the application

Aside: people wonder why I post to this blog. The prime reason is exactly the type of comment Stephen has made and what it encourages and enables me to think about. The duality Stephen mentions is important and not something I would have thought to closely about without his spark.

Summary

My response to Stephen’s criticism is that I recognise that this duality is a big problem. It’s also the most likely outcome of the project, i.e. those lone-wolves who aren’t seen as being “aligned” are also seen as “poor quality”. My intent, however, is to use alignment as an idea acceptable to higher education that can be used to modify a broken system to increase the level of reflection and discussion around L&T that occurs as part of the everyday practice of teaching academics. That “alignment” is a means, not the end.

I also think it unlikely that this is what will happen.

Qualms about constructive alignment

The alignment project as described draws heavily on John Biggs work on constructive alignment. I’ve always had to qualms about constructive alignment:

  • Assumption of plan-driven or teleological design.
    Constructive alignments is a teleological design process. It assumes you can identify the outcomes at the start and use that as a basis to identify/design the actions necessary to get students there. A number of aspects of learning, even in its limited form of university-based learning, make me doubt whether or not this is really all that possible. The next point picks up on human agency, but other aspects include the inherent diversity in learners backgrounds, capabilities and aims. Even in higher education the talk is of the increasing diversity of students. Given that diversity, how do you claim to identify a set of outcomes that is suitable for all, let alone develop activities and assessments that are suitable for all?

    I have a long standing preference for ateleological design processes and have a (potentially forlorn) hope that the alignment project might be more ateleological than teleological.

  • Assumption that you can “force” students to learn.
    Here’s a quote from Biggs (2001) that illustrates the assumption that troubles me

    In aligned teaching, where all components support each other, students are
    “trapped” into engaging in the appropriate learning activities, or as Cowan (1998) puts it, teaching is “the purposeful creation of situations from which motivated learners should not be able to escape without learning or developing” (p. 112). A lack of alignment somewhere in the system allows students to escape with inadequate learning.

    At one level I’m worried about this perspective because of what words such as “trapped” and “inescapable” can mean, what it says about the people in charge of such a system.

    My more pragmatic problem with this perspective is that I don’t think it can work. People always have some level of agency. Many university students are highly pragmatic, they will use their agency to subvert the system to achieve their ends with means they find acceptable. I’m not convinced that even the best constructively aligned course can escape the effects of compliance and task corruption.

Qualms about the reflective institution

The alignment project is essentially aimed at implementing something that approaches Biggs (2001) idea of a reflective institution. The application gives a summary of the stages involved in achieving Bigg’s goal. I’ve actually written about the idea previously back in February last year I wrote

However, the detail of his suggested solution is, I think, hideously unworkable to such an extent as likely to have a negative impact on the quality of teaching if any institution of a decent size tried to implement it. As Biggs (2001) says, but about a slightly different aspect, “the practical problems are enormous”.

I’ve been involved with the underbelly of teaching and learning at universities to have a significant amount of doubt about whether the reality of learning and teaching matches this representation to the external world. I’ve seen institutions struggle with far simpler tasks than the above and individual academics and managers “game the system” to be seen to comply while not really fulfilling (or even understanding) the requirements.

The project’s assumptions

In my head, the project is based on the following assumptions:

  • For most academics, the majority of teaching is copying a previous course, making some minor modifications and teaching it.
  • Preparation for teaching is generally driven by administrative deadlines (the bookshop needs to order textbooks on date X, teaching starts on Y etc) and systems (you use system X to order the textbook, the LMS to create your course site).
  • Most, if not all, of these systems do not encourage or enable the academic to think about the concepts of learning underpinning these decisions. They just have to choose the textbook, make sure the assignment is different from last year and ensure that there are no egregiously out of date references in the lectures.
  • Consequently, most academics (upwards of 50%) just do what they did last time, teach the way they were taught. (There are exceptions, but they are the minority).
  • If you add into these systems some minor tweaks that encourage and enable academics to reflect on the conceptions of learning within the course, and provide some appropriate support, then you might encourage the majority to start reflecting and eventually improving their teaching.

To some extent the project is based on the nature of the teaching context at the participating institutions. For example, my current institution has specified that every course will have a Moodle course site.

Qualms about the project and some possible responses

The following are the qualms that I can think about the project. Can you suggest more? For each of these, I’ve tried to describe what I think is a response.

It will die within a week

Qualm: The alignment project is still in pre-application days. There are on-going discussions with various institutional leaders about whether or not they think that this is an idea that they can support. There’s always a chance that by this time next week (or not long after) the project will be dead due to lack of support.

Response? If that happens, I’m hoping we can continue with the project on a smaller scale. Perhaps with just a single program or two at my current institution to explore some of the ideas and their impacts. At the very least, I’m interested in how/if some form of curriculum mapping can be put into Moodle.

Teleological design

Qualm: As I’ve stated above, I don’t think teleological design works. In particular, I don’t think this is the way most academics approach the design of the teaching. The draft alignment project application actually cites literature that shows academics are mostly making minor changes to existing courses. This post expands on this literature. When I’ve seen constructive alignment in action, it’s typically been as part of a large redesign of a course. For all sorts of reasons I think this is a failing.

Response? The methodology expressed in the project proposal recognises this and seeks to introduce the question of alignment in a way which fits with the focus academics have on minor modifications to existing teaching. The hope is that by making alignment a visible part of the tools and support around making minor modifications, then staff may start thinking about alignment and be able to make minor modifications that improve alignment. The aim is not massive redesign to ensure alignment. It’s about minor changes that improve alignment.

And, if I’m honest, the aim for me is not really to get them implementing constructive alignment. It’s just to create an environment that encourages and enables academics to reflect on their teaching and how they are doing it.

Using it as a stick

Qualm: This is one of my biggest fears. Theoretically the project will result in the alignment (or lack thereof) to be readily visible to all folk associated with a course/program. This is going to include people in formal leadership positions. This could very easily lead to this being used as a stick to beat the “bad” teachers. In Bigg’s (2001) words, a focus on the teacher, rather than the teaching.

Response? The only defense I can see against this is ensuring that the folk in those leadership positions are intelligent folk who can see the problems with this. Or, perhaps at the least, ensure that their actions are visible enough so other more enlightened folk can mitigate their effects. In some situations, I’m not sure we’d be able to convince teaching academics of the potential success of this approach.

In the project, I think this can be addressed by having people on the project reference group that are broadly recognised as being experts in this field and having them interact closely with the members of the institutional steering committees (containing formal leaders). Hopefully during the project they can learn the lessons which inform latter practice.

Corruption

Qualm: In terms of a likely outcome, I can see – given my comments about human agency above – that some/most academics would employ task corruption once the alignment project was in place. Task corruption is where an group or individual, consciously or unconsciously, adopts an approach to a task that either avoids or destroys the task. White (2006) talks about two approaches

  • amputation – where parts of the task are no longer performed; and
  • simulation – the emphasis is on being seen to have done the task, not actually to have done it.

Response? In the end, I don’t think there’s anything that can prevent this from happening. All you can do is provide an environment in which the practice becomes valued. That’s what I think this project is about, making alignment part of the culture, the way things are done. This will never entirely successful and is likely not to succeed. However, the responses currently in the project include:

  • making alignment and the responses to it visible;
    Much current corruption occurs because university teaching is a primarily solitary act. There are questions around whether this is a good thing, about forcing one’s views on others. But then academia is supposed to be about peer review.
  • enable and encourage;
    The focus is on creating an environment that helps academics engage in this. If they aren’t engaging then the environment needs to be tweaked, hence the focus on action research. This needs to be on-going.

Technological gravity

Qualm: Related to the above is the conception of technological gravity that McDonald and Gibbons (2009) define and which I’ve posted about (and linked to edupunk). This idea is based on the idea of three major assumptions around learning and teaching:

  1. Technology I – different technologies automatically lead people to develop quality instruction.
    i.e. Moodle is an LMS designed with social-constructivist principles and it’s open source. If the institution adopts Moodle then the quality of instruction will improve.
  2. Technolgy II – different techniques/methods lead people to develop quality instruction.
    i.e. if all our courses are designed using constructive alignment, then student learning outcomes will improve.
  3. Technology III – characteristics of a local situation are used to identify the technologies and methods that will have a practical, positive influence in solving a defined problem/improve learning.
    i.e. this is the bit I think has some connections with edupunk – perhaps what Stephen describes as the lone-wolf approach above.

Technological gravity is defined as the force that seems to suck people and institutions away from Technology III and towards Technology I and II. This project is just as likely to suffer from technological gravity as anything else. McDonald and Gibbons (2009) identify three reasons for technological gravity:

  1. distracted focus;
    i.e. the institution has to get ready for a quality audit and needs to focus on that.
  2. status quo adherence;
    i.e. the changes introduced are re-interpreted (or mis-interpreted) and slightly adapted to fit with current practice. “Of course, my 3 hour lecture on the basics of theory are help the students develop critical evaluation skills.”
  3. over-simplification.
    i.e. doing X is too hard, we need to make it simpler for folk to do routinely. By making it simpler, something is lost.

Responses? In terms of focus, the aim of the project is to embed this in the institutional systems. It should just happen. Buy-in of leadership, building it into institutional systems (the LMS) etc are all steps being taken. This will be hard.

With status quo adherence, if the project works then this should hopefully be the type of problem with which the quality enhancement process focuses on. That process is focused on encouraging people to reflect, in part visibly.

In terms of over simplification, this is perhaps where the quality feasibility stage comes in. Mm, weak

The quality and lone wolf duality

Qualm: Stephen’s qualms include

the duality the approach presupposes, between ‘quality’ (alignment) and the disorganized ‘lone wolf’ approach to teaching.

I think this is based on the idea that any disorganised “lone wolf” approach to teaching is by definition not aligned and consequently can’t be thought of quality teaching.

Response: I don’t think the project can respond to this. The first stage of Bigg’s (2001) reflective institution (and consequently of this project) is to make the quality model clear. The model in this case is some sense of alignment. Being aligned is by definition quality.

If you strongly believe in constructive alignment, then this is probably not a problem. However, as I outlined above, I have qualms about constructive alignment. If you asked most people around here whether I am a lone-wolf or a quality-focused/aligned kind, most would answer lone-wolf. Given these how can I justify this project and my involvement with it?

Having thought about this, my current response has two themes:

  • the alternatives are even worse;
    Anything has to be better than what I see as increasingly common practice within Australian higher education. I’ll pick up on this more below.
  • I’m not as dogmatic as Biggs.
    In the following, I’ll argue that I’m my perspective is not as black and white as the duality Stephen has identified. Instead, I’m taking a more gray perspective. Perhaps seeing the quality model as not a dichotomy or end-point, but encouraging a dialectic.

My impression of Biggs (solely from his writings) and some of the constructive alignment practitioners I’ve met (perhaps they influence my perspective of Biggs) are of a very dogmatic perspective. Alignment provides the answer, and the answer is good. If you are not a follower of the answer, you are a heathen. If you follow alignment, you need to re-design your course so that it is 100% aligned – as approved by the constructive alignment church. There’s almost a touch of Technology II about constructive alignment.

To me, learning and teaching is much more complex. I can see how a perfectly aligned course could have horrendously horrible outcomes, depending on the context. I can also see how an apparently mis-aligned course can generate good quality outcomes. For me, the aim of the alignment project is not to achieve perfectly aligned courses and hence quality student learning outcomes. The aim of project is to modify everyday teaching practice so that it encourages and enables academics to start asking questions, rather than simply following administrative processes. Am I trying to make use of a Technology III perspective of alignment?

So, why use alignment at all? Because the system is broken – I pick this up in the next section.

So why do it

All of the above has got me thinking about why I’m pushing this project. Here are my answers.

I have a job

At the simplest and most pragmatic level, I have a job. The institution pays me to do certain things that it deems important and ALTC grants are pretty high on the list of importance. If I want to continue to have a job, I need to demonstrate I’m fulfilling their goals.

That’s not sufficient though. I also think I could enjoy the job (eventually) and generate some benefit broader than my personal employment.

The system is broken

As it happens earlier this week I was listening to this discussion Stephen had in Argentina. One of the topics covered was that, in his opinion, the current educational system in which his “audience” were working within, is broken. One point being that it is very difficult, if not impossible, to use a lot of his approaches within such a broken system.

In terms of the higher education sector in Australia, I agree with Stephen, the system is broken. Even worse, for some time I have been dismayed at the increasingly prevalent broken approaches that are being adopted in the quest for improving the quality of L&T within that broken system. It’s a common theme on this blog. What is currently being done within Australian universities to improve L&T will, at best, offer slight improvements for the folk who were already improving, or at worst, significantly decrease the overall quality of L&T (while at the same time showing “evidence” of improvement).

At this stage I come back to some thinking about inside-out versus outside-in that was sparked by questions from Leigh Blackall. In my job I think I have to come up with approaches that can change the system from the inside-out. It’s an aim likely to fail.

To do this, you have to have some connection with what is being done within the system. Alignment is broadly and generally unquestioningly accepted within higher education, especially amongst the leadership. This project will be attractive because of this acceptance. Most have accepted the value of alignment. More importantly, in my experience most people can, from a common sense perspective (“Common sense is the collection of prejudices acquired by age eighteen”), accept the idea of alignment as a good thing. It’s also a fairly simple thing to understand (though difficult to implement).

What this means is:

  1. Management can see the rationale for this and how it fits with the external demands they are having to deal with.
  2. Teaching academics can (hopefully) see the initial sense of the idea, at least enough to start talking about it.

i.e. this helps get the idea accepted and helps the project introduce into everyday practice some discussion about L&T that moves beyond administrative tasks. It helps introduce a change that might move the system in the right direction (but won’t fix it).

In summary, alignment is a means to an end. It’s not what is fundamental about this project. What is fundamental is encouraging a bit more reflection around L&T into everyday practice. I don’t really care how aligned the courses are, as long as academics are working in an environment that helps them reflect on their L&T and do something about it.

Of course, translating that view into reality and avoiding alignment being seen as an ends, is another story all together.

References

Biggs, J. (2001). The Reflective Institution: Assuring and Enhancing the Quality of Teaching and Learning. Higher Education, 41(3), 221-238.

McDonald, J., & Gibbons, A. (2009). Technology I, II, and III: criteria for understanding and improving the practice of instructional technology Educational Technology Research and Development, 57(3), 377-392.

White, N. (2006). Tertiary education in the Noughties: the student perspective. Higher Education Research & Development, 25(3), 231-246.

Getting started with Col’s indicators block

Col has been playing around with some ideas for a Moodle indicators block. This is a record of my first attempt to install and play with the block. Might also do a bit of reflection and setting up of processes etc so we can go further with this.

The long term goal is to promote the Indicators project, help some folk and do some research.

Warning: Much of the following is intended only for the indicators project team. At this stage, there’s probably not a lot of value in anyone outside the project trying to use the block. It’s very early days.

Installing the block

Installing the single PHP file provided by Col in the right place in my local Moodle install, setting permissions, visiting the “admin” page for my Moodle install and she’s all right to go. Go to a dummy course, login as a staff member, add the block and it’s all working. The block currently shows some idea of effort on the part of students, so logged in as a staff member, I don’t see much. Login as a dummy student and this is what I see. (Click on it to see a bigger version)

Indicators block version 0

It seems to work, though with a few errors. The dummy student I’m using hasn’t done a lot and the arrow indicates that. The errors include:

  • The PHP error re: undefined variable.
  • The [[Indicators]] as the label.
  • The quite large amount of screen space being taken up by the right hand block column – only since the indicators block was added.
  • The white background for the graph, rather than transparent.

The aim is to make this open source and let anyone work on it – or at least anyone in the indicators project as a first step. This means we need to get this under version control.

New code – effort tracking during early stages

Col’s just sent some new code, installed it and refreshed the page for the dummy student. I get the following

Next step in indicators block

The background colour has been improved. However, the interesting observation is that the one page reload has catapulted this student from a fairly low effort level, to a fairly high effort level.

My first guess, without even having looked at the code is that this is because this is a dummy course, there are no real students and I only use it occasionally for testing. This means very low levels of usage by “students”. At these levels, depending on the maths used, a single extra page refresh can make a huge difference.

This is something the block should recognise and address, some solutions might include:

  • Having a “too low to show” option, so that effort isn’t tracked in a state of low usage.
  • Or showing that overall usage is low and liable to wild swings. Perhaps a visible “confidence” level that indicates how confident the block is that it is showing you something meaningful.

Putting the block under git

If we’re going to work collaboratively on this, and allow other people to use it, we need some sort of support for version control and a range of other features. I’ve been using git and github for BIM, so I think we should use those for the block. I’m still a newbie at this, but I’m slightly ahead of the other guys in the indicators project. So the following shares what I did to get this up and going in the hope that it is useful for them and that they (and others) can pick up any errors I made.

Getting started

I’ve only done this once before, a month or so ago, and can’t remember anything. So, I’m starting with the github help.

I’ve already set up my laptop to use github which from memory involved: creating a github account, setting up some environment stuff and generating some ssh keys. Just follow the guides in the right hand menu on help.github.com. Just found the learn.github site.

The process

Here’s what I did

  • initialised a new git repo for the block;
    bash$ cd blocks/indicators
    bash$ git init
    Initialized empty Git repository in /Applications/XAMPP/xamppfiles/htdocs/moodle/blocks/indicators/.git/
    

    This is the empty git repository

  • add and commit the file
    bash$ git add block_indicators.php
    bash$ git commit -m 'initial commit'
    [master (root-commit) c1a7051] initial commit
     1 files changed, 103 insertions(+), 0 deletions(-)
     create mode 100644 block_indicators.php
    
  • Quick double check
    bash$ git log
    commit c1a70517f09d2f86de53e9e1c6a056d864e7622d
    Author: David Jones <davidthomjones@gmail.com>
    Date:   Thu May 13 10:17:50 2010 +1000
    
        initial commit
    
  • Add a new repository on github
    Actually, when you create a new repository, github presents you with the full set of instructions. (Point I didn’t remember, is to make the name of the project match the folder name.) Part of those instructions include what I’ve already done. Here’s the rest.
    bash$ git remote add origin git@github.com:djplaner/indicators.git
    bash$ git push origin master
    Counting objects: 3, done.
    Delta compression using up to 2 threads.
    Compressing objects: 100% (2/2), done.
    Writing objects: 100% (3/3), 1.63 KiB, done.
    Total 3 (delta 0), reused 0 (delta 0)
    To git@github.com:djplaner/indicators.git
     * [new branch]      master -> master
    
  • github like to have a README file, so let’s add one.
    bash$ vi README
    bash$ git add README
    bash$ git commit -m "Added readme"
    [master a08e84a] Added readme
     1 files changed, 5 insertions(+), 0 deletions(-)
     create mode 100644 README
    bash$ git push origin master
    Counting objects: 4, done.
    Delta compression using up to 2 threads.
    Compressing objects: 100% (3/3), done.
    Writing objects: 100% (3/3), 419 bytes, done.
    Total 3 (delta 0), reused 0 (delta 0)
    To git@github.com:djplaner/indicators.git
       c1a7051..a08e84a  master -> master
    
  • I have a second local Moodle install, here’s how I get the code from git into the appropriate place for that second Moodle install
    bash$ cd cm_moodle/blocks
    bash$ git clone git://github.com/djplaner/indicators.git
    Initialized empty Git repository in /Applications/XAMPP/xamppfiles/htdocs/cm_moodle/blocks/indicators/.git/
    remote: Counting objects: 6, done.
    remote: Compressing objects: 100% (5/5), done.
    remote: Total 6 (delta 0), reused 0 (delta 0)
    Receiving objects: 100% (6/6), done.
    bash$ cd indicators
    bash$ ls
    README			block_indicators.php
    

    At this stage, I think you need to create a local git repository to work with this locally.

  • Let’s fix that undefined variable problem, caused by a variable not being declared/initialised before an “if”, and then update github?
    bash$ vi block_indicators.php
    bash$ git commit -m "fixed undefined canview"
    bash$ git push --mirror git@github.com:djplaner/indicators.git
    Counting objects: 5, done.
    Delta compression using up to 2 threads.
    Compressing objects: 100% (3/3), done.
    Writing objects: 100% (3/3), 341 bytes, done.
    Total 3 (delta 1), reused 0 (delta 0)
    To git@github.com:djplaner/indicators.git
       a08e84a..caf4b54  master -> master
     * [new branch]      origin/HEAD -> origin/HEAD
     * [new branch]      origin/master -> origin/master
    
  • Okay, now to get that change showing up in my original indicators block
    bash$ git pull origin master
    From github.com:djplaner/indicators
     * branch            master     -> FETCH_HEAD
    Updating a08e84a..caf4b54
    Fast forward
     block_indicators.php |    1 +
     1 files changed, 1 insertions(+), 0 deletions(-)
    

    A similar thing on my second Moodle install (which I think others will be doing), would’ve been just “git pull”.

Reflections and what’s next

That all seems to work. I do realise that I’ve probably just identified some of the basic commands without really groking the full capabilities of git and github. Perhaps I’ll learn.

Tasks to do later on include:

  • Identify how to add the other members of the indicators project to the github repository (so they have write access).
  • Take a look at Col’s code and start thinking about what we can do to add features.

More thinking about the alignment project

The following is the latest, and first close to (but not there) complete, draft of the proposal explaining the alignment project. While informed by good discussions with a range of folk, the following is still a bit limited. Should be improved over the next couple of weeks.

Even if the application doesn’t get off the ground it has helped me make connections bit a range of different bodies of work (complex adaptive systems, connectivism, distributive leadership and distributed cognition). Some of which I’ve been aware of and some I’ve ignored. It has helped develop my interest in thinking about how to combine some of the principles underpinning these bodies of work with behaviour change, hopefully to do some interesting things in the future.

As always, any comments/suggestions are more than welcome.

Executive summary

The aim of this project is to build distributive leadership capacity into institutional systems and processes to encourage and enable alignment and quality enhancement. It aims to make consideration of alignment a regular, transparent, supported and integrated part of common teaching practice, supported by effective systems and processes. The project aims to fulfil the suggestion by Biggs (1996), that attempts to enhance teaching should seek to address the system as a whole, rather than simply adding “good” components such as new curriculum or methods. It seeks to build distributive leadership to empower academics to actively engage in alignment and move towards achieving what Biggs (2001) calls ‘the reflective institution’.

For most teaching academics, the consideration of alignment in their courses and programs is not a part of everyday teaching practice. Consideration of alignment is typically limited to events such as significant re-design of courses and programs of visits from accreditation or quality assurance organizations. The dominant teaching experience for academics is teaching an existing course, generally one the academic has taught previously. In such a setting, academics spend most of their time fine tuning a course or making minor modifications to material or content (Stark, 2000). Given this focus, it does not appear surprising when Green et al (2009) report that “many academic staff continue to employ inappropriate, teacher-centered, content focused strategies”. If the systems and processes of university teaching and learning practice do not encourage and enable everyday consideration of alignment, is it surprising that many academics don’t consider alignment?

Instructional (Cohen, 1987), curriculum (Anderson, 2002) and constructive (Biggs, 1996) alignment are all built on a similar foundation: the recognition that student learning outcomes are significantly higher when there are strong links between those learning outcomes, assessment tasks, and instructional activities and materials. Cohen (1987) argues that limitations in learning are not mainly caused by ineffective teaching, but are instead mostly the result of a misalignment between what teachers teach, what they intend to teach, and what they assess as having been taught. The importance of achieving and demonstrating alignment with expected outcomes is also a central component of outcomes-based accreditation and quality assurance approaches that are increasingly widespread within higher education.

Consequently, the main tasks of this project are based on the three stages which Bigg’s (2001, p. 221) identified as encouraging institutional reflective practice. These are:

  1. Make explicit the quality model.
    Alignment should be explicit if it is to be seen as a key to quality student learning outcomes. The systems, technology, processes and support practices around learning and teaching should therefore enable and encourage alignment to be an everyday consideration. This support will enable: a) the level of alignment within a course, or group of courses, to be mapped and understood; and b) information about the alignment of a course or courses to be used in the everyday learning and teaching practice.
  2. Build in support for quality enhancement.
    An institution must also establish mechanisms that allow it to review and improve current practice, as it is not sufficient to simply make the quality model explicit (Biggs (2001, p. 223). This stage aims to help teachers to ‘teach better’ through the provision of responsive, appropriate, and contextualised support that responds to insights gained as a result of a greater focus on alignment and other factors.
  3. Institute a process for quality feasibility.
    An institution can only enhance quality if it actively identifies and removes factors that inhibit quality learning (Biggs, 2001, p. 229). This requires formal leadership, processes and hierarchies at the participating institutions to be actively involved in the removal of these inhibiting factors. For the project this involves factors identified through the quality enhancement process and also, more broadly, factors inhibiting the project’s aim of building distributive leadership capacity.

This project will help teaching academics to more regularly consider alignment through context sensitive and collegial methods by building distributive leadership capacity into the participant institutions. . This improved capacity will empower and encourage teaching academics to develop and grow their conceptions of teaching and learning and engage in ongoing improvement of teaching. This process is aided by the active removal of inhibiting factors. The combination of all these actions should lead to significant improvements in student learning outcomes.

Background and rationale

While it is common to describe leadership as a concept that eludes comprehensive definition (Southwell & Morgan, 2009), Parker (2008) suggests that some level of conceptual clarity around leadership within higher education has emerged from the ALTC leadership grants. This emerging view sees leadership in universities as inclusive and distributed, as opposed to the “deeply entrenched association of leadership with hierarchy and authority” (Parker, 2008). Lakomski (2005) argues that the growing recognition of distributed leadership within organisational theory is helping debunk the leader myth of traditional leadership theories. This project, like a number of previous ALTC Leadership projects, is based on the concept of distributed or distributive leadership.

Parrish et al (2008) define distributive leadership as the distribution of power through a collegial sharing of knowledge, of practice, and reflection within a socio-cultural context. Zepke (2007) argues that this is more than the delegation of tasks and responsibilities, and more than collaborative practice. Spillane et al (2004, p. 9) argue that, based on its foundations in distributed cognition and activity theory, distributive leadership is not limited to people, but can also be attributed to artefacts such as language, notational systems, tools and buildings. Leadership activity is distributed through an interactive web of actors, artefacts and situation (Spillane et al., 2004, p. 20). Spillane et al (2004, p. 11) define Leadership as

the identification, acquisition, allocation, co-ordination, and use of the social, material, and cultural resources necessary to establish the conditions for the possibility of teaching and learning.

Over thirty years of research (Prosser, Ramsden, Trigwell, & Martin, 2003; Ramsden, Prosser, Trigwell, & Martin, 2007) has produced abundant empirical inquiry and theory that links the quality of student learning outcomes with: (1) the approaches to learning taken by students; (2) the students’ perceptions of the learning context; and (3) the approaches to teaching practiced by teaching staff. In turn, this research confirms the findings of other leadership studies by illustrating that variation in teaching approaches is associated with perceptions of the academic environment (Ramsden et al., 2007). As Biggs (1999) argues, it is the alignment of all aspects of the system that contributes to higher quality outcomes. Conversely, misalignment within an institutional system is likely to contribute to a lowering of quality outcomes. In particular, while pedagogues may hold a higher-level view of teaching other contextual factors may prevent use of those conceptions (Leveson, 2004).

A fundamental assumption of this project is that there is a misalignment between the importance of instructional and curriculum alignment to student learning outcomes and its prevalence within the teaching and learning systems and processes of universities. This misalignment is seen as a major contributing factor to Barrie’s et al (2009) observation that despite significant espoused intentions around graduate attributes,

Australian universities have not generally been successful in deliberately and systematically refocussing the curriculum in ways that foreground the development of these attributes as opposed to the acquisition of factual disciplinary content or the accumulation of isolated and unrelated knowledge, skills and dispositions

This project aims to address this misalignment through making alignment a prevalent component of the teaching and learning systems of the participant institutions. It seeks to move consideration of alignment beyond a focus on program review or accreditation purposes, towards making consideration of alignment as a part of everyday teaching practice. To achieve this goal, the project must deal with a number of problems. The approaches this project will adopt to address these problems are described in the following.

Most teaching practice is not alignment focused

The practice of most academics does not separate planning from implementation, and rather than starting with explicit course objectives, starts with content (Lattuca & Stark, 2009). The dominant setting for academics is teaching an existing course for which they spend most of the time making minor modifications to material or content (Stark, 2000). For most staff teaching a course starts with the existing course materials such as outlines, assignments and website. The general description of these existing courses embedded in these materials may be non-specific and not systematically explain the content of teaching and the outcome of learning (Levander & Mikkola, 2009). This make it difficult to understand just how aligned a course is both within itself and with other courses in the program. This problem is compounded by the increasing casualisation of academic staff that leads to a context where there is high staff turnover, lack of ownership and lack of institutional support (Green et al., 2009).

The project will embed consideration of alignment into everyday practice by modifying the main institutional learning and teaching information system used by teachers and students, the LMS. The intent is to map alignment of a subset of existing courses within the LMS through a collaborative process between teaching academics and support staff. As described above, standard practice for most academics is to copy the course site from the last offering and make minor modifications to material and activities. The LMS modifications will enable and encourage teaching academics to modify the alignment mapping of their course as they make these minor modifications. Importantly, the project also aims to identify and experiment with additional LMS modifications that enable teaching staff and students to make use of the alignment mapping within the LMS.

Teaching is an isolated, solitary practice

The norms of the higher education community encourage autonomy and independence (Uchiyama & Radin, 2009). Lowe and Marshall (2004) describe academic life as often isolated and that even when this isolation is overcome, few academics will discuss course design and teaching practices with peers. The planning and implementation of teaching has largely been a private issue creating the possibility that the actual delivered teaching represents the teacher’s implicit, internalised knowledge and not that described in published course descriptions (Levander & Mikkola, 2009).

Enabling examination, comparison and discussion about the alignment and how it was achieved amongst groups of courses, teaching academics and other stakeholders is a major aim of the project. Initially this may focus on leveraging the alignment information for staff teaching courses within the same program, including program coordinators. The L&T support section below describes how the project hopes to enable and encourage connections between teaching academics and L&T support staff.

Alignment is difficult

Levander and Mikkola (2009) describe the full complexity of managing alignment at the degree level which makes it difficult for the individual teacher and the program coordinator to keep connections between courses in mind. von Konsky et al (2006) describe how the sharing of courses between programs and a variety of outcome types (e.g. graduate attributes and course, program, discipline accrediting body learning outcomes) significantly complicates curriculum design and review. In reporting on the status of curriculum mapping, a significant task associated with alignment, Willet (2008) reports on the need for more research on effective political and electronic strategies for the construction and maintenance of curriculum maps, especially those that improve faculty participation and buy-in.

The overarching aim of the project is to build distributive leadership capacity into the systems (mostly in the form of modifications to the LMS) and processes (mostly aimed at helping teaching staff overcome these difficulties) of the participant institutions. The project aims to reduce, if not remove, the difficulties associated with this task. It seeks to achieve this by adopting an action research methodology that draws heavily on the skills, experience and insights from a broad array of project participants. The action research methodology recognises that a major part of this project is focused on learning about these difficulties and how best to reduce them within the host institutions. The following table summarises how participant selection will help reduce the impact of difficulties.

Participants Contribution
Reference group Members: chosen due to expertise and experience gained from previous ALTC leadership grants (e.g. ???) and related alignment and mapping work (Lowe & Marshall, 2004; Oliver, Jones, Ferns, & Tucker, 2007).
Responsibilities: critique and offer suggestions for improvement of project plans and results.
Institutional steering committees
(1 per institution)
Members: Institutional members with expertise/responsibility for aspects of institutional strategic aims or operational environment.
Responsibilities: planning how project activities are integrated into each institution, and fulfilling quality feasibility task.
Project team Members: Institutional L&T support staff with expertise and insight into alignment and related issues.
Responsibilities: collaborating with and helping participating teaching academic staff map and respond to course alignment.
Teaching academic staff Members: Teaching staff responsible for courses selected (using process developed by institutional steering committee and reviewed by reference group) for participation in the project.
Responsibilities: Engage reflectively on the process and its outcomes.

Concerns around learning and teaching (L&T) support

Academics come to teaching with immense amounts of content knowledge but little or no knowledge of teaching and learning (Weimer, 2007). Given this limited knowledge and the complexities and importance of learning and teaching knowledge universities have provided various types of L&T support (e.g. staff development, instructional design etc). How this support is provided and questions about its impact of the quality of L&T remain problematic. Parker (2008) identifies the on-going tension between centralised and devolved L&T support. It is widely recognised that the activities and resources associated with L&T support are used by small numbers of teaching academics, and usually not those most in need of the support (The National GAP, 2009). Weimer (2007) argues that despite nearly 30 years of effort, L&T support roles have had little impact on the instructional quality of higher education.

By making alignment an everyday consideration of teaching practice, the project aims to directly address some concerns around L&T support by drawing on important insights from the literature. Numerous authors (Biggs, 1999; Michael Prosser & Trigwell, 1999; Ramsden, 1998) have argued that the focus of L&T support should shift from techniques and technologies towards the facilitation and support of a more reflective approach to teaching. Encouraging reflection at all levels is a fundamental components of the project’s aims to move towards Bigg’s (2001) idea of the reflective institution. The quality enhancement task of the project is most closely associated with encouraging a reflective approach to teaching. Biggs (2001, p. 227) argues that the fundamental problem with L&T support is the focus on individual teachers, rather than on teaching. Following his approach, this project maintains the on-going focus on the alignment of courses, not on individual teachers. Boud (1999) argues that L&T support needs to be embedded within the context of academic work, that it needs to occur in or close to the teaching academics sites of practice. The aim of the quality enhancement phase is to make consideration of alignment an important site of practice for teaching academics and to provide the L&T support necessary as part of this site of practice.

Limitations of quality assurance

While outcomes-based quality assurance has been a prevalent component of higher education for a number of years, there remain significant concerns about how it is implemented and the subsequent outcomes. Raban (2007) observes that the quality management systems of most universities employ procedures that are retrospective and weakly integrated with long term strategic planning. He continues to argue that the conventional quality management systems used by higher education are self-defeating as they undermine the commitment and motivation of academic staff through an apparent lack of trust, and divert resources away from the core activities of teaching and research (Raban, 2007, p. 78). Barrie et al (2009) identify a bureaucratic approach to quality assurance as a potential contributor to the limited engagement of university staff in graduate attributes curriculum renewal. Biggs (2001) defines this type of quality assurance as retrospective and argues that its procedures are frequently counter-productive for quality and that most of its indicators concentrate on administrative procedures. He cites Bowden and Marton’s (1998) opinion that “retrospective QA actually damages teaching”.

Bigg’s (2001) conception of the reflective institution and its use of prospective quality assurance is presented as a solution that can make retrospective QA redundant. This project seeks to build distributive leadership capacity that enables the development of prospective quality assurance based around the everyday teaching practice of academic. Bigg’s (2001) defines prospective quality assurance as being, in part, as a bottom-up, systemic and supportive process with a priority on educational or scholarly outcomes. Such an approach has a focus on the teaching, not the teacher. These characteristics have significant connections with Southwell and Morgan’s (2009) description of Fullan’s (2008) “new leadership”, which they describe as having many of the hall marks of distributed leadership.

Long-term systemic change

As an attempt to build distributive leadership capacity the fundamental problem facing the project is to encourage long-term, systemic change. The change should not disappear once the project completes, it should become part of everyday operations. To achieve long-term, systemic change the project will:

  1. Ensure participation of formal institutional leadership and integration with institutional priorities.
    Beyond simply expressing support for a project, this project requires the active participation of formal institutional leadership roles in the institutional steering committees. These committees are responsible for developing the institutional implementation plans for two cycles of alignment embedding. These plans are intended to ensure that the project integrates appropriately with institutional priorities and practices. They are tasked with Bigg’s (2001) quality feasibility task that aims to increase institutional alignment.
  2. Action research perspective, flexible responsive.
    There is recognition that the type of fundamental change being attempted by this project is difficult, complex and replete with uncertainty. A critical success factor for the project is the ability to identify and respond to new insights. The projects action research methodology and the very nature of Bigg’s (2001) idea of a reflective institution aims to achieve on-going learning and improvement.
  3. Having a scholarly, not bureaucratic focus.
    As described above, the very nature of prospective quality assurance (Biggs, 2001) is bottom-up, systemic, supportive, and with a priority on educational or scholarly outcomes.
  4. Modifying an institutional information system.
    A fundamental enabler of this project is the presence of an information system that is embedded into the everyday practice of teaching and learning (for both students and staff) that encourages and enables consideration of alignment. Rather than develop a stand alone tool, this project seeks to modify the institutional LMS, a system to which the institutions are already significantly committed. In addition, both institutions have adopted the open source LMS Moodle as their institutional LMS. As an open source system, it is not only possible to make the changes, the subsequent changes will become available within the broader Moodle community. This increases the likelihood of on-going support both within and outside the participant institutions.

Project outcomes

The project aims to build leadership capacity within two institutions that enables consideration of alignment to become part of everyday teaching practice. The outcomes of that aim will include:

  • Within both institutions a number of courses that have had their instructional alignment mapped, made visible and reflected upon.
  • Increased availability and knowledge of resources around alignment and course mapping, especially those produced by ALTC projects, within the participant institutions.
  • For some of these courses, evidence of changes over time in the alignment and structure of the course.
  • Evidence of whether or not there have been changes in the conceptions of learning and teaching held by teaching staff participants.
  • Evidence of whether or not there have been changes in student learning experience or outcomes.
  • Availability of extensions to the Moodle LMS that enable the mapping of instructional alignment within and between courses.
  • Availability of extensions to the Moodle LMS that leverage course alignment information to provide a diverse collection of learning and teaching services.

Methodology

The project will use an eight stage process that has at its core two action research cycles. Each action research cycle consists of 3 stages:

  • plan,
    The institutional steering committee with input from other institutional project members formulates a plan for the research cycle. Institutional plans are shared between participant institutions and reviewed by the reference group.
  • embed, and
    At its core, the project team work with selected teaching academic participants to map, understand and respond to the alignment within their courses. A key part of this stage will be identifying how having the alignment information of the course within the LMS can be leveraged for improving L&T. This will typically proceed over the course of an entire term.
  • review.
    A formal process of reviewing what happened during the embed stage involving all project participants.

Given that two action research cycles with the above three stages, there are two remaining stages. These are focused on the broader tasks of establishing and completing the project.

References

Anderson, L. (2002). Curricular alignment: A re-examination. Theory into Practice, 41(4), 255-260.

Barrie, S., Hughes, C., & Smith, C. (2009). The national graduate attributes project: integration and assessment of graduate attributes in curriculum. Sydney: Australian Learning and Teaching Council.

Biggs, J. (1996). Enhancing teaching through constructive alignment. Higher Education, 32(3), 347-364.

Biggs, J. (1999). Teaching for quality learning at university. Buckingham: Open University Press.

Biggs, J. (2001). The Reflective Institution: Assuring and Enhancing the Quality of Teaching and Learning. Higher Education, 41(3), 221-238.

Boud, D. (1999). Situating academic development in professional work: Using peer learning  International Journal for Academic Development, 4(1), 3-10.

Bowden, J., & Marton, F. (1998). The University of Learning. Oxford: Routledge.

Cohen, S. A. (1987). Instructional alignment: Searching for a magic bullet. Educational Researcher, 16(8), 16-20.

Fullan, M. (2008). The six secrets of change. San Francisco, CA: John Wiley and Sons.

Green, W., Hammer, S., & Star, C. (2009). Facing up to the challenge: why is it so hard to develop graduate attributes? Higher Education Research & Development, 28(1), 17-29.

Lakomski, G. (2005). Managing without Leadership: Towards a Theory of Organizational Functioning: Elsevier Science.

Lattuca, L., & Stark, J. (2009). Shaping the college curriculum: Academic plans in context. San Francisco: John Wiley & Sons.

Levander, L., & Mikkola, M. (2009). Core curriculum analysis: A tool for educational design. The Journal of Agricultural Education and Extension, 15(3), 275-286.

Leveson, L. (2004). Encouraging better learning through better teaching: a study of approaches to teaching in accounting. Accounting Education, 13(4), 529-549.

Lowe, K., & Marshall, L. (2004). Plotting renewal: Pushing curriculum boundaries using a web based graduate attribute mapping tool. Paper presented at the 21st ASCILITE Conference, Perth.

McKinney, L. (2010). Evaluability assessment: Laying the foundation for effective evaluation of a community college retention program. Community College Journal of Research and Practice, 34(4), 299-317.

Oliver, B., Jones, S., Ferns, S., & Tucker, B. (2007). Mapping curricula: ensuring work-ready graduates by mapping course learning outcomes and higher order thinking skills. Paper presented at the Evaluations and Assessment Conference. Retrieved 17 Feb, 2010, from http://www.eac2007.qut.edu.au/proceedings/proceedings_ebook.pdf.

Parker, L. (2008). Leadership for excellence in learning and teaching in Australian higher education: Review of the Australian Learning and Teaching Council (ALTC) Program 2006-2008. Sydney: Australian Learning and Teaching Council.

Parrish, D., Lefoe, G., Smigiel, H., & Albury, R. (2008). The GREEN Resource: The development of leadership capacity in higher education. Wollongong: CEDIR, University of Wollongong.

Prosser, M., Ramsden, P., Trigwell, K., & Martin, E. (2003). Dissonance in experience of teaching and its relation to the quality of student learning. Studies in Higher Education, 28(??), 37-48.

Prosser, M., & Trigwell, K. (1999). Understanding learning and teaching: The experience in higher education. Buckingham: SRHE / Open University Press.

Raban, C. (2007). Assurance versus enhancement: less is more? Journal of Further and Higher Education, 31(1), 77-85.

Ramsden, P. (1998). Learning to Lead in Higher Education. London: Routledge.

Ramsden, P., Prosser, M., Trigwell, K., & Martin, E. (2007). University teachers’ experiences of academic leadership and their approaches to teaching. Learning and Instruction, 17(2), 140-155.

Southwell, D., & Morgan, W. (2009). Leadership and the impact of academic staff development and leadership development on student learning outcomes in higher education: A review of the literature. Sydney: Australian Learning and Teaching Council.

Spillane, J., Halverson, R., & Diamond, J. (2004). Towards a theory of leadership practice: a distributed perspective. Journal of Curriculum Studies, 36(1), 3-34.

Stark, J. (2000). Planning introductory college courses: Content, context and form. Instructional Science, 28(5), 413-438.

The National GAP. (2009). Key issues to consider in the renewal of learning and teaching experiences to foster graduate attributes. Sydney: The National Graduate Attributes Project.

Uchiyama, K. P., & Radin, J. L. (2009). Curriculum Mapping in Higher Education: A Vehicle for Collaboration. Innovative Higher Education, 33(4), 271-280.

von Konsky, B., Loh, A., Robey, M., Gribble, S., Ivins, J., & Cooper, D. (2006). The benefit of information technology in managing outcomes focused curriculum development across related degree programs. Paper presented at the 8th Australian Conference on Computing Education, Hobart, Australia.

Weimer, M. (2007). Intriguing connections but not with the past. International Journal for Academic Development, 12(1), 5-8.

Wholey, J. S. (2004). Evaluability Assessment. In J. S. Wholey, H. P. Hartry & K. E. Newcomer (Eds.), Handbook of practical program evaluation. San Francisco: Jossey-Bass.

Willett, T. (2008). Current status of curriculum mapping in Canada and the UK. Medical Education, 42(8), 786-793.

Zepke, N. (2007). Leadership, power and activity systems in a higher education context: will distributive leadership server in an accountability driven world? International Journal of Leadership in Education, 10(3), 301-314.

Requirements for an “indicators” Moodle block

The Indicators project team are off to Moodlemoot AU 2010. All three are giving presentations, but only two (i.e. not me) are on the indicators project. In thinking about how we might make the most of this opportunity, we’ve floated the idea of announcing an “indicators” Moodle block at the conference. This post is an attempt to make concrete our thinking and take some steps towards making it happen, hopefully even generate some interest and external comments.

What is an “indicators” Moodle block?

At least for me, at the moment, an “indicators” Moodle block is a simple extension to Moodle that just about anyone can install into their Moodle install and start getting some insights into whether or not what they (teaching staff, students or administrators) are doing is good, bad or indifferent.

Such a block would probably fulfill some/all of the following:

  • Can be embedded into a normal Moodle view.
    As a teaching staff member, or a student, the block would be part of my normal Moodle interface. It’s not located somewhere special that I have to remember to visit, it’s always there.
  • Is very visual.
    It sends a strong, very clear message. I don’t have to apply special knowledge or spend a lot of time trying to understand what it is telling me. If I’m doing something wrong, it should be red or something similar.

    This does NOT mean that is obtrusive. As a part of a normal Moodle view it can take my attention away from other stuff.

  • Enables comparisons.
    It doesn’t tell me how many times I’ve posted to the discussion forum, it tells me how much less (or more) I’ve posted than the best students, or all the students, or the worst students. In a friendly way it helps me understand how my use of the LMS compares to that of others.
  • Only uses what data is already in Moodle.
    The aim is for anyone with a Moodle install to be able to add this block and use if straight away. No need to modify/connect with external data sources (at least not yet).
  • Serves as a stepping stone to more functionality.
    The first block is our foray into providing such a service. Over time we might add more functionality. The block has to be a good open source project, something others can add to.

    Also, the block has to allow the use to provide more than just visualise the data. It should help them to plan actions they might take, to talk to others about this, to track history etc.

Other tasks

Apart from thinking about what it actually is, we also need to think about what we need to do to start implementation. Here’s my first list.

  • Identify some visualisation software.
    Most of these will be graphs or perhaps networks. We need to find a way to generate these graphs/network diagrams from PHP and in a way we can include in a Moodle block.
  • Figure out the Moodle database structure.
    Whatever we do we’ll be pulling data from Moodle. So need to find out the format for the bit we’re interested in.
  • Data caching?
    Most of these examples are likely to require getting a large bunch of data and doing some calculations before generate the visualisation. The block can’t be a performance hog, so we’re going to have to figure out some way to minimise performance impact. Caching?
  • Moodle block programming.
    Have done a little, there’s a bit of doco out there. So shouldn’t be too hard.
  • Managing the code.
    We haven’t done any joint development yet. Doing something like this would require us to figure out how we manage the development process.
  • Who does what?

Blurb for the alignment project

The following is an early attempt at an “executive summary” for the alignment project. It’s meant to get folk who know nothing about the project excited, or at least interested, in the project. The main audience, at the moment, is probably limited to institutional leaders around learning and teaching and those likely to be evaluating ALTC grant applications.

Suggestions and criticisms more than welcome.

I’m particularly interested in literature references that support some of the observations/claims e.g. that consideration of alignment is not part of everyday teaching practice for teaching academics.

Executive summary

Instructional (Cohen, 1987), curriculum (Anderson, 2002) and constructive (Biggs, 1996) alignment are all built on the recognition that student learning outcomes are significantly higher when there is a strong link between those learning outcomes, the assessment and the instructional activities and materials. Cohen (1987) argues that limitations in learning is not mainly caused by ineffective teaching, but instead is mostly the result of a misalignment between what teachers teach, what they intended to teach, and what they assessed as having been taught. The importance of achieving and demonstrating alignment with expected outcomes is also a central component of outcomes-based accreditation and quality assurance approaches that are increasingly widespread within higher education.

For most teaching academics, however, the consideration of alignment in their courses and programs is not part of everyday teaching practice. Consideration of alignment is typically limited to events such as significant re-design of courses and programs of visits from accreditation or quality assurance organizations. This lack of regular consideration of alignment may be a significant contributing factor to the on-going limitations of university learning and teaching and its quality assurance processes. For example, Barrie et al (Barrie, Hughes, & Smith, 2009) make the observation

despite the rhetoric of graduate attributes policy and despite the espoused claims of statements of course learning outcomes, the reality is that teaching in some courses has not changed from a model of transmission of factual content.

Through a collaborative action research process this project seeks to build and grow distributive leadership capacity within the systems and processes of the two participating institutions that encourages and enables consideration of alignment, and action-based on that consideration, to become a regular, transparent, supported and integrated part of common teaching practice. Through this the project seeks to explore steps towards adoption Biggs’ model of the reflective institution. As a result the project aims to develop approaches that can systemically enhance learning and teaching through enabling and encouraging teaching academics to question and change their conceptions and practice of learning and teaching and through this on-going consideration of alignment significantly improve student learning outcomes.

References

Anderson, L. (2002). Curricular alignment: A re-examination. Theory into Practice, 41(4), 255-260.

Barrie, S., Hughes, C., & Smith, C. (2009). The national graduate attributes project: integration and assessment of graduate attributes in curriculum. Sydney: Australian Learning and Teaching Council.

Biggs, J. (1996). Enhancing teaching through constructive alignment. Higher Education, 32(3), 347-364.

Cohen, S. A. (1987). Instructional alignment: Searching for a magic bullet. Educational Researcher, 16(8), 16-20.

Leadership as appreciating resistance

Am busy reading and trying to do up a grant application, when I should be working on the PhD. However, I couldn’t bypass this quote from FUllan (2001: 65)

Defining effective leadership as appreciating resistance is another of those remarkable discoveries: dissent is seen as a potential source of new ideas and breakthroughs. The absence of conflict is a sign of decay … investing only in likeminded innovators is not necessarily a good thing. They become more likeminded … If you include the naysayers, noise in the early stages will yield later, greater implementation.

It resonates strongly with me for two reasons:

  1. the increasing prevalence of the opposite definition of leadership; and
    The trend towards increasingly corporate approaches within universities means that increasingly there are short-term management positions which have to deal with increasing demands for accountability from government etc. The simple and increasingly prevalent approach is to stomp all over resistance and naysayers. “You’re not a team player” and “Why so negative” are the common statements I’ve heard from this approach.

    This type of success is the “I deny your reality and substitute my own” approach to leadership. It’s an approach that only ends up annoying people and failing in the long-term.

  2. the importance I place on this definition.
    Given the above, it should be no surprise that diversity of opinion is important to me. Increasingly, it is something I seek to encourage in the groups I work with, though it can often be very difficult to do. First in terms of people recognising the value of diversity of opinion. For example, I was on an interview panel where one member refused to consider someone for a job because they didn’t know a particular body of literature that the panel member thought important.

The really difficult distinction to make is between “appreciating resistance” and recognising the idiots. To often resistance is equated as being an idiot, and there’s a danger that appreciating resistance may mean paying too much attention to idiots.

References

Fullan, M. (2001). Leading in a culture of change. San Francisco: Jossey-Bass

Can BIM support the use of Moodle blogs?

Problem

BIM is a Moodle activity module that helps academic staff manage the use of individual student blogs, where those blogs are hosted on the students’ choice of blog service (e.g. WordPress.com etc.). An in-built assumption/limitation in how BIM works is that each student’s blog must be open to the world. If access to the blog is restricted, BIM can’t mirror it.

For many folk this is not a problem. However, there are occasions where teaching staff want students to use individual blogs, but what the students post should be kept private, to some extent.

One Solution?

It’s been suggested that allowing students to register their Moodle blog with BIM might get around this problem. This post is an attempt to explore that option.

Potential constraints

I don’t have a lot of time to spend on this, so it has to be simple and not involve major modifications to BIM. In addition, I’m reluctant to add to BIM special support for Moodle blogs. The aim of BIM is to encourage and enable the use of real tools through reliance on “standards” i.e. RSS and ATOM feeds. I don’t want to add special support for any particular blog.

The Moodle blog itself is not much of a blog and is likely to change soon.

The Moodle blog

Focusing on 1.9, I can find out the following:

  • Site configuration for blogs.
    Each Moodle site can configure its policies around blog visibility. Options range from turning the blog facility off entirely through to all site users being able to see all blog entries and everyone being able to see entries specified as world readable.
  • Users can also configure entries for just them to see.
  • RSS security by obsufucation, not real security.
    My understanding is that Moodle blog’s can generate RSS, and that restrictions are based on the location of the RSS being secret as opposed to really secure.
  • As per this post on Moodle.org site the Moodle site needs to have RSS enabled at the site level via: Admin blog > Server > RSS > Enable RSS

Moodle Blog RSS

With RSS enable at the site level, an RSS icon is displayed on a user’s blog. If they click on that they get an RSS feed with the URL not being all that private. i.e. confirming above, you have to know somethings to work it out, but it can be worked out.

In terms registering a Moodle blog feed, you can’t rely on auto-detect as Moodle doesn’t use the right “HTML”. You actually have to register the RSS feed.

But now I have another problem. I can’t seem to get the Moodle blog to add posts/items to the RSS feed.

Conclusions

I’m going to leave this investigation now without having worked it out. The reasons are:

  • If you can get RSS being generated from the Moodle blog you should be able to register it with BIM, as long as the permissions are set correctly.
  • If you can do this, then the students’ RSS feeds are only going to be very slightly more secure than those on WordPress.com or similar.
  • It will be very easy for other students within the course to see each other posts, so not a lot of privacy.