The Weblog of (a) David Jones

Web 2.0 tools in assessment in higher education

davidtjones — Sun, 15 Nov 2009 23:51:35 +0000

Next Monday I will be at the University of Melbourne participating in a “National Roundtable” title “Web 2.0 Authoring Tools in Higher Education Learning and Teaching: New Directions for Assessment and Academic Integrity”. This is being run as part of an ALTC project titled Web 2.0 authoring tools in higher education learning and teaching: New directions for assessment and academic integrity

The purpose of the roundtable is to “review experience and make joint recommendations for good practice guidelines”. The aim of this post is two fold:

Force me to actually think a bit about what I know/think about this topic.
Encourage others to share, disagree and improve what I think through their own contributions and experience.

I’m going to use social media, rather than Web 2.0.

I’m also traveling today, so I don’t think I’ll finish this entirely today.

A matter of cultures or mindsets

The way I’m currently thinking about this is the question of cultures or mindsets. There are (at least) two different cultures/mindsets involved here:

The social media culture.
Assessment/learning and teaching within higher education.

For me, good practice around using social media in assessment in higher education is about making sure that these cultures match. I also see this as potentially the biggest hurdle. I’m not sure that the culture of assessment/learning and teaching in higher education can easily match the culture of social media.

This blog post is titled “The impact of corporate culture on social media (An IBM Case Study)” and includes the slides from a presentation at a conference. The blog post includes the following

That culture is, in my view, the most overlooked, underestimated factor determining whether social media succeeds or fails in a company. And when corporate culture and social media are pitted against each other, social media will always fail. Always.

Disclaimer: I recognise that there are problems with using culture in this way. e.g. some claim there can be no culture within organisations, others disagree. I’m using it, I guess, in terms of the “way we do things around here”.

Mono, multi and the majority culture

I do recognise that learning and teaching/assessment in higher education is not a mono-culture. There are many and varied cultures that differ in many ways. However, I think there is a fairly large group of folk involved in learning and teaching within higher education that have much in common. For this I turn to Moore’s chasm and Geoghegan (1994). This is the idea shown in the figure below that shows a chasm between the innovative folk and the pragmatic folk.

The chasm idea is that the folk to the left of the chasm are very different to those to the right. Geoghegan’s (1994) idea is that most of what happens in higher education around instructional technology is designed for the folk to the left of the chasm. This is why most use of instructional technology has little adoption and what is adopted is of poor quality.

Geoghegan (1994) describes the difference between these two groups via the following table.

Innovators	Pragmatists
Like radical change	Like gradual change
Visionary	Pragmatic
Project oriented	Process oriented
Risk takers	Risk averse
Willing to experiment	Need proven uses
Self sufficient	Need support
Relate horizontally (inter-disciplinary)	Relate vertically (within discipline)

Rather than focus on the culture of the innovators. I’m going to focus on the “culture” of the pragmatists. I do this because they represent by far the majority of people within higher education. Also because the innovators, to a large extent, will take care of themselves. Lastly and perhaps most importantly, because I increasingly see that management and the Information Technology folk within many universities are, for various reasons, increasingly more like the pragmatists, than the innovators. As such, they can and do constrain what happens.

Culture/Principles of social media

I’m sure there’s a lot of this stuff out there. But I’m going to use these principles and in particular the remixing of them here. The 10 principles are (quoted from here):

Decentralization is freedom: Freedom enables us to pursue our thoughts and interests in a social space. Thus decentralization is of primary value.
Information wants to be free: The cost of obtaining information is rapidly declining, but still capable of providing and creating value. Freedom is necessary for free information.
Findability is power: Without findability, the information’s ability to provide and create value is greatly diminished.
Karma is real: The more you give, the more you get. You just don’t know what at the point at which you’re giving.
Rules beget rules: At some point, organization happens so that common understanding of interactions are possible.
Economies have currencies: Trade is possible with Karmic infrastructure and rules of engagement.
Communication is blood: Communication is the transport mechanism for information flow.
Immediacy in all things: Acting on new, validated information when appropriate moves things forward more quickly than before.
Context is fluid: Things change more often, as does your frame of reference. Think about the information you have at various points and look at developments along a continuum.
Associations are inherently good.

The mismatch with the culture of the pragmatists

I’m running out of time, so cutting this short. My current position is that the culture of the pragmatic, corporate university and the perceptions of the majority of staff are not a good match with the principles of social media.

For example, decentralisation, free information, karma, communication, immediacy and others don’t fit.

Flight being called.

Drawbacks of good practice and the need for theory

Dave Snowden is not a big fan of best practice and I like his reasoning. The folk sponsoring this roundtable use the “best practice lite” term that seems to be quite popular in higher education at the moment – “good practice”. It seems to remove the strong one and only nature of “best”, but it retains the main problem.

The main problem is the attempt to duplicate a practice from one context to another. If these contexts are 100% the same then this might be possible, depending on how much you can learn about the practice. But with most universities being complex systems, this is not possible. Snowden takes the approach of understanding the underpinning theory and using the theory as the basis to design an intervention that makes sense within the new context. Rather than repeating what worked in a different context.

This is what underpins my preference for understanding the culture of social media and universities, for understanding the theory.

Twitter back channels, conferences, sessions and engaging the audience

davidtjones — Fri, 13 Nov 2009 01:57:04 +0000

A couple of days ago we did an experiment around presentations that included the use of a twitter back channel (hashtag: #eair). The following is a reflection on what that allowed me to do and implications for conferences.

The question I end up at is “Should conferences have twitter hash tags for individual sessions?”

What I could do

After I finished the presentation I was able to review the tweet stream and see what people had said. This gave me a different and much greater understanding of what some of the audience were thinking and talking about. It was also frustrating because it revealed that I hadn’t effectively engaged some of the audience, they didn’t get “it”. I did end up replying to a number of tweets to expand on ideas or give alternate perspectives. The tweet stream enabled me to extend the conversation.

This ability has given me a better understanding of what worked and didn’t work for the presentation. I plan to try and use this for all my presentations.

The problem

It was easy to do this because I used a hash tag that was unique to my presentation – #eair. As I’m the only one use that tag for my local presentations, that should be ok. But conferences are difference. I’m just back from EDUCAUSE’09 which used the tag #educause09 for all discussion. There wasn’t a tag for individual sessions. So I couldn’t easily couldn’t easily find what people had said about a particular session, including the one I gave.

This absence/difficulty of tracking tweets for a particular session reduces the number of possible conversations.

Ignorance alert

I haven’t really gotten into the Twitter thing as much as others and was pre-occupied during much of EDUCAUSE’09. Is there a simple, well-known solution that I’m missing?

Solutions oriented

I keep being told I should be solutions oriented – apparently I’m cynical/pessimistic. So here’s my initial idea.

Each session at a conference get a unique number. That session number get added to the conference hash tag.

Example, ASCILITE’09 is using (I think) #ascilite09. If I’m tweeting in session 55, I’d be using a tag something like #ascilite09s55 or #ascilite09#55.

The drawback, based on very little testing, is that this would split off the conference tweets into the sessions. Perhaps the conference would need an aggregator? Perhaps, people just have two different searches? Perhaps the conference creates a Twitter list?

Don’t know enough to say. But the ability for each session tweet stream to be identifiable would be something I’d use.

Posted in icddu Tagged: twitter conferences

Reflection on alternatives and experiments

davidtjones — Tue, 10 Nov 2009 23:34:41 +0000

I’ve just completed a presentation which included an experiment with some different technologies for lectures and a talk I gave at EDUCAUSE09. The slides, video and other related resources are available from the main presentation page.

The purpose of this post is an attempt at a debrief/reflection on the experience.

Lecture experiment

The wrapper around this presentation was an experiment in using some different technologies to deliver and support the presentation including:

ustream to stream and record the presentation video.
Votapedia as an audience response system that breaks the constraints on place of clickers.
Twitter as a back channel.

The purpose of this experiment was to put an initial toe in the water. To do something with the technology to increase awareness of what it could/couldn’t do.

ustream

The video of the presentation (including powerpoint slides, views of the lecturer and audience) was broadcast via this ustream.tv channel. The recorded video is now hosted on ustream.tv.

Some statistics from the ustream dashboard:

At it’s pead there were 11 people watching the broadcast.
The broadcast went for 1 hour and 42 minutes (and 1 second).

Additional metrics will be available after processing.

I gave the presentation in one of the Interactive System-wide Lecture (ISL) theaters at the CQUni Rockhampton campus. There were audience members in the room with me and also at the CQUni theaters in Bundaberg and Mackay.

Folk from the CQUni IT division set up a flash encoder in the ISL theater so that the traditional ISL stream was sent to ustream. This was a major benefit as I am familiar with the ISL system and it provides some additional features including the ability to view the presenter, the member of the audience (at any of the campuses) asking a question, my slides, another computer and a document camera.

With little or no change on my part, my presentation broke the place constraint on a traditional ISL presentation. i.e. anyone from across the world with a web browser could watch the presentation. This is of significance for CQUni since it has a significant number of distance education students who never make it to a campus.

With ustream, these students could view lectures which they can currently only (in a small percentage of cases) access as podcasts or streaming video after the fact.

There are some organisational issues to be addressed such as how the university could automate/support the use of ustream and whether or not using ustream would be better than a university hosted service.

I don’t expect ustream to become an accepted option. The question of restricting access to just our students will be an issue. It will also likely be seen to be easier/less risk to support our own service than use an external service.

The ISL “console”

The presenter “console” in these ISL theaters offers some significant benefits over simply using ustream in my office with my computer. I can’t easily see members of the audience, it’s a bit more difficult to get a shot of me and to show other physical objects. The ability to use ustream with the console, especially because I’m familiar with using it for presentations, was a big plus.

However, it’s also a draw back. There are only so many of these consoles on campus. So there’s a limit to how many people can do this.

To get the best of both worlds there seems to be a need to create a system I can use on/with my laptop and get the same functionality as the console. Then the place restriction is entirely gone.

An important part of this would be the ability to see the participants when they ask questions. This is starting to get into elluminate territory and raises the question of what is the difference?

One perspective might be that elluminate is the “integrated system” approach to this where I’m potentially talking about a more best of breed approach – which is showing my prejudice.

Votapedia

Votapedia was used as an audience response system/clickers for a couple of simple questions. For example, the first question asked the audience how long they had owned a mobile phone. The question and results can be seen here (50% of the 17 responses had owned mobile phones for longer than 5 years).

The advantage of Votapedia over clickers is that Votapedia uses mobile phones or a web browser for answering the question. As such it also breaks the place constraint of clickers. Plus the combination of mobile phones or a web browser is something that most people already have, further reducing costs.

Setting up the votapedia surveys, from a technical perspective, was a little quirky. i.e. I had to get used to the way it works. Once familiar, it was fairly simple to manage and set them up. Even then, I did create a user error that prevented the final questions from being usable. A simple experience thing, the sort of insight that isn’t readily available in the Votapedia information and more importantly in the process. i.e. the system should be able to pick this sort of thing up and offer a warning. (though it is expecting a bit too much from a free service).

In terms of real, effective use in a course, the most difficult part of Votapedia – as with any audience response system – will be redesigning the lectures to make effective use of the technology and the interactions that it supports. In particular, the additional time such interactions would take which would appear to remove time from the presenting of information. A big challenge for many staff, including myself.

Twitter

Given there was only a small audience any use of social media was going to be somewhat limited. We had about 30/40 tweets from a half a dozen people or so. I had organised for one of my colleagues to act as “moderator” for the Twitter back channel. This was a good move. Col kept me aware of anything I needed to know and also encouraged some discussion with tweets.

ustream’s integration with Twitter was also a benefit. You can see the tweet stream on the ustream channel which hosted the stream. One drawback, this stream isn’t associated with the stored recording, would’ve been good to see some association, even some connection.

A twitter back channel appeared to be beneficial to some, but there could be some drawbacks. I did find myself replying to some of the tweets after the presentation. It gave me a chance to offer a perspective on some issues which may not have previously been possible.

There seemed to be value in twitter, however, the cost of having a moderator makes it difficult. You could get around that by perhaps tasking it to students in a rotating capacity, but there are drawbacks there. Is this a way to open up the private act of teaching by having other academic staff sit in on classes?

The future

It looks like we’ll probably do something similar to this in a couple of weeks when we do a trial run of an ASCILITE’09 presentation. Some ideas for things to think about:

Use Keynotetweet to automatically inject questions and comments into the tweet stream as certain slides are presented.
For example, references and links to resources used in the presentation. Questions for those on twitter.
Publicise the twitter ID of the moderator.
Use Votapedia for immediate evaluation of the session.

Posted in elearning, futures, icddu, iLecture, lterc Tagged: ustream twitter, votapedia

Future of universities – an age old problem

davidtjones — Mon, 09 Nov 2009 00:04:34 +0000

I’m in the midst of preparing some additional slides for a presentation/experiment tomorrow around alternatives for the LMS and the lecture. In part, this presentation connects with the future of universities – and perhaps learning in general – a topic that seems to have gotten increasing airplay in the last year or so. Especially in the form of pundits predicting problems with current practice. This post is in part about showing that this is not a new thing, but also about saving some nice quotes for future use.

As part of the work on the slides, I was doing a quick Google on “origins of the lecture”. One of the the bits I came across was a book titled In search of the virtual class: education in an information society in which I found the following quotes leading off chapter 4

Learning processes are lagging appallingly behind and are leaving both individuals and societies unprepared to meet the challenge posed by global issues. This failure of learning means that human preparedness remains underdeveloped on a global scale. Learning is in this sense far more than just another global problem: its failure represents, in a fundamental way, the issue of issues. (Botkin et al, 1979, p9)

I like the idea of “issues of issues”. Placing this problem of education as a fundamental problem for so much else. I also like that this comment was made 30 years ago. Something that illustrates one or more of

The long-term importance of the idea.
The on-going difficulty of doing anything meaningful about this problem.
The on-going market that exists for people to profess fundamental flaws within the education system.

A similar quote in the same location

There is only one problem and that is education, all other problems are dependent on this one (President Domingo Faustino Sarmiento – founder of Argentina’s national education system)

This one adds the potential observation that if all you have is a hammer (i.e. you are in the education business) then everything you see is a nail (education is the solution).

The book

The book these quotes come from seems, from my current limited reading, to be on the predictive books from the mid-1990s arguing for how technology can/will radically improve/change learning and teaching. The following is from page 73

Why is education out of step with society’s needs? Does the problem lie in the way education is administered, the methods of instruction and the content of the curricula? These are the issues that advanced industrial societies focus on as they attempt to find a solution. Our concern si with the extent to which the problem lies with the classroom as a communication system for learning. Our argument is that the classroom is a technology that emulates the way people live and work in an industrial society. It does not relate to the way people will live and work in an information society. Some countires are sufficiently into a transition to an information society for the discrepancy to be obvious.

Part of the argument is that the classroom approach is wasteful of resource of space and time.

Posted in crc, elearning, futures, icddu

Lectures, alternatives, poll everywhere and unexpected events

davidtjones — Mon, 26 Oct 2009 04:15:21 +0000

This Wednesday I’m involved with an experiment and presentation that is seeking to test out some alternatives for lectures/presentations. As it happens, the last week has brought a couple of events that are (so far) helping the case for the experiment. These are described below.

And now for a word from our sponsors…

The aim of the experiment it to break out of the geographic limitations of participation in lectures/presentations. Anyone with a web browser can participate (a Twitter account and mobile phone will increase your ability to participate, but aren’t necessary). The more people who use these medium, the better. So you are invited.

More detail on the experiment/presentation page.

We return now to your regularly scheduled program

Being bumped

I work at CQUniversity. The university has 4/5 regional campuses spread across a fairly broad geographic area. A significant number of courses are offered across all of those campuses. A common approach for some years has been for lectures for these courses to be given from one campus and broadcast across the other campuses via the Interactive System-wide Learning (ISL) system. Essentially a video-conference system with specially built rooms at each of the campuses.

This approach is becoming embedded into the operations of the institution. To such an extent that the ISL rooms are becoming a resourcing bottleneck. Apart from teaching, these rooms are also used for research presentations and meetings. It’s getting to the stage that trying to get these rooms during campus is simply impossible.

Originally, the experiment was scheduled to use one room on each of the campuses

Rockhampton – 33/G.14. Bundaberg – 1/1.12. Gladstone – MHB 1.09. Mackay – 1/1.01.

On Friday I was told that we’ve been bumped from the Mackay room. Apparently someone senior needs the Mackay room for an ISL session that is more important than my experiment.

Normally, this would have meant Mackay staff would miss out on the live presentation. They’d have to rely on the recorded presentation.

Not now. Theoretically, they should be able to participate the same as people off campus. I’m actually happy about this, it gives me a practical story to tell about why this approach might be useful. It will be interesting to see what problems arise.

PollEverywhere Polls and results

Over the weekend, while avoiding work on the presentation I came across this post from Wes Fryer. It describes how they used PollEverywhere in a conference presentation. PollEverywhere is essentially a commercial version of Votapedia which I plan to use on Wednesday.

Some things I found interesting:

The graphs.
The PollEverywhere graphs look much nicer than Votapedias (minor point).
A comment that students like this approach because it is a legitimate use of their mobile phones in class.
The idea that this type of experiment was an “a-ha” moment for some.

Posted in elearning, icddu, iLecture, lterc, presentations

The bureaucratic model and the grammar and future of universities

davidtjones — Mon, 26 Oct 2009 02:49:27 +0000

Last week I attended a presentation by a colleague at CQUniveristy titled The Bureaucratic Model of Adult Instructional Design. The stated purpose of the presentation was

present and explore the Bureaucratic Model as a narrative that we must understand if we are to influence the direction of adult education.

The talk resonated with me as much of my current struggles/work is trying to make folk aware of a range of unstated assumptions that guide their thinking about learning and teaching within a university context. As Jay says, we have to understand those assumptions before we can think of influencing the future of learning and teaching – and somewhere in that, universities.

Since Jay’s talk I’ve come across and/or been reminded of a range of related work. Please feel free to add more here.

A vision for the future

Tony Bates has recently posted the second of his blog posts title Using technology to improve the cost-effectiveness of the academy: Part 2 within which he gives his vision for the future of universities.

A number of his implications seek to remove many of the basic assumptions that underpin university operation (e.g. semesters, fixed exams). However, a number of them show connections with an existing orthodoxy (e.g. all PhD students will have 6 months training in L&T).

That’s one of the problems I have with visioning. Too often it excludes interesting possibilities because it is held back by the background, preferences, ideas and prejudices of the people doing the visioning. My preference would be to let it emerge through a institution/setting that is flexible, open and questioning. I think much more interesting things can emerge from that situation than can ever happen because of the visioning of experts.

That’s because, no matter who you are, you have unstated assumptions that define what you can think of. Often this is addressed by having lots of different people do the visioning, but too often such attempts use approaches that to quickly focus on a particular vision, closing out future possibilities.

The grammar of school

In this post I mentioned a 1995 article by Seymour Papert on Why school reform is impossible. In this article Papert draws on Tyack and Cuban’s (1995) idea of the “grammar of school”

The structure of School is so deeply rooted that one reacts to deviations from it as one would to a grammatically deviant utterance: Both feel wrong on a level deeper than one’s ability to formulate reasons. This phenomenon is related to “assimilation blindness” insofar as it refers to a mechanism of mental closure to foreign ideas. I would make the relation even closer by noting that when one is not paying careful attention, one often actually hear the deviant utterance as the “nearest” grammatical utterance a transformation that might bring drastic change in meaning.

This sounds very much like what is happening in Jay’s bureaucratic model.

The need for experiments

A lot of the current debate about the future of universities is built on the comparison with print media. i.e. look, newspapers are a long-running institution that are dieing. Look, Universities, they are a long-running institution, they must be dieing also.

Clay Shirkey has written a long blog post title “Newspapers and Thinking the Unthinkable”. A major point that he makes in his post, seems to apply directly to the future of universities and the limitations of attempts at visioning like those of Tony Bates. In particular, this

Revolutions create a curious inversion of perception. In ordinary times, people who do no more than describe the world around them are seen as pragmatists, while those who imagine fabulous alternative futures are viewed as radicals. The last couple of decades haven’t been ordinary, however. Inside the papers, the pragmatists were the ones simply looking out the window and noticing that the real world was increasingly resembling the unthinkable scenario. These people were treated as if they were barking mad. Meanwhile the people spinning visions of popular walled gardens and enthusiastic micropayment adoption, visions unsupported by reality, were regarded not as charlatans but saviors.

He then draws on the development of the printing press to talk about revolutions

That is what real revolutions are like. The old stuff gets broken faster than the new stuff is put in its place. The importance of any given experiment isn’t apparent at the moment it appears; big changes stall, small changes spread. Even the revolutionaries can’t predict what will happen

Dede’s metaphors of learning

Lastly, the following recording is of talk by Professor Chris Dede and some metaphors of learning. It is the current underlying assumption of consistency in delivery of learning that underpins much of what universities are currently doing which is my biggest bugbear. It’s what is contributing to university learning and teaching approaching what Dede describes as “the worst of fast food”.

Chris Dede: Human behaviours and metaphors for learning

Posted in cddu, elearning, icddu, lterc, PsFramework

Participation, impact, collecting data and connecting people

davidtjones — Fri, 23 Oct 2009 22:58:00 +0000

A couple of colleagues and I are trying to kickstart a little thing we call the Indicators project. We’ve developed a “tag line” for the project which sums up the core of the project.

Enabling comparisons of LMS usage across institutions, platforms and time

The project is seeking to enable different people at different institutions to analyse what is being done with their institutions learning management system (LMS, VLE, CMS) and compare and contrast it with what is happening at different institutions with different LMS.

To some extent this project is about improving the quality of the data available to decision makers (which we define to include students, teaching staff, support staff and management). In part this is about address the problem identified by David Wiley

The data that we, educators, gather and utilize is all but garbage.

But it’s not just about the data. While the data might be useful, it’s only going to be as useful as the people who are seeing it, using it and talking about it. David Warlick makes this point about what’s happening in schools at the moment

not to mention that the only people who can make much use of it are the data dudes that school systems have been hiring over the past few years.

And then this morning George Siemens tweeted the following

Collecting data less valuable that connecting people” http://bit.ly/3SMJCT agree?

If it’s an either/or question, then I agree. But with the indicators project I see this as a both/and question. For me, the indicators project is/should be collecting data in order to connect people.

What follows is an attempt to map out an example.

The link between LMS activity and grades

There is an established pattern within the literature around data mining LMS usage logs. That pattern is essentially

the higher the grade, the greater the usage of the LMS

The order is reversible as I don’t think anyone has firmly established a causal link, it’s just a pattern. My belief (yet to be tested) is that is probably, mostly good students get good grades and do everything they can do to get good grades, including using the LMS.

With our early work on the indicators project we have found some evidence of this pattern. See the two following graphs (click on them to see bigger versions).

The X axis in both graphs is student final grade at our current institution. From best to worst the grades are high distinction (HD), distinction (D), credit (C), pass (P), and fail (F).

In the first graph the Y axis is the average number of hits on either the course website or the course discussion forum. Hopefully you can see the pattern, students with better grades average a higher number of hits.

In the next graph, the Y axis is the average number of posts (starting a discussion thread) and the average number of replies (responding to an existing discussion thread) in the course discussion forum. So far, the number of replies is always greater than the number of posts. As you can see, the pattern is still there, but it is somewhat less evident for replies.

Importance of staff participation

Fresen (2007) identified the level of interaction or facilitation by teaching staff as a critical success factor for web-supported learning. We though we would test this out using the data from the project by dividing courses up into categories based on the level of staff participation.

The previous two graphs are actually for the 678 courses (the high staff particiaption courses) for which teaching staff had greater than 3000 hits on the course website during the term. The following two graphs show the same data, but for the super-low staff participation courses (n=849). A super-low course is one where teaching staff had less than 100 hits on the course website during term.

What do you notice about the pattern between grade and LMS usage?

First, the hits on the course site and the course discussion forum

Now, the average number of posts and replies in the course discussion forum

For me, the pattern is not there. The HD students appear to have decided there’s no value on the course website and decided they need to rely upon themselves. They’ve still been able to get a HD in spite of the super low staff participation. More work needs to be done.

I’m also interested in what the students in these super low courses might be talking about and what networks they are forming. The SNAPP tool/work at Wollongong might be useful here.

How to bring people together

My fear is that this type of finding will be used to “bring people together” in a way that is liable to be more destructive than anything. i.e. something like this:

The data mining dweebs (I do recognise that this probably includes my colleagues and I) will bring it to the attention of university management.
After all, at least at my institution it’s increasingly management that have access to the dashboards, not the academic staff.
The data mining dweebs and management will tell stories about these recalcitrant “super-low” academics and their silliness.
A policy will be formulated, probably as part of “minimum standards” (aka maximum requirements), that academics must average at least X (probably 3000 or more) hits on their course website in a term.
As with any such approach task corruption will reign supreme.

While the indicators project is a research project focused on trying to generate some data, we also have to give some thought and be vocal about how the data could be used appropriately. Here are some initial thoughts on some steps that might help:

Make it visible.
To some extent making this information visible will get people talking. But that visibility can’t be limited to management or even teaching staff. All participants need to be able to see it. We need to give some thought about how to do this.
Make it collaborative.
If we can encourage as many people as possible to be interested in examining this data, thinking about it and working on ways to harness it to improve practice, then perhaps we can move away from the blame game.
Be vocal and critical about the blame game.
While publicising the project and the resulting data, we need to continuously, loudly and effectively criticise the silliness of the “blame game”/policy approach to responding to the findings.
Emphasise the incompleteness and limitation of the data.
The type of indicators/data we gather through the LMS is limited and from some perspectives flawed. An average doesn’t mean a great deal. You can’t make decisions with a great deal of certainty solely on this data. You need to dig deeper, use other methods and look closer at the specifics to get a picture of the real diversity in approaches. There may be some cases where a super-low staff participation approach makes a lot of sense.

References

Fresen, J. (2007). A taxonomy of factors to promote quality web-supported learning. International Journal on E-Learning, 6(3), 351-362.

Posted in elearning, icddu, indicators Tagged: vle cms lms

Alternate ways to get the real story in organisations

davidtjones — Wed, 21 Oct 2009 03:50:51 +0000

I’ve just been to a meeting with a strangely optimistic group of people who are trying to gather “real stories” about what is going on within an organisation through focus groups. They are attempting to present this information to senior management in an attempt to get them to understand what staff are experiencing, to indicate that something different might need to be done.

We we asked to suggest other things they could be doing. For quite some time I’ve wanted to apply some of the approaches of Dave Snowden to tasks like this. The following mp3 audio is an excerpt from this recording of Dave explaining the results of one approach they have used. I recommend the entire recording or any of the others that are there.

Why do we shit under trees?

Imagine this type of approach applied to students undertaking courses at a university as a real alternative to flawed smile sheets.

Posted in cognitiveEdge, icddu

Choosing a research publication outlet

davidtjones — Sun, 18 Oct 2009 23:12:19 +0000

I’m reluctant to post this. It’s part of a pragmatic approach to figuring out where, as an Australian academic, I should try and target publications. It seeks to identify publications in the higher education and educational technology areas that would be “best”.

I’m well aware of the questionable aspects of this approach, but if this is the game…. Especially when your institution is starting to discuss definitions of research active staff – the implication being that if you aren’t research active you don’t get time to do research – that include requirements for fixed numbers of A and A* journals within a 3 year period.

My mitigation strategy against this type of pragmatism is that I am fairly open when it comes to my research. Much of it gets an airing here first. It’s not much, but better than nothing (or at least that’s what I keep telling myself).

For my immediate purposes, it looks like AJET is a good fit. A journal that is open access.

Work to do

Find out how much value is placed on the difference between A and A* journals.
Check the final lists from the government to see if rankings have changed.

What’s your suggestion?

What’s the “best” publication outlet?

I’m assuming that when it comes to writing a paper based on that research that the first step is to choose the outlet. Which journal or conference are you aiming the paper at? I think you need to answer this question as there is a part of the writing process that has to respond to the specifics of the outlet (e.g. address the theme of a conference etc.).
In answering this question, I can think of at least the following dimensions to consider:

Quality.
There are two common strategies I’ve heard: top down or bottom up. Bottom up folk go for the “worst” journal based on the hope that their poor article will get accepted. The top down folk suggest starting at the top because you never know, you might get lucky, and if you don’t you will at least get good feedback to improve the paper. At this stage you prepare it for submission to outlet #2.
Fit.
i.e. the one which best fits the topic or point of your paper. Which may be to visit Hawaii (conference) or might be a topic match (the paper “Gerbils preference in social software” might be a good fit for the journal “Studies in Gerbil Selection of Social Software”.
Speed of review.
How quickly will the journal accept and publish your paper.
Openness.
Are the papers published in a closed or open manner? Can you circulate copies? Is the journal an open access journals .

The rankings approach that is increasingly prevalent tends to suggest that “Quality” is the first choice. The following focuses on the quality dimension, however, in operation there needs to be an appropriate balance with the other factors.

How to judge the top quality publication?

The “top quality publication” dimension begs the question, “How do you know what is the top quality publication?”. In some disciplines this is a clear cut thing. You can’t be a researcher within a field without knowing. The trouble is that in some other fields, it’s not so clear. Especially if you’re new to the field.

Those wonderful folk in the Australian government, following the lead of their British colleagues, are making it easier for us poor Australian academics. As part of this work they are developing “discipline-specific tiered outlet rankings”. i.e. if you want to play the game, you follow their rankings – while trying to balance the other dimensions.

While the Oz government lists are still under development John Lamp is providing a nice interface to view the rankings as part of his broader site. There’s a but field of research method and a search. This is provided for two lists from the Australian Research Council – an early draft one and a more recent one. The recent one isn’t that integrated into the database – so the following information is a bit out of date, but it gives an indication.

In the following I’ve selected those journals of potentially most interest to me – I could be mistaken and have left some important ones out – but it’s a start. I’ve added a link to the journal home page and made some comments from my look at their online information.

My main interests are in educational technology within higher education, so that’s the focus. Suggestions and comments welcome.

One of the outstanding tasks I have, is to determine how much of a difference folk are making between A and A* journals.

Higher education

Most of these are selected from this list

Ranking	Journal	Comments
A*	Higher Education Research and Development	Max 7000 words Closed access 6 issues a year
A*	Studies in Higher Education	Max 7000 words Closed access 8 issues a year
A	Higher Education Quarterly	Associated with Society for Reseach in Higher Education Closed
A	Higher Education Review	5K to 10K words Copyright is assigned to Tyrrell Burgess Associates with a fee? to cover all rights. Author allowed to circulate with acknolwedgement This is interesting HIGHER EDUCATION REVIEW is committed to a problem-based epistemology. In all countries there is an urgent need to formulate the problems of post school education, to propose alternative solutions and to test them. The policy and practice of governments and institutions require constant scrutiny. New policies and ideas are needed in all forms of post school education as new challenges arise.
A	International Journal of Teaching and Learning in Higher Education	he specific emphasis of IJTLHE is the dissemination of knowledge for improving higher education pedagogy. Review process ~ 3 months Open 4K to 7K words 3 types of article: research, instructional (designed to explain and clarify innovative higher education teaching methods) and review
A	Journal of Higher Education	6 issues a year Paper-based submission!!! Max 30 pages, double-spaced 12 months submission to publication (usuaully) closed
A	Teaching in Higher Education	One aim of journal “identifies new agendas for research” 3K to 6K words 6 issues a year closed

Coming out of that table, the International Journal of Teaching and Learning in Higher Education sounds interesting, at least for me. It’s open access, shortish review times, promise of good feedback, has a couple of types of articles and is related to the scholarship of learning and teaching which is connection to an aspect of my current position.

Educational technology journals

Most of these came from this list

Ranking	Journal	Comments
A*	British Journal of Educational Technology	Closed Various suggestions it’s the top journal in this sort of field. Only 4000 words Not clear about hosting your own sites 6 issues a year
A*	Computers & Education	Closed 8 issues a year Impact factor higher than BJET? Apparently horrible restrictions on reuse Authors suggest reviewers! No max length
A	ALT-J	3 times a year Basically closed 5K words
A	Australasian Journal of Educational Technology	Open access 5k to 8K words, with occasional flexibility
A	Australian Educational Computing	2 issues a year Closed
A	Educational Technology & Society	Open access 7K words About 4 issues a year
A	Educational Technology Research and Development	Closed Claimed two month review process 5K to 8K words
A	Journal of Computer Assisted Learning	Closed 3K to 7K words
A	Technology Pedagogy and Education	Closed 3 issues a year
B	International Journal on E-Learning	Closed AACE journal
B	Internet and Higher Education	Closed 10 to 30 pages double spaced
C	Studies in Learning Evaluation Innovation and Development	Open 3K to 6K Disclaimer: I’m associated with this journal

Discipline specific and curriculum

Sometimes I do work with discipline folk, some of the following might be interesting. More of these journals here. I’ve only included links for these.

Ranking	Journal
A*	Management Learning
A*	Nursing outlook
A*	Science Education
A	Computer Science Education
A	Journal of Engineering Education

Posted in icddu, publication Tagged: era

Podcast for presentations at the PLEs & PLNs symposium

davidtjones — Thu, 15 Oct 2009 23:41:12 +0000

The following basically tells the rationale and approach used to create a (audio) podcast of the presentations from the Personal Learning Environments & Personal Learning Networks Online symposium on learning-centric technology.

I don’t know if anyone else has already done this, but just in case will share.

If you don’t want to be bored by the background, this is the link for the podcast.

Rationale

I’ve hated the idea of the LMS for quite some time. I even had the chance to briefly lead a project looking at investigating how PLEs could be grown and used within a university, at least before the organisational restructure came. In its short life the project produced a symposium, a number of publications, various presentations and a little bit of software.

Due to the background I had some significant interest in the symposium being organised by George Siemens and Stephen Downes. However, due to other responsibilities, odd times (given my geographical location) for the elluminate presentations and the low speed of my home Internet connection I knew I was unlikely to actively engage. Some of these factors have already prevented my on-going engagement with CCK09.

I probably would have left it there, however, over the last 24 hours two separate folk have mentioned the symposium and almost/sort of guilted me into following up. The one thing I can do at the moment, due to a fitness kick involving a great deal of walking, is listen to mp3s. So, I wanted an easy way to get the mp3s. A podcast sounds ideal for my current practices.

The podcast

Last night I did a quick google and found this page that seems to provide a collection of links to video and audio recordings of presentations associated with the CCK09 course. Including some mp3s from the presentations at the PLEs & PLNs symposium

Rather than download and play silly buggers with iTunes I decided to recreate an approach we used on our first “Web 2.0 course site”. Using del.icio.us the students and staff in the course could tag audio/video for inclusion in a podcast created by Feedburner.

So I followed the same process for these:

Tagged the mp3 files linked to from this page with del.icio.us using a “pleplnPodcast” tag
Used feedburner to create the podcast
Subscribed via iTunes, this podcast is now part of all the other podcasts I follow.
Synced with the iPhone and I’m away.

I just hope now that I have the time to reflect and write about what I listen to.

Thank you Deidre and Maijann for the encouragement to engage with the symposium. Thanks to those organising the symposium and CCK09 for the resources.

Posted in cck09, elearning, icddu, ple, ples@CQUni