Daily Archives: February 27, 2009

Featured on slideshare – the give and take of social computing

The editorial team over at Slideshare have decided my recent PhD presentation is worth of featuring on the Slideshare home page. The following screen shot of the Slideshare home page is included as proof and also as a record. 105 views before being featured. How high can you go?

Featured on slideshare

This is another example of the “you scratch my back, I’ll scratch your back” approach to networking, links and social software. In the email I got from Slideshare with the good news was this

p.s. Why not blog/twitter this and let the world know about your awesome creation?

So here I am, blogging about it. So, maybe I’ll raise views on the presentation. Maybe I’ll make more people aware of Slideshare as a service. Plus, by featuring my presentation on the home page, my presentation gets more hits.

Of course the $64K is, “what value is all of this”?. Well, apart from the small amount of pleasure I get from “running up some good numbers” (apparently there is a bit of brain science or psychology that explains this good feeling – must look that up at some stage), the main benefit is the unknown.

There may not be any positive value in it, at least not to me, at least not this time. However, there could also be something really valuable and something that would never have happened otherwise.

Just this week, a post of mine on patterns for e-learning has started a conversation with the head of an Art and Design Research centre in the UK about design and some of the very interesting work that he and his students/colleagues are doing. Without the blog post, I would never have made that connection.

Punya Mishra posted something this week on this same topic. His post makes the point much more poetically than mine and he uses a good term for it “serendipitous connectabilty”.

In the time it has taken me to write this post, the number of views on the presentation has gone to 122 views – 17 views while writing.

Update

A couple of days after being feature, and by now having moved from the Slideshare home page, the presentation statistics are now: 1265 views, 118 downloads, 5 favourites (including the co-founder of slideshare), 5 comments and being added to 1 group.

PhD update #1 – the start of a tradition?

Tradition

In attempt to encourage on-going work and solve the problem of keeping various folk updated I’m going to start a new Friday tradition – posting a summary of the PhD related work I did in the last week. Here’s to the first of many.

20th to 27th February

In the last week I’ve

  • Given a presentation on the thesis at the ANU
  • Caught up on some literature on higher education.
  • Had a related paper accepted at an international conference.
  • Created a to do list/overview of the thesis.
  • Given some thought to the research note describing how to write up information systems design research work.

I close with what I currently think I’ll aim for next week.

Presentation

129 slides in 30 minutes, given, recorded and placed onto slideshare and turned into a slidecast. In the last week, the slidecast has just gone past the 100 views.

Literature on higher education

For some reason, earlier in the week I ended up looking at the online archives of the journal Higher Education. Found a number of papers of interest, both to the PhD and other work.

I’ve incorporate the work of a number of these papers into Chapter 2. I’ve found this much easier now that I have a firm grasp on the Ps Framework. Having that structure makes it much easier (but not straight forward) to build the argument.

What I’ve been reading has also led to a couple of blog posts. Some of these have been trying to get arguments for the thesis down in to prose.

To do list

Today I’ve achieve the major thing I wanted to do this week. Get some overall idea of where the thesis is up to, what its structure is and what is left to do. The to do list serves this purpose. I hope to keep this updated, both adding and striking out, to dos as time progresses.

Presenting design research work

Originally, the submission to JAIS of Gregor and Jones (2007) included an appendix that offered a suggested structure for a design theory/research thesis/paper. They didn’t want to publish the appendix so we put it online a couple of months ago.

At least one of us is keen to get this published. Today I blogged another post providing a potential argument about why the research note is important, even though “experts” will think it problematic.

Next week

For the foreseeable future (a few weeks at least) I think my major aim will be two fold:

  1. Make significant progress, if not complete a draft of Chapter 2 on the Ps Framework.
  2. Similar progress on Chapter 3 – especially because I want to get feedback on the ideas I currently have, which may prove to be novel and consequently somewhat limited.

Why formulaic guidance annoys experts and why they ignore the needs of the novice

Shirley is keen to do some more work on the “research note” suggesting a structure for a design theory/research thesis/paper. This was the appendix that was original part of Gregor and Jones (2007) but which JAIS decided not to publish. The main reason was that the reviewers thought it boiled down the process of publishing to a formula, and they obviously thought this was a bad thing.

The rejection by the reviewers is interesting because the research note has proven to be useful and interesting for a range of folk. Even before the 2007 publication we had received requests from at least one information systems Professor for a copy he could use in a doctoral seminar. I should point out that this gentleman (who shall remain nameless) has some significant runs on the board in terms of publications around information systems design theory. Shirley also continues to get requests for it.

We put the research note onto my blog in early Oct 2008 and have not widely publicised this. Even though, it has been viewed 114 times and is sitting at #10 of the top posts on my blog.

So why the disconnect?

The blindness of experts

I’m suggesting that this difference in response to the paper is based on a mismatch of requirements between a novice researcher (e.g. someone taking a doctoral seminar) and an expert researcher (e.g. a reviewer for a Tier 1 journal). What the novice researcher thinks is a god send, a way out of the swamp. The expert thinks is simplistic and demeans their expertise.

My argument is that the expert researcher is blind to the requirements of the novice, because the expert thinks differently and is not taking seriously the needs of the novice. The expert is doing the novice a disservice.

I’ll use the Dreyfus model of skills acquisition as the basis for my conclusion.

Dreyfus model of skills acquisition as an explanation

In earlier post I pointed to a video that was my introduction to the Dreyfus model of Skills Acquisition. Based on a paper title “A five-stage model of the mental activities involved in directed skill acquisition” the model identifies 5 stages of learning a new skill and the characteristics and requirements of people in each of those stages. See the following table for a summary of the model.

Novice-to-Expert scale
(Adapted from Lester (2005))
Stage Characteristics
Novice Rigid adherence to taught rules or plans
Little situational perception
No discretionary judgement
Advanced Beginner Guidelines for action based on attributes or aspects
Situational perception still limited
All attributes and aspects are treated separately and given equal importance
Competent Copying with crowdeness
Now sees actions at least partially in terms of longer-term goals
Conscious, deliberate planning
Standardised and routinised procedures
Proficient Sees situations holistically rather than in terms of aspects
Sees what is most important in a situation
Perceives deviations from the normal pattern
Descision-making less laboured
Uses maxims for guidance, whose meanings vary according to the situation
Expert No longer relies on rules, guidelines or maxims
Intuitive grasp of situations based on deep tacit understanding
Analytic approaches used only in novel situations or when problems occur
Vision of what is possible

One of the ideas underpinning this work is that if you want to help someone develop their skill you have to

  • Identify where they are located in terms of the skill?
    Are they a novice? Competent? etc.
  • Customise your “training” so that it fits what people at that level need and to help them move onto the next level.

So at which level would you place a reviewer for a Tier 1 journal? At which level would you put a doctoral student starting out on his PhD? Do you see how these two very different people require different assistance?

Conclusions

There is most definitely a place for the research note. It would serve a good purpose in helping novices move up the Dreyfus skill levels. However, it should be written to ensure that it helps novices move beyond the level of applying a formulaic approach to a deeper understanding.

One suggestion/thought that arises from this is that perhaps the research note should not be a straight forward formula. i.e. give the impression that the authors have agreed on a single structure that is suitable across all situations. Perhaps the research note needs to have some disagreement or other strategies to “appropriately” move the novices from the lower levels to the higher levels. Rather than present the end result of the “experts” formulation, perhaps it should show some of the questions and reasoning that informs the “experts” formulation.

References

Stan Lester, Novice to expert: the Dreyfus model of skill acquisition

Minute papers – encouraging relection?

Part of the rationale for developing and using BAM had its origins in this unpublished paper (Jones, 2005). A part of the paper talks about minute papers, that content is reproduced below.

Minute papers

The minute paper is one way to help promote meta-cognitive thinking amongst students and to provide academics with ungraded, anonymous, immediate feedback from their students in order to assess how well and how much they have learned (Murphy & Wolff, 2005). Empirical tests have found that students completing minute papers scored higher than those who did not (Murphy & Wolff, 2005). For academic staff, minute papers raise the awareness of student experience and misunderstandings and provide an opportunity to reflect on teaching. Also it is a mechanism through which the academic demonstrates respect for and interest in student opinion and encourages the student’s active involvement in the learning process (Angelo & Cross, 1993). Unlike other forms of course evaluation the minute paper can be explained to students as a vehicle for improving their own on-going instruction rather that that of future students (Chizmar & Ostrosky, 1998).

A minute paper asks students to take a minute at the end of a class or topic to answer, traditionally on paper, a small number, usually one or two, of questions about the class. The most common two questions are:

  1. What was the most important thing you learned during today’s class?
  2. What question(s) remain upper-most in your mind? Or, what is the muddiest point still remaining at the conclusion of today’s class?

The anonymous student responses are handed into the academic who makes use of these responses to make some adjustment in the course. If minute papers are overused or poorly used it can be seen by students as a gimmick or pro forma exercise in polling (Angelo & Cross, 1993). Murphy and Wolf (2005) found that as the semester progressed a few students became “bored” with the minute papers and gave rushed and trivial responses to the questions. It is difficult to prepare questions that can be easily understood and quickly answered (Angelo & Cross, 1993). Implementing the one-minute paper in an online form did not achieve the same response rate as a paper-based version but was superior in that students provided longer responses, provided the instructor with greater flexibility with replies and were automatically archived for future use (Murphy & Wolff, 2005).

The author has used minute papers in face-to-face teaching and found them to be useful. It is thought that asking distance education students to blog a minute paper each time they do some study will provide a minimal level of structure and help the coordinator be aware of how each student is progressing.

The need for observable change

The minute paper idea has some similarity with the concept of a course barometer (Jones, 2002) through its use a simple, regular set of questions asked regularly during term to provide academic staff with feedback from students that can form a basis for improvement.

The commonality continues in terms of the importance of observable change. The barometer paper (Jones, 2002) found that students were much more likely to contribute to a barometer if they could see observable change happening as a result of barometer feedback. This overlaps with the point made above about minute papers

If minute papers are overused or poorly used it can be seen by students as a gimmick or pro forma exercise in polling (Angelo & Cross, 1993). Murphy and Wolf (2005) found that as the semester progressed a few students became “bored” with the minute papers and gave rushed and trivial responses to the questions.

The observable part of “observable change” also suggests that the course barometer, at least in its original format, is likely to be an improvement over minute papers. This is due to the fact that student comments are visible to all students on the course barometer, however, student comments in minute papers are typically not visible to others.

Use of minute papers with BAM

The initial BAM assignment had 3 of 9 questions use minute paper like questions. For example

Post an entry to your blog that answers the following questions:
  1. What were the most important concepts you learnt about data and process modelling this week?
  2. Why do you think those concepts are important?
  3. What are the data and process modelling concepts that are still causing you the greatest problems?
  4. How might the problems you are having be solved?

There is much research still to be done on the use of BAM. One avenue of interest might be to investigate the quality of the answers given to these “minute paper” questions and any correlation with final results. (yes, there are all sorts of limitations with that sort of research, but still some small amount of value). Perhaps Chizmar and Ostrosky (1998) can provide some insight into this.

References

Angelo, T. and K. Cross (1993). Classroom Assessment Techniques: A Handbook for College Teachers. San Francisco, Jossey-Bass.

Chizmar, J. and A. Ostrosky (1998). “The one-minute paper: Some empirical findings.” Journal of Economic Education 29(1): 3-10.

David Jones, Student feedback, anonymity, observable change and course barometers, World Conference on Educational Multimedia, Hypermedia and Telecommunications, Denver, Colorado, June 2002, pp. 884-889.

David Jones (2005), Enhancing the learning journey for distance education students in an introductory programming course

Murphy, L. and D. Wolff (2005). “Take a minute to complete the loop: using electronic Classroom Assessment Techniques in computer science labs.” Journal of Computing Sciences in Colleges 21(1): 150-159.