Trying out a new writing process

It’s Wednesday, that means writing day. Time to continue work on a paper that’s been mentioned previously. In this process I’m exploring being open about the writing process and attempting to create a collection of artifacts (evidence?) leading up to the final paper. This post documents today’s steps with a particular focus on first steps with a new Smallest Federated Wiki (SFW) enabled writing process that fruitfully merges with use of this blog.

An idea SFW enabled writing process

Perhaps it’s my programming background, but when I start writing a paper or a presentation I start by developing the structure. The sequence of ideas or points that I want to make in the paper. Once I have that structure in place, once I know what I want to say, I find writing that much easier. Of course the structure will change over time, but having the outline of ideas helps.

In the past I’ve used a range of adhoc methods for developing this structure. Ranging from the purely mental approach of reciting and reframing it in my head through to drawing upon artifacts such as pen & paper, text files edited with vi, and Word documents. As the title suggests the plan here is to use SFW. To create a page/article in SFW that consists of a list of ideas/concepts. Each of those ideas/concepts are separate pages in SFW.

The idea being that the writing process will be a sequence of reading, thinking and writing about each of those ideas/concepts. Of fleshing out each of the pages for the single ideas and perhaps moving them around, removing some and adding others. At some stage when I’m happy with the raw material on SFW, the idea will be translate that into Word or some other editing software.

On the plus side, this approach seems to offer better support (the linking of ideas and their separate development, versioning at a paragraph level, enabling others to build on what I’ve written) for a writing approach I’m comfortable with. However, it’s also likely to offer some challenges and misfits. For example, I found it difficult to come up with labels for the ideas/concepts that make up the paper that are both meaningful to me (and the writing process) and general enough to encourage/enable others to contribute and build upon those ideas.

If other people do contribute on build on those ideas, what implications might this have for authorship? At the moment, I don’t see the SFW ideas being in the form of the final paper. They will be more general. When I move to the word processing software I will re-write/write my own version, but it will likely be influenced by the input of any other contributors.

I also wonder about the affordances of the SFW interface for writing. i.e. the level of technical support it offers for entering and organising large amounts of text and my understanding of how best to leverage that. There’s still not the level of “fluency” (I feel somewhat dirty using that word) I’d like.

Associated with that I wonder about the question of citation management. Important to academic writing, but not directly supported by SFW. I’ll have to develop some practices and wonder how those will scale, especially as I move into the writing of the formal paper.

Time will tell.

Case studies in relevant journals

The next phase of writing is to start filling in some of the ideas in the paper structure by looking at the literature. The first area I’m looking at will be the question of case study research in the journals I’m targeting and also around the question in the paper (what’s involved in the reality of trying to develop high quality learning environments in higher education? Does that explain why there’s limited widespread quality?) The point is to identify prior work to build on and learn from.

Raising the questions of how to best search through articles in specific journals. Does Google scholar support this or do I have return to using the old style (horrendous) “library database search”. Ahh, appears the institutional library has moved a step beyond some of my memories from years ago and my local collection of papers is revealing some tidbits.

Time to start sifting and reading.

Progress made. To early to tell how it will go.

At least one of the papers I skimmed resonated with a negative local experience and generated a rant.

Barriers to higher education technology adoption: Digital fluency or usefulness?

Motaghian et al (2013) in talking about “web-based learning systems” (LMS) conclude that (p. 167)

perceived usefulness was the most important influential factor on instructors’ intention and their actual use of the systems (adoption)

This is seen as important since earlier they’ve argued (p. 158)

despite the emerging trend of using web-based learning systems to facilitate teaching and learning activities, the number of users of web-based learning systems is not increasing as fast as expected (Wang &Wang, 2009). Eventually, while e-learning has been promoted to various levels of users, the intention to continue using such system is still very low. Although initial acceptance of e-learning is an important first step toward achieving e-learning success, actual success still needs continued usage (Lee, 2010). However, since the web is a new medium (for educators and learners alike) for course delivery and learning, it is not well known which mediating and moderating factors in the online environment contribute more to its acceptance and use (Sanchez-Franco, Martínez-López, & Martín-Velicia, 2009)

They found that how useful an instructor perceived the institutional web-based learning system to be, was the most important factor influencing use.

They found that “perceived usefulness (0.50) contributed two times more to intension to use than the perceived ease of use (0.25)” (p. 66) and concluded (emphasis added) (p. 66)

As a result, instructor’s perceived ease of use might not be as important as instructor’s perceived usefulness in this context. Thus, instructors will be more likely to continue to use the system if they consider it useful. Hence, instructors’ requirements should be taken into consideration when developing web-based learning systems (Wang & Wang, 2009).

What might that suggest about the idea from the 2014 Horizon Report that the #1 barrier to higher education technology adoption is the low digital fluency of academic staff?

Might it suggest that low levels of system adoption says more about the usefulness of the technology, than the fluency of the instructors?Might it suggest that the requirements of instructors aren’t being taken into consideration in their development?

Given my recent experience (at the same time as reading this paper) with institutional approaches to e-learning, I know how I’d answer those questions.

What about you?

Note: In talking about this I’m generally focusing on how institutions implement these systems (e.g. Moodle), rather than the development of those systems (e.g. Moodle). I think institutions (at least the ones that I’ve experienced) are particularly incompetent and producing systems that I would perceive as useful.

The importance of bricolage

Not to surprisingly, this observation has me thinking about the differences between the SET (in their way) mindset used by most institutional e-learning and the (breaking) BAD mindset used by people like myself.

A SET mindset is takes a Strategic approach to deciding what work gets done. It’s focus is on achieving the institutional plan or vision. For example, making sure that every course uses the standard look and feel template. Instructors’ perceived usefulness of the system is not the prime concern of a SET mindset.

A BAD mindset uses bricolage as a way of deciding what work gets done. Ciborra (1992) defined bricolage as the “capability of integrating unique ideas and practical design solutions at the end-user level” (p. 299). Bricolage is about solving the problems experienced by users. Bricolage focuses on enhancing perceived usefulness.

So, would the SET mindset or the BAD mindset contribute to greater levels of adoption?

Seems like a no-brainer to me.

References

Ciborra, C. (1992). From thinking to tinkering: The grassroots of strategic information systems. The Information Society, 8(4), 297–309.

Motaghian, H., Hassanzadeh, A., & Moghadam, D. K. (2013). Factors affecting university instructors’ adoption of web-based learning systems: Case study of Iran. Computers & Education, 61, 158–167. doi:10.1016/j.compedu.2012.09.016

Learning about case study methodology to research higher education

The following is a summary and some thoughts on Harland (2014). The abstract for the paper is

Learning about teaching through inquiry is a sound practice for professional development and the university teacher-as-researcher is now commonplace. With the proliferation of inquiry-based postgraduate programmes in university teaching, more academics from across the disciplines are learning new ways of doing research. In this paper, I draw on 10 years’ experience of teaching research methods in higher education. I teach on a one-year part-time course that aims to help academics change their practice and contribute to the theories of teaching in higher education through publication of their work. The preferred research method is case study and there are four questions that both inexperienced and experienced participants can find challenging: what is the potential of case study; what forms of data are acceptable; when does analysis stop and what makes a quality case study? I conclude with a set of recommendations for the new researcher aimed at enhancing the quality of research. Suggestions include properly integrating existing theory into published work, avoiding positivist analogues to judge research, using multiple methods in research design and avoiding writing descriptively or without a critical audience in mind.

My interest

As outlined previously, I have to write more journal articles. The current paper idea I have goes under the following working title “BAM, BIM, Blogs and Breaking BAD: What does it take to create quality e-learning?”. It’s currently conceptualised as a case study of the development and use of BAM and BIM to support the use of individual student blogs from 2006 through 2015. The basic argument is that it’s no surprise that most e-learning is not that great, given the difficulty of doing anything decent within the current institutional mindset around e-learning. The idea is to draw on the Breaking BAD paper, a presentation to MoodleMoot and various other publications round BIM/BAM over the years.

Given it’s a case study and they have limitations, it’s probably a good idea to be able to write up the method in a way that ticks all the right boxes. Hence my interest in Harland (2014).

Thoughts

Case study is a dominant/well accepted method. Tick.

How often does Computers and Education accept case study work?

Points out some interesting points for consideration by early case study researchers in higher education. Not a “how to” guide such as Baxter and Jack (2008)

Summary

Introduction

Researching own teaching practice one way academics learn to teach. Teacher-as-researcher, link to high schools and Boyer’s SoTL. Defines “dual researchers” people rsearching both their discipline and the teaching of that discipline.

Author from hard sciences. Research in academic development didn’t follow the science rules.

cites Tight (2012) that qualitative inquiry remains dominant research method in higher ed journals.

As a reviewer, author finds most articles are sub-par. Case study used in most articles read.

# of case studies published in four higher education journals: 2007-2012
adapted from Harland (2014, p. 1114)
Journal Location Case study Conceptual study Total #
Higher Education Europe 344 181 525
Studies in Higher Education Europe 238 62 300
Teaching in Higher Education Europe 176 111 287
Higher Education Research & Development Australasia 146 101 247
Total: 904 455 1359

“Case study consists of empirical inquiries of single cases that are contextually unique (Stake, 1995)” (Harland, 2014, p. 1114) – my emphasis added – typically addressing something of interest to the authors professional practice. Has instrinic value to those that benefit from the professional practice, but can also contribute to “the theories and practices of higher education”

There are four sticking points in learning case study research methods

  1. What is the potential of the case study?
  2. What forms of data are acceptable?
  3. When does analysis stop?
  4. What makes a quality case study?

These are used to structure the rest of the paper.

Has a para which appears essentially the research method para. Autoethnography is used. The four sticking points are addressed through a personal narrative.

What is the potential of the case study?

Specificity of case study research seen to limit contribution to theory. But that type of certainty of knowledge is very techno-rational. Case study inquiry involves individuals/teams interpreting data. Requiring new standards of judgement (Flyvbjerg, 2006) who contrasts rule-based and case-based knowledge. Case-based is always context-dependent.

As I see it, no two practice contexts are ever genu- inely the same and so rules and deterministic models for guiding thinking and action are not that useful. (p. 1115-1116)

Case study research cannot be truly replicated given the uniqueness of context, but it can be learned from. What each reader may learn will differ.

While its possible to generalise from case studies (Denzin, 2009), it’s unusual. Though it is argued that cases provide an opportunity for generalisation.

Case study research seen as better for generating hypotheses than theory building (Flyvbjerg, 2006) – depending on the definition of theory. Options include

  1. Explanatory and predictive of cause & effect and thus can direct action
  2. pragmatist perspective that has theory/practice intertwined. People generate theories to seek meaning in practice. A form of personal theory building.

Theoretical relevance enhanced if existing theories are integrated. Through which contribution can be made through new interpretation of data. “Existing theory should be seen as an integral part of the case” (p. 1116)

What forms of data are acceptable?

“case study may rely on multiple sources of evidence and be practiced as multi-method research (Denzin & Lincoln, 1994)” (p. 1117) as long as it helps answer the research question. i.e. it can use quantitive research methods. Apparently a surprise to some.

Tight (2012) found only 5% of 440 published articles in 15 higher education journals used a mixed quantitative/qualitative methodology.

What does analysis stop?

Outcomes of any analytic technique will depend on intentions, background knowledge, cognitive processes, mindset etc. Hence analysis is recursive.

Minimise the time between collecting data and writing the research account. “disciplined writing seems to be the most essential part of the analytical process” (p. 1118).

No genuine endpoint.

What makes a quality case study?

Better to engage wider theories than just describe practice. But not sufficient

Quality case research:

  • requires imagination (Dewey, 1938)
  • requires creativity (Morse, 1995)
  • must bring the reader as close as possible to the experience (Fossey et al, 2002)
  • provide conceptual insight (Siggelkow, 2007)
  • should be believable, which requires coherence and provide new theory and instrumental utility (Eisner, 1991)
  • “potential to create an impact on the field of practice” (p. 1118)
  • have something important to say
  • well structured and clearly writte

Argues that a case study should enable someone to learn from it.

Author explores the impact (from this measure) of one of his case study publications. Somewhat sobering results.

Conclusion

Summarises the key points to be “attentive to” and “cautious of” against the four challenges and some more general comments e.g. research must fulfil its purpose and this needs to be known before time to help align process and outcomes.

Does make the point that case study is a form of learning and that this can be seen in daily practice, more so than in research articles.

Also points out that publications on case study methods are “often complex or underpinned by unstated assumptions, following a procedure is never straightforward” (p. 1121). Case study research does allow you to learn from experience, so these methods should be seen as guidelines.

References

Baxter, P., & Jack, S. (2008). Qualitative case study methodology: Study design and implementation for novice researchers. The Qualitative Report, 13(4), 544–559.

Harland, T. (2014). Learning about case study methodology to research higher education. Higher Education Research & Development, 33(6), 1113–1122. doi:10.1080/07294360.2014.911253

Choosing a research publication outlet: Part II

My name is David. I’m an academic. It’s been 8 years since my last journal paper (but what a paper it was, thanks Shirley).

In my defence, for three of those years I was finishing the PhD, 1 year finishing a GDL&T, three years getting used to a new discipline, and generally just cynical about the whole journal process, and not having much I considered useful to say. But that has to change, otherwise I’ll have some explaining to do. This year is about writing some journal papers. So where should I publish?

I asked and answered a similar question in 2009, hopefully this time I might actually go the next step and write something.

(Note: I start the writing process by getting to know the destination. The theory being that a paper is more likely to get accepted if it fits the requirements, aims, tone etc of the destination. I already have a few ideas re: papers, finding a suitable destination is the first task.)

What’s changed since 2009?

I’ve officially moved from the Information Systems discipline to Education. But given I was originally focused on educational technology, the journals identified in that original post are still suitable.

I’m at a new university that has recently introduced awards for journal articles “designed to encourage .. researchers to make strategic decisions about the journals in which they seek to publish, in order to help them get the best return on their research efforts and as a crucial step in improving the research performance of … as an institution”. Obviously linked to the on-going ERA. This is judged largely on the Scopus SNIP (Source-Normalized Impact per Paper) index.

Obviously there is more to research impact than the journal’s “citation potential”, but it’s the current measure of choice. At the very least I’ll have to consider it.

Updating the list

For the following list I’ve

  1. Taken a list of journals mentioned in the original post.

    Have excluded a couple I no longer deem interesting.

  2. Added any other journals I think potentially relevant.
  3. Identified/updated the following information about the journals
    • Ranking as per the Oz government approach.
      This was used in the 2009 ERA, but wasn’t used in the 2012 exercise.
    • SNIP
      Provided by this search form.
    • h5-index and h5-median.
      In my travels I’ve discovered various other types of rankings such as Google scholar’s method which generates the following for

    • Are the papers open or closed?

      This remains an important personal consideration.

    • Position on article copyright.

      i.e. is it ok for me to put a copy of the article on my personal site? A potential proxy for being open.

    • Max paper size.
    • # Issues per year.
    • Turnaround time on review.
    • ERA FoR
      Using John Lamp’s interface to the ERA data.

Once I have that information, the plan is to look more closely at the current editorial directions and the types of papers being accepted for publication in each of the different journals.

The list and observations

The complete list is available as a Google spreadsheet.

The top 10 journals ordered by SNIP are

  1. Computers and Education
  2. Internet and Higher Education
  3. Studies in Higher Education
  4. Educational Technology Research & Development
  5. IRRODL
  6. British Journal of Educational Technology
  7. Educational Technology & Society
  8. Higher Education Research & Development
  9. Teaching in Higher Education
  10. AJET

The top 10 journals ordered h5-index as per Google Scholar are

  1. Computers and Education
  2. British Journal of Educational Technology
  3. Educational Technology & Society
  4. Internet and Higher Education
  5. IRRODL
  6. Educational Technology Research & Development
  7. Studies in Higher Education
  8. AJET
  9. Higher Education Research & Development
  10. Teaching in Higher Education

Interesting to see BJET and ET&S climb the ladder in terms of h5-index.

Could I win an institutional award?

This is not the purpose for writing, but it’s an interesting exercise in exploring how level the playing field is between various disciplines (other than the sciences).

The only results I can find for the institutional publication awards had the following SNIP values

  • 1st – 3.154
  • 2nd – 3.01
  • 3rd – 2.181
  • Student award – 3.273
  • Special mention – 8.2

    An article in Nature that was in the wrong time frame.

Computers and Education is by far the highest ranked of these journals using either SNIP of h5-index. C&E’s SNIPP value was 3.29. Meaning a first place, at least for the above period.

Internet and Higher Education has a SNIP value of 2.55, making 3rd possible.

BJET, IRRODL and AJET (the most familiar journals to me) have SNIPP values of: 1.71, 1.77 and 1.16 respectively.

While a sample size of one isn’t great, it appears possible but only just if I focus on higher education or general educational technology journals.

The couple of teacher education related journals that get a mention either don’t have a SNIPP or have one just above 1. Not likely to please the “impact police”. Of course, I didn’t go searching for more general teacher education journals.

Other misc. observations on journals

Computers and Education

Apparently has a “liberal copyright policy”. Which appears to permit posting to open websites but doesn’t exactly trumpet that position, e.g. doesn’t appear to be mentioned on this page.

There is a choice to publish an article as open access, but there’s a fee required. $1800!!!!!! But as an author you do get a 30% discount on Elsevier books.

Recommends a clearly defined article structure. Wonder how many published articles follow that? Apparently not many.

Has a “turnaround time” of around 10 weeks.

Internet and Higher Education

Audience – “faculty, administrators, and librarians charged with the responsibility of fostering the use of information technology and the Internet on their respective campuses.” Potentially a good fit.

Seems Elsevier have a standard structure they like. Same here as Computers & Education.

About a 5 week turnaround time.

Educational Technology Research and Development

Two month turn around claimed.

The fee for your article being open access is $USD3000!!!!

BJET

$USD3000 cost for open. Not real clear about whether it’s okay to share versions of articles via personal websites.

So where might I publish?

No clear winner, but some thoughts include

  • Publishing in Computers and Education would satisfy the “impact police” but the initial topic I have planned may not fit well.
  • My preference is in open access journals and IRRODL is perhaps a closer fit for what I’m thinking of writing.
  • Internet and Higher Education is a closed journal, but also a reasonable fit and it promises the fastest decision time.

Time will tell.