The great website move of 2008

Update: On 21 Oct, 2008 I redirected my old website here. There are links to the old site included below. If you click on them, you’ll be brought back to this blog.

As 2008 comes to a close I have a fairly strong external motivation to move my website from its current location to one without a connection to any specific institution. i.e. a true personal website. This tells some of the plans of the move.

I’m somewhat reluctant to make this move as I’ve had the current website location since 1994. The Wayback Archive only seems to go back to around 1997 but you can see the evolution of the site

All of these have been hosted on a UNIX box (first Linux and currently Mac OS X) sitting under a desk in my office at the University I work(ed)? for. A very techie, roll your own sort of thing.

The Web has moved on. Social media, Web 2.0, the cloud, software as a service (SaaS) are all symptoms of the continual climb up the abstraction layer which is starting to change some of the fundamental assumptions of the practice around information technology.

As someone who likes what they see with these changes, especially with what they mean for the practice of organisational information technology, it’s time to practice what I sometimes preach. To use the cloud and social media for my site. To escape the varagies of organisational information technology.

The plan

The plan is to use this blog as my personal home page. To leverage off the community around WordPress to do what I used to do, only better and with a greater level of freedom, ease of use and with potential of greater capabilities.

Initially, I plan to move all of the information (at least most of it) off the current site and place it as “pages” on the blog. Some early moves have already begun.

Once most of that information is moved, time to redirect Google and other search engines away from the old site to the new one.

Maintaining my google ranking

Now comes the ego thing.

My name is David Jones. A fairly common name with a range of famous connections. For example,

  • The David Jones department store in Australia.
  • Various famous folk, Davy Jones from the Monkeys etc.

Currently, if I Google “david jones” my website is 7th on the list (though with the wrong hostname). The first 3 are related to the David Jones depatment store (I’m guessing they pay google a bit). Then there is

Obviously, my ego would like that to be maintained.

The Anatomy of a Design Science Paper: A Research Note

(A Suggested Structure for a Design Science Article or Thesis)

Shirley Gregor and David Jones

This note is a supplement to Gregor and Jones (2007). It is a “work-in-progress” and the authors welcome input.

When we prepared our 2007 paper it originally had an appendix that showed a suggested structure for a journal article or thesis reporting an information systems design theory (“design-science-type knowledge” using more general terms). When writing a design science article it is necessary to include additional material that is not strictly part of the design theory/knowledge. That is, it is necessary to add the components that “position the article so that it is worthy of publication, showing why the topic is significant, that the work is credible, and that it is an advance on prior work (demonstrating novelty and a contribution to knowledge).

The reviewers were not comfortable with the appendix, in part because it was thought that it might appear too formulaic and be used inappropriately without reflection. Thus, the appendix did not appear in the published article. Subsequently, we have come to believe that colleagues might find it useful to have such a structure presented, at least for the purpose of discussion. Each of us has struggled with presenting design work in articles and having these articles accepted for publication. Portions of our work over a number of years remain unpublished, partly we think because we did not understand how to position and present the work satisfactorily. There has also of course been the problem that design-type work was seen by some as not being legitimate research in Information Systems. This problem has motivated our papers about theory and design theory.

Table 1 is an update of the original appendix and shows an view of the idealized structure of a paper (article or thesis) that presents design knowledge, with the eight components of a design theory as in Gregor and Jones (2007). Variations on this idealized structure would be expected in practice, both in content and the order of presentation. Following the table is an analysis of an existing article in a leading journal to examine whether it conforms to this structure.

It should be noted that this structure does not vary greatly from a traditional article or thesis (for example, see Perry 1998). The difference is that Chapters 5, 6 and 7 (or similar) in a design science paper show how an artifact was (or is to be) constructed, reflect on the principles underlying its construction and possibly advance hypotheses about its behaviour. In contrast, in a non-design article, these chapters would describe the results of an experiment, survey or case study in a standard empirical study. In both cases, these chapters are presenting the “evidence” that is the basis for the knowledge claims in the article.

In our discussion above we have indicated that what we term “design theory” may not be generally recognized as “theory”, but may be given some other label such as “design-type knowledge”. We appreciate that many in our community are uncomfortable with using the word “theory” in connection with artifact design. While this issue remains open, we ourselves prefer to use the word “theory”. In this we follow Gregor (2006), who distinguishes theory for design and action as a specific and recognizable category of theory (Type V theory).

It can be seen from the example given in a following section that the structural elements of a design theory can be detected in an article that does not even claim to be “design science”. We would argue that well-developed design papers that have been accepted in leading journals are likely to have many of the aspects of design theory (in our terms), rather than presenting only the description of a single artifact. The paper by Chiang and Mookeerjee (2004) does not just describe the artifact construction, but gives underlying background theory to show why it works and reflects on how applicable the design might be across a number of contexts. Thus, the generalization required of a theory is provided, even though it does still have a relatively narrow scope. These elements of reflection on underlying justificatory knowledge and the limits of applicability of the artifact need to be present to show that the research is in fact research, and not “consulting” or routine industry problem solving. Of course the demonstration of novelty, significance and credibility is also required for the paper to be accepted as a research contribution.

Table 1 Idealized structure of a design science paper

Section/Chapter Contents Notes
1. Introduction The purpose or goal and scope of the theory. ISDT component
  Definition of constructs ISDT component (2)
  Motivation, significance, outline of article. Similar to conventional articles
2. Literature review/background What was known about these systems before the work in the article was begun (that is, prior design theory), including similar existing systems. Similar to conventional articles
  Problems, gaps in knowledge and reasons for a new theory being needed. Similar to conventional articles
3. Research methodology Description of design science/constructive research approach. Many existing articles in Software Engineering and Computer Science omit this component. See Nunamaker et al., 1991; Walls et al. 1992); March and Smith 1995); Hevner at al. 2003; Gregor and Jones 2007.
4. Justificatory knowledge Theory from IS and natural and social science that informs the design theory. The placing of this component is debatable. It could be in Section 2 or 5. ISDT component (7)
5. Specification of the Designed artifact Meta-requirements – the purpose or goal of the class of artifacts addressed by the theory ISDT component (1)

  Process by which the designers arrived at their solution. This component could be optional. However, it might assist in demonstrating credibility. For example,  it could describe an iterative design with intermediate test stages and so on (the “search” process).
  Principles of form and function ISDT component (3)
  Consideration of artifact mutability ISDT component (4)
  Principles of implementation ISDT component (6)
  Testable design propositions ISDT component (5)
6. Instantiation Description of any working system or method in use
  ISDT component (8) This material could be partly combined with the previous section for exposition.
7. Evaluation Describe tests of system/ method in use. The tests could show performance, functionality, user acceptance and so on.
  Evaluate against criteria for design science/constructive research. See Burstein and Gregor (1999), March and Smith (1995), Hevner et al., (2004).
8. Discussion and conclusions Summarize work and findings, discuss limitations, establish significance. Similar to conventional articles

Example Design Article Structure (Chiang and Mookeerjee, 2004)

This paper was used as an example in Gregor and Jones (2007, p. 324) as presenting a design theory for a method, although the article did not explicitly present itself as being “design science”.

Analysing the Chiang and Mookeerjee paper shows a structure as follows.

1. Introduction (3 pages)

The significance and importance of incremental development is established.

It is claimed that existing research does not give much guidance on how to manage incremental development (that is, a “knowledge gap is identified).

States that the paper advances a “fault threshold policy for incremental improvement” (p. 4) (that is, the purpose of the design theory).

The term used in the paper are also explained (constructs).

Gives an overview of the policy problem, with a diagram (showing the scope of the problem addressed). Our experience with other articles of this type suggests that this is an important aid to readability, as giving a practical, easily understandable example of the nature of the artifact being dealt with helps to “ground the problem for the reader early on.

2. Literature Review (1.5 pages)

The problem is shown as connected to work on coordination and team integration (theory base and justificatory knowledge).

Prior research on improving software team productivity and related work is reviewed.

This section concludes with a statement that methods to manage software faults in an evolving system have not yet been prescribed.

Missing – Methods Section

There is no description of what methodology was employed in the study. This omission is observed in many design science type papers.

3. Fault Threshold Policy (5 pages)

This section provides an analytic description of the new software fault threshold policy.

It gives a detailed description of the way the policy operates (principles of form and function). An example is provided (a hypothetical instantiation).

Missing – principles pf implementation?

Not a great deal of detail is given on how to build a concrete version of this abstract policy in specific projects. An example is given where the formulae in the policy are applied to an imaginary scenario. It is stated that it might be necessary to build some randomness into the model in a real-life project and this is left for further work.

4. Learning effects (3 pages)

The behaviour of the policy in the context of multiple construction cycles, as learning benefits occur, is investigated (artifact mutability).

This section also includes a sub-section with a comparison of the new policy with another method, a “fixed-release method. It could be questioned whether this sub-section might have been placed differently (eg in the evaluation section), as it appears more to do with a comparative evaluation than learning effects.

5. Simulation experiments (3 pages)

This section provides an evaluation of the new method.

It is stated that this section has three purposes:

  1. that the new method/model makes accurate predictions;
  2. that it is robust with respect to underlying assumptions;
  3. that it is applicable to heterogeneous systems (that is, it has generalizability).

An overall assessment of the policy is provided and future directions for research are identified.

6. Summary and conclusions (.75 page)

A summary of the paper is given and it is speculated that most organizations would benefit from frequent system testing and use of the model, as it has economic benefits ( a testable hypothesis).

Note that there are 18 pages in the body of the paper in total. Almost a third of the paper describes the software fault threshold policy itself.

References

Burstein, F. and S. Gregor (1999) “The Systems Development or Engineering Approach to Research in Information Systems: An Action Research Perspective”, in Proceedings of the 10th Australasian Conference on Information Systems, B. Hope and P. Yoong, (eds.), Victoria University of Wellington, pp.
122-134.

Chiang, I. R. and V. S. Mookerjee (2004) “A Fault Threshold Policy to Manage Software Development Projects”, Information Systems Research, (15)1, pp. 3-21.

Gregor, S. and Jones, D. (2007). The anatomy of a design theory. Journal of the Association of Information Systems, 8, 5, Article 2, 312-335.

Hevner, A. et al. (2004) “Design Science in Information Systems Research”, MIS
Quarterly, (28)1, pp. 75-105.

March, S. T. and G. F. Smith (1995) “Design and Natural Science Research on Information Technology”, Decision Support Systems, (15), pp. 251-266.

Perry, C. (1998). A Structured Approach for presenting theses. Australasian Journal of Marketing, 6(1), 63-85.

The emergence of design research in IS in North America

Continuing previous posts looking at/summarising recent design research literature this post gives a summary of the following paper

William Kuechler, Vijay Vaishnavi, The emergence of design research in information systems in North America, Journal of Design Research, 7(1), 2008, pp 1-16

As with previous posts, this is not intended to be a generally useful post for anyone but me.

This paper is published in a journal associated with the broader design research discipline. It attempts to explain the origins of Information Systems Design Research (ISDR) to that broader audience and identifies constraints/limitations of that practice and suggests ways forward.

Origin of IS and emergence of ISDR

IS origins in the raise of computers in the 1950s and the academic Management Science communities move into studying management information systems. Design not initially considered a relevant topic.

Greater significant of MIS arose with failures in 1960s. An early key paper (Ackoff, 1967) on MIS’s primary criticism was of the design and design criteria for the systems. Early MIS program arose…..continues with brief history of the IS field.

Suggests that the rise of design as a research direction in MIS was due to limited numbers of MIS academics in the 1980s. Moves to “boot strap” academics from other fields (e.g. computer science and engineering) brought new folk into the field who came with a strong design tradition.

A quote that attracts my attention

Perhaps more importantly, during that time research about design, built on the conceptual basis set forth in Nunamaker et al. (1991), Walls et al. (1992), and others, continued to formalise the distinctions between design research and empirical research as practiced in the natural and management sciences (March and Smith, 1995). Following Nunamaker et al. (1991), the DR theory papers were ecumenical, suggesting that IS research in total was a multi-methodological endeavour and proposing an important place for design research alongside traditional empirical studies.

The on-going positioning of design research as separate from “traditional” research is something I’m not sure about. I wonder if the origins of design research, as having to battle an established pre-dominant “paradigm”. A paradigm, which in turn, was constrained by a limited view of research that only allowed the development of a separate paradigm. If this limited view of research is rejected, then the “separation” of design and “traditional” research disappears.

It’s description of Hevner et al (2004) is

The paper expresses the view of the nature and place of IS design research that we believe is currently held by a majority of those practicing in the field; this view is self-consciously pre-paradigmatic and so Hevner et al. have chosen to present its core concepts as guidelines that circumscribe the domain without fully explicating it.

Current state of ISDR

Another interesting quote, which connects to some of my reservations about ISDR, is

many of the earliest advocates for and most widely published proponents of design in IS research have backgrounds in computer science and engineering. It is no surprise then that the current majority view of IS design research closely adheres to the engineering model (Eekels and Roozenburg, 1991).

I wonder if this emphasis limits the perspectives enshrined in current ISDR.

They describe the current ISDR worldview as including the following

  • IS aims to improve the efficiency and effectiveness of organisations. IS research aims to improve the ability of information systems to achieve their goal.
    I have some qualms about the potential “efficiency” emphasis that this embodies. I can see this coming through in some of the practices of IS within organisations.
  • IS research looks at the confluence of people, organisations and technology and thus needs to distinct and complementary paradigms to improve information systems: behavioural (natural) science and design research
    This seems a big leap. There is certainly a need for a broad array of knowledge, but that this necessarily means distinct and complementary paradigms…
  • Design involves the building and evaluation of artefacts intended to meet business needs identified during behavioural research.
  • Design research is different from design because it addressed “important unsolved problems in unique or innovative ways” or “development of more effective and efficient solutions to previously solved problems”. Design research addressed wicked problems.
    Does this mean that the design of ISes in organisations does not include addressing wicked problems? I tend to feel most do.
  • Explaining the theoretical basis for design may lag implementation.

ISDR is explained as having

  • epistemology – “obviously pragmatist”
  • implicit axiology – utilitarian and pragmatic – “methodologies do not describe any external reality . . .their scientific merit should be evaluated on the basis of their practical value” (Iivari, 1986)
    I wonder in what context this Iivari quote was made.

Does it imply that all ISDR does connect with these views. Are there strengths/weaknesses of this view? What are the alternatives? Who’s arguing an alternative?

Mentions Hevner et al’s (2004) 7 guidelines which “are strongly bound to commonsense notions of business utility and forcefully promote the strongly felt need of the entire IS academic cohort for legitimacy through
relevance”.

The 7 in a quick summary are

  1. Design research in IS (ISDR) produces artefacts
  2. ISDR must be relevant.
  3. The design of the ISDR artefact must be rigorously evaluated.
  4. ISDR must provide a novel contribution.
  5. ISDR must balance rigour and relevance.
  6. An ISDR contribution must be functional.
  7. Design research results must be communicated to both technical and management oriented audiences.

Again mentions that ISDR is pre-paradigmatic – meaning a “lack of common understanding of the definition and scope of design research in IS”

The lack of a common understanding of the definition and scope of design research in IS was patently evident at the most recent IS design research conference (DESRIST, 2007).

Discussion was whether or not ISDR was research with

  • design as a topic of investigation
    A broader topic and can include a range of any topics including psychological attributes of good designers or design education.
  • design as a method of investigation
    More narrowly defined to the use of IS design techniques to construct an improved IS artefact. Many experience IS design researchers focus on this

Authors express a concern about the premature convergence on the definition of ISDR to be on the “constructivst” design research methodology. The aim to change this and suggest that ISDR should be expanded to include the other aspects while retaining a focus on relevant IT artefacts.

This view seems to have the broader IS research field, within which ISDR sits. They are then wanting to make ISDR encompass all aspects of design.

An alternate view is that the aim of IS research is to improve the design and use of information technology in society. To do this research seeks to generate knowledge which answers/solves important questions/problems. One of the important classes of questions/problems to solve is those associated with design knowledge – i.e. how to do something. This knowledge is expressed in design theory.

They mention something like this further down the paper

That fundamental design issues within IS – what an information system design is and how best to effect one, and to understand, represent and teach the design process – are a topic of current interest is well brought out in Bajaj et al. (2005), ‘Systems analysis and design: should we be researching what we teach?’

The case for brodening ISDR

The nature of IS

Define IS as a broad discipline “that takes responsibility for the effective implementation and management of ICTs within organisations”. Divides implementation (development of systems) from management (responsibility for day-to-day operations). Does recognise that the latter includes a strong design component.

I have a couple of problems with this

  • Limits IS to just organisations, not broader societal uses.
  • Not comfortable with the artificial separation between implementation and management – makes me think of Snowden’s distinction.

Talks about the impact of social/organisational issues on the implementation/operation of IS and how this affects design and also differentiates it, somewhat from computer science.

Many important design decisions are made based on socially constructed notions of cost, effectiveness, need, organisational culture, etc. and so the ‘designerly’ (non-scientific in the sense of being difficult to derive logically and rigorously from first principles) aspect in IS is more influential than in computer science and many engineering disciplines.

Distinguishes between two broad “schools” of design work

  • US based – which due to its orgins within business schools has been separate from other design literatures and focuses more on design as rigourous, objective, formal and somewhat detached.
  • European – broader recognition of subjective aspects of multi-stakeholder views of projects (e.g. soft systems methodology) and with a greater emphasis on the business environment and client interactions in IS design.

A survey of other design fields suggests to them

  • All design practices share a large common grounding – a meta level
  • Much IS can learn from all of those
  • Each field stresses different topics and there are different details in the material aspects of the designed artefacts
  • Each field has unique elements that are important only to that field

The differences in IS are

  • relative invisibility of the effects of its artefacts until after deployment
  • mutability of the organisational environments in which its artefacts are deployed.
    Interestingly, it appears that it is the mutability of the IT artifact that many of the IS researchers are currently quoting, at least the ones I’m hearing. This is different from the organisational environment – it’s possibly more the fact that the organisational environment requires mutability and the IT artifact can, increasingly, deliver this.

Open issues on ISDR

Sees the requirement that ISDR requires artefacts be designed, built and evaluated as major limit. Should move to the broader conception of design.

Positions the emergence of this view with

  • current IS-wide perceived need for increased relevance in IS research (Benbasat and Zmud, 1999; Orlikowski and Iacono, 2001).
  • The desire to distinguish ISDR from behavioural research
  • An over-reaction to the denigration of artefact-related research in IS
  • Call from organisational researchers in IS to distinguish the IS academic field from other forms of organisational studies.

Illustrate breadth of design research by looking at articles from two broad design research journals, two design area specific journals and a few others.

Develop a table listing representative (not exhaustive) topic areas, outputs of research in those topic areas and references. Would be illustrative to look at this list and determine whether or not it can be mapped to Gregor’s five types of theory (assuming that design is common enough that it can be applied to Gregor which is specific to information systems).

A summary of the table

  • user-designer communication – group communication, effects on requirements gathering and system design
    Output is study results and methodologies. It isn’t clear what study results are. However, methodologies would probably fit with a design theory.
  • Training of effective designers
    Study results and course curricula – does this become design theory?
  • Design project management
    Case studies, principles and techniques – design theory
  • The nature of reliable designs
    Study results, taxonomies – Type 1 theory
  • Adoption and adaptation of design ideas from other design disciplines

    Case studies, principles, frameworks – Type 1 theory

Much of it appears to be able to go into the different types of theory.

They suggest that the above outputs are not IT artefacts, and are hence evidence of a need to move beyond the IT artefact emphasis of design research. Of course, I’d suggest this comes back to Gregor’s theory of theories.

A proposal

The talk about resistance to the broadening of design research from IS design researchers for fear of losing the progress made in its acceptance.

This can be accomplished simply by redefining ISDR outputs as ‘IS design practice relevant artefacts or knowledge directly useful in the construction of IS design practice relevant artefacts’. This position is similar to but expands on Iivari’s proposals for IS research as the exploration of ‘meta-artefacts’ (Iivari, 2002, 2007).

Appears to have some connections to the idea of connecting this with the theory of theories.

Expanding the allowable outputs will allow the richness from other areas while retaining the focus.

Quote Whinston and Geng (2004) as arguing the case for IS being inclusive, rather than exclusive.

Concluding remarks

Hevner et al (2004) give a solid definitional base point from which to evolve. But it lacks any notion of a grounding meta-level. It is still a work in progress.

They wish it to expand in scope to provide the IS field with the full range of benefits that DR has provided for older design-based fields. Suggest ISDR can mature more quickly be taking relevant research and philosophical discussion of older design fields – especially

  • relation to theory in design fields
  • relevance of ‘scientific’ methodologies, ontologies and epistemologies to design (Cross, 2006; Cross et al, 1981)

If the core of information systems is the artefact then surely its heart is increasing understanding of every aspect of the design process by which these artefacts are brought into being.

References

Bajaj, A., Batra, D., Hevner, A., Parsons, J. and Siau, K. (2005) ‘Systems analysis and design: should we be researching what we teach?’, Communication of the AIS, Vol. 15, pp.478–493.

Cross, N. (2006) Designerly Ways of Knowing, London: Springer.

Cross, N., Naughton, J. and Walker, D. (1981) ‘Design method and scientific method’, in R. Jacques and J.A. Powell (Eds), Design: Science: Method, Guilford, UK: Westbury House, pp.18–29.

Whinston, A. and Geng, X. (2004) ‘Operationalizing the essential role of the information technology artifact in information systems research: grey area, pitfalls and the importance of strategic ambiguity’, MIS Quarterly, Vol. 28, No. 2, pp.149–159.