Daily Archives: October 25, 2008

A Paradigmatic Analysis of Information Systems As a Design Science

The following is a summary of and reflection upon

Juhani Iivari, (2007), A Paradigmatic Analysis of Information Systems As a Design Science, Scandinavian Journal of Information Systems, 19(2):39-64

Reflection

This paper is somewhat similar, at a very abstract level, to one I’ve been thinking about. However, it’s told from a different perspective, with a different intent and different outcomes (and probably much better than I could). There is enough difference that I think I can still contribute something.

One aspect of that difference would come from the fact that the foundation of my thoughts will be Shirley’s types of theories which Juhani identifies as being more complete than the framework he developed.

Questions and need for further thinking

In the epistemology of design science section the author outlines a framework to structure IS research. Somewhat equivalent to Shirley’s theory of theories. Does this structure belong in the “epistemology” section or the “ontology” section?

The question of truth value and truthlikeness is something I need to read on further

The 12 theses

The author summarises his view in 12 theses, I’ve listed them below with, where it exists, some early indication of some of my problems and/or thoughts. At least those that currently exist.

  1. Information Systems is ultimately an applied discipline.
    I agree. Juhani mentions the problems with the term “applied science” in the first footnote.
  2. Prescriptive research is an essential part of Information Systems as an applied discipline.
    Agreed. I would add that there has been significantly too much focus on the other forms of research – descriptive and explanatory – at the expense of prescriptive research. A flaw that has negatively impacted on the IS discipline.
  3. The design science activity of building IT artifacts is an important part of prescriptive research in Information Systems.
    I agree, however, I don’t see it as the main output or purpose of prescriptive research in information systems. At least not any more than building a quantitative survey is the main contribution/output of descriptive/explanatory research. For me, building an IT artifact is a method to test the theory being developed.
  4. The primary interest of Information Systems lies in IT applications and therefore Information Systems as a design science should be based on a sound ontology of IT artifacts and especially of IT applications.
    There’s a glimmer of agreement here. Not sure how I far that goes. I see IS as having a main interest in how IT applications are used/impact organisations/groups/people. For me the focus on just IT applications is computer science.
  5. Information Systems as a design science builds IT meta-artifacts that support the development of concrete IT applications.
    Agree, but with meta-artifacts expressed as information systems design theories.
  6. The resulting IT meta-artifacts essentially entail design product and design process knowledge.
    Yes.
  7. Design product and design process knowledge, as prescriptive knowledge, forms a knowledge area of its own and cannot be reduced to the descriptive knowledge of theories and empirical regularities.
    Not certain about this one. Mention a bit more below.
  8. Constructive research methods should make the process of building IT meta-artifacts disciplined, rigorous and transparent.
    Agree.
  9. Explication of the practical problems to be solved, the existing artifacts to be improved, the analogies and metaphors to be used, and/or the kernel theories to be applied is significant in making the building process disciplined, rigorous and transparent.
    Agree, but need more time to think about whether this is complete.
  10. The term ‘design theory’ should be used only when it is based on a sound kernel theory.
    Probably disagree, see more discussion below. Need more thought.
  11. Information Systems as a design science cannot be value-free, but it may reflect means-end, interpretive or critical orientation.
    Yes agree. I wonder if there are any other additional ethical perspectives.
  12. The values of design science research should be made as explicit as possible.
    Yes.

What distinguishes design science from IT development practice

Juhani suggests the use of rigorous constructive research methods as what distinguishes practice from design science. Which leads him to admit that if a practitioner uses a constructive research method, then they are doing research.

I find this vaguely troubling

I would suggest that need to move to the output. My view assumes that an artifact is not a sufficient output for design science. If you accept that the expected output of research is the generation or testing of theory (knowledge). Then the output of design science should be design theory (though I don’t like the phrase design science). An artifact can be part of the design theory, but not the sole output.

An IT practitioner will not (typically) generate design theory. They generate artifacts. A researcher aims to go the next step and generate design theory.

Does DSR have a positivistic epistemology

Juhani argues that action research and design science research are very different in terms of history, practice, ontology and epistemology. As part of this he suggests that DSR (especially from engineering and medicine) is based on positivistic epistemology and that argues against Cole et al that it might be possible for some applications of DSR around IS within organisations to have a different epistemology.

This argument is based on his work on the paradigmatic assumptions regarding systems development approaches which found that al 7 IS development approaches shared a fairly realistic ontology and positivistic epistemology.

However, earlier in the paper he argues that systems development approaches are not a good match for use as constructive research methods. Hence how can an analysis of systems development approaches be used to argue anything about DSR? Yes, there is likely to be some strong overlap, but it doesn’t seem to be strong evidence.

Also, simply because historically these systems development approaches (and one assumes IS developers/researchers) have held this particular view that this excludes some practice of DSR which has a different epistemology.

Test artifacts in laboratory and experimental situations as far as possible

It is suggested that action research can be used to evaluate artifacts and provide information on how to improve these artifacts. However, Juhani also suggests that design science artifacts should be tested in laboratory studies as far as possible.

I believe this closes off a major fruitful way of developing design theory. An approach that ties very much into Juhani’s first major source of ideas for design science research – practical problems and opportunities. DSR that uses action research as a methodology to not only evaluate but also inform the design of an artifact/ISDT can lead to very fruitful ideas.

Does a design theory need a kernel theory

Juhani says yes. If we do without there is a “danger that the idea of a ‘design theory’ will be (mis)used just to make our field sound more scientific without any serious attempt to strengthen the scientific foundation of the meta-artifacts proposed”.

There is something to this, but I also have some qualms/queries which I need to work through. The queries are

  • Situations where descriptive theory has to catch up with prescriptive theory.
    i.e. physics of powered flight being figured out after the Wright brothers flew.
  • Situations where descriptive theory is closing off awareness or insight.
    Someone deeply aware of descriptive theories will have a set of patterns established in their head which may limit there ability to be aware of the situation or envision different courses of action (i.e. inattentional blindness aka perceptual blindness).

    Awareness of a situation, or the ability to avoid established descriptive theories may highlight new and interesting solution (yes, I think this occurence might be rare).

There is an argument to be had about the difference between the final version of the ISDT and its formulation. It may be that a complete/formal ISDT does need to have a kernel theory or two. However, it may not have been there at the beginning.

For example, the work that forms that basis of my design theory for e-learning started without clearly stated and understood kernel theories based on formal descriptive research. However, a very early paper (Jones and Buchanan, 1996) on that work included the following

It is hoped that the design guidelines emphasising ease of use and of providing the tools and not the rules will decrease the learning curve and increase the sense of ownership felt by academic staff.

It’s not difficult to see in that statement a connection with diffusion theory and TAM. Descriptive knowledge that has informed later iterations of this work and diffusion theory certainly gets a specific inclusion as a kernel theory in the final ISDT.

What’s the kernel theory for the IS development life cycle

In footnote 7 the author writes that Walls et al (1992) “suggest that the information systems development life-cycle is a design theory, although I am not aware of any kernel theory on which it is based.”

I agree, in so much as I’m not aware of an clear statement of the kernel theories that underpin the SDLC. I also think that the absence of such a clear statement is a potential short-coming.

There is a world view embodied in the SDLC. For example, I believe that the SDLC assumes that the world fits into the simple or complicated fields of the Cynefin Framework and is completely inappropriate when used in other types of systems – even in the complicated field it can be difficult. Agile/emergent development methodologies appear to be a better fit for the complex section of Cynefin.

Which raises the question, is there value in going back and developing an ISDT for the SDLC which makes clear the assumptions that underpin it by providing kernel theories.

Irreducibility of prescriptive knowledge to descriptive knowledge. Juhani states, that since most IT artifacts aren’t strongly based on descriptive knowledge

This makes one to wonder whether the IS research community tends to exaggerate to the significance of descriptive theoretical knowledge for prescriptive knowledge of how to design IT successful artifacts. In conclusion, in line with Layton (1974) I am inclined to suggest that prescriptive knowledge forms a knowledge realm of its own and is not reducible to descriptive knowledge.

That seems to be a rather large leap to me. The questions it brings to mind include

  • Does the absence of strong links mean its irreducible?
    I don’t understand how Juhani has gotten from “most IT artifacts have weak links to descriptive knowledge” to “prescriptive knowledge is not reducible to descriptive knowledge”.

    Not to suggest it’s wrong. It’s just that I’m not smart enough to make the conenction, yet.

  • Is there more to this statement that meets the eye?

    Despite this weak reliance on descriptive theories people design reasonably successful IT artifacts.

    • What types of artifacts are reasonably successful? Who says? Why are they successful?
      There’s a large amount of literature about the failure of large scale information systems. Is that failure due to the weak reliance?

      We can all point to systems that are being used by people to perform tasks. But does use mean success? Does it mean that the need of the folk is strong enough that they will adapt and work around the system enough to do the task they wish to achieve? Is success generating the best possible system? How do you evaluate that?

      Perhaps the success of some systems, even with weak reliance on descriptive knowledge, simply proves how adaptable people are.

    • Does weak reliance, mean none?
      The example I give above shows a situation where without knowledge f a specific type of descriptive knowledge (diffusion theory) a practitioner was already aware of something very similar, a need to go that way. An example of the relevance/rigor gap?

If you haven’t noticed, I lost my way in the above. Need to come back to it. I feel there is more to unpack there.

Summary

Abstract

Discusses

  • ontology – suggests ontology of IT artifacts, draws on Popper’s three worlds as a starting point
  • epistemology – emphasizes the irreducibility of the prescriptive knowledge of IT artifacts to theoretical descriptive knowledge, suggests a 3 level epistemology for IS – conceptual knowledge, descriptive knowledge and prescriptive knowledge
  • methodology – expresses a need for constructive research methods for disciplined, rigorous, transparent building of IT artifacts as outcomes of design science research (so as to distinguish design research from simply developing IT artifacts), also discusses connections between action research and design science research.
  • ethics – points out IS as a design science cannot be value free, distinguishes three ethical positions: means-end oriented, interpretive and critical

of design science.

Introduction

Computer science has always been doing design science research. Much of the early IS research focused on systems development approaches and methods – i.e. design science research

But the last 25 years of mainstream IS research has lost sight of these origins – due to the “hegemony of the North-American business-school-oriented IS research” over leading IS publication outlets.

The dominant research philosophy has been to develop cumulative, theory-based research to be able to make prescriptions.

A pilot analysis of practical recommendations of MISQ articles between 96 and 2000 showed they were weak (Iivari et al. 2004)

Current upsurge in interest in design science may change this. Also important that these papers have turned attention onto how to do design science research more rigorously.

IS is increasingly being seen as an applied science, a quote from Benbasat and Zmud (2003)

our focus should be on how to best design IT artifacts and IS systems to increase their compatibility, usefulness, and ease of use or on how to best manage and support IT or IT-enabled business initiatives.

Iivari’s (1991) previous work on applying paradigms to IS development approaches or schools of thought used the Burrell and Morgan (1979) framework but expanded it in two ways to encapsulate his design science background.

  1. Added ethics as an explicit dimension
  2. incorporated constructive research to complement nomothetic and idiography research

This essay revisits that work and applies it directly to design science research.

Ontology

States design research should be based on sound ontology. However, does not state explicitly (at least at this stage) why this is the case. Not suggesting that it should be based on sound ontology, but I want to know why Juahni thinks it should be.

Talks about Poppers (1978) three worlds as the basis for this ontology (a lecture delivered by Popper)

  • World 1 – physical objects and events, including biological entities
  • World 2 – mental objects and events
  • World 3 – products of the human mind, includes human artifacts and also covers institutions and theories

Popper talks about World 3 include “also aeroplanes and airports and other feats of engineering.”

Iivari argues

  • institutions are social constructions that have been objectified (Berger and Luckman, 1967)
  • truth and ‘truthlikeness’ (Niiniluoto 1999) can be used in the case of theories, but not artifacts
  • Artifacts are only more or less useful for human purposes

Disciplines of computer are interested in IT artifacts. Dahlbom (1996) adopts a broad and possibly confusing interpretation of the concept of the artifact including people and their lives. Coming back to just IT he says

When we say we study artifacts, it is not computers or computer systems we mean, but information technology use, conceived as a complex and changing combine of people and technology. To think of this combine as an artifact means to approach it with a design attitude, asking questions like: Could this be different? What is wrong with it? How could it be improved? (p. 43).

Dahlbom also claims the discipline should be thought of as “using information technology” instead of “developing information systems” (p.34). Need to look at this more to see if there is much more to this claim than the surface interpretation.

Starts thinking about developing a sound ontology for design science. Identifies the need to answer the question about what sort of IT artifacts IS should build, especially if we wish to distinguish ourselves from computer science. In terms of ontology of artifacts mentions

  • Orlikowski and Iacono (2001) – the names from the IT artifact
    And their list of views of technology: computational, tool, proxy and ensemble.
  • March & Smith (1995)/Hevner et al (2004) from design research
    And their constructs, models, methods and instantiations. Iivari suggests this is a very general classification, its application is not always straightforward
  • diffusion of innovations – Lyytinen and Rose (2003), refining Swanson (1994) identify
    • base innovations
    • systems development innovations
    • services – administrative process innovations (e.g. accounting systems) technologyical process innovations (e.g. MRP), technological service innovations (e.g. remove customer order entires), and technological integration innovations (e.g. EDI).

In my view the primary interest of Information Systems lies in IT applications.

Defines 7 archetypes of IT applications. As archetypes they may not occur in practice in their pure forms.

Role/function Metaphors Examples Connection with Orlikowski & Iacono
To automate Processor Many embedded or transaction processing systems technology as labour substitution tool
To augment Tool (proper) Many personal productivity systems; Computer aided design technology as productivity tool
To mediate Medium Email, instant messaging, chat rooms, blogs, electronic storage systems (e.g. CDs and DVDs) technology as socail relations tool
To informate Information source Information systems proper technology as information processing tool
To entertain Game Computer games
To artisticize Piece of art Computer art
To accompany Pet Digital (virtaul and roboting) pets

The above table interprets information system

  • is a system whose purpose “is to supply its group of users with information about a set of topics to support their activities” (Gustafsson et al, 1982, p100)
  • implies that an IS is specific to the organisational/inter-organisational context in which it is implemented
  • information content is also a central aspect

Differences between IT artifacts include

  • In design – different design approaches used for different purposes
  • In their diffusion – Swanson (1994) and Lyytinen and Rose (2003)
  • In their acceptance – Iivari’s conjecture

Proposes that IT artifacts have invaded all of Popper’s worlds

  1. IT artifacts are embedded in natural objects, e.g. to measure physical states, and nanocomputing may open up new opportunities. How IT artifacts affect natural phenomena is likely to become a significant research problem.
  2. IT artifats are influencing our consciousness and mental states, our perceptions.
  3. Significant constituents of organisations and societies – make it feasible to develop more complex theories.

Research phenomena below influence epistemology and methodology

  1. How does the use of a mobile phone affect one’s brain temperature?
  2. How does the use of a mobile phone affect one’s perception of time and space?
  3. How do mobile phones affect the nature of work in organisations?

An ontology for design science

World Explanation Research Phenomena Examples
World 1 Nature IT artifacts + World 1 Evaluation of IT artifacts against natural phenomena
World 2 Conciousness and mental states IT artifacts + World 2 Evaluation of IT artifacts against perceptions, consciousness and mental states
Institutions
Theories
Artifacts: IT artifacts, IT applications, meta IT artifacts
IT artifacts + World 3 Institutions
IT artifacts + World 3 Theories
IT artifacts + World 3 artifacts
Evaluation of organizational information systems
New types of theories made possible by IT artifacts
Evaluation of the performance of artifacts comprising embedded computing

Epistemology of design science

Truth, utility and pragmatism. Argues against the adoption of the idea from pragmatism that truth is seen as practical utility. Artifacts, if theories are excluded, do not have any truth value. Practical action informed by theory may develop some level of truth if it consistently proves to be successful.

Draws on his earlier work in adopting a framework from economics to structure research within IS. It’s again based the type of knowledge being produced, in his case there are three types

  1. Conceptual knowledge – which has no truth value
    Includes concepts, constructs, classifications, taxonomies, typologies and conceptual frameworks.
  2. Descriptive knowledge – has truth value
    Includes observational facts, empirical regularities and theories/hypothesis which group under causal laws.
  3. Prescriptive knowledge – which has no truth value
    Design product knowledge, design process knowledge and technical norms.

The author suggests the following mapping between his framework and Shirley’s types of theory

  1. Conceptual – “Theories for analysing and predicting”
  2. Descriptive – theories for explaining and predicting and theories for explaining (as empirical regularities)
    Can include

    • observational facts – who invented what, when.
    • descriptive knowledge – TAM, Moore’s law
    • empirical regularities and explanatory theories identify causal laws that are either deterministic or probablistic
  3. Prescriptive – theories for design and action
    Relatively speaking, prescriptive knowledge is the least well understood
    form of knowledge in Table 3.

Suggests that theories of explaining, in the form of grand theories such as actor-network theory, do not fit into his framework. But the do in Shirley’s.

On the question of truth value or truthlikeness

  • Conceptual – the goal is essentialist, to identify the essence of the research territory and the relationships. May be more or less useful in developing theories at the descriptive level (quotes Bunge 1967a here).
  • Prescriptive – artifacts and recommendations do not have a truth value. Only statements about their efficiency and effectiveness have such a value

Beckman (2002) identifies four criteria of artefacts

  1. Intentional – the knife is a knife because it is used as a knife
  2. Operational – it is a knife because it works like a knife
  3. Structural – is a knife because it is shaped and has the fabric of a knife
  4. Conventional – is a knife because it fits the reference of the common concept of a ‘knife’

Juhani does not include the conventional with artifact as it may not achieve community acceptance until years after invention and construction.

Prescriptive knowledge is irreducible to descriptive knowledge

Suggests that most IT systems are built divorced from descriptive knowledge. There is only a weak link between IT artifacts and descriptive knowledge. And yet IT systems are still reasonably successful.

This makes one to wonder whether the IS research community tends to exaggerate to the significance of descriptive theoretical knowledge for prescriptive knowledge of how to design IT successful artifacts. In conclusion, in line with Layton (1974) I am inclined to suggest that prescriptive knowledge forms a knowledge realm of its own and is not reducible to descriptive knowledge.

Kernel theories

Believes the presence of a kernel theory is the defining characteristic of a “design theory”.

This is seen as difficult and leads to a softening of requirements for a kernel theory – e.g. Markus et al (2002) allowing any practitioner theory-in-use to serve as a kernel theory. Implying the design theory is not based on scientifically validated knowledge.

Methodology of design science

Classifications of IS research methods (Benbasat, 1985; Jenkins, 1985; Galliers and Land, 1987 and Chen and Hirschheim, 2004) do not recognise anything resembling constructive research methods. Iivari (1991) suggested constructive research as the term to denote the research methods required for constructing artifacts.

Positions building artifacts as a very creative task. Hence it is difficult to define an appropriate method for artifact building. Having constructive research methods is essential for the identity of IS as a design science. The rigor of methods distinguishes the design science from the practice of building IT artifacts.

Suggests two ways to identify the difference

  1. There is no constructive research methods, instead the difference is the evaluation. Design science requires scientific evaluation of the artifacts.
    drawback may lead to reactive research where IS as a designs cience focuses on the evaluation of existing artifacts, rather than building new ones.
  2. Define a rigorous approach for constructive research and use this to differentiate design science from invention in practice.

Iiivari didn’t specify the constructive research methods. Talks about Nunamaker et al (1990-1991) and their suggestion that systems development methods could serve this role. Iivari doesn’t appear to think so. Pitfalls include:

  • Do SDMs allow sufficient room for creativity and serendipity which are essential for innovation?
    A significant concern when attempting to make the building process more disciplined, rigorous and transparent.
  • Most serious weakness of the Nunamaker et al suggestion is that it integrates systems development quite weakly with research activities.

Hevner et al (2004) suggests rigor in designs cience research is derived from the effective use of prior research – using the existing knowledge base. Iivari claims it is in making the construction process as transparent as possible.

The source of ideas

Iivari suggests four major sources for ideas for design research

  1. Practical problems and opportunities
    Emphasizes the practical relevance of this research. Customers known as significant source of innovations (von Hippel 2005). But practice problems may be abstracted or seen slightly differently. Design science can also create solutions long before a problem is seen/understood.
  2. Existing artifacts
    Most design science research consists of incremental improvements to existing artifacts. Must understand what has gone before, if only to evaluate contribution.
  3. Analogies and metaphors
    Known that analogies and metaphors stimulate creativity.
  4. Theories
    i.e. kernel theories can serve as inspiration

Design science and action research

Many authors associated design science and action research, since they both attempt to change the world. Iivari suggests that they are different in a number of ways

  • Historically
    Action research – socio-technical design movement. Design science – engineering.
  • Practically
    Action research – focused on “treating social illnesses” within organisations and other institutions. Technology change may be part of the treatement, but the focus is more on adopting than building technology.
    DSR – focus on the construction of artifacts, most having material embodiment. Usually done in laboratories, clearly separated from potential clients.
  • Ontologically,
    DSR – in engineering/medicine adopts a realistic/materialistic ontology
    Action research – accepts a more nominalistic, idealistic and constructivist ontology

    Materialism attaches primacy to Popper’s World 1, idealism to World 2. Action research is also interested in the institutions of World 3.

  • epistemologically, and
    Consequently, design research, especially in engineering and medicine, have a positivistic epistemology in terms of knoweledge applied from reference disciplines and knowledge produced. Action research is strongly based on an anti-positivistic epistemology. The very idea of AR is anti-positivistic as each client is unique.
  • methodologically.

Cole et al (2005) take the alternate perspective that design science and AR share important assumptions regarding ontology and epistemology. Cole et al implicitly limit design science to IS in an organisational context, if so then shouldn’t the ontology and epistemology of DSR be different. Juhani is doubtful about this, based on his work evaluating systems development approaches – but he’s said earlier that systems development approaches aren’t a good match for constructive research – for DSR. Can he make this connection here?

Ethics of design science

Design science shapes the world. “Even though it may be questionable whether any research can be value-free, it is absolutely clear that design science research cannot be.” which suggests that the basic values of research should be expressed as explicitly as possible.

Juhani then uses his own work (1991) to identify three roles (?types of ethics?)

  1. Means-end oriented
    Knowledge is provided to achieve an ends without questioning the legitimacy of the ends.
    Evaluation here is interested in how effectively the artifact helps achieve the ends
  2. interpretive
    The goal is to enrich understanding of action. Goals are not clear, focus on unintended consequences.
    Evaluation seeks to achieve a rich understanding of how an IT artifact is really appropriated and used and what its effects are, witout focusing on the given ends.
  3. Critical
    Seeks to identify and remove domination and ideological practice. Goals can be subjected to critical analysis.
    Evaluation focuses on how the IT artifact enforces or removes unjustified domination or ideological practices.
  4. Most DSR is means-end oriented, but it can be critical (e.g. Scandinavian trade-unionist systems development approaches)

    Question values of IS research – whose values and what values dominate?

    Conclusions

    Introduces the 12 theses summarised right up the top

    References

    Benbasat, I., & Zmud, R. (2003). The Identity Crisis within the IS Discipline: Defining and Communicating the Discipline’s core properties. MIS Quarterly, 27(2), 183-194.

The many Ps

For those of you who don’t know, I’m developing something called the Ps Framework for sensemaking around e-learning (read the earlier post for more information.

Jocene, one of my current colleagues and someone who is having to engage with the Ps Framework, has somewhat regularly complained about IT folk and their interest in the Ps, Cs, etc.

She seems to have some connection with someone else with a similar line of thought. Jason Woodruff in this post outlines and complains about some of the “P Words” that are floating around. Always useful to keep things in perspective.

The other reason I mention this is the “synergistic” way I came across Jason’s post from about a year ago. I’ve just moved my website from a personal server to WordPress. When I posted my most recent post about the Ps Framework wordpress.com included links to what it thought were related posts at the bottom. Jason’s was one of them.

The “web 2.0″ move, to a site more connected with others, seems to have paid off in a small way.