Research Method – Overview

The following is the first part of chapter 3 of my thesis. The aim of this part is to explain the broad view of research that informs the work. The second part will give more specific details about the specific method used. Over the next week, I’m re-reading this chapter, when the fixes are done, I will upload a completed version.

Update: The latest version of the complete chapter is available from this page

Introduction

This thesis aims to answer the “how” question associated with the design, development and evolution of information systems to support e-learning in universities. It seeks to achieve this by using an iterative action research process (Cole, Purao et al. 2005) to formulate an information systems design theory (ISDT) (Walls, Widmeyer et al. 1992; Walls, Widmeyer et al. 2004; Gregor and Jones 2007). This chapter aims to situate, explain and justify the nature of the research method adopted in this work. It starts by examining the question of research paradigm and its connection with theory (Section 3.2). In particular, it seeks to explain why the choice of paradigm is seen as secondary to deciding the type of theory to be produced, in terms of selecting a research method. The chapter then uses four questions about a body of knowledge identified by Gregor (2006) to describe the particular perspectives that inform the research method to formulate the ISDT developed in this thesis (Section 3.3).

The formulation of an ISDT is one example of design research (Simon 1996; Hevner, March et al. 2004). At the start of this work, design research was not a dominant research methodology within the field of information systems (Lee 2000). There was a reluctance to accept the importance of this type of knowledge within information systems (Gregor 2002) and to this day there remain diverse opinions and on-going evolutionary understanding about the nature, place and process associated with design research and design theory (Baskerville 2008; Kuechler and Vaishnavi 2008). Consequently, the thinking underlying this thesis, and the content and structure of this chapter, has undergone a number of iterations as understanding has improved throughout the entire research process. For example, initial descriptions of this work (Jones and Gregor 2004; Jones and Gregor 2006) used the structure of an ISDT presented by Walls, Widmeyer and El Sawy (1992). This thesis uses the improved specification of an ISDT presented by Gregor and Jones (2007), an improvement that arose, in part, from work associated with this thesis. For these reasons, this chapter may delve into greater detail about these issues than traditional.

Paradigms and theory

It seems traditional at this point to describe the type of research paradigm that has informed the research for this work based on the assumption that the paradigm embodies a world view that has provided the fundamental assumptions to guide this research project and its selection of method. This section takes a slightly different approach.

This section seeks to argue that the question of research paradigm is of secondary importance to the matching of the research question, to the type of theory that best fits and subsequently the most appropriate research methodology or paradigm. This section argues that the aim of research is the generation and evaluation of knowledge (Section 3.2.1) and that this knowledge is typically expressed as different types of theory (Section 3.2.2). Lastly, the section seeks to connect this view with similar views of research paradigms (Section 3.2.3).

What is reseaerch?

The sixth edition of the OECD’s (2002) Frascati Manul defines research and experimental development as a

creative work undertaken on a systematic basis in order to increase the stock of knowledge, including knowledge of man, culture and society, and the use of this stock of knowledge to devise new applications

Vaishnavi and Kuechleer (2004) define research as “an activity that contributes to the understanding of a phenomenon”. Research, in its most conceptual sense, is nothing more than the search for understanding (Hirschheim 1992). Research is systematic, self-critical inquiry that is founded in curiosity and driven by a desire to understand arising from a stable, systematic and sustained curiosity and subjected to public criticism and, where appropriate, empirical tests (Stenhouse 1981).

Based on these perspectives it appears that a major aim of research is to generate and evaluate knowledge. Various perspectives on the nature of that knowledge, it’s purpose, validity, novelty, utility etc exist. Returning to the OECD (2002), they define research and development to cover three activities:

  1. 1. basic research;
    Experimental or theoretical work, without practice application in view, that aims to acquire new knowledge of the foundations of phenomena and observable facts
  2. applied research; and
    Original investigation aimed at acquiring new knowledge primarily for a specific practical aim or objective.
  3. experimental development.
    Systematic work based on existing knowledge that is directed towards producing new, or improving existing, processes, systems or services.

Even with these differences, a major aim of research appears to be to make a contribution to knowledge. If this is the case, then how is that knowledge represented. In creating and validating knowledge, scientists rely on the clear and succinct statement of theory, theory that embodies statements of the knowledge that has been developed (Venable 2006). Developing theory is what separates academic researchers from practitioners and consultants (Gregor 2006).

The role of theory and method

If an aim of research is to make a contribution to knowledge, should theory be used to represent that knowledge? Theory should be a primary output of research (Venable 2006). Theory development is a central activity in organisational research (Eisenhardt 1989). There is value in theory because it is practical. The practicality of good theory arises because it advances knowledge in a scientific discipline and guides research towards crucial questions (van de Ven 1989). Theories are practical as the enable knowledge to be accumulated in a systematic manner and the use of this knowledge to inform practice (Gregor 2006).

While there is recognition of the importance of theory, there remains questions about what it is. There has been a long-running search for the meaning of “theory” (Baskerville 2008). DiMaggio (1995) identifies at least three views of what theory should be and suggests that each has some validity and limitations. There is and has been disagreement about whether a model and a theory are different, whether or not a typology is a theory and other questions about theory (Sutton and Staw 1995). Many researchers within information systems use the word theory, but fail to give any explicit definition (Gregor 2006). This lack of consensus about what theory is, may explain why it is difficult to develop strong theory in the behavioural sciences (Sutton and Staw 1995).

Types of theory

Part of the confusion around theory has been around its purpose, around whether or not there are different types of theory. Within the information systems field there has been several different approaches to identifying different types of theory. Iivari (1983) described three levels of theorising: conceptual, descriptive and prescriptive. A number of authors (Nunamaker, Chen et al. 1991; Walls, Widmeyer et al. 1992; Kuechler and Vaishnavi 2008) have used the distinction of kernel and design theories. Taking a broad view of theory Gregor (2006) identified five inter-related categories of theory based on the primary type of question at the foundation of a research project. These five categories and their question of interest are summarized in Table 3.1.

Table 3.1 – Gregor’s Taxonomy of Theory Types in Information Systems Research (adapted from Gregor 2006)
Theory type Distinguishing attributes
I. Analysis Says “what is”.
The theory does not extend beyond analysis and description. No causal relationships among phenomena are specified and no predictions are made.
II. Explanation Says “what is”, “how”, “why”, “when”, “where”.
The theory provides explanations but does not aim to predict with any precision. There are no testable propositions.
III. Prediction Says “what is” and “what will be”.
The theory provides predictions and has testable propositions but does not have well-developed justificatory causal explanations.
IV. Explanation and prediction (EP) Says “what is”, “how”, “why”, “when”, “where” and “what will be”.
Provides predictions and has both testable propositions and causal explanations.
V. Design and action Says “how to do something”.
The theory gives explicit prescriptions (e.g., methods, techniques, principles of form and function) for constructing an artifact.

The taxonomy presented in Table 3.1 is based on little prior work and there exists opportunities for further work and improvement (Gregor 2006). There also remains some disagreement about the designation of design theory to Theory type V (Venable 2006). However, it does seem to provide a foundation on which to build sound, cumulative, integrated and practical bodies of theory within the information systems discipline (Gregor 2006).

Relationship between theory and method

Gregor (2006) suggests that research begins with a problem to be solved or a question of interest. The type of theory that is to be developed or tested depends on the nature of this problem and the questions the research wishes to address (Gregor 2006). This connection is made on the basis of the primary goals of theory (Gregor 2006). Assuming this image of the research process then it seems logical that the next step is the selection of research methods or paradigms most appropriate to develop or test the selected theory type. This is not to suggest that there is a one-to-one correspondence between a particular theory type and a particular method or paradigm. Gregor (2006) argues that none of the theory types necessitate a specific method, however, proponents of specific paradigms do favour certain types of theory over others. While there is no necessary correspondence between theory types and methods/paradigms, it is suggested that certain methods/paradigms are better suited to certain types of theory, research problems and researchers.

Recognising different types of theory makes it possible to see the differences as complementary and consequently enable integration into a larger whole (Gregor 2006). It is possible for research to make a contribution to more than one type of theory. Baskerville (2008) argues that there is clearly more to design research than design theory alone. Kuechler and Vaishnavi (2008) show how a design research project is contributing to both design theory (Gregor’s Type V) and kernel theory (Gregor’s other types). The possibility for a research project to make contributions to different types of theory suggests that a research project may draw upon several different methods or paradigms.

The role of research paradigms

Having briefly summarised the perspective on research, theory and method in previous sections, this section makes some connections between this perspective and the views on research paradigms expressed by Mingers (2001) and the pragmatic view of science/paradigm (Goles and Hirschheim 2000).

Research methodology attempts to approximate a compatible collection of assumptions and goals which underlay methods, the actual methods, and the way the results of performing those methods are interpreted and evaluated (Reich 1995). The assumptions or beliefs about the world, how it works and how it may be understood has been termed a paradigm (Kuhn 1996; Guba, 1999). Numerous authors have sought to identify and describe different research paradigms. Lincoln and Guba (2000) identify five major paradigms: positivism, postpositivism, critical theory, constructivism and participatory action. Within the information systems discipline, Orlikowski and Baroudi (1991) identify three broad research paradigms: positivist, interpretive and critical. Within information systems and in connection with the rise of design research, numerous authors (Nunamaker, Chen et al. 1991; March and Smith 1995; Hevner, March et al. 2004) have suggested that it is possible to identify two broad research paradigms within information systems: descriptive and prescriptive research. Where descriptive research is seen as traditional research where prescriptive research is design research. There are some who take issue with seeing design research as a separate paradigm (McKay and Marshall 2007).

Just as there are differing views on the number and labels of different research paradigms, there are differences on how to describe them. Guba and Lincoln (1994) describe the beliefs encompassed by a paradigm through three, interconnected questions: ontology, epistemology and methodology. Mingers (2001) describes a paradigm as being a general set of philosophical assumptions covering ontology, epistemology, ethics or axiology and methodology. Gole and Hirschheim (2000) use ontology, epistemology and axiology.

Mingers (2001) describes three perspectives on paradigms. These are:

  • isolationism;
    Views paradigms as based on contradictory assumptions which makes them mutually exclusive and consequently a researcher should follow a single paradigm.
  • complementarist; and
    Paradigms are seen as more or less suited to particular problems and selection is based on a process of choice.
  • multi-method.
    Paradigms are seen to focus on different aspects of reality and can be combined to provide a richer understanding of the problem.

Minger’s (2001) multi-method perspective seems to fit well with a research project seeking to address a research problem through making contributions to different types of theory (as described in Section 3.2.2). Such a perspective suggests that the question of whether a researcher is positivist, interpretivist or critical is of secondary importance to the question of fit between problem, theories and methods.

Such a perspective seems to have connections with that of the pragmatist perspective of research described by Goles and Hirschheim (2000). Pragmatists consider the research question as more important then the method used or the worldview meant to underpin the method (Tashakkori and Teddlie 1998). Table 3.2 compares four important paradigms, including pragmatism. It has been suggested that pragmatism draws on a philosophical basis of pluralism to undercut the traditional dichotomous battle between conflicting paradigms (Goles and Hirschheim 2000). It facilitates the construction of connections and interplay between conflicting paradigms (Wicks and Freeman 1998).

If a paradigm must be chosen, then that of pragmatism seems the best fit. This research puts the question of “how to design and support an information systems for e-learning within universities” as the focus. The type(s) of theories, the methods to be used and their appropriateness should flow and align with that question. The following section provides a explanation of this alignment and describes the choices made for this work.

Table 3.2 – Comparisons for four important paradigms used in the social and behavioural sciences (adapted from Tashakkori and Teddlie 1998)
Positivism Postpositivism Pragmatism Constructivism
Methods Quantitative Primarily quantitative Quantitative + qualitative Qualitative
Logic Deductive Primarily deductive Deductive + inductive Inductive
Epistemology Objective point of view, Knower and known are dualism Modified dualism. Findings probably objectively “true” Both objective and subjective points of view Subjective point of view. Knower and known are inseparable.
Axiology Inquiry is value-free Inquiry involves values, but they may be controlled Values play a large role in interpreting results Inquiry is value-bound
Ontology Naive realism Critical or transcendental realism Accept external reality. Choose explanations that best produce desired outcomes Relativism
Causal linkages Real causes temporally precedent to or simultaneous with effects There are some lawful, reasonably stable relationships among social phenomena. These may be known imperfectly. Causes are identifiable in a probalistic sense that changes over time There may be causal relationships, but we will never be able to pin them down. All entities simultaneously shaping each other. It’s impossible to distinguish causes from effects.

References

Baskerville, R. (2008). "What design science is not." European Journal of Information Systems 17(5): 441-443.

Cole, R., S. Purao, et al. (2005). Being proactive: Where action research meets design research. Twenty-Sixth International Conference on Information Systems: 325-336.

DiMaggio, P. (1995). "Comments on "What theory is not"." Administrative Science Quarterly 40(3): 391-397.

Eisenhardt, K. (1989). "Building theories from case study research." The Academy of Management Review 14(4): 532-550.

Goles, T. and R. Hirschheim (2000). "The paradigm is dead, the paradigm is dead…long live the paradigm: the legacy of Burrell and Morgan." Omega 28: 249-268.

Gregor, S. (2002). "Design Theory in Information Systems." Australian Journal of Information Systems: 14-22.

Gregor, S. (2006). "The nature of theory in information systems." MIS Quarterly 30(3): 611-642.

Gregor, S. and D. Jones (2007). "The anatomy of a design theory." Journal of the Association for Information Systems 8(5): 312-335.

Hevner, A., S. March, et al. (2004). "Design science in information systems research." MIS Quarterly 28(1): 75-105.

Hirschheim, R. A. (1992). Information Systems Epistemology: An Historical Perspective. Information Systems Research: Issues, Methods and Practical Guidelines. R. Galliers. London, Blackweel Scientific Publications: 28-60.

Iivari, J. (1983). Contributions to the theoretical foundations of systemeering research and the Picoco model. Oulu, Finland, Institute of Data Processing Science, University of Oulu.

Jones, D. and S. Gregor (2004). An information systems design theory for e-learning. Managing New Wave Information Systems: Enterprise, Government and Society, Proceedings of the 15th Australasian Conference on Information Systems, Hobart, Tasmania.

Jones, D. and S. Gregor (2006). The formulation of an Information Systems Design Theory for E-Learning. First International Conference on Design Science Research in Information Systems and Technology, Claremont, CA.

Kuechler, B. and V. Vaishnavi (2008). "On theory development in design science research: anatomy of a research project." European Journal of Information Systems 17(5): 489-504.

Kuhn, T. S. (1996). The Structure of Scientific Revolutions. Chicago, University of Chicago Press.

Lee, A. S. (2000). "Irreducibly Sociological Dimensions in Research and Publishing." MIS Quarterly 24(4): v-vii.

March, S. T. and G. F. Smith (1995). "Design and Natural Science Research on Information Technology." Decision Support Systems 15: 251-266.

McKay, J. and P. Marshall (2007). Science, Design and Design Science: Seeking Clarity to Move Design Science Research Forward in Information Systems. 18th Australasian Conference on Information Systems, Toowoomba.

Mingers, J. (2001). "Combining IS Research Methods: Towards a Pluralist Methodology." Information Systems Research 12(3): 240-259.

Nunamaker, J. F., M. Chen, et al. (1991). "Systems development in information systems research." Journal of Management Information Systems 7(3): 89-106.

OECD (2002). Frascati Manual: Proposed standard practice for surveys on research and experimental development. Paris, France, Organisation for Economic Co-operation and Development: 254.

Orlikowski, W. and J. Baroudi (1991). "Studying information technology in organizations: Research approaches and assumptions." Information Systems Research 2(1): 1-28.

Reich, Y. (1995). "The study of design research methodology." Transactions of the ASME.

Simon, H. (1996). The sciences of the artificial, MIT Press.

Stenhouse, L. (1981). "What counts as research?" British Journal of Educational Studies 29(2): 103-114.

Sutton, R. and B. Staw (1995). "What theory is not." Administrative Science Quarterly 40(3): 371-384.

Tashakkori, A. and C. Teddlie (1998). Mixed methodology: combining qualitative and quantitative approaches. Thousand Oaks, California, SAGE.

Vaishnavi, V. and B. Kuechleer. (2004, 18 January 2006). "Design Research in Information Systems."   Retrieved 20 April 2004, 2004, from http://www.isworld.org/Researchdesign/drisISworld.htm.

van de Ven, A. (1989). "Nothing is quite so practical as a good theory." The Academy of Management Review 14(4): 486-489.

Venable, J. (2006). The role of theory and theorising in design science research. First International Conference on Design Science Research in Information Systems and Technology, Claremont, CA.

Walls, J., G. Widmeyer, et al. (2004). "Assessing information system design theory in perspective: How useful was our 1992 initial rendition." Journal of Information Technology, Theory and Application 6(2): 43-58.

Walls, J., G. Widmeyer, et al. (1992). "Building an Information System Design Theory for Vigilant EIS." Information Systems Research 3(1): 36-58.

Wicks, A. and R. E. Freeman (1998). "Organization studies and the new pragmatism: Positivism, anti-positivism and the search for ethics." Organization Science 9(2): 123-140.

About these ads

3 thoughts on “Research Method – Overview

  1. estelaripa

    Thanks!! I`m trying to make a similar task (acomplish my thesis, write my blog as an aid to this) and your blog makes me feel less lonely and provides to me very interesting inputs!!! (Sorry about my english ;)

    Reply
    1. davidtjones Post author

      G’day,

      Happy to find a fellow traveler on the PhD journey. Thanks for the comments on the blog, I’m sorry to say that in recent months I’ve neglected the PhD reports. Must get back into those.

      It’s interesting to see the connections that form, and how unexpected they can be. It must be over a year now that Diego and I first connected, and now another connection is formed. Also interesting to see some of the familiar names and discussions in your blog, but also some new and interesting ones.

      What’s your thesis topic? From your blog, I’m guessing something around e-learning, online communities?

      David.

  2. Pingback: Blog de tesis, por David Jones « Estelaripa's Blog

Leave a Reply

Please log in using one of these methods to post your comment:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s