Skip to content. | Skip to navigation

  • Home
  • Archive
  • 6. Issue
  • The E-learning Circle – a holistic software design tool for e-learning
Citation and Metadata
Document Actions
Full Text

1 Introduction

Looking at research in the field, there are different opinions about where e-learning design will turn in the future. Some claims that activity-based e-learning is the next-generation e-learning (Griffiths 2004), while others argue that mobile learning (Dye et al 2005; Traxler 2006), personalization (Johnson et al 2006) or immersive digital games (Kickmeier-Rust et al 2007) is the next-generation e-learning.

There is a need for a holistic approach to e-learning, because the paradigms of instructional technology show how changing paradigms reduce the accepted models of instruction and instructional technology types (Koschmann 1996), which in turn reduce the opportunities for variation and individualization within e-learning. The holistic approach is also needed to avoid the overexposure of few parts of an e-learning system such as learning objects or assessment.

The study’s main objective is to investigate how to assure the quality of the development process of e-learning applications by implementing pedagogical principles into the software design process, more specifically the pedagogical principles individualization, variation and meta-learning.

Quality assurance means in this study systematic and planned actions in order to ensure that the product should be suitable for the intended purpose and to eliminate mistakes. Quality assurance will improve the end user satisfaction and reduce mistakes during the software development.

The structure of the article is as follows; first the research method is presented, with focus on research design, data collection and data analysis. The E-learning Circle is then described with text and figures (sector by sector), and an explanation of the use of the E-learning Circle is provided. The article then continues with a presentation of two projects, where the E-learning Circle is used. The results and the trustworthiness of the study are discussed, and reflections upon the research method are provided. Finally the article concludes with an overview of conclusions and further work.

2 Research method

2.1 Research design

The problem statement of the study is exploratory and open-ended, and requires an exploratory design. The research method used to develop the E-learning Circle is Grounded Theory (Glaser & Strauss 1967, Strauss & Corbin 1998). The inductive, theory discovering approach of Grounded Theory, allowing the grounding of theory in empirical data, is appropriate for exploratory studies.

2.2 Data collection

The selection of research sites followed the technique of theoretical sampling, described as “data gathering driven by concepts derived from the evolving theory and based on the concept of ‘making comparisons’, whose purpose is to go to places, people, or events that will maximize opportunities to discover variations among concepts and to densify categories in terms of their properties and dimensions” (Strauss & Corbin 1998). The empirical data, upon which this article is based, were collected through interviews, focus groups, and expert groups.

2.2.1 Interviews

Interviewing as data collection method was performed throughout the whole study. The study consists of 21 face-to-face interviews with 23 interviewees. The selection criteria of interviewees were designed to cover the users’ perspectives, including students, instructors, researchers and system developers.

The interviews were semi-structured, which means flexible, but based on a framework of themes to be explored. Questions were planned ahead, in order to find themes and open-ended questions and to prepare for flexibility during the interview. This was important in the exploratory study in order to allow new ideas and questions to emerge. The interviews lasted ca one hour and the interviews were recorded and transcribed.

2.2.2 Focus groups

The second method of data collection was the use of three focus groups, defined as “a small group of people assembled by a researcher to identify through informal discussion the key issues and / or themes related to a research topic” (Reitz 2004). The focus groups were arranged to collect data at an early stage of the study and each focus group consisted of 8-10 persons (students, teachers and researchers). The mix of user groups in the same focus group was avoided due to different interests, levels of ‘expertise’, and user needs.

The focus group sessions first consisted of a brainstorming session, where the focus groups got ‘next generation e-learning’ as clue, and then continued with a group discussion based on the ideas from the brainstorming.

2.2.3 Expert groups

The third data collection method was the use of three expert groups in a problem solving process. The expert group participants were system developers and researchers working on specific problems related to the emerging categories. The researchers were active contributors in the problem solving activities. The expert groups contributed to the development of three prototypes.

The first prototype was exemplifying how to implement best practice by developing wizards based on pedagogical design patterns (Kolås & Staupe 2004; Saatz & Kolås 2005). The second expert group developed the E-learning ontology (Kolås 2006) and the PLExus prototype (Kolås & Staupe 2007), which is a running prototype of a personal learning environment implemented in the semantic technology of topic maps. The third expert group developed a paper prototype using stereotype modelling to model ambient learners (Kofod-Petersen et al 2008), based on the emerging E-learning Circle.

2.3 Data analysis

Grounded Theory provides the researcher with analytical tools when analyzing the data in the iterative research process. “As grounded theorists, we begin our analyses early to help us focus further data collection. In turn, we use these focused data to refine our emerging analyses” (Charmaz 2005). In the processes of open, axial and selective coding (Strauss & Corbin 1998) tools like questioning, constant comparison, diagrams, and memos were valuable.

2.3.1 Open coding

During open coding concepts are identified and their properties and dimensions are discovered in data, then the concepts are categorized. There are three different ways to perform open coding; a word-by-word or line-by-line analysis, analysis of a sentence or paragraph and analysis of a document, observation or interview as a whole (Strauss & Corbin 1998). In the process of analyzing the interviews open coding was mainly performed by analysis of sentences and paragraphs. The open coding of the interview transcripts created a large number of concepts, e.g. ‘Picking pedagogy’, ‘Choosing media type’ and ‘Freedom of relationships’.

The categorization of concepts also belongs to open coding, and is important in order to reduce the number of data units. We used in vivo categories (named by the respondents), e.g. ‘me-learning’, in addition to in vitro categories (named by the researcher) (Alvesson & Schiöldberg 2008) e.g. ‘meta-learning’. The focus group participants together initialized the categorization of ideas / concepts during brainstorming, and during the discussion afterwards the categories and concepts were questioned and compared.

2.3.2 Axial coding

Axial coding is the process of relating categories to their subcategories (Strauss & Corbin 1998). Diagrams proved to be effective in the axial coding, relating categories to each other and to subcategories. Diagrams also provided visualizations, which were useful finding subcategories among the main categories. “Subcategories answer questions about the phenomenon such as when, where, why, who, how, and with what consequences” (Strauss & Corbin 1998). The process of questioning, in combination with diagrams, was productive in the analysis in order to move the productive and creative work further.

2.3.3 Selective coding

“Selective coding is the process of integrating and refining categories” (Strauss & Corbin 1998). In this analyzing stage the attention is focused on the key components, with the goal of reaching theoretical saturation and moving from categories to theory. The writing of memos was one useful tactic. Codes and their relationships are not obvious in the coding phases, and it has been useful to go back to the memos to find old ideas that appear in a new way after more research.

A second tactic was to find the core category among which as many other categories as possible are related to and which occurs frequently in the empirical data. In this project the core category for a long time was ‘the student’ and the initial version of the E-learning Circle presented the student in the circle’s centre. After more research it was however obvious that ‘the subject’ as core category was more fruitful than ‘the student’ in order to cover the pedagogical principles of variation and individualization. ‘The subject’ as core category was hidden in the empirical data, as several interviewees claimed ‘the student’ to be the main focus. The ‘student’ category remains important, now as one of the circle’s main subcategories, but the subject’s characteristics were mentioned over and over again during the interviews. The importance of the subject’s characteristics was not said directly, but came through during the analysis. The ‘subject’ became during the selective coding to be considered as the core category.

A third tactic was to draw diagrams or models of how the categories are related to each other. The E-learning Circle illustrates how the use of integrative diagrams was useful in this study. Diagrams were helpful in the analyzing process to visualize the relationships between categories and based on the selective coding’s tactics the three sectors of the E-learning Circle emerged.

3 The E-learning Circle

The E-learning Circle is a contribution to the software design process of e-learning applications, more specifically a tool to ensure early focus on the pedagogical principles of variation and individualization, including learning, teaching and assessment. The tool visualizes the connection between pedagogical principles and technological solutions.

The E-learning Circle is presented with “Subject” at the centre (Figure 1), which includes both a specific subject (e.g. Object-oriented programming or English literature) and a complete subject field (e.g. Computer science or English). Then the circle is divided into three sectors (illustrated with different colours):

  • Learning objectives,

  • The student,

  • The teacher.

Each sector has four levels, where pedagogy is the focus of the three inner circles turning to technology in the outer circle.

Figure 1 - The main parts of the E-learning Circle. The centre of the circle is dedicated to the subject taught, and the E-learning Circle has three main sectors.

3.1 The “learning objectives” sector

The “learning objectives” sector illustrates the connection between learning objectives and assessment tools (Figure 2). The learning objectives are divided into four types; skills, knowledge, attitudes and meta-learning. The use of the terms skills, knowledge and attitudes are inspired by well-known taxonomies (Bloom 1956; Kratwohl 1964; Dave 1970). Meta-learning is the state of “being aware of and taking control of one’s own learning” (Biggs 1985) and is added as a learning objective because also in e-learning settings it is important to focus on students’ ability to ‘learn to learn’.

The learning objective “Knowledge” is based on Bloom’s taxonomy for the cognitive domain (1956), and has the following subcategories; Knowledge, Comprehension, Application, Analysis, Synthesis and Evaluation. Several articles (Conole 2004; Conole 2005) have been formulating verbs that belong to each level in Bloom’s taxonomy e.g. the verbs reproduce, arrange, and memorise, which belong to the knowledge level, while the verbs categorize, combine, create and design belong to the ‘synthesis’ level. Similar categorization of verbs can also be found for other taxonomies, e.g. the taxonomies for the affective and the psychomotor domains.

Figure 2 - The learning objective sector. The sector describes how the learning objectives via taxonomies are concretized into technological solutions.

The outer circle presents existing assessment tools to cover the assessment needs of different learning objectives on specific levels. A procedure to find existing assessment tools covering the different learning objectives is to consider each verb mentioned at each level of the four learning objective types and then connect assessment tools to these, e.g. the verbs reproduce, arrange and memorise (from the Knowledge level of Bloom’s taxonomy) have assessment tools like short answer, match and memory connected to them. This exercise is performed on each level of all the taxonomies and the results are shown in the E-learning Circle. Doing this exercise we find that it is necessary to develop more assessment tools within e-learning systems in order to assess the different learning objective levels.

3.2 The “student” sector

This sector contains elements that describe the student group’s heterogeneity. The “student” sector is divided into the main categories “multiple intelligences”, “proficiency stages” and “culture” (Figure 3). To be able to individualize e-learning to a heterogeneous student group we need to quality assure the production of learning objects based on all categories mentioned.

Figure 3 - The student sector. This sector describes how learning objects need to take into account the various needs of a heterogeneous student group.

The ”multiple intelligences” theory (Gardner 1985) provided a contribution to the discussion about who the learner is. The theory defines eight different intelligences: the visual / spatial intelligence, the verbal / linguistic intelligence, the logical / mathematical intelligence, the bodily / kinaesthetic intelligence, the musical / rhythmic intelligence, the interpersonal intelligence, the intrapersonal intelligence and the naturalistic intelligence. The idea is that all persons have eight intelligences, where some intelligences are more developed than others. The student sector describes what production tools the different intelligences require. Producing learning objects for different intelligences requires a variety of media types and production tools. Learning objects for the visual intelligence can be produced in tools like presentation software, mind maps, graphics tool (raster and vector graphics) and motion graphics tool (animation tool, screen capture tool, motion graphics tool and video editing tools). For the verbal intelligence we have tools like word processors, web editors, and tools to record audio. The logical intelligence needs tools like spread sheets, databases and modelling software.

The kinaesthetic intelligence has until now had few useful tools for e-learning, since simulators (e.g. a flight simulator) have been too expensive to use in e-learning settings. TV-games etc use cheap technology for kinaesthetic in-data with motion sensitive tools, where a web camera captures the user movements, which the system interprets and the user interacts with the system “waving” his hands. The e-learning systems also need to look to TV-games to satisfy the needs of the musical intelligence e.g. karaoke-systems that interprets if the singer hits the right tone. In addition there are midi tools to produce music. The interpersonal intelligence is covered by communication, coordination and cooperation tools (Studio Apertura 2006), while the intrapersonal intelligence is covered by tools like mind maps. The naturalistic intelligence is represented by tools like databases and hypertext editors.

The students also differ when it comes to proficiency stages; novice – advanced beginner – competence – proficiency – expert (Dreyfus 1998). The novice has different needs, e.g. need the help provided by a wizard, compared to a student on the advanced beginner stage, where e.g. a toolkit is useful, or the competence stage, where e.g. a framework is sufficient.

The cultural context also needs to be considered in e-learning. The E-learning Circle uses Hofstede’s (2001) five cultural dimensions; Power distance index, individualism, masculinity, uncertainty avoidance index and long-term orientation. In addition, “Location” is added to the E-learning Circle because one of the main features of e-learning is the opportunity for learning “anywhere”.

3.3 The “teacher” sector

The teacher facilitates e-learning, and will use a variety of pedagogical methods based on different theories of learning; Behaviourism, information processing theory, cognitive constructivism and socially oriented theories of learning (Koschmann 1996). Heinich et al. (2002) have specified ten main pedagogical methods. In this article two of them; problem solving and discovery, are merged. The teacher sector (Figure 4) illustrates the nine methods’ relations to different theories of learning. Pedagogical methods are traditionally reusable, and the “teacher” sector shows how the pedagogical methods are implemented in an e-learning environment as learning activities, such as procedural simulation (Alessi & Trollip 2001), chat, animation etc.

The teacher sector illustrates that it is possible to vary the pedagogical methods also in an e-learning setting, and can make it easier to understand and accept that an e-learning system not necessarily needs to be dedicated to one learning theory (e.g. socio-constructivism) if the goal is to vary according to the different needs of the student group and the specific subject’s characteristics. The teacher sector also makes it possible to detect what learning activity tools an e-learning system requires.

Figure 4 - The teacher sector. This sector implies an eclectic view of learning theories within e-learning, and emphasizes that different pedagogical methods must be used when designing learning activities.

4 Using the E-learning Circle as a design tool

The E-learning Circle’s (Figure 5) main application area is to assure the quality in the design process of e-learning systems ensuring that new systems support variation and individualisation. This section describes how this can be done.

Figure 5 - The E-learning Circle – A holistic approach to e-learning.

4.1 Dealing with complexity

The E-learning Circle is a tool dealing with complexity. Vendelhaven (2002) describes how mistakes, information loss, work duplication and misunderstandings are typical problems when the responsibility of segments is moved between persons. The E-learning Circle provides a systematic overview and merges both pedagogical principles and technological solutions, and as a design tool in the process of designing e-learning systems, this is useful dealing with complexity.

4.2 Bridging the gap – common language

In the design of e-learning systems IMS-LD (IMS 2003), a method for modelling learning processes, has been important the last years. IMS-LD critics concern reusability and the teachers’ difficulties to use the specification (Downes 2003, Griffiths & Blat 2005). IMS-LD is not bridging the gap between teachers and software developers, as it is based on specifications from the software engineering field. “New tools and representations are needed if teachers are to intervene in editing and creating units of learning” (Griffiths & Blat 2005). The E-learning Circle is an attempt to provide a tool, which can work as a common language between the teacher and the software developer, which in turn also will be helpful to succeed with participatory design.

4.3 Best practice

Best practice is a process or method that is more effective delivering a specific outcome than other processes or methods. The E-learning Circle has best practice embedded, e.g. the pedagogical methods and proficiency stages are best practice within instructional design.

4.4 Useful in several design models and processes

The E-learning Circle is not a prescriptive method, but a design tool, which may be used to support several processes, e.g. development of requirement specification, user modelling, interface design and choice of system architecture.

The use of the E-learning Circle is not connected to one specific software design model, but it supports different software design models in several phases. In agile methods, the E-learning Circle can be useful in costumer collaboration, which is valued in the Agile manifesto (Beck et al 2001). Using the Unified Process (Booch et al 1999) the E-learning Circle is useful from the perspectives of use case and design views, and specifically within the workflows of Requirements and Analysis and design. The ‘Grimstad model’ (Crossley and Green 1985) was an early software design model with focus on teachers as developers mainly focusing on smaller applications (lessonware), and in this model the E-learning Circle may be useful in phases of idea generation and goal formulation, as well as the phases of metaphor and market design. In prototyping the E-learning Circle is useful for e.g. requirements analysis, user modelling, choice of architecture and interaction.

4.5 Examples of use

The use of the E-learning Circle as a design tool of course depends on the software design model in use. The following sections describe the use of the E-learning Circle in two prototyping projects.

4.5.1 The PLExus prototype

In the process of theoretical sampling a prototype of a personal learning environment called ‘PLExus’ (Kolås & Staupe 2007) was developed. PLExus is based on the semantic technology of topic maps, a choice based on the emerging E-learning Circle. The semantic-based navigation of topic maps enables variation and individualization in e-learning through e.g. efficient context-based retrieval, customized views, information visualizations and deeper understanding of the domain conceptual relations (Dichev et al 2003).

To be able to develop a topic map it is necessary to build a topic map ontology, described as “the set of privileged topics and their characteristics, including associations between them” (Grønmo 2006) and a unique PSI (published subject identifier). PLExus was developed using the E-learning Circle in the requirement analysis and as a framework developing the e-learning ontology and unique PSI (Kolås 2006). PLExus provides a student interface allowing customized views of learning objects and learning activities based on pedagogical method, learning objective type, proficiency stage etc. An online wizard is provided to add metadata and learning objects to the PLExus topic map. The choices provided by the wizard are directly connected to the E-learning Circle.

4.5.2 Stereotype modelling of ambient learners

The E-learning Circle is also used in the work of stereotype modelling of learners in an ambient intelligent learning environment (Kofod-Petersen et al 2008). The aim of this project was to use stereotype modelling as a mean of modelling ambient learners so that the learning resources could be quickly and efficiently adapted to the learner. The student sector of the E-learning Circle was specified by a set of facets with a value and a rating in the process of stereotype modelling, e.g. the facets of a spatial intelligent person are images, shapes and 3D-spaces, based on a person’s capability of conjuring up mental images and transforming them, working with shapes and navigating in three-dimensional spaces.

The two developing projects both based the work on the E-learning Circle, despite the major differences of the systems. The PLExus prototype provides the student a user interface where learning resources are accessible in a web interface with multiple access opportunities. The ambient intelligent learning environment on the other hand will provide the student an adaptive interface. This shows that the E-learning Circle is not system dependent, but is able to contribute to the design process of a variety of e-learning systems.

4.6 Alternative application areas

Other application areas of the E-learning Circle are to evaluate existing e-learning systems and to quality assure development of e-learning courses.

4.6.1 Evaluation method

It is possible to use the E-learning Circle as an evaluation method, to evaluate existing e-learning systems in order to find the systems strengths and weaknesses and in order to compare e-learning systems based on pedagogical considerations. The drawback of using the E-learning Circle in evaluations is that such a process is reactive, while the use of the E-learning Circle in a design process is proactive. Advantages using the E-learning Circle as an evaluation tool are that the evaluation is independent of the evaluated system and that the evaluation will have a holistic approach.

4.6.2 Quality assuring instructional design

Newby et al (2006) define instructional design as “the process of translating principles of learning and instruction into plans for instructional materials and activities. The emphasis is on creating a plan for developing instructional materials and activities that increase an individual’s learning”. Examples of instructional design models are the ADDIE model, A.S.S.U.R.E (Heinich et al 2002) and rapid prototyping.

The E-learning Circle may also be useful for teachers, and can contribute to quality assure the course development in an online learning environment. “Many first-time users of VLEs (Virtual Learning Environments) seek to adapt the way that they work to the way that the software needs things to be done” (Britain & Liber 2004). As a tool in the course development, the E-learning Circle illustrates and provides a systematic approach to the teacher how it is possible to vary the learning based on the different needs of the heterogeneous student group when it comes to e.g. teaching methods / learning activities, multiple intelligences, proficiency stages and cultural background. The E-learning Circle also enables the teachers to evaluate their own practice and make their pedagogical choices more explicit and in addition it visualizes the connection between theory and practice. The E-learning Circle makes teachers more conscious about the different aspects an e-learning course needs to cover, like identifying learning objectives and the needs of the students and selecting the most suitable teaching methods. The circle will then be useful e.g. in the process of ensuring individualization, variation and meta-learning in an e-learning environment.

The alternative use of the E-learning Circle to assure the instructional design quality may be seen together with rapid prototyping (Batane 2010), and the E-learning Circle can be regarded as a tool in the process of needs analysis within rapid prototyping.

5 Discussion

The E-learning Circle emphasizes the importance of content knowledge (knowledge about the subject taught) as a basis, which later is connected to pedagogical knowledge and technological knowledge (Koehler et al 2005) to achieve effective e-learning where learning resources, communication and collaboration take place via technology.

5.1 The pedagogical theories

The E-learning Circle uses specific pedagogical theories e.g. Bloom’s (1956) taxonomy and Gardner’s (1985) multiple intelligences. Some of these theories are controversial within the pedagogical field. The experience of this work, however, is that the theories are useful connecting technology to pedagogical theories. There is however possible to replace theories in the circle, for example the taxonomies used in the sector “learning objectives” could be replaced by Anderson’s (2001) revised taxonomy for the cognitive domain, Harrow’s (1972) or Simpson’s (1972) taxonomies for the psychomotor domain. Such a replacement will not influence the outer circle in a large extent.

Also concerning the student there are many pedagogical theories; Coffield et al. (2004) identified 71 models of learning styles. Some of these models are alternatives on how to describe the student. The pedagogical discussion of different theories, e.g. student descriptions, is therefore not the main issue. The important issue is that we are able to describe the student and able to connect technology to these descriptions. This can be regarded as both a weakness and strength of the E-learning Circle. The strength is that the circle remains the focus on the subject’s characteristics, and does not regard one pedagogical theory as the single most useful theory. The drawback is that the circle can be diluted if different users keep replacing the theories.

It can also be regarded as a weakness to use Dreyfus’ (1998) proficiency stages since these originally are used in skill acquisition. In most learning situations the learners are on different stages in the learning process, and therefore Dreyfus’ stages are used generically in the E-learning Circle. The E-learning Circle also shows that there are not many technological tools to support the fact that students are on different stages in the learning process, and indicates that more research should be conducted on these questions.

One may discuss whether meta-learning could be placed under ‘skills’. Learning to learn, e.g. learning to work in groups, learning to give constructive critics etc. can be considered skills, but the E-learning Circle keeps meta-learning as an additional learning objective to face the challenges of e-learning in life-long learning. Adding meta-learning next to the traditional learning objective types (knowledge, skills and attitudes) is one of the contributions to the e-learning field made through the E-learning Circle.

5.2 Reusability

Robson (2004) claimed that “context is the friend of learning and the enemy of reuse”, and explains the problems of learning objects and reuse. The E-learning Circle suggests instead of specific context some generic student “types”, similar to “personas”, that we need to have in mind both when developing e-learning systems and when designing a course.

The E-learning Circle illustrates how learning objects must be retrievable for the students, based on different criteria like degree of difficulty, intelligence support and cultural dimensions. It is important that the learning objects are presented to the student without the danger of information overload. This is of great importance planning the human-computer interface of e-learning systems.

5.3 Individualization

One aim of the study was to implement the pedagogical principle of individualization and the question “Why is not the student placed in the circle’s centre?” then becomes apparent. If the student is the main focus, many of the sectors would disappear. Focus on the subject taught is empirical-based and provides a holistic approach to e-learning, including learning objectives, assessment, student needs, learning objects, teaching methods and learning activities. The E-learning Circle implies an eclectic view of learning theories, and illustrates that different subjects have various characteristics and needs when it comes to learning theories and technology, but is still able to implement individualization.

5.4 Future predictions and the E-learning Circle

The UNFOLD project claims activity-based learning is the next generation e-learning (Griffiths 2004). The E-learning Circle shows that learning activities are just one of many factors that are important in e-learning.

Dye et al (2005) suggests that mobile learning is the next-generation e-learning. Their argument is understandable if the technological solution is the main factor in the transition from one generation to another. The E-learning Circle focuses on pedagogy connected to technology, and illustrates how mobile learning is one of many student group heterogeneities. If mobile learning is to be considered the next generation e-learning, it is important that lessons learned from computer-based learning are remembered when moving to another technological platform. The E-learning Circle can contribute to quality assure that pedagogical principles are covered also in a mobile e-learning environment.

6 Trustworthiness

Validating qualitative research is not as clear cut a matter as it is with quantitative research and some argue that the term ‘validity’ should not be applied to qualitative research (Thagaard 1998; Salomon & Vavik 2008). We prefer to use the term ‘trustworthiness’.

To ensure credibility, we used Grounded Theory, a well-established research method within IS research (Orlikowski 1993; Pandit 1996; Urquhart 2001; Smit 1999). Triangulation also ensures credibility (Shenton, 2004) and this study uses triangulation by data sources (Patton 2002). Accuracy is ensured as far as possible through the thorough description of the research process.

6.1 Data collection techniques

Wibeck (2000) argues that brainstorming is a technique which does not belong within the definition of focus groups, because it does not allow criticizing each other’s ideas. The experience was, however, that brainstorming provided valuable data, encouraging creativity and creating new ideas. Data collection through focus groups was chosen because it provides multiple user perspectives, and because the group interaction allows creative ideas to thrive. This allows the researcher to ‘step back’ to a larger extent than in ordinary interview settings, because as focus group participants share insights and ideas, each individual member will respond, interact and continue the process. This was useful in an exploratory study in order to move ahead.

In this study, where focus is on how to improve the design process of e-learning systems, expert groups of software developers and the problem solving activities were valuable in order to stay close to the studied world (Charmaz 2005) and to provide data, which cover the intersection between pedagogy and technology. The aim of the study was to bridge pedagogy and software development, and expert groups were useful in order to bring pedagogical ideas into the situation of a software design process.

6.2 Dealing with literature

How to deal with the literature within Grounded Theory is often discussed, and Urquhart (2001) claims that “one of the oft quoted misconceptions about Grounded Theory is that the researcher does not do any literature searching”, but emphasizes that the Grounded theory researcher has to relate to literature in a slightly different way to a conventional researcher. “So literature is used to help build the theory, and the substantive theory is related to the literature, but only once the substantive theory has been developed” (Urquhart 2001). The E-learning Circle includes well-known pedagogical theories, but these were included in a late stage of the development of the E-learning Circle and are based on the empirical data, where coding and comparisons revealed that the pedagogical theories were hidden in the empirical data.

7 Reflections on the Grounded Theory

7.1 The Grounded Theory divergence

Since the Grounded Theory approach originated with the work of Barney Glaser and Anselm Strauss in their book “The Discovery of Grounded Theory” in 1967, not only the two originators of Grounded Theory have moved in slightly different directions, but also other researchers have adopted and adapted the Grounded Theory, which has lead to alternative versions of Grounded Theory (Denscombe, 2003).

Smit (1999) describes the divergence between Glaser and Strauss, the two originators of the Grounded Theory method. “Strauss and Corbin mention that they set out to provide clear, straightforward, and basic information on the knowledge and procedures needed by researchers who want to build their first theory at a substantive level” (Smit, 1999). “Glaser argues that what Strauss and Corbin describe will not produce a Grounded Theory, but rather ‘a forced, preconceived, full conceptual description, which is fine, but is not Grounded Theory’ (Smit, 1999). Locke (1996) notes that there are no differences between Glaser and Strauss’s positions on the key analytical procedures (constant comparison and theoretical sampling) involved in Grounded Theory methodology. “However, they do write subsequently different renditions of researchers’ relationships to the worlds they study” (Locke, 1996). “With the Glasarian approach the researcher allows the theory to emerge from the data, whilst the Straussian approach the researcher interrogates the data in order to arrive at a theory” (Smit, 1999). This research is inspired by the Strauss’s version of Grounded Theory, based on the interpretive approach and the interrogation with the data.

7.2 Critics of the Grounded Theory

Critics of the Grounded Theory approach include the epistemological positions. Denscombe (2003) describes how Glaser’s version is positivistic, while Strauss’ version is interpretive. “Glaser’s version rest on the belief that: (a) the researcher should maintain a distance and independence from the data; and (b) the meaning of the data will emerge inductively from the data if studied using a suitably neutral methodology... Contrasting with this, there is Strauss’s version, which is more in line with interpretivism, in that the role of the researcher is to go looking for the meaning that the data hold, possibly probing beyond their superficial meaning” (Denscombe, 2003).

Pandit (1996) describes some of the problems with Grounded Theory; First, the Grounded Theory research is extremely time-consuming; second, the Grounded Theory research involves long periods of uncertainty. Third, Grounded Theory research requires certain qualities of the researcher e.g. confidence, creativity and experience. Accordingly, the novice researcher are likely to find the approach more difficult than more conventional methodologies and the more experienced researcher is likely to produce better theory” (Pandit, 1996).

One disadvantage using the Grounded Theory method is that “the approach does not lend itself to precise planning” (Denscombe, 2003). Not being strictly dependent on a plan created early in the project has been interesting, and our opinion this made the results of this project better. Grounded Theory is experienced as useful in this exploratory project, and instead of regarding no precise plan as a weakness of the method; this allows an exploratory and creative approach and strengthens the research project.

Also the fact that Grounded Theory allows a variety of qualitative data collection methods (e.g. interviews, observations, document analysis) is an advantage. Together with the rich tools and techniques, the method helps the novice researcher in the process of analyzing the data.

7.3 Grounded Theory research in IS

“Grounded Theory has been increasing in popularity in Information Systems as a research method. This is evidenced by the growing literature that is either discursive on philosophy and application or detailed about the method” (Hughes & Jones, 2003). Hughes and Howcroft (2000) point out that there are four inconsistencies in the understanding and application of the Grounded Theory method in IS research. First, the projects range from those concerned with organizational change to those concerned with the practical use of the method to inform knowledge based systems design. Second, some use the method prescriptively, whilst others use some of its procedures to supplement other research strategies. Third, the underlying assumptions made explicit by the researchers range from qualitative-interpretive to qualitative-positivist and finally, Grounded Theory is used on its own or alongside other methods (Hughes & Howcroft, 2000).

8 Conclusions and further work

The aim of the study was to investigate how to assure the e-learning system design quality by implementing the pedagogical principles of variation, individualization and meta-learning. The proposed E-learning Circle is a tool in the design process of e-learning systems, bridging pedagogy and technology by providing a common language for teachers and system developers. It is also a tool dealing with complexity in the process of designing e-learning systems and has best practice embedded. The strengths of the circle are the compact presentation and the overview it provides. It does not only provide a pedagogical toolkit or a technology-based syntax, but in concrete terms illustrates the connection between pedagogical theories and technology. This is done by connecting specific technological tools to well-known pedagogical theories.

The E-learning Circle is not a prescriptive method, and may be used in different design models and different e-learning systems. This is exemplified by two projects; the PLExus prototype and stereotype modelling of ambient learners.

In the future it will be interesting to test the E-learning Circle in other design processes. Further work will also include making the circle user-friendly by designing questions belonging to each part of the sectors and by performing user tests.

There is a need for a holistic approach to the view of next generation e-learning, where the R&D focus must turn from small parts like “learning objects”, “learning activities” or “mobile learning”, to these parts understood in the relation to the whole. The E-learning Circle is such a holistic contribution to the e-learning field.


This work is supported through the QUIS project, carried out with the support of the European Community - The e-learning programme.

9 References

Alessi, S.M.; Trollip, S.R.: Multimedia for learning – Methods and development. 3rd edition, Allyn and Bacon, Boston, 2001

Alvesson, M.; Schiöldberg, K.: Tolkning og reflection. Vetenskapsfilosofi och kvalitativ metod 2nd edtion. Naranaya Press, 2008.

Anderson, L.W.; Krathwohl (Eds.): A Taxonomy for Learning, Teaching and Assessing: A Revision of Bloom's Taxonomy of Educational Objectives. Longman, New York, 2001.

Batane, T.: Rapid prototyping for designing and delivering technology-based lessons. In: Oray, M., Jones, S.A., Branch, R.M. Springer (Ed.): Educational Media and Technology Yearbook, vol. 35, 2010.

Beck, K.; Beedle M.; van Bennekum, A.; Cockburn, A.; Cunningham, W.; Fowler, M.; Grenning, J.; Highsmith, J.; Hunt, A.; Jeffries, R.; Kern, J.; Marick, B.; Martin, R.C.; Mellor, S.; Schwaber, K.; Sutherland, J.; Thomas, D.: Manifesto for Agile Software Development. 2001. (last check: 2010-06-23)

Biggs, J. B.: The role of metalearning in study processes. British Journal of Educational Psychology, vol. 55, 1985, pp 185-212.

Bloom, B. (Ed.): Taxonomy of educational objectives: The classification of educational goals: Handbook I, cognitive domain. Longmans, Green, New York Toronto, 1956.

Booch, G.; Rumbaugh, J.; Jacobsen, I.: The Unified Modeling Language User Guide. Addison-Wesley, Reading, Mass., 1999.

Britain, S.; Liber, O.: A Framework for the Pedagogical Evaluation of eLearning Environments. 2004. (last check: 2010-06-23)

Charmaz, K.: Grounded Theory in the 21st century. In: Denzin, N.; Lincoln, Y.S. (Ed.): The Sage Handbook of qualitative research, 3rd edition. SAGE Publications, Thousand Oaks, 2005, p.507-537.

Coffield, F.; Moseley D.; Hall E.; Ecclestone K.: Learning styles and pedagogy. In: Post-16 learning. A systematic and critical review. Learning and Skills. Research Centre, 2004.

Conole, G.: Report on the effectiveness of tools for e-learning. JISC, 2004. (last check 2010-06-23)

Conole, G.: Mediating artefacts to guide choice in creating and undertaking learning activities. CALRG seminar 2005.11.01.

Crossley, K.; Green, L.: A Practical Guide for Teachers: Designing Computer Lessonware. Crossley and Green, 1985.

Dave, R. H.: Psychomotor levels. In: Armstrong R.J. (ED.): Developing and Writing Behavioral Objectives. Educational Innovators Press, Tuscon, AZ, 1970.

Denscombe, M.: The good research guide for small-scale social research projects, 2nd edition. Open University Press, Buckingham, 2003.

Dichev, C.; Dichva, D.; Aroyo, L.: Using topic maps for e-learning. Paper presented at the IASTED International Conference CATE, July 1-2, 2003 in Rhodes, Greece.

Downes, S.: Design, standards and Reusability. Stephen’s web. 2003. (not available at 2010-06-23) (last check 2010-06-23)

Dreyfus, H. L.: Intelligence Without Representation. University of Housten. 1998. (last check 2010-06-23)

Dye, A.; Fagerberg, T.; Rekkedal, T.: Designing an Always-Online Learning Environment for Mobile Learners and Teachers. EU Leonardo Project “Mobile learning: The next generation of Learning”, NKI Distance Education. 2005. (last check 2010-06-23)

Gardner, H.: Frames of Mind: The Theory of Multiple Intelligences. Basic Books, New York, 1985.

Glaser, B. G.; Strauss, A. L.: The discovery of Grounded Theory: strategies for qualitative research. Aldine publishing company, Chicago, 1967.

Griffiths, D.: UNFOLD: Participate in creating the next generation of eLearning. LCCN Newsletter 8th issue, 2004. (last check 2010-06-23)

Griffiths, D.; Blat, J.: The Role of teachers in editing and authoring units of learning using IMS learning design. Advanced Technology for Learning, Vol.2, 2005, No. 4.

Grønmo, G.O.: Creating semantically valid topic maps. Ontopia. 2006. (last check 2010-06-23)

Harrow, A. J.: A taxonomy of the psychomotor domain. David McKay Co, New York, 1972.

Heinich, R.; Molenda, M.; Russell, J.D.; Smaldino, S.E.: Instructional media and technologies for learning. 7th edition. Merrill Prentice Hall, Upper Saddle River, NJ, 2002.

Hofstede, G.: Culture’s consequences: Comparing values, behaviors, institutions and organizations across nations. 2nd ed. Saga Publications, Inc., Thousand Oaks , 2001.

Hughes, J.; Howcroft, D.: Grounded Theory – never knowingly understood. In: Information Systems Review, vol 4, 2000, no.1, pp 181-197.

Hughes, J.; Jones, S.: Reflections on the Use of Grounded Theory in Interpretive Information Systems Research. Paper presented at ECIS 2003. Paper 62. (last check 2010-06-23)

IMS. 2003. Learning Design Specification. IMS. (last check 2010-06-23)

Johnson M.; Liber O.; Wilson S.; Sharples P.; Milligan C.; Beauvoir P.: Mapping the future: The PLE reference model and emerging technology. ALT-C 2006. Sept 5-7, 2006 in Edinburgh, Scotland.

Kickmeier-Rust, M.; Pierce, N.; Conlan, O.; Schwarz, D.; Verpoorten, D.; Albert, D.: Immersive Digital Games: The Interfaces for Next-Generation E-learning? In: Stephanidis, C. (Ed.): Universal Access in HCI, part III, HCII 2007, Lecture notes in computer science ; 4556. Springer, Berlin, 2007.

Koehler, M. J.; Mishra, P.; Yahya, K.: Tracing the development of teacher knowledge in a design seminar: Integrating content, pedagogy and technology. Computers and Education, vol. 49., 2005, issue 3, p. 740-762.

Kofod-Petersen, A.; Petersen, S. A.; Bye, G. G.; Kolås, L.; Staupe, A.: Learning in an Ambient Intelligent Environment – Towards modelling learners through stereotypes. Revue d'intelligence Artificielle, vol. 22, 2008, no.5, p. 569-588.

Kolås, L.; Staupe, A.: Implementing pedagogical methods by using pedagogical design patterns. Paper presented at AACE Ed-Media 2004 in Lugano, Switzerland.

Kolås, L.: Topic maps in e-learning: An ontology ensuring an active student role as producer. Paper presented at E-learn 2006, October 13-17, in Honolulu, USA.

Kolås, L.; Staupe, A.: A requirement specification of a next generation e-learning system, TISIP, 2006.

Kolås, L.; Staupe, A.: The PLExus Prototype: A PLE realized as Topic Maps. Paper presented at the 7th IEEE International Conference on Advanced Learning Technologies (ICALT 2007), 2007, July 18-20, Niigata, Japan.

Koschmann, T.: CSCL: Theory and Practice of an emerging Paradigm. Southern Illinois University, 1996.

Kratwohl, D.R.; Bloom, B. S.; Masia, B. B.: Taxonomy of educational objectives, Handbook II: Affective domain. David McKay Company, Inc., New York, 1964.

Locke, K.: Rewriting the discovery of Grounded Theory after 25 years. Journal of Management Inquiry, Vol. 5, 1996, no. 3, pp 239-245.

Newby, T.J.; Stepich, D.A.; Lehman, J.D.; Russell, J.D.: Educational technology for teaching and learning. 3rd edition, Pearson Merrill Prentice Hall, 2006.

Orlikowski, W.J.: CASE tools as Organizational Change: Investigating Incremental and Radical Changes in Systems Development. MIS Quarterly Vol. 17, no. 3, 1993, pp 309-340.

Pandit, M. R.: The Creation of Theory: A Recent Application of the Grounded Theory Method. The Qualitative Report, 2(4), 1996.

Patton, M. Q.: Qualitative Research & Evaluation methods. 3rd ed., Sage Publications, Thousand Oaks, 2002.

Reitz, J. M. 2004. Dictionary for Library and Information Science. Libraries Unlimited.

Robson, R.: Reusability Guidelines for Authors, Designers and Repositories. Eduworks Corporation, 2004. (not available at 2010-06-23)

Saatz, I.; Kolås, L.: Support for the instructor - from technical to pedagogical point of view. Paper presented at the IASTED International Conference on Internet and Multimedia Systems and Applications (EuroIMSA 2005), 2005, February 21-23 in Grindelwald, Switzerland.

Salomon, G.; Vavik, L.: Introduction to Research in Education and ICT – Chapter 4: Why should I believe you? 2008. (not available at 2010-06-23, secure web site)

Shenton, A. K.: Strategies for ensuring trustworthiness in qualitative research projects. Education for information 22, 2004, pp 63-75.

Simpson, E. J.: The Classification of Educational Objectives in the Psychomotor Domain. The Psychomotor Domain. Washington DC: Gryphon House, 1972.

Smit, J.: Grounded Theory Methodology in IS research: Glaser vs. Strauss. SART/SACJ, No 24, 1999, pp 219-222.

Strauss, A.; Corbin, J.: Basics of Qualitative Research: Techniques and procedures for developing grounded theory. Sage Publications, Thousand Oaks, Calif. 1998.

Studio Apertura – NTNU. 2006. Funksjonell klassifisering. NTNU. (not available at 2010-06-23)

Thagaard, T.: Systematikk og innlevelse – en innføring i kvalitativ metode (3rd edition). Fagbokforlaget, 2009.

Traxler J.: The evaluation of next generation learning technologies: the case of mobile learning. Paper presented at ALT-C 2006: The next generation. 2006, September 5-7, in Edinburgh, Scotland.

Urquhart, C.: An encounter with Grounded Theory: Tackling the Practical and Philosophical issues. In: Trauth, E. M. (Ed.): Qualitative Research in IS: Issues and Trends. Idea Group Publishing, Hershey, Penn., 2001.

Vendelhaven, T.: Objektorienteret systemudvikling med UML. Ingeniøren bøger, København, 2002.

Wibeck, V.: Fokusgrupper – Om fokuserade gruppintervjer som undersökningsmetod. Lund: Studentlitteratur, 2000.