Posted on

Entity linking Systems for Literature Reviews

Let’s face it. In busy academic lives there is hardly any time to do some deep reading, let alone stay up to date with everything that is published in your area in real time. Yet, staying up to date with latest knowledge and reviewing literature regularly is our bread and butter as academics.

In addition, published literature reviews help establish your expertise of a particular area. Researchers increasingly automate the coding process in literature reviews and accelerate the literature review process by using computer-assisted tools like Leximancer, topic modelling, Bibliometrix, R packages, NVivo, etc.

However, existing approaches for coding textual data do not account for lexical ambiguity; that is, instances in which individual words have multiple meanings.

To counter this, we developed a method to conduct rapid and comprehensive analyses of diverse literature types by using entity linking in literature reviews. We present a new literature review framework that embeds entity linking.

See the framework step by step below:

In the same paper, we present an example where we apply the framework to review the literature on digital disruption and digital transformation.

On how to adapt the framework to your needs, see the full paper:

Marrone, M., Lemke, S., Kolbe, L.M. (2022), Entity linking Systems for Literature Reviews, Scientometrics. Forthcoming.,

Posted on

Developing interdisciplinary research maps from business/management and the environmental sciences

Summary of “Interdisciplinary Research Maps: A new technique for visualizing research topics”(Marrone & Linnenluecke, 2020)

Interdisciplinary research is challenging, in part due to the sheer magnitude of knowledge embedded within disciplines, and also to the lack of a common shared understanding across them.

To bridge the gap in understanding between the disciplines of business/management and the environmental sciences, Marrone & Linnenluecke (2020) developed a ‘map’ of topics, concepts, and ideas discussed in top publications in these fields of research.

Developing the map

  • Data for the study was sourced by selecting articles published since 2011 in the top four journals by impact factor in each field, through the Scopus database. The abstracts, titles and publication years of 4,827 environmental sciences articles and 2,671 business and management articles were downloaded.
  • These data were exported to two separate Comma Separated Values (CSV) files, one for each of the areas of interest. The titles of the publications were then merged with their respective abstracts and the files were analysed using TAGME entity linking tool to compile a list of all possible topics from the text in the abstracts and titles.
  • The researchers ‘cleaned’ the results of the analysis by deleting topics that made little meaningful sense, given the context in which they were used. After cleaning the results, 7,915 topics were retained in the environmental sciences articles, and 4,293 in business/management articles.
  • A map was created (see figure below) to show the frequency with which topics are mentioned in each field.

What the map tells us

1.     Topics that are frequently identified in one literature, but not the other

Some topics are represented almost exclusively in the environmental sciences articles, many of them linked to concerns about climate change. Meanwhile, in the business/management journals, the topics which arise most frequently are related to firm structure and expansion. There is an opportunity here, for future research to connect topics across these two fields. For example, further interdisciplinary research could serve to explore the impacts of climate change on business and management decisions, such as asset valuations and investments.

2.     Topics that are frequently associated with both literatures

Several topics are common to both sets of literature. For example, “decision-making” is a frequently discussed topic in both business/management and the environmental sciences. The topic of “China” also arises frequently in both disciplines, but in different ways. Articles in the business/management journals address management challenges and economic opportunities in China, while the those in the environmental sciences journals address the role of emerging economies like China’s in climate adaptation and mitigation efforts. Such areas of topic convergence may provide fruitful avenues for future interdisciplinary research.

Figure 1: Topics are represented as dots; those associated with the business/management literature are coloured red, and those associated with the environmental sciences blue.

Reference:

Marrone, M. and Linnenluecke, M.K., 2020. Interdisciplinary Research Maps: A new technique for visualizing research topics. Plos one, 15(11), p.e0242283.

Posted on

Intelligent Machines: Will AI replace academic researchers?

According to a recent article in Forbes magazine,we can expect to see most people collaborating on their work with an AI (Artificial Intelligence) counterpart by 2030. Will these ‘counterparts’ simply enhance what we do, or will they ultimately come to replace human beings altogether?

Transhumanism

Transhumanists examine the possibilities presented by the interactions between people and technology. They tend to welcome technological developments as means of enhancing our intellectual, physical, and psychological capacities (Bostrom, 2003; figure 1).

Figure 1. Adapted from Bostrom (2003, p.12)

An ethical minefield?

It’s not all good news, however. The concept of merging people with technology throws up a host of ethical considerations.

  • Who will access technological advantages? Those who can afford to pay for AI implants, wearable technology and tools? Will a group of enhanced humans emerge at the expense of everyone else?
  • If a person performs a criminal act, while coupled with an AI element, who bears responsibility? Should it be the person linked to the AI, the AI itself, or the person who developed or programmed the AI?

How close are we to developing human-like intelligence?

The idea of people integrating with computers is far from new. The famous computer pioneer J.C.R. Licklider coined the term ¨man-computer symbiosis” in the 1960s. Licklider predicted that computing technology would eventually advance to the extent that you would be able “to think in interaction with a computer in the same way that you think with a colleague whose competence supplements your own.” (Licklider, quoted in Lesh et al., 2004). Almost 60 years on, are we there yet?

The sort of technology which could result in human-like intelligence and the sorts of ethical conundrums outlined above, is still some way off, yet there are some very interesting possibilities available to us. Researchers, for example, may approach a form of symbiosis with computers. AI-based search tools are developing, which can help us navigate the broad swathes of literature available to us, speeding up our engagement with information (Extance, 2018). This is where we believe ResGap comes in.

Harnessing the potential of AI

Technology such as that available through ResGap can enhance your research performance, by allowing you to quickly find research gaps, understand how your research field has evolved over time, and identify hot and cold topics. By letting ResGap do the filtering for you, pointing you to where you will most productively direct your attention, you are able to engage with an astonishing breadth of information.

The kind of technology offered by ResGap doesn’t replace you as a researcher. You can work withResGap, but not yet in Licklider’s “Man-computer symbiosis”. Rather, you are still in the driving seat, harnessing the power of an extremely useful tool. While you won’t be able to access the whole space available to posthumans in Bostrom’s model (above: The Space of Possible Modes of Being), but you’ll certainly be pushing at the envelope of what is accessible to humans. It’s not a bad place to start.

Bostrom, N. (2003) Introduction to transhumanism. Presented at the Intensive Seminar on Transhumanism, Yale University, 26 June 2003. Available at:  https://www.slideshare.net/danila/introduction-to-transhumanism?from_action=save

Extance, A. (2018) How AI technology can tame the scientific literature. Nature Available at: https://www.nature.com/articles/d41586-018-06617-5

Lesh, N., Marks, J., Rich, C., Sidner, C.L. (2004) Man-Computer Symbiosis ´Revisited: Achieving Natural Communication and Collaboration with Computers. IEICE Transactions on Information and Systems

Posted on

A tale of two research fields: from Team Mental Model theory to research model creation

Team Mental Model Theory

Team mental models are organized mental representations of the team’s relevant environment, shared across team members (Klimoski & Mohammed, 1994). They emerge because individual team members tend to categorise elements of their environments, such as tasks, situations, response patterns or relationships. These categorisations then become shared over time, thanks to communication within the team.

The extent to which mental categorisations are shared across team members can vary widely. They may be highly consistent with one another or completely incongruent. Importantly, when researchers talk about shared mental models, they do not suggest that an identical set of categorisations is held by every member of the team. Rather it is suggested that there exists some degree of consistency or convergence between individuals’ mental models (Kang, 2006; Rentsch, 2008).

Teams with higher levels of convergence of mental models can perform better. A shared mental model enables team members to anticipate the needs and actions of others in the team (Cannon-Bowers, 1993). Thus, the team can coordinate its actions, enhancing its decision-making capacity (Stout, 1999).

From TMM to research models

If we conceptualise research communities as teams, we can begin to see how TMM theory applies to researchers seeking to identify what is researched by individuals members of the team. A degree of sharedness will exist across the mental models held by researchers in each field. They are likely to use the same terms in reference to the same concepts. However, mental models which are shared within one field may diverge significantly from those in a neighbouring field. An understanding of the ways in which different groups think about a topic has vast potential in the pursuit of interdisciplinary research.

Our map, generated by resgap.com help illustrate the idea that mental models lie on a continuum, rather than as a dichotomy (say, very infrequently discussed in one literature to smilingly identical mentions in both literatures). As an example, we could study how the term “techno-stress” is studied by Psychology and Information Systems researchers. Our tool would help uncover what are the topics that are frequently discussed in both fields, as well as topics that are frequently discussed in one field, but unfrequently discussed in the other.

By mapping what is discussed among researchers in separate fields, we may increase the effectiveness of our research. It is possible to develop a systemic understanding of who is doing what, increasing the coordination of our actions with those of other members of the research community. We become able to anticipate the needs and actions of our fellow researchers.

The unique contribution of resgap.com lies in its use of entity linking. Because resgap.com does not rely on keyword identification alone, it identifies discussion of concepts, not just the usage of similar terms. This helps overcome the challenge presented differences in the use of terminology. Different terms may be used in different fields to describe the same concept, or the same terms may be used in different fields to describe completely different concepts. However, resgap.com can cut through the confusion, allowing us to see where fields overlap, and where relevant and valuable research may be directed.

References

Cannon-Bowers, J. A., Salas, E., & Converse, S. (1993). Shared mental models in expert team decision making. In N. J. Castellan, Jr. (Ed.), Individual and group decision making: Current issues(pp. 221-246). Hillsdale, NJ, US: Lawrence Erlbaum Associates, Inc.

Kang, H.-R., Yang, H.-D., & Rowley, C. (2006). Factors in team effectiveness: Cognitive and demographic similarities of software development team members. Human Relations, 59(12), 1681–1710. https://doi.org/10.1177/0018726706072891

Klimoski, R., & Mohammed, S. (1994). Team Mental Model: Construct or Metaphor? Journal of Management, 20(2), 403–437. https://doi.org/10.1177/014920639402000206

Marrone, M, Hammerle, M (2018) Smart Cities: A Review and Analysis of Stakeholders’ Literature. Bus Inf Syst Eng60: 197. https://doi.org/10.1007/s12599-018-0535-3

Rentsch, J. R., Small, E. E. & Hanges, P. J. (2008) Cognitions in organizations and teams: What is the meaning of cognitive similarity? In D. B. Smith (Ed.),LEA’s organization and management series. The people make the place: Dynamic linkages between individuals and organizations (pp. 127-155). New York, NY, : Taylor & Francis Group/Lawrence Erlbaum Associates.

Stout,R.J., Cannon-Bowers, J. A., Salas, E., & Milanovich, D. M. (1999). Planning, Shared Mental Models, and Coordinated Performance: An Empirical Link Is Established. Human Factors, 41(1), 61–71. https://doi.org/10.1518/001872099779577273

Posted on

Finding research gaps in IT Service Management (and in ITIL research)

Summary of “Relevant Research Areas in IT Service Management: An Examination of Academic and Practitioner Literatures” (Marrone & Hammerle, 2017)

IT Service Management is a field of IS research which is widely used and popular in practice (Iden & Eikebrokk, 2013; Marrone & Kolbe, 2011). This study compares business and academic literature in the field. Since the behaviour of practitioners is influenced by the (business) literature they read (Carroll & McCombs, 2003), the comparison uncovers aspects of professional behaviour or practices not explored by academic research, termed “practice-oriented research gaps” (Müller-Bloch and Kranz, 2015).

Academic literature used in the study comprised abstracts of papers from selected information systems publications, identified by database searches using keywords related to ITSM. Practitioner literature was identified through searches of selected popular press and specialist IS publications. Both sets of literature were published during a time span of 16 years (from 1 January, 2000, to 1 May, 2016). A semantic entity annotator(the technology used by resgap.com) was employed to identify topics in the two groups of identified literature, then keyword analysis was applied to identify statistically significant topics.

For each set of literature, eight of the 10 most frequently used topics also appeared regularly in the other set, suggesting that academics and practitioners view many of the same topics as highly important. However, several of the most frequently used topics differed, suggesting a degree of misalignment. Practitioner literature tended to focus on topics associated with the physical implementation and application of ITSM, while, academic literature highlighted the ideaof implementation.

Research gaps identified

The study uncovered four broad practice-oriented research gaps. For each of these, three examples of possible research questions are provided, using a taxonomy proposed by Jarvinen (2000). They are categorised as conceptual-analytical, theory creating or testing, and artefact building or evaluating.

1.    Combining Frameworks

The combination of different frameworks by organisations, to support the use of ITSM, is frequently discussed in practitioner literature. In contrast, most academic papers only consider the use of one framework at a time. Some papers do present evidence that firms combine two frameworks: CoBIT (Control Objectives for Information and Related Technologies) and Information Technology Infrastructure Library (Cater-Steel, Tan, & Toleman, 2006; de Espindola, Luciano, & Audy, 2009; Lapão, 2011; Vogt, Küller, Hertweck, & Hales, 2011). However, practitioners discuss a wide range of frameworks that organizations use simultaneously.

Potential research questions:

  • How does co-implementing frameworks help strengthen areas that a single framework does not cover, such as business-IT alignment, knowledge management, organizational learning, outsourcing, and competitive advantage? (Conceptual analytical)
  • What theory can best reflect why different organizations consider strategic and technical factors when choosing to co-implement ITSM frameworks? (Theory creating)
  • Can one develop a model that indicates the most appropriate mix of ITSM frameworks based on an organization’s specific requirements? (Artefacts building)

2.    Infrastructure

Little academic research has addressed how improvements in infrastructure help organizations achieve beneficial outcomes of implementing ITSM. Further, research has not described the impact that implementing ITSM has on an organization’s infrastructure or cloud computing.

Potential research questions:

  • Which ITSM processes, if any, contribute to the effective management of cloud services? (Conceptual analytical)
  • As organizations increase their reliance on cloud service providers, what is the impact on the benefits that they receive when implementing ITSM? (Artefacts evaluation)
  • How do ITSM frameworks help organizations implement cloud services? (Conceptual analytical)

3.    Software and gamification

The practitioner literature often warns that IT departments implementing ITSM may prioritise software tools over processes, to their disadvantage. It further proposes that gamified tools may offer significant benefits in implementation. However, the advantages and difficulties associated with relying on tools when implementing ITSM are not discussed in the academic literature, nor are the effects of gamification examined.

Potential research questions:

  • How can an organization best use tools to support the implementation of ITSM? (Conceptual analytical)
  • Which kind of model could explain the benefits received due to the use of ITSM tools? (Theory creating)
  • How effectively does gamification help train staff in the ITSM processes—specifically as it concerns content retention and engagement and staff retention? (Artefacts evaluating)

4.    Regulation compliance

Practitioner literature suggests that several organizations implemented ITSM, motivated by the need to comply with regulations, such as the  Sarbanes-Oxley Act (SOX) introduced in in 2002. The impact of regulation on the implementation of ITSM is less evident in the academic literature.

Potential research questions:

  • What is the relationship between the types of regulations introduced and the ITSM organizations implement? (Conceptual analytical)
  • Which kind of model could explain how organizations implement ITSM due to the introduction of different regulations compared to other rationales for adoption? (Theory creating)
  • How effectively did SOX encourage organizations to pay closer attention to their IT governance?(Artefacts evaluating)

References

Carroll, C. E., & McCombs, M. (2003). Agenda-setting effects of business news on the public’s images and opinions about major corporations. Corporate Reputation Review, 6(1), 36-46.

Cater-Steel, A., Tan, W.-G., & Toleman, M. (2006). Challenge of adopting multiple process improvement frameworks. In Proceedings of the European Conference on Information Systems.

de Espindola, R. S., Luciano, E. M., & Audy, J. L. N. (2009). An overview of the adoption of IT governance models and software process quality instruments at Brazil—preliminary results of a survey. In Proceedings of the 42nd Hawaii International Conference on System Sciences.

Iden, J., & Eikebrokk, T. R. (2013). Implementing IT service management: A systematic literature review. International Journal of Information Management, 33(3), 512-523.

Jarvinen, P. (2000). Research questions guiding selection of an appropriate research method. In Proceedings of the European Conference on Information Systems.

Lapão, L. V. (2011). Organizational challenges and barriers to implementing IT governance in a hospital. Electronic Journal of Information Systems Evaluation, 14(1), 37-45.

Marrone, M., & Kolbe, L. M. (2011). Uncovering ITIL claims: IT executives’ perception on benefits and Business-IT alignment. Information Systems and E-Business Management, 9(3), 363-380

Marrone, M., & Hammerle, M. (2017) Relevant Research Areas in IT Service Management: An Examination of Academic and Practitioner Literatures. Communications of the Association for Information Systems 41(1), 517-543

Müller-Bloch, C., & Kranz, J. (2015). A framework for rigorously identifying research gaps in qualitative literature reviews. In Proceedings of the International Conference on Information Systems.

Vogt, M., Küller, P., Hertweck, D., & Hales, K. (2011). Adapting IT governance frameworks using domain specific requirements methods: Examples from small & medium enterprises and emergency management. In Proceedings of the Americas Conference on Information Systems.

Posted on

How do you find a research gap?

The aim of all research is to add to or enhance existing knowledge. Arguably, we can only achieve this once we understand the work that has already been carried out in a given field. There are varying opinions, depending on the field of inquiry and methodological approach involved, regarding the level of familiarity a researcher should have with existing literature prior to commencing fieldwork, however it is generally accepted that research should fill gaps in the literature. It is perhaps surprising, then, that so little is written about just how to find a “research gap” in the first place.

Finding the literature

The most obvious way to find a research gap is simply to read and analyse the relevant literature. However, this is easier said than done, as the volume of published literature can be staggering. Fortunately, there are some excellent bibliographic databases, which can speed the process of searching for relevant literature. Literature analysis may then be approached either qualitatively or quantitively.

Qualitative literature analysis

A qualitative analysis may involve the development of a concept matrix (Webster & Watson, 2002) or similar.

Figure 1: Example of a concept matrix (adapted from Webster & Watson, 2002)

Legend: O (organisation), G (group), I (individual)

The concept matrix assists researchers to organise the literature they have read, according to the concepts it relates to. It can be adapted, depending on the area of interest. In the example above, for instance, units of analysis are included.

Quantitive literature analysis

Quantitative analysis of literature may be carried out using a variety of tools, from systematic reviews to meta-analyses, citation analyses, and text mining (Marrone, 2017). Choice of tools may be determined to some extent by the ability of the researcher to acquire or access the technical expertise to leverage them.

Find the gap, or create one?

Reading and analysing the literature may reveal gaps which can be explored, however Alvesson & Sandberg (2011) suggest that research gaps may also be created by the researcher. By linking together work which has previously been considered separately, a researcher can uncover uncharted territory. In this way, opportunities to contribute to existing knowledge are constructed (Locke & Golden-Biddle, 1997) rather than merely identified.

What does a “research gap” look like?

Several authors have sought to characterise research gaps, describing the various forms they can take, whether considered from the perspective of objectively identifiable existing gaps in research (gap finding) or as opportunities to construct new “gaps” (gap creation). Some examples are summarised below.

Figure 2: Characteristics of research gaps (Click to see image bigger)

Gap-finding frameworks

Step-by-step guides to finding research gaps are hard to come by, however in the field of medical epidemiology, a framework for identifying research gaps from systematic reviews of literature has been published (Robinson et al., 2011). In this field, the PICOS framework (Population, Intervention, Comparison, Outcome, Setting) is commonly used to characterise a research gap. Robinson et al. (2011) suggest that an analysis of the reason for the existence of the gap can further inform the development of research questions. The reasons elucidated by Robinson et al. (2011) for the existence of research gaps are similar to the characteristics of gaps described by other authors, as shown in figure 2 (Characteristics of Research Gaps).

There are many ways to go about identifying research gaps, perhaps so many that the options may on occasion be overwhelming. A considered approach, coupled with knowledge and utilisation of the tools available to assist in research gap-finding, is likely to result in improved research design.

References

  • Alvesson, M., & Sandberg, J. (2011). Generating research questions through problematization. Academy of Management Review. 36(2), 247-271 [doi:10.5465/AMR.2011.59330882]
  • Hallgren, M. (2012) The construction of research questions in project management. International Journal of Project Management, 30(7), 804-816.
  • Locke, K., & Golden-Biddle, K. (1997) Constructing opportunities for contribution: Structuring intertextual coherence and “problematizing” in organizational studies. Academy of Management Journal, 40: 1023–1062.
  • Muller-Bloch, C. & Kranz, J. (2015) A framework for rigorously identifying research gaps in qualitative literature reviews. International Conference on Information Systems 2015 [available at: https://aisel.aisnet.org/icis2015/proceedings/ResearchMethods/2/]
  • Marrone, M., & Hammerle, M. (2017). Relevant Research Areas in IT Service Management: An Examination of Academic and Practitioner Literatures. Communications of the Association for Information Systems, 41, 517-543.
  • Robinson, K. A., Saldanha, I. J. & McKoy, N. A. (2011) Development of a framework to identify research gaps from systematic reviews. Journal of Clinical Epidemiology, 64(12), 1325-1330.
  • Sandberg, J., & Alvesson, M. 2011. Ways of constructing research questions: Gap-spotting or problematization? Organization, 18: 23–44.
  • Webster, J., & Watson, R. T. (2002). Analyzing the past to prepare for the future: Writing a literature review. MIS Quarterly, 26(2), 13-23