Posted on

Blowing Hot and Cold: What can topic dynamics tell us about research gaps?

What are “hot” and “cold” topics and how do we identify them?

The amount of scientific interest shown in different topics varies over time. Those generating increasing levels of interest are “hot” topics, while topics with falling levels of interest are “cold” (Antons, 2016).

The level of scientific interest in a topic is reflected by its prevalence in published literature, so analysis of the occurrence of topics in the literature over time can provide an indication of topic dynamics (Griffiths & Steyvers, 2004).Entity linking is an incredibly useful tool for identifying topic occurrence in a body of literature. By taking account of the context in which a concept is mentioned, entity linking can pick up trends which would be missed by analyses based on keywords alone.

Why should researchers care about topic dynamics?

Information about topic dynamics can give us an insight into the ways in which scientific interest in different topics has waxed and waned in the past. This can be useful in historical research. The topic landscape can also be mapped for different journals, allowing researchers to select the most appropriate journals to which they should submit their papers (Lee, 2018).

Further, we can use data on topic dynamics to make predictions about future research trends. We may build predictive models, or we may simply assume past growth to be an indicator of future potential. An understanding of topic dynamics can therefore help researchers and funders to direct research efforts to areas where there is most likely to be demand.

Finding research gaps

Perhaps of greatest interest to potential researchers are hot topics associated with relatively small numbers of published articles. These represent emerging areas of scientific interest and may be the richest source of research gaps, waiting to be filled. However, hot topics already associated with a large body of published work, may still present opportunities to apply different or multiple levels of analysis, while cold topics may represent areas where interest can be revived (Antons, 2016).

Analysis of topic dynamics may also reveal be opportunities to connect related areas of research which have previously been considered separately. Hopp (2018) mapped the topic landscape of disruption research between 1975 and 2016 to discover two increasingly disconnected subnetworks within the field. They suggest that reconnecting these areas should be a research priority.

Technological advances have resulted in the development of excellent tools for analysing the vast bodies of literature available to us. This presents researchers with the opportunity to understand the dynamics of scientific interest, allowing them to direct their work towards the areas where it may have the greatest impact.

Antons, D., Kleer, R. & Salge, T. O. (2016) Mapping the Topic Landscape of JPIM, 1984-2013: In Search of Hidden Structures and Development Trajectories. Journal of Product Innovation Management, 33(6), 726-749.

Griffiths, T.L. and Steyvers, M., 2004. Finding scientific topics. Proceedings of the National academy of Sciences, 101(suppl 1), pp.5228-5235.

Hopp, C., Antons, D., Kaminski, J. & Salge, T. O. (2018) The Topic Landscape of Disruption ResearchA Call for Consolidation, Reconciliation, and Generalization. Journal of Product Innovation Management, 35(3), 458-487.

Lee, H. & Kang, P. (2018) Identifying core topics in technology and innovation management studies: a topic model approach. Journal of Technology Transfer, 43(5), 1291-1317.

Posted on

Smart Cities: a literature review (in plain English!)

Summary of “Smart Cities: A Review and Analysis of Stakeholders’ Literature” (Marrone and Hammerle, 2018)

There has been increasing interest in recent years in the use of digital technology help deal with the “wicked problems” of environmental degradation and poverty in towns and cities. Cities where attempts are made to achieve this are known as “smart cities”.

This literature review compared the views of different groups of people on the idea of “smart cities”, seeking to compare diverse perspectives by examining the topics discussed in different categories of publication. Since what people read, hear and see will influence and reflect their views, analysis of the publications they are exposed to can give us an insight into those views (McCombs and Shaw 1972; Carroll and McCombs 2003). In this study, for example, the views of those who live in towns and cities were considered by reviewing news media, while the views of those involved in research organisations were analysed using academic publications (see table).

Group Literature category
Citizens News media
People involved in business Trade publications
People involved in research Academic publications
People involved in government Government reports

The topics arising in different categories of literature were compared using resgap.com technology. Key topics forallcategories of literature were:

  • Internet of Things
  • Technology
  • Infrastructure
  • Smart grid
  • Urban planning
  • Energy
  • Transport
  • Innovation
  • Sustainability

Key topics which arose frequently in news media but less so in other categories of literature, suggesting that citizens were concerned about them but that other groups did not consider them to be of such high importance, were:

  • Autonomous car
  • Hackers
  • Start-up company

Further analysis of these topics revealed some interesting differences between the ways in which they were discussed in news media and in other categories of literature. In the case of the “Autonomous car” topic, all categories of literature addressed the benefits of autonomous cars. However, while other literature types focused more on how a reliance on autonomous vehicles might come about, news media tended to present this transportation method as potentially disruptive, considering the risks associated with it. News media was also the only literature category to focus on how peoplemight be involved in the use of autonomous cars.

On the topic of “Hackers”, news media presented more detail regarding the intricacies of hacking, compared with other types of literature, and suggested reasons why hackers have not yet become widespread in smart cities. News media expressed the importance of preventing hacking to protect the people who use smart city services and emphasised how lack of action on the part of companies and governments could leave smart city services open to attack from hackers.

Regarding “Start-up company”, although all categories of literature highlighted the importance of start-ups in the development of smart cities and of fostering connections between different groups to enable start-ups to be successful, news media alone specifically highlighted how innovations brought about by start-ups may help to serve people and impact their everyday lives. Other literature types were more focused on the opportunities for economic growth and profits brought about by developments in the smart city space.

Existing academic research suggests that the perspectives of citizens are often ignored in the development of smart cities (Hollands 2015). The results of this review suggest that citizens are under-represented, rather than being completely ignored. The research gaps identified here are in  person-centred topics, such as privacy, which important to citizens. These should be addressed by practitioners involved in developing and marketing smart city services and by government and academic bodies involved in producing smart city policies.

References:

Carroll CE, McCombs M (2003) Agenda-setting effects of business news on the public’s images and opinions about major corporations. Corp Reput Rev6:36–46. https://doi.org/10.1057/ palgrave.crr.1540188

Hollands RG (2015) Critical interventions into the corporate smart city. Camb J Reg Econ Soc8:61–77

Marrone, M, Hammerle, M (2018) Smart Cities: A Review and Analysis of Stakeholders’ Literature. Bus Inf Syst Eng 60: 197. https://doi.org/10.1007/s12599-018-0535-3

McCombs ME, Shaw DL (1972) The agenda-setting function of mass media. Public Opin Q36:176–187

Posted on

A thousand cultures: bridging interdisciplinary divides

In his “Two Cultures” lecture, C.P. Snow (1961) spoke of a “gulf of mutual incomprehension” between scientists and scholars of the humanities, resulting from their disparate perspectives on the world and what we can hope to know about it. Since the enlightenment, scientists have sought to unearth objective facts and present them in an unbiased manner, while scholars of the humanities have celebrated the role of human interpretation in building knowledge and understanding. In the succeeding decades, interdisciplinary or even transdisciplinary work has been called for, to solve increasingly complex problems, yet divides persist.

Interdisciplinary divides are not only in evidence between the sciences and humanities. They exist between quantitative and qualitative researchers, between practitioners and theorists, and between academic and business sectors (Kahn, 2011). Echoing the title of Deleuze and Guattari’s (1987) seminal book. “A Thousand Plateaus”, it is perhaps more accurate to speak of a thousand cultures than of two. Deleuze and Guattari used the analogy of rhizomes (modified plant stems which run underground horizontally) to describe aggregations of multiplicity as an alternative to “arboreal” conceptions of diverging roots or branches. We may consider “disciplines” in a similar way (see Figure 1). Within a rhizomatic conception of research, groups of researchers may share enough qualities that they may be considered to form a discipline, but each discipline will still be connected to the others.

Figure 1: Possible means of conceptualising disciplines. The image on the left is of a traditional, “arboreal” conception of disciplines, represented as a branching tree diagram. The image on the right is of an alternative, rhizomatic conception of disciplines, a horizontally interconnected network of researchers, represented here by numbered circles.

When we broaden our conception of cultures or disciplines, we acknowledge a wider range of types of interdisciplinary divide. Beyond the intellectual divides described by Snow (1961), structural divides emerge, including gulfs in the understanding and use of technology. There are institutional divides concerning conventions in career progression and journal publication, differences in academic calendars, and differences in working practices between academia and business (Kahn, 2011). Further, there is variation in conventions and language across different geographic locations.

How entity linkers can help us cross the divides

Different terms may be used in different disciplines to describe the same concept, or the same term may refer to different concepts depending on its disciplinary context. Both phenomena were encountered in a recent review of literature on the concept of “green prescriptions”. In New Zealand, the term, “green prescription” is used to describe the prescribing of physical activity as therapy (e.g. Anderson, 2015). In Europe, the term has been similarly adopted to describe the prescribing of activities carried out in natural settings as therapy (Van den Berg, 2017). This practice is also referred to in Europe as a performing “nature-based” (e.g. Barton, 2015) or “ecotherapy” (Bibby, 2013) interventions. In Germany, however, “green prescriptions” refer to a type of medical prescription, usually for complementary or supplementary medication (e.g. Heyde, 2014).

As the purpose of the literature review was to explore the prescribing of activities in natural environments, papers describing the New Zealand concept of a “green prescription” and papers describing the European concepts of green prescriptions and nature-based or ecotherapy interventions were relevant, while papers on the German “green prescription” were not. An entity linker would enhance a literature review of this kind, by expediting the uncovering of associated terms such as “nature-based intervention” while discarding unrelated papers, such as those using the term “green prescription” to describe the prescribing of supplementary medication.

Entity linkers have great potential aligning the different ways in which a term is discussed across disciplines, by removing ambiguity. Harnessing this capacity to search for references to the entity in question, discarding references to unrelated entities and including references which use different terminology, we may begin to bridge the divides between a thousand cultures or disciplines.

 

References

Anderson, Y. C., Taylor, G. M., Grant, C. C., Fulton, R. B. & Hofman, P. L. (2015) The green prescription active families programme in Taranaki, New Zealand 2007–2009: Did it reach children in need? Journal of Primary Health Care, 7(3), 192-197.

Barton, J., Sandercock, G., Pretty, J. & Wood, C. (2015) The effect of playground- and nature-based playtime interventions on physical activity and self-esteem in UK school children. International Journal of Environmental Health Research, 25(2), 196-206.

Bibby, P., Wild, A. & Bodell, S. (2013) The benefits of ecotherapy interventions on mental health conditions. British Journal of Occupational Therapy, 76, 47-47.

Deleuze, G. & Guatarri, F. (1987) A Thousand Plateus: Capitalism and Schizophrenia. (2ndEdn) University of Minnesota Press

Heyde, I., Böschen, D., Dicheva, S., Hinrichs, A. & Peters, H. (2014) Folic acid on green prescription: When adjunctive therapy is reimbursed. Deutsche Apotheker Zeitung, 154(19), 36-38.

Kahn, J. (2011) The two (institutional) cultures: a consideration of structural barriers to interdisciplinarity. Perspect Biol Med.,54(3):399-408.

Snow, C.P. (1961) The Two Cultures and The Scientific Revolution. New York: Cambridge University Press

Van den Berg, A. E. (2017) From green space to green prescriptions: Challenges and opportunities for research and practice.Frontiers in Psychology, 8.

Posted on

Literature reviews – Keyword searches don’t work

The trouble with keyword searches

Keywords are expected to help us identify relevant papers, when we conduct literature searches. Unfortunately, they’re not as effective at doing this as we might hope, since they aren’t always representative of the content of an article.

Subjectivity in keyword  selection

Keywords may be selected by the author of a paper, in which case are likely to represent the themes which the author deems most important in their article (Névéol et al. 2010).  However, these may not necessarily correspond with the dominant themes found in the paper itself.  When authors do not provide keywords to accompany their own publications, they may be selected by editors, who then add their subjective interpretations of the text (Gerdsri et al., 2013; p.420).

Inconsistent terminology

Terminology used in keywords may vary according to preference, so that different terms are used by different authors  to represent the same concept. Where standardised indexing terms are used, such as the Medical Subject Headings (MeSH®) in the bibliographic database, MEDLINE®, these can be substantially different from the author-selected keywords (Figure 1).

Author keywords MEDLINE Indexing Terms
Decision-making

Rural health services

Interhospital transport

Survival analysis

 Adult

Aged

Cohort studies

Decision making

Diagnosis-related groups

Female

Health services accessibility

Hospital mortality

Hospitals, community/organisation & administration

Hospitals, rural/organisation & administration

Humans

Intensive care units/utilisation

Length of stay

Male

Middle aged

New Hampshire/epidemiology

Outcome assessment (health care)

Patient transfer/statistics & numerical data

Prospective studies

Survival analysis

Figure 1: Author keywords and MeSH indexing terms assigned to a sample article indexed in MEDLINE (Névéol et al. 2010)

Limited number of keywords

Author-selected keywords are also usually limited in number, typically to between 4 and 8 per article. This small number of keywords is unlikely to provide a comprehensive overview of the topics or themes in an article. Indeed, indexers assigned an average of 13.0 (+/-11.9) terms to papers in a collection of 14,398 open-access articles in PubMed Central®, suggesting that a greater number of terms is required to capture the thematic content of most papers.

An alternative approach

Fortunately, there’s a better way to enhance literature searches. Entity linking allows us to consider the context of words as well as the relationships between them. By linking words which carry the same meaning to an entity, we can extract entities from text, rather than relying on subjectively assigned keywords. Entities represent the themes contained in the text, removing the ambiguity associated with varying use of terminology.

References

Névéol, A., Doğan, R. I., & Lu, Z. (2010). Author Keywords in Biomedical Journal Articles. AMIA Annual Symposium Proceedings, 2010, 537–541.

Gerdsri, N., Kongthon, A. & Vatananan, R. S. (2013) Mapping the knowledge evolution and professional network in the field of technology roadmapping: a bibliometric analysis. Technology Analysis & Strategic Management, 25(4), 403-422.

Posted on

Can entity linking be used for literature reviews?

Entity linking is a term used to describe the automated process, carried out by a computer, of identifying objects or concepts mentioned in a body of text. Take the following text, for example:

It’s difficult to remember a time when I wasn’t conducting literature searches and looking for research gaps to fill.

Concepts mentioned in this text include literatureand gaps. Each could refer to several different entities. Literature could represent the concept of writing as an art form, written work in general, or specifically academic literature. Because it is mentioned here in the context of a literature search however, it is likely to refer to academic literature. Gaps may be physical spaces between objects or conceptual breaks in continuity, but since they are mentioned in this text as research gaps, we can infer that these gaps are conceptual.

When a computer carries out the task of entity linking, it uses the context in which an entity is mentioned to identify which specific entity the text refers to. It does this by referring to a knowledge base, such as Wikipedia. If you haven’t heard of entity linking before, you may have seen it referred to by one of its other names: named entity linking, named entity disambiguation, named entity recognition and disambiguation, or named entity normalization.

Examples of the use if entity linking to assist in comparing groups of literature

1.    Comparing perspectives and attitudes

In a recent study of perspectives on smart cities, Marrone & Hammerle (2018) compared topics across news media, trade publications, academic articles and government reports. This allowed them to compare sources to which citizens, businesses, research organisations and governments were exposed, thus gaining insight into the attitudes and perspectives of these groups. The comparison was carried out using a entity linker, TAGME, which allowed search strings which referred to the same entity to be merged.

2.    Comparing practitioner and academic literature

In a second study by the same authors (Marrone & Hammerle, 2017), misalignment between practitioner and academic literatures was examined, again using an entity linker. Topics were compared across the two groups of literature, focusing on those which were salient in practitioner literature. This facilitated identification of areas on interest to practitioners which are not discussed regularly in academic literature. In short, it elicited research gaps – areas where research is needed by practitioners or is likely to be relevant to practice.

How entity linking is done

According to Piccinno and Ferragina (2014), the entity linking process, as carried out by the tool, TAGME, may be divided into three stages: spotting, disambiguation, and pruning.

  • Spotting involves scanning of the text for meaningful sequences to produce a set of possible mentions (such as literature searchesin the example text given above). The SEA then retrieves a list of candidate entities from its knowledge base for each mention. This list will contain all the possible meanings that it can associate with the mention (such as literature as art, as all writing or as academic literature).
  • Disambiguation then takes place, where the SEA connects a score with each candidate entity in the list, by modelling how strongly the entity correlates with the mention in its context. The connections with the highest scores become the candidate annotation (in the case of the mention, literature, in the example text, the candidate annotation could be academic literature).
  • Pruning is the final stage, in which the SEA decides if it will discard a candidate annotation based on the other annotations that it has made to the text. This decision will therefore depend on whether the annotation makes sense given the overall context of the text.

By removing ambiguities, entity linking can improve the performance of your data analysis. As an automated process, it prevents the introduction of bias, which occurs when we manually code text.

Sources:

Marrone, M. & Hammerle, M. (2017) Relevant research areas in IT Service Management: An examination of academic and practitioner literatures. Communications of the Association for Information Systems: Vol. 41 , Article 23. Available at: http://aisel.aisnet.org/cais/vol41/iss1/23

Marrone, M. & Hammerle, M. (2018) Smart Cities: A review and analysis of stakeholders’ literature. Business and Information Systems EngineeringAvailable at: https://doi.org/10.1007/s12599-018-0535-3

Piccinno, F., & Ferragina, P. (2014). From TagME to WAT: A new entity annotator. In Proceedings of the 1st International Workshop on Entity Recognition & Disambiguation(pp. 55-62).

Posted on

How do you find a research gap?

The aim of all research is to add to or enhance existing knowledge. Arguably, we can only achieve this once we understand the work that has already been carried out in a given field. There are varying opinions, depending on the field of inquiry and methodological approach involved, regarding the level of familiarity a researcher should have with existing literature prior to commencing fieldwork, however it is generally accepted that research should fill gaps in the literature. It is perhaps surprising, then, that so little is written about just how to find a “research gap” in the first place.

Finding the literature

The most obvious way to find a research gap is simply to read and analyse the relevant literature. However, this is easier said than done, as the volume of published literature can be staggering. Fortunately, there are some excellent bibliographic databases, which can speed the process of searching for relevant literature. Literature analysis may then be approached either qualitatively or quantitively.

Qualitative literature analysis

A qualitative analysis may involve the development of a concept matrix (Webster & Watson, 2002) or similar.

Figure 1: Example of a concept matrix (adapted from Webster & Watson, 2002)

Legend: O (organisation), G (group), I (individual)

The concept matrix assists researchers to organise the literature they have read, according to the concepts it relates to. It can be adapted, depending on the area of interest. In the example above, for instance, units of analysis are included.

Quantitive literature analysis

Quantitative analysis of literature may be carried out using a variety of tools, from systematic reviews to meta-analyses, citation analyses, and text mining (Marrone, 2017). Choice of tools may be determined to some extent by the ability of the researcher to acquire or access the technical expertise to leverage them.

Find the gap, or create one?

Reading and analysing the literature may reveal gaps which can be explored, however Alvesson & Sandberg (2011) suggest that research gaps may also be created by the researcher. By linking together work which has previously been considered separately, a researcher can uncover uncharted territory. In this way, opportunities to contribute to existing knowledge are constructed (Locke & Golden-Biddle, 1997) rather than merely identified.

What does a “research gap” look like?

Several authors have sought to characterise research gaps, describing the various forms they can take, whether considered from the perspective of objectively identifiable existing gaps in research (gap finding) or as opportunities to construct new “gaps” (gap creation). Some examples are summarised below.

Figure 2: Characteristics of research gaps (Click to see image bigger)

Gap-finding frameworks

Step-by-step guides to finding research gaps are hard to come by, however in the field of medical epidemiology, a framework for identifying research gaps from systematic reviews of literature has been published (Robinson et al., 2011). In this field, the PICOS framework (Population, Intervention, Comparison, Outcome, Setting) is commonly used to characterise a research gap. Robinson et al. (2011) suggest that an analysis of the reason for the existence of the gap can further inform the development of research questions. The reasons elucidated by Robinson et al. (2011) for the existence of research gaps are similar to the characteristics of gaps described by other authors, as shown in figure 2 (Characteristics of Research Gaps).

There are many ways to go about identifying research gaps, perhaps so many that the options may on occasion be overwhelming. A considered approach, coupled with knowledge and utilisation of the tools available to assist in research gap-finding, is likely to result in improved research design.

References

  • Alvesson, M., & Sandberg, J. (2011). Generating research questions through problematization. Academy of Management Review. 36(2), 247-271 [doi:10.5465/AMR.2011.59330882]
  • Hallgren, M. (2012) The construction of research questions in project management. International Journal of Project Management, 30(7), 804-816.
  • Locke, K., & Golden-Biddle, K. (1997) Constructing opportunities for contribution: Structuring intertextual coherence and “problematizing” in organizational studies. Academy of Management Journal, 40: 1023–1062.
  • Muller-Bloch, C. & Kranz, J. (2015) A framework for rigorously identifying research gaps in qualitative literature reviews. International Conference on Information Systems 2015 [available at: https://aisel.aisnet.org/icis2015/proceedings/ResearchMethods/2/]
  • Marrone, M., & Hammerle, M. (2017). Relevant Research Areas in IT Service Management: An Examination of Academic and Practitioner Literatures. Communications of the Association for Information Systems, 41, 517-543.
  • Robinson, K. A., Saldanha, I. J. & McKoy, N. A. (2011) Development of a framework to identify research gaps from systematic reviews. Journal of Clinical Epidemiology, 64(12), 1325-1330.
  • Sandberg, J., & Alvesson, M. 2011. Ways of constructing research questions: Gap-spotting or problematization? Organization, 18: 23–44.
  • Webster, J., & Watson, R. T. (2002). Analyzing the past to prepare for the future: Writing a literature review. MIS Quarterly, 26(2), 13-23