What people say versus what people do: Developing a methodology to assess conceptual heterogeneity in a scientific corpus [iSchool Research Showcase]

Fu, Yuanxi; Schneider, Jodi

Permalink

https://hdl.handle.net/2142/124969 Copy

Description

Title

What people say versus what people do: Developing a methodology to assess conceptual heterogeneity in a scientific corpus [iSchool Research Showcase]

Author(s)

• Fu, Yuanxi
• Schneider, Jodi

Contributor(s)

• Clarke, Caitlin
• Lischwe Mueller, Natalie
• Manasi Ballal, Joshi
• Hsiao, Tzu-Kun

Issue Date

2024-11-06

Keyword(s)

• conceptual heterogeneity
• text embedding
• graph embedding
• knowledge graph
• research objectives
• evidence selection practices

Abstract

Conceptual heterogeneity poses significant challenges to constructing and reasoning over knowledge graphs for science. In commonsense knowledge, yoga is a type of exercise. However, in our Exercise and Depression dataset, 8 out of 27 systematic review reports EXCLUDED research on yoga to avoid confounds with yoga’s meditation elements already known to alleviate depression. Thus, conclusions drawn by those 8 publications will not apply to yoga. Due to this conceptual heterogeneity, reasoning engine should not use claims about exercise extracted from these 8 publications and “yoga is a type of exercise” to reason about the relationship between yoga and depression. This example shows why assessing conceptual heterogeneity is critical. We propose a novel methodology for assessing conceptual heterogeneity in a domain of science, based on conventional wisdom: do not listen to what people say; watch what they do. We transform our Exercise and Depression dataset into a similarity network using a metric we previously proposed, adjusted Jaccard similarity, to construct the “what-people-do” network. In the “what-people-do” network, there is an edge between each pair of systematic reviews weighted to indicate the extent (0-1) of the similarity between the two reviews’ evidence selection practices. We are experimenting with text and graph embedding to create the “what-people-say” network by embedding the research questions of the 27 systematic review reports to obtain the similarities between the research questions. We hypothesize that the correlation between the “what-people-do” network and the “what-people-say” network will provide a quantitative assessment of conceptual heterogeneity in a domain of science. If this hypothesis is proved, we argue that this methodology should be applied to a corpus of a scientific domain to gauge conceptual diversity before the corpus is digested into a knowledge graph.

Has Part

• https://doi.org/10.13012/B2IDB-4614455_V4
• https://doi.org/10.13012/B2IDB-6128763_V3
• https://doi.org/10.1162/qss_a_0028

Type of Resource

• still image
• text

Language

eng

Sponsor(s)/Grant Number(s)

• NSF CAREER 2046454
• Campus Research Board of the University of Illinois at Urbana-Champaign RB21012
• the 2024–2025 Perrin Moorhead Grayson and Bruns Grayson Fellow, Harvard Radcliffe Institute for Advanced Study

Owning Collections