UW Interactive Data Lab
IDL logo

Concept Induction: Analyzing Unstructured Text with High-Level Concepts Using LLooM

Michelle Lam, Janice Teoh, James A. Landay, Jeffrey Heer, Michael Bernstein. ACM Human Factors in Computing Systems (CHI), 2024
Figure for Concept Induction: Analyzing Unstructured Text with High-Level Concepts Using LLooM
A summary of the LLooM concept induction algorithm. Status quo topic models tend to produce topics aligned with low-level keywords (e.g., “feminist, feminism”). We introduce LLooM, a concept induction algorithm that takes in unstructured text and produces high-level concepts (e.g., “Criticism of Feminism”) defined by explicit inclusion criteria. We instantiate this algorithm in the LLooM Workbench, a mixed-initiative text analysis tool that can amplify the work of analysts by automatically visualizing datasets in terms of interpretable, high-level concepts.
Materials
Abstract
Data analysts have long sought to turn unstructured text data into meaningful concepts. Though common, topic modeling and clustering focus on lower-level keywords and require significant interpretative work. We introduce concept induction, a computational process that instead produces high-level concepts, defined by explicit inclusion criteria, from unstructured text. For a dataset of toxic online comments, where a state-of-the-art BERTopic model outputs “women, power, female,” concept induction produces high-level concepts such as “Criticism of traditional gender roles” and “Dismissal of women's concerns.” We present LLooM, a concept induction algorithm that leverages large language models to iteratively synthesize sampled text and propose human-interpretable concepts of increasing generality. We then instantiate LLooM in a mixed-initiative text analysis tool, enabling analysts to shift their attention from interpreting topics to engaging in theory-driven analysis. Through technical evaluations and four analysis scenarios ranging from literature review to content moderation, we find that LLooM’s concepts improve upon the prior art of topic models in terms of quality and data coverage. In expert case studies, LLooM helped researchers to uncover new insights even from familiar datasets, for example by suggesting a previously unnoticed concept of attacks on out-party stances in a political social media dataset.
BibTeX
@inproceedings{2024-lloom-concept-induction,
  title = {Concept Induction: Analyzing Unstructured Text with High-Level Concepts Using LLooM},
  author = {Lam, Michelle AND Teoh, Janice AND Landay, James AND Heer, Jeffrey AND Bernstein, Michael},
  booktitle = {ACM Human Factors in Computing Systems (CHI)},
  year = {2024},
  url = {https://idl.uw.edu/papers/lloom-concept-induction},
  doi = {10.1145/3613904.3642830}
}