UW Interactive Data Lab
IDL logo

Uncertainty Displays Using Quantile Dotplots or CDFs Improve Transit Decision-Making

Michael Fernandes, Logan Walls, Sean Munson, Jessica Hullman, Matthew Kay. ACM Human Factors in Computing Systems (CHI), 2018
Figure for Uncertainty Displays Using Quantile Dotplots or CDFs Improve Transit Decision-Making
The One Bus Away Android Interface as seen by users as of September 2017 (left) alongside our adaption of the interface for testing the impact of uncertainty visualizations like quantile dotplots on transit decisions (right).
Materials
PDF | Honorable Mention Award
Abstract
Everyday predictive systems typically present point predictions, making it hard for people to account for uncertainty when making decisions. Evaluations of uncertainty displays for transit prediction have assessed people’s ability to extract probabilities, but not the quality of their decisions. In a controlled, incentivized experiment, we had subjects decide when to catch a bus using displays with textual uncertainty, uncertainty visualizations, or no-uncertainty (control). Frequency-based visualizations previously shown to allow people to better extract probabilities (quantile dotplots) yielded better decisions. Decisions with quantile dotplots with 50 outcomes were (1) better on average, having expected payoffs 97% of optimal (95% CI: [95%,98%]), 5 percentage points more than control (95% CI: [2,8]); and (2) more consistent, having within-subject standard deviation of 3 percentage points (95% CI: [2,4]), 4 percentage points less than control (95% CI: [2,6]). Cumulative distribution function plots performed nearly as well, and both outperformed textual uncertainty, which was sensitive to the probability interval communicated. We discuss implications for realtime transit predictions and possible generalization to other domains.
BibTeX
@inproceedings{2018-uncertainty-bus,
  title = {Uncertainty Displays Using Quantile Dotplots or CDFs Improve Transit Decision-Making},
  author = {Fernandes, Michael AND Walls, Logan AND Munson, Sean AND Hullman, Jessica AND Kay, Matthew},
  booktitle = {ACM Human Factors in Computing Systems (CHI)},
  year = {2018},
  url = {https://idl.uw.edu/papers/uncertainty-bus},
  doi = {10.1145/3173574.3173718}
}