Stanford CS547 HCI Seminar | Spring 2026 | Show It or Tell It? Text, Visualization, and Combination
Stanford Online · 56:18 · 4 days ago
Text and visualization should be treated as a combined, co-equal system rather than separate elements, yet establishing clear design rules remains difficult because user preferences are highly context-dependent and cognitive models regarding their interaction remain conflicting.
-
Design minimalism — Research suggests that viewers prefer charts with more descriptive text, which contradicts traditional "minimalist" interface design principles that prioritize reducing ink .
-
Captions vs. visuals — When captions contradict the visually most salient regions of a chart, the text often dictates the viewer's final takeaway rather than the prominent visual features .
-
Perceived bias — While text annotations have little effect on a viewer's prediction of data trends, they serve as a potent signal for the audience to judge the author's potential bias .
-
Accessibility preferences — Sighted readers generally prefer high-level interpretations in descriptions, whereas blind and low-vision users often prefer low-level, concrete data points .
-
Reading fluency — Inserting non-text visuals directly into paragraphs can interrupt the "parafovea"—the area of vision that processes upcoming words—which is essential for maintaining fluent reading .
-
Cognitive theories — Conflicting psychological models, such as Dual Coding theory (which suggests combined modes enhance learning) and Cognitive Load theory (which suggests combined modes overwhelm memory), prevent a consensus on the best way to present integrated information .
-
Model architecture — Multimodal language models generally integrate inputs by either concatenating visual and textual embeddings into a shared vector space or using "cross-attention" layers to allow information to flow dynamically between the two data types .
-
How do multimodal language models utilize cross-attention to process images and text?