The world is awash in data visualizations, from charts accompanying news stories on the economy to graphs tracking the weekly temperature to scatterplots showing relationships between baseball statistics.
At their core, data visualizations convey information, and everyone consumes that information differently. One person might scan the axes, while another may focus on an outlying data point or examine the magnitude of each colored bar.
But how do you consume that information if you can’t see it?
Making a data visualization accessible for blind and low-vision readers often involves writing a descriptive caption that captures some key points in a succinct paragraph.
“But that means blind and low-vision readers don’t get the ability to interpret the data for themselves. What if they had a different question about the data? Suddenly a simple caption doesn’t give them that. The core idea behind our group’s work in accessibility has been to maintain agency for blind and low-vision people,” says Arvind Satyanarayan, a newly tenured associate professor in the MIT Department of Electrical Engineering and Computer Science (EECS) and member of the Computer Science and Artificial Intelligence Laboratory (CSAIL).
Satyanarayan’s group has explored making data visualizations accessible for screen readers, which narrate content on a computer screen. His team created a hierarchical platform that allows screen reader users to explore various levels of detail in a visualization with their keyboard, drilling down from high-level information to individual data points.
Under the umbrella of human-computer interaction (HCI) research, Satyanarayan’s Visualization Group also develops programming languages and authoring tools for visualizations, studies the sociocultural elements of visualization design, and uses visualizations to analyze machine-learning models.
For Satyanarayan, HCI is about promoting human agency, whether that means enabling a blind reader to interpret data trends or ensuring designers still feel in control of AI-driven visualization systems.
“We really take a human-centered approach to data visualization,” he says.
An eye for technology
Satyanarayan found the field of data visualization almost by accident.
As a child growing up in India, Bahrain, and Abu Dhabi, his initial interest in science sprouted from his love for tinkering.
Satyanarayan recalls his father bringing home a laptop, which he loaded with simple games. The internet grew up along with him, and as a teenager he became heavily engaged in the popular blogging platform Movable Type.
A teacher at heart even as a teenager, Satyanarayan offered tutorials on how to use the platform and ran a contest for people to style their blog. Along the way, he taught himself the skills to develop plugins and extensions.
He enjoyed designing eye-catching and user-friendly blogs, laying the foundation for his studies in human-computer interaction.
When he arrived at the University of California at San Diego for college, he was interested enough in the HCI field to take an introductory class.
“I’d always been a student of history, and this intro class really appealed to me because it was more about the history of user interfaces, and tracing the provenance and development of the ideas behind them,” he says.
Almost as an afterthought, he spoke with the professor, Jim Hollan — a pioneer of the field. Even though he hadn’t thought much about research beforehand, Satyanarayan ended up spending the summer in Hollan’s lab, studying how people interact with wall-sized displays.
As he prepared to pursue graduate studies (Satyanarayan split his PhD between Stanford University and the University of Washington), he was unsure whether to focus on programming languages or HCI. When it came time to choose, the human-centered focus of HCI and the interdisciplinarity of data visualization drew him in.
“Data visualization is deeply technical, but it also draws from cognitive science, perceptual psychology, and visual arts and aesthetics, and then it also has a big stake in civic and social responsibility,” he says.
He saw how visualization plays a role in civic and social responsibility through his first project with his PhD advisor, Jeffrey Heer. Satyanarayan and his collaborators built a data visualization interface for journalists at newsrooms that couldn’t afford to hire data departments. That drag-and-drop tool allowed journalists to design the visualization and all the data storytelling they wanted to do around it.
That project seeded many elements that became his thesis, for which he studied new programming languages for visualization and developed interactive graphical systems on top of them.
After earning his PhD, Satyanarayan sought a faculty job and spent an exhausting interview season crisscrossing the country, participating in 15 interviews in only two months.
MIT was his very last stop.
“I remember being exhausted and on autopilot, thinking that this is not going well. But then, the first day of my interview at MIT was filled with some of the best conversations I had. People were so eager and interested in understanding my research and how it connected to theirs,” he says.
Charting a collaborative course
The collaborative nature of MIT remained important as he built his research group; one of the group’s first graduate students was pursuing a PhD in MIT’s program in History, Anthropology, and Science, Technology, and Society. They continue to work closely with faculty who study anthropology, topics in the humanities, and clinical machine learning.
With interdisciplinary collaborators, the Visualization Group has explored the sociotechnical implications of data visualizations. For instance, charts are frequently shared, disseminated, and discussed on social media, where they are stripped of their context.
“What happens as a result is they can become vectors for misinformation or misunderstanding. But that is not because they are poorly designed to begin with. We spent a lot of time unpacking those details,” Satyanarayan says.
His group is also studying tactile graphics, which are common in museums to help blind and low-vision individuals interact with exhibits. Often, making a tactile graphic boils down to 3D-printing a chart.
“But a chart was designed to be read with our eyes, and our eyes work very differently than our fingers. We are now drilling into what it means to design tactile-first visualizations,” he says.
Co-design is a driving principle behind all his group’s accessibility work. On many projects, they work closely with Daniel Hajas, a researcher at the University College of London who has been blind since the age of 16.
“That has been really important for us, to make sure as people who are not blind, that we are developing tools and platforms that are actually useful for blind and low-vision people,” he says.
His group is also studying the sociocultural implications of data visualization. For instance, during the height of the Covid-19 pandemic, data visualizations were often turned into memes and social artifacts that were used to support or contest data from experts.
“In reality, neither data nor visualizations are neutral. We’ve been thinking about the data you use to visualize, and the design choices behind specific visualizations, and what that is communicating besides insights about the data,” he says.
Visualizing a real-world impact
Interdisciplinarity is also a theme of Satyanarayan’s interactive data visualization class, which he co-teaches with faculty members Sarah Williams and Catherine D'Ignazio in the Department of Urban Studies and Planning; and Crystal Lee in Comparative Media Studies/Writing, with shared appointments in the School of Arts, Humanities, and Social Sciences and the MIT Schwarzman College of Computing.
In the popular course, students not only learn the technical skills to make data visualizations, but they also build final projects centered on an area of social importance. For the past two years, students have focused on the housing affordability crisis in the Boston area, in partnership with the Massachusetts Area Planning Council. The students enjoy the opportunity to make a real-world impact with their work, Satyanarayan says.
And he enjoys the course as much as they do.
“I love teaching. I really enjoy getting to interact with the students. Our students are so intellectually curious and committed. It reassures me that our future is in good hands,” he says.
One of Satyanarayan’s personal interests is running along the Charles River Esplanade in Boston, which he does almost every day. He also enjoys cooking, especially with ingredients he has never used before.
Satyanarayan and his wife, who met while they were graduate students at Stanford (her PhD is in microbiology), also delight in tending their plot in the Fenway Victory Gardens, which is overflowing with lilies, lavender, lilacs, peonies, and roses.
Their newest addition is a miniature poodle puppy named Fen, which they got when Satyanarayan earned tenure earlier this year.
Thinking toward the future of his research, Satyanarayan is keen to further explore how generative AI might effectively assist people in building visualizations, and its implications for human creativity.
“In the world of generative AI, this question of agency applies to all of us,” he says. “How do we make sure, for these AI-driven systems, that we haven’t lost the parts of the work we find most interesting?”
