Brain Experience

By Baihan Lin © 2022, All Rights Reserved. Presented at the Data Visualization Keynote of Parsons School of Design.

Live ebsite: howbrain.work

Demo video: https://youtu.be/T88bKQzu0P4

Which brain regions light up at which moments when seeing which images or listening certain music? And inversely what do brain imagine seeing or hearing given certain types of neural responses?

In Brain Experience, the data visualization enables the users to experience how human brains perform complicated tasks that we as people find easy. The mechanisms and processes beneath it, surprisingly, are highly complex and involve many organic elements. The projects can help the users unravel these beautiful mysteries that intrigue scientists and philosophers for ages. The cognitive tasks specific for this visualization are mostly likely to be either viewing images or listening to music.

Research Topics:

This project will:

• introduce the foundations of the perception process of the brains with visual storytelling and
• visualize the dynamic process of how different brain regions interact to perform these cognitive tasks,

Research Significance:

in order to help thereaders understand how the human brains process information in an organized, systematic and hierarchical way, so that they can:

1. have a knowledge of the underlying mechanisms when they perform daily tasks;
2. gain access to one of the research frontiers in the fields of cognitive and computational neuroscience;
3. come up with new research questions and pursue a scientific research;
4. build better neural network structures inspired by these neuroscientific discoveries; and
5. revisit their philosophical stances of life about how our consciousness and thinking processes are formed.

<br><br>

Research Outline:

<br><br>

Design Outline:

<br><br>

Design Wireframe:

<br><br>

References (ongoing):

• Lin, Baihan, Marieke Mur, Tim Kietzmann, and Nikolaus Kriegeskorte. "Visualizing representational dynamics with multidimensional scaling alignment." arXiv preprint arXiv:1906.09264 (2019).
• Kriegeskorte, N., & Kievit, R. A. (2013). Representational geometry: integrating cognition, computation, and the brain. Trends in cognitive sciences, 17(8), 401-412.
• de Vries, Saskia EJ, Jerome A. Lecoq, Michael A. Buice, Peter A. Groblewski, Gabriel K. Ocker, Michael Oliver, David Feng et al. "A large-scale standardized physiological survey reveals functional organization of the mouse visual cortex." Nature neuroscience 23, no. 1 (2020): 138-151.
• Kriegeskorte, Nikolaus. "Deep neural networks: a new framework for modeling biological vision and brain information processing." Annual review of vision science 1 (2015): 417-446.