From Noise to Meaning: How Visual Information Makes Sense

2018-03-28T04:58:07Z (GMT) by Elizabeth Zavitz
A digital exhibition by Elizabeth Zavitz.
Faculty of Medicine, Nursing and Health Sciences

Presented by Monash University Library
March - June 2018
Hargrave-Andrew Library

Over our lifetimes, we get to see a varied collection of images that reflect the huge diversity of things around us. In reality, we are only likely to experience a very small subset of possible images. The things we end up seeing are not only constrained, but constrained in very predictable ways. Objects tend to have continuous edges and surfaces, and are constructed in ways consistent with the laws of the physical world. We also tend to see fewer, larger objects close to us, rather than the many smaller objects that are further away. These constraints produce regularity that brain can take advantage of to improve how it processes visual information.

Visual information comes in two forms: energy, and spatial structure. The ‘energy’ is the combination of frequencies and proportions present in the image, analogous to the combination of notes played simultaneously by an orchestra, or the palette of colours available to a painter. Spatial structure refers to how that energy is distributed across the space of an image: it is the melody the orchestra plays, or the scene the painter creates.

In this exhibit, I illustrate some of the regularities in which our brains specialise. The video begins with static-like noise: there is no structure. Gradually, the noise begins to look cloudy as the palette of energies is made more consistent with what we see in the natural world. The noise thins until the image becomes sparse and individual patterns emerge. The patterns then arrange themselves into elongated contours, which snap into focus as the energy across different spatial scales aligns at one point in space. Notice how the viewing experience becomes more comfortable as the visual information is arranged in increasingly natural ways and becomes more consistent with what the brain anticipates.

The regularities presented here have shaped the nature of our visual brains on both evolutionary and developmental timescales, and can provide an important lever towards understanding the inner working of neural networks.