Ramblings

These articles discuss various aspects of the relationship between sight and sound, and aim to provide a theoretical basis for the design and creation of audiovisual material. Links to scientific studies referenced in these articles can be found in the Related Research section of this site.

Audiovisual Correlations

When attempting to create an engaging, dynamic and aesthetically pleasing visual representation of an audio signal, one of the fundamental aims is to ensure that specific aspects of an audio signal can be recognized and relatable to what is being seen. One method of enacting this is to select single attributes of each perception and methodically link them together, so that changes in one auditory attribute will result in similar changes applied to a corresponding visual attribute, and vice versa.  Read More

The Audiovisual Experience

Pierre Schaeffer stated that linking music with an image is more complicated than linking a single sound to an image, as there is no cause and effect relationship between what is seen and what is heard1. If a simple sound is paired with its accompanying visual cause, there is an obvious basis in reality for the two stimuli to become fused together. However, with music, the realistic joining of audio and visual elements does not occur, instead the two are fused together in a wholly different way. Read More

Synchronicity = Simplicity?

It may seem simplistic and obvious to suggest that events occurring at the same time seem more connected, but the idea of synchronicity becomes more interesting when explored further. Temporal relationships between auditory and visual events are the building blocks of an audiovisual composition, and understanding how these can be created and the way we perceive simultaneous events can help towards a more effective combination of audio and visual material. Read More

Gestalt Visual Perceptions

Advancements in computer graphics and programs began a tendency for audiovisual sequences to be composed entirely of abstract geometrical and graphical objects, shapes and figures. The absence of realistic video footage containing natural, relatable visual content, such as human emotion, within these pieces requires a different set of ideas and theories towards the recognition of individual visual objects and events.

Theories regarding the way humans perceive visual objects and shapes were first developed by Max Wertheimer, Kurt Koffka and Wolfgang Koher during the 1920s. This group of German psychologists outlined principles that describe the natural, automatic recognition and categorization of visual elements into unified groups or whole structural entities, and these ideas became what are known as the principles of Gestalt visual perception. Read More