New Structure, New Piece.

New projection mapping project underway, finished making the physical version of the designed structure. This has been built using card board from cereal boxes … I have so much cereal now;

The visuals will be split between the figure and ground, with audio-reactive visual effects being mapped onto the objects in the foreground and pre-composed textures and effects in the background. One video is going to be split into 4 x 4 grid, creating sixteen sections that will be mapped to the front two faces of the cubes. The second video will be split in two and layered behind the objects.

The audio for this piece will be ‘Yesterday, Tomorrow, Today’ by Ital Tek, check it out below;

 

New piece coming soon!

Pierre Schaeffer’s contribution to Audiovisual Theory

Link to an article that documents the work of Pierre Schaeffer, and describes his ideas that informed an unofficial audiovisual theory. He discusses how music and realistic, natural sounds combine differently with visual events, and also the idea that synchronous auditory and visual events can have the same effect as additive synthesis.

Download

Visual Illusion Induced by Sound

The interconnectivity of our auditory and visual senses is an important aspect to consider when composing audiovisual material, as the simultaneous perception of both auditory and visual events can result in one sense changing and manipulating the other. Research undertaken to investigate the affect and influence each sense can exert on the other highlights interesting aspects of the relationship between auditory and visual phenomena.

The human visual system perceives events in three-dimensional space, selecting, identifying and obtaining information regarding an object and its movement. The eye is more spatially aware of events and activity occurring in the surrounding environment (Chion, 1994, p. 11). In contrast, the human hearing system can only provide limited information concerning the spatial aspects of an environment, instead being described by Salter (2011, p. 205) as having a ‘higher temporal resolution than the eye’. This description is supported by Chion (1994, p. 10), who explains that the hearing system processes information faster than the eye. Although these differences exist, Salter (2011, p. 205) argues that perception is seen as co-structuration, with sensorial events being created by the instantaneous connection between bodily, cognitive and environmental stimuli.

Research focused directly on the interconnectivity of auditory and visual senses has proven that the perceived intensity of a visual stimulus is enhanced in the presence of sound, showing that perceiving auditory and visual material simultaneously generates a more effective experience for the viewer.

Once such experiment that emphasizes the interconnectivity of auditory and visual perceptions, and the way one sense can influence and manipulate the other, has been carried out by Shams et al. (2002). This research explored how the perception of visual objects can be influenced by an auditory stimulus. Test subjects were presented with a flashing white circle on a computer screen and a number of audible beeps, and were asked to count the amount of flashes that they saw. When multiple beeps accompanied a single visual flash, subjects reported seeing multiple flashes. The correct number of flashes could be perceived once the time difference between each stimulus exceeded 70ms, thus exposing the temporal limits of this effect.

The results of this experiment prove that auditory stimuli can have a significant influence over the sense of sight, and, through an illusion, actually add to what we visually perceive. This research also demonstrates the interconnectivity of each perception and the ability of sound to influence and enhance vision, and could be manipulated for the creation of complex, perceptually challenging audiovisual compositions.

Link to PDF download of research article:

Download

The Audiovisual Experience

Pierre Schaeffer stated that linking music with an image is more complicated than linking a single sound to an image, as there is no cause and effect relationship between what is seen and what is heard1. If a simple sound is paired with its accompanying visual cause, there is an obvious basis in reality for the two stimuli to become fused together. However, with music, the realistic joining of audio and visual elements does not occur, instead the two are fused together in a wholly different way. The relationship between music and the image has a much stronger basis in subjective emotional and aesthetic responses than in realistic objectivity, causing the simultaneous perception of musical audio and imagery to create a whole new experience entirely; the audiovisual experience. Read More

Early Projection Mapping Tests

Messing around with some projection mapping:

and: