Wednesday, January 7, 2009

Visual Language

Jacques Bertin was a pioneer of visual language with his book, Semiology of Graphics. Bertin, a map maker, set forth a system for using and interpreting signs and symbols in various forms of graphic presentation. His concern was primarily with printed graphics.

Jock Mackinlay proposed a practical implementation of some of Bertin's work in his 1986 paper, Automating the Design of Graphical Presentations of Relational Information. Many others have pushed the area of technology called information visualization forward, notably Edward Tufte in his books and lectures.

In an age of virtual reality, though, the types of graphics that can be presented are as rich and varied as reality itself, and even beyond, and the interpretation just as subtle and ambiguous. Experiencing virtual reality (VR) graphics is, in some ways, like watching a movie, and perhaps the best guides to interpretation are those on the language of cinema. In particular, James Monaco's classic How to Read a Film comes to mind, along with Jennifer van Sijll's Cinematic Storytelling.

One factor is that electronic graphics displays can show animation and other time-based effects. This involves a different set of conventions from printed graphics. For example, in a comic strip, adjacent panels

indicate a sequence of events, separated in time.

However, in a movie, a split-screen like this,

usually represents simultaneous action. The actions in the two portions of the screen are going on concurrently.

That's just a simple example, but it illustrates how our interpretation of very similar visual effects depends on the context they appear in. We'll talk about that more later.

No comments: