Monday, July 13, 2009

Public Service Message: No Sponsor Bias (so far!)

In the wake of this New York Times article on sponsorship of bloggers and social media contributors, let me state emphatically that this blog has not taken a dime from any sponsors. (As if!)

It's not for want of trying. I've made a few half-hearted attempts to monetize this blog, to use the term of art, but so far, all efforts have been for naught. (I'm open to any suggestions.)

If the situation changes in the future, I'll be sure to let you know. But for now, though you may see ads alongside the posts here, rest assured that you're getting pure content, unadulterated by any sponsorship or other commercial interests. (Well, except for blatant self-aggrandizement on the part of the author.)

Friday, March 6, 2009

The Graphics Pipeline (again)

I've been thinking more about the graphics pipeline, which I wrote about here. Sometimes a picture is worth a bunch of words, so here's a simplified diagram that may explain the idea better:



The particular inspiration for my current thinking was a documentary, The Pixar Story, currently airing on Showtime. This hour-and-a-half film gives a good overview of the evolving technology of computer animation, as well as the histories of some of the major players. It's worth a look.

By the way, speaking of art and technology, the above diagram was drawn with a program called Graphviz (http://www.graphviz.org). It reads a simple text file that describes your diagram, and then draws it automatically. For example, the text file for the above diagram was simply this:


digraph G {
node [shape=box];
edge [labeljust="l" arrowhead="empty"];
"3D Computer\nModel" -> "2D Image" [label="Computer\nRendering"];
"2D Image" -> "Final Image" [label="Retouching"];
Imagination -> "2D Image" [label="Artist\nRendering"];
"Real World" -> "2D Image" [label="Capture"];
"Real World" -> Imagination [label="Perception"];
"Real World" -> "3D Computer\nModel" [label="Scene/Motion\nCapture"];
Imagination -> "3D Computer\nModel" [label="CAD"];
}

Tuesday, March 3, 2009

Taxonometry?

On another blog, I posted about Taxonometrics, the measure of how well various organization schemes can classify and categorize information to make it accessible. This is certainly relevant to art, which encompasses much of the on-line content that is the subject of all these attempts at classification.

Monday, March 2, 2009

Any Rights Left?

The gradual erosion of creators' rights to their creations is as good a topic as any for this forum, for it certainly sits right on the frontier between art (or at least creativity) and technology. The recent kerfuffle over the Amazon Kindle 2 text-to-speech feature highlights this. Briefly, the second version of Amazon's electronic bookreader introduced the feature of automatically reading books out loud. The Author's Guild and others jumped on this, seeing it as an infringement of the audiobook rights for the books in question. In the face of this, Amazon backed down somewhat, and said the publishers' permission will be required for the speech feature to work.

But this is only the latest salvo in the escalating war between technology and creativity. The disturbing trends for authors, artists, and other creators are twofold:
  1. People expect more and more content to be free, and
  2. Technology enables more and more mash-ups, in which one person's content is re-purposed in some way.
The evidence of the first issue is the gradual erosion of the music recording industry, the newspaper industry, and others. As it becomes harder and harder to attract readers to paid content, it likewise becomes harder to attract advertisers, so revenue dries up. Why can't free content be paid for by advertising? That's pretty much what everyone's counting on, but so far, it's still just optimism. Also, the advertising model is at risk because of the second trend.

As more and more people scrape content from other Web sites, piece it together, re-package it, and present it without attribution or back links, that content gets stretched thinner and thinner, so the likelihood of its earning any ad revenue for the creator diminishes.

I'll talk about all these in more detail later, so this is just an overview. But, I'll throw out one more nugget for thought: Shepard Fairey. In a sense, Fairey represents both sides of the coin. He's the creator who wants unfettered use of other assets as sources. But he also staunchly defends his own intellectual property rights. This duality goes beyond his celebrated Obama image. You can read more about it here.

Thursday, February 19, 2009

Data + Art exhibit, Pasadena, CA

Some folks might be interested in the Data + Art exhibit currently at the Pasadena Museum of California Art. Subtitled Science and Art in the Age of Information, this appears to be an exhibit of work related to data visualization and other forms of data expression. To quote the blurb on the exhibit page, "These interpretations of data will empower the average person to see the invisible, hear the inaudible and understand the impossibly complex."

Personally, I find it interesting that this museum is devoted specifically to California art, as if that were a genre unto itself. I can understand museums devoted to local history and culture, and museums devoted to specific genres, styles or philosophical outlooks. But California as a category of art seems odd to me.

The exhibit runs through April 12.

Wednesday, February 18, 2009

Declarative vs. Procedural Formats (and farewell, Freehand and PostScript)

About two months ago, I posted a short note about Gaussian distributions and their uses in, for example, simulating airbrush painting. I included a couple of sample images based on an implementation I wrote in the PostScript programming language.

PostScript was interesting in that it was a full programming language that was designed to run on the printer. In other words, back in 1984, the printer ceased to be just an output device (WOM - write-only memory). It became a programmable computer that was part of the workflow.

PostScript was extremely successful, and in many ways revolutionized the printing industry. However, because all programs are susceptible to bugs, and bugs in a printer were difficult to trouble-shoot and correct, some PostScript workflows had problems. This, among other things, led Adobe to try to find a more stable print format. What they arrived at was a subset of PostScript in which everything was pre-defined so that no custom programs would be running in the printer. This format had been developed for use as Adobe Illustrator's particular dialect of PostScript, but Adobe now called it Portable Document Format (PDF), and began pushing that. In fact, they let PostScript languish, and have not added any new features or updated the language since PostScript 3, about 10 years ago.

This is a somewhat long-winded background to the real point, which has to do with the difference between procedural formats and declarative formats. Procedural formats can contain actual procedures, or scripts, to detail what the receiver should do with the data. Declarative formats are static in the sense that there's no programming involved. The receiver is supposed to know all the possible variants of data, and what to do with them. If some unfamiliar item of data is encountered, it's either ignored or treated as an error. Or, in other words, procedural formats can contain behavior. Declarative formats just contain data, and the receiver has to know how to behave in every possible situation.

So PostScript is procedural. There are some built-in operators for drawing basic shapes, images and text. But most PostScript documents are really programs that use procedures to describe the appearance of the page. Even fonts are procedures in PostScript.

Since procedural formats are prone to bugs, developers often react by seeking a declarative format, such as PDF. Everything you can put in a PDF is spelled out in the definition of the PDF format, and programs and devices that receive PDF are supposed to know what to do with all that stuff.

The problem with procedural formats, though, is the same as the benefit. Everything's spelled out in the definition. You can only do what's in the definition. The airbrush simulation I talked about would not be possible in PDF, because it relies on procedures. Now Adobe or others could define an airbrush feature in PDF, but then a new version of PDF would have to come out, and all the software and devices that can receive PDF would have to be upgraded to add this feature. More importantly, if you wanted to send an airbrush PDF document to someone, you would have to know if they already have upgraded hardware and software to be able to handle it. PostScript doesn't have this problem because the airbrush feature uses the PostScript language, so any PostScript language receiver would process it.

Oddly, later versions of PDF are adding features to support scripting and Flash, which are procedural. In other words, having gone to the trouble of cutting the procedural aspects of PostScript out to make PDF, PDF is now being made procedural again.

In the same way, the HTML standard that was developed for Web pages has been augmented with Java, JavaScript and other procedural content intended to add that programmability back into a format that was deliberately kept declarative. So maybe we should just admit that data formats, for the Web and for printing at least, have to include some procedural elements. Purely declarative formats are just too inflexible.

While I'm waxing nostalgic for PostScript, I'll also tip my hat again to the late, lamented Altsys Aldus Altsys Macromedia Freehand program. Among its many great features, this competitor of Adobe Illustrator allowed users to specify PostScript for each shape that was drawn, so you could create your own effects (e.g., airbrush) in PostScript, and then use them for any shape created in Freehand.

Oh, well.

Tuesday, February 17, 2009

Visual Thinking?

Kevin Cheng is an illustrator/cartoonist/human-computer interface designer/hmmm, probably could think of a few more. In particular, he's a big advocate of the use of comic strips in many contexts, including as a tool for designing computer user experiences. He blogs about this, and much else, at http://kevnull.com.

He's also posted about his upcoming presentations on the use of comics at Rosenfeld.com. The first is at VizThink, the conference on visual thinking. Doesn't that sound cool? It's next week, Feb. 22-25, in San Jose. I'm afraid I won't be able to be there, but it certainly sounds interesting. But the really cool thing is that VizThink is intended to be an on-going community, not just a one time, or even annual, conference. You can read more about it here.

Friday, January 23, 2009

Art Genomes?

Structuralist and post-deconstructionist and neo-post-pseudo-intellectualist-structuralites have been advancing the idea that we can classify and thus, to some extent, interpret or evaluate works of art by their semiotics. Briefly, this means we think of these art works as systems of signs that stand for things, and then consider the way the signs are configured within a work of art, and how that might reflect something about the things the signs stand for. Language is a semiotic system ... we use words to stand for things, and in writing, the combinations and sequences of words says something about the subject we're writing about. At least, we hope it does. Much of the 20th century philosophizing about this dealt with movies or, as it's more pompously known, the art of cinema.

That's all well and good, but now there's a very practical problem with is leading to a new way of thinking about this. Basically, the problem is that you've been listening to music through Pandora, or renting movies via Netflix, and you'd like to know what other music or movies would appeal to you.

These online content sources have tons of movies and music, but the problem is how to classify it so that you can identify which ones might appeal to audience members with certain tastes. Netflix, for example, asks you to rate each movie you rent, and over time, it builds up a database of what movies you've liked.

But there's still the similarity problem: If your favorite movies were Ishtar and Gigli, how to you decide which other movies are similar?

One promising answer is the movie genome. Basically, the idea is to identify a slew of properties that a movie can have, like plot, cast, awards, box office success, etc. Comparisons of movies on the basis of their genomes are likely to good matches and non-matches.

Now, you might think "Isn't this just a fancy way of comparing all these properties? Calling them a genome doesn't really change anything."

The answer is "Well, yes." It is just a fancy way of comparing these things. But treating all these properties as genes accomplishes two things:
  1. It makes the classification of movies more systematic, and
  2. It makes it possible to use some elaborate algorithms for doing the comparisons, finding near-matches, etc.
Obviously this genome idea could also be applied to books, plays, paintings, etc. It may lead to some new ideas about how to classify and interpret works in these art forms.

Or not.

Wednesday, January 7, 2009

Visual Language

Jacques Bertin was a pioneer of visual language with his book, Semiology of Graphics. Bertin, a map maker, set forth a system for using and interpreting signs and symbols in various forms of graphic presentation. His concern was primarily with printed graphics.

Jock Mackinlay proposed a practical implementation of some of Bertin's work in his 1986 paper, Automating the Design of Graphical Presentations of Relational Information. Many others have pushed the area of technology called information visualization forward, notably Edward Tufte in his books and lectures.

In an age of virtual reality, though, the types of graphics that can be presented are as rich and varied as reality itself, and even beyond, and the interpretation just as subtle and ambiguous. Experiencing virtual reality (VR) graphics is, in some ways, like watching a movie, and perhaps the best guides to interpretation are those on the language of cinema. In particular, James Monaco's classic How to Read a Film comes to mind, along with Jennifer van Sijll's Cinematic Storytelling.

One factor is that electronic graphics displays can show animation and other time-based effects. This involves a different set of conventions from printed graphics. For example, in a comic strip, adjacent panels



indicate a sequence of events, separated in time.

However, in a movie, a split-screen like this,



usually represents simultaneous action. The actions in the two portions of the screen are going on concurrently.

That's just a simple example, but it illustrates how our interpretation of very similar visual effects depends on the context they appear in. We'll talk about that more later.

Sunday, January 4, 2009

Signals and Samples

The area of technology that seems most closely related to art is called signal processing. Signal processing is really a collection of techniques and concepts that are used for manipulating pictures, sounds, TV, movies, and other kinds of electronic information. We don't need to know all the technical stuff, but there are two ideas that are useful for understanding digital art media. They're also kind of cool ideas that can be understood visually.

Signals


The first, and biggest, idea in this area is signals themselves. What are they? Why should we care?

For our purpose, we can say a signal is a sequence of numbers. There are other kinds of signals that don't quite fall into this definition, but the ones we're thinking about with digital art media are sequences of numbers.

As one simple example, think about the time shown on a clock, compared with actual time that has passed. For example, since the start of this year, 2009, clocks in the U.S., that measure time in 12 hour cycles, produce a signal like this:



The green line shows what time is displayed on the clock. One hour after midnight, it shows 1:00, then 2:00, 3:00, and so on. When the clock display reaches 12:00, we switch from AM to PM (or vice versa) and start counting again. So even though time itself just keeps moving forward, hour after hour and day after day, the time shown by the clock repeats the cycle from 12:00 to 12:00.

I cheated a little bit here, because the green line doesn't look like a series of numbers, which is what we said a signal is. If you think about a digital clock, the time displayed by the clock only changes once per minute: 12:00, 12:01, 12:02, etc. So if this signal represents the time shown on a digital clock, then looking really closely at this signal, we'd see something more like this:



We get this kind of stairstep effect as the time on the clock jumps from one minute to the next. If the digital clock showed seconds, there would be 60 tiny stairsteps for each of the big ones shown here.

A lot of signals are measurements of something over time. For example, this shows the average high and low temperatures in New York City over the course of a year.



You can see that in January, both the high and low temperatures are on the cool side, but they both go up around July and August, and then down again by the end of the year. These could be considered two signals.

But not all signals compare measurements with time. I'll talk in more detail later about treating images as signals which are measurements of color compared with location ... the upper left corner is one color, the pixel next to it is another color, etc.

Samples

So if we think of a signal as a sequence of numbers, then each number is a sample. Simple, isn't it? Usually, we get the samples by measurement ... we look at the clock face, or we check the thermometer, or we scan the points in an image to measure the color at each point.

There's an old joke that a clock that's completely stopped is better than one that loses one minute every hour ... the stopped clock is right twice a day! Suppose you had a clock that stopped at 8:17, and you only looked at it at 8:17 AM and 8:17 PM every day. You'd have no way of knowing if the clock was working or not. In other words, you wouldn't be able to tell this signal





from this one



The second signal, the stopped clock, is an alias for the original signal. So the fact that, based on the samples you have, the second signal looks like the first one, is called aliasing. The way to get around this is to take more samples ... look at the clock more than twice a day! Then you'll be able to tell which of these two signals the clock is producing. This way of getting around aliasing is anti-aliasing. It's just like the jagged edges you see in low resolution images. Higher resolution images don't have the same jaggedness, because there are more samples.

There are other ways to reduce the effect of the jaggies with low resolution images, and we'll talk about that later.