## Sunday, January 4, 2009

### Signals and Samples

The area of technology that seems most closely related to art is called signal processing. Signal processing is really a collection of techniques and concepts that are used for manipulating pictures, sounds, TV, movies, and other kinds of electronic information. We don't need to know all the technical stuff, but there are two ideas that are useful for understanding digital art media. They're also kind of cool ideas that can be understood visually.

### Signals

The first, and biggest, idea in this area is signals themselves. What are they? Why should we care?

For our purpose, we can say a signal is a sequence of numbers. There are other kinds of signals that don't quite fall into this definition, but the ones we're thinking about with digital art media are sequences of numbers.

As one simple example, think about the time shown on a clock, compared with actual time that has passed. For example, since the start of this year, 2009, clocks in the U.S., that measure time in 12 hour cycles, produce a signal like this:

The green line shows what time is displayed on the clock. One hour after midnight, it shows 1:00, then 2:00, 3:00, and so on. When the clock display reaches 12:00, we switch from AM to PM (or vice versa) and start counting again. So even though time itself just keeps moving forward, hour after hour and day after day, the time shown by the clock repeats the cycle from 12:00 to 12:00.

I cheated a little bit here, because the green line doesn't look like a series of numbers, which is what we said a signal is. If you think about a digital clock, the time displayed by the clock only changes once per minute: 12:00, 12:01, 12:02, etc. So if this signal represents the time shown on a digital clock, then looking really closely at this signal, we'd see something more like this:

We get this kind of stairstep effect as the time on the clock jumps from one minute to the next. If the digital clock showed seconds, there would be 60 tiny stairsteps for each of the big ones shown here.

A lot of signals are measurements of something over time. For example, this shows the average high and low temperatures in New York City over the course of a year.

You can see that in January, both the high and low temperatures are on the cool side, but they both go up around July and August, and then down again by the end of the year. These could be considered two signals.

But not all signals compare measurements with time. I'll talk in more detail later about treating images as signals which are measurements of color compared with location ... the upper left corner is one color, the pixel next to it is another color, etc.

### Samples

So if we think of a signal as a sequence of numbers, then each number is a sample. Simple, isn't it? Usually, we get the samples by measurement ... we look at the clock face, or we check the thermometer, or we scan the points in an image to measure the color at each point.

There's an old joke that a clock that's completely stopped is better than one that loses one minute every hour ... the stopped clock is right twice a day! Suppose you had a clock that stopped at 8:17, and you only looked at it at 8:17 AM and 8:17 PM every day. You'd have no way of knowing if the clock was working or not. In other words, you wouldn't be able to tell this signal

from this one

The second signal, the stopped clock, is an alias for the original signal. So the fact that, based on the samples you have, the second signal looks like the first one, is called aliasing. The way to get around this is to take more samples ... look at the clock more than twice a day! Then you'll be able to tell which of these two signals the clock is producing. This way of getting around aliasing is anti-aliasing. It's just like the jagged edges you see in low resolution images. Higher resolution images don't have the same jaggedness, because there are more samples.

There are other ways to reduce the effect of the jaggies with low resolution images, and we'll talk about that later.