Color and Infrared

Color, Monochrome, and IR

MDCS3 and DCSG devices come in two varieties, color and monochrome. The basic imagers are the same, but the color type adds a striped color pattern on top of the imager photo sites. While color information can be useful in an application, it degrades the spatial resolution of the luminance signal, which is used by the stereo algorithms.

The CMOS imagers are infrared sensitive out to about 900 nm (near IR). For some applications, near-IR response is useful. In most cases, however, near-IR response degrades the sharpness of the image, since the IR wavelengths focus at a slightly different place. The MDCS3 includes IR cut filters to eliminate near-IR wavelengths.

Color vs. Monochrome

The MDCS3 and DCSG devices come in either color or monochrome versions. They both use the same underlying CMOS megapixel imaging array. The color version incorporates a Bayer pattern color filter on the pixels. Every set of four pixels has the color pattern:

R	G
G	B

A color image is reconstructed from this pattern by assigning R, G, and B values at every pixel of the color image. A luminance (monochrome) value can also be assigned by a suitable function of the four pixels. If the color image is the same size as the original Bayer image, then the colors and luminance must be interpolated. Because of the high resolution of the MDCS3, non-interpolated color images at 640 x 480 are also possible.

By contrast, in a monochrome imager each element of the imager array records just the luminance at that pixel. There is no color information, but the luminance information is recorded at the full resolution of the imager.

There are several important differences between the two types of stereo heads, in terms of image quality and stereo algorithms.

Because the color imagers have filters on the pixels, they receive less light, and thus have less sensitivity than the monochrome imagers.
The luminance spatial resolution of the color imagers is less than that of the monochrome imagers. Since the stereo algorithms use luminance information, the monochrome head is better for stereo.

Although the monochrome imagers give better spatial resolution than the color imagers, the large number of pixels means that there is almost always adequate spatial resolution for an application. Hence we do not recommend monochrome over color, except where low light levels may be a factor.

The response curves for the MDCS3 imagers are given just below.

Color Algorithm

Color interpolation from the Bayer pattern to RGB images is done on the host PC. The Small Vision System and the DCAM drivers incorporate three different algorithms for color interpolation.

Color binning. This algorithm is used when the image from the device has twice the resolution of the required output image. For example, if the device is sending a 640x480 image to the PC, and the output resolution is 320x240, then the binning algorithm is used. Binning gives the best color quality.
Edge-weighted color interpolation. This algorithm takes edges into account, and gives a good rendition of color when input and output images are the same resolution. It is a slower algorithm than (3), but the results are better.
Color averaging. This algorithm averages colors from neighboring pixels to complete the color image. It is very fast, but produces results that are less satisfactory than (2).

Below are blow-ups showing individual pixels, of algorithms (2) and (3), on black lettering. Note how much crisper the top (2) image is. The bottom (3) image shows a "zipper" effect on the vertical lines, where alternating lines are offset slightly.

Infrared Characteristics

The CMOS imagers in the MDCS3 are highly sensitive to near-infrared radiation (up to 950 nm). The following images show a scene with just IR light admitted to the imager, just visible light, and a combination of visible and IR.


Visible spectrum light only (< 700 nm)	Visible + infrared light	Infrared light only (> 700 nm)

Some objects reflect IR energy, e.g., vegetation and the lacquer in the Japanese tansu (the TV images were different over the three images, so ignore them).

Most lenses are optimized for visible light, and longer IR wavelengths are not focused at the same distance as visible light. This creates blur when both IR and visible light are admitted to the imager, for both color and monochrome stereo heads. If you click on the first two images above to get larger images, and look at the books on the bookshelf, you can see much more detail in them.

The Bayer color filter does not block near IR, so colors will be "washed out" if IR is admitted to the color imager.

For these reasons, both the monochrome and color MDCS3 devices incorporate a high-efficiency IR cut filter, which essentially eliminates all radiation over 700 nm.

For near-IR work, the filter can be removed, and IR pass filters placed over the lenses to eliminate visible light, as in the third image above. Please contact Videre Design if you are interested in this option.