Small Vision System

SVS Features

The Small Vision System (SVS) is an efficient software implementation of the SRI stereo algorithms, running on standard PC hardware, either MS Windows or Linux.  These algorithms are 3 to 4 times faster than similar algorithms, and have high-quality filtering to reject false stereo matches.  Coupled with stereo cameras and an IEEE 1394 (Firewire) interface, it is a complete, low-cost development environment for realtime stereo applications. 

You can use the SVS with any of the stereo heads available from Videre design.  You can also use images available in the computer memory, under both MS Windows and Linux operating systems.

SVS Benefits

• Efficient

SRI International's patent-pending algorithms are the most efficient ones available -- up to 6 times faster than competing solutions.  Realtime capable: 320x240, 32 disparities at frame rates (30 Hz) with a PIII 700 MHz

• Accurate

Excellent range resolution and accuracy, using imagery from Videre Design cameras.  Range is interpolated to 1/16 pixel, and texture and consistency filters remove unreliable results.

• Flexible

Simple, fast, accurate stereo head calibration using a printable planar target.  You can change lenses, change baseline, even verge your cameras, and still calibrate them accurately.  Lens distortion, all internal camera parameters, and external camera parameters are computed.

• Convenient

Easy-to-use C++ library API for MS Windows or Linux makes programming applications simple.  Cross-platform display with the FLTK GUI and OpenGL 3D windows let you display results immediately.

SVS has an interface to MatLab, so stereo images and disparity results can be sent directly to MatLab for further analysis.

The XVision2 system, developed by Greg Hager at Johns Hopkins University, has an interface to SVS.  XVision2 concentrates on realtime segmentation and tracking in video images.

• Affordable

SVS comes bundled with Videre Design digital stereo heads.  It is also available as a software system for a modest licensing fee.

SVS Kit Contents

• Calibration software tool
• Stereo processing libraries
• Driver and capture application software for the MDCS line of stereo cameras
• Interface to MatLab
• Application and display software (sources included)
• User's Manual

SVS Requirements

• Pentium-compatible PC (Pentium MMX, AMD K6-2 or better) running MS Windows 98/ME/2000/XP with VC++ 6 or VNET++ 2003, or Linux

NOTE: For MDCS cameras, the interface uses SSE instructions.  These instructions are available in PIII-type processors, including AMD K7 processors and VIA Eden processors.

• Videre Design stereo camera head* or stereo images in host computer memory
• Digital frame grabber* for acquiring live images, if using a stereo camera.
   

* items are available from Videre Design, see Ordering

Accurate Range Results

This 640x480 color image, taken with a MEGA-D stereo head, shows the fine detail of the imagery.  Below, the SVS system has converted the stereo pair into an accurate 3D point cloud.  Click on the bottom image to view an animation of the point cloud.  The red ray is the camera line-of-sight.

SVS was run in standard mode, with no post-processing of the disparity or 3D image, other than the standard texture and L/R filters.

Realtime Performance

To fully appreciate the impact of full-motion stereo, you can download these Quicktime movies, which show several people moving in a corridor -- click on the images to show the movie. The left-hand movie is the video image, the right is the stereo disparity image. The frame size is 320 x 240, with 24 disparities, at 15 fps.

 

XVision2 Interface

XVision2 is the newest version of the XVision system developed by Greg Hager, now at Johns Hopkins University.  It is an open-source vision system, concentrating on realtime segmentation and tracking algorithms for video data.  The new version incorporates and interface to SVS, enlarging its scope with realtime stereo input.  XVision2 is a convenient way to experiment with new algorithms and applications using stereo.

XVision2 also has a MatLab interface to SVS data, so that the power of MatLab analysis can be brought to bear on your stereo data.  

Relevant subdirectories for SVS users are: 

    src/Devices/matlab
    src/Examples/showdisp

Software Specifications

• General
  • Two-image stereo computation
  • Arbitrary frame sizes
  • Area correlation algorithm
  • Video-rate implementation at up to 320x240 frames on standard PCs
• Calibration
  • Calibration routines by presentation of a simple planar target
  • Internal parameters - radial and tangential distortion, lens decentering, focal length, pixel aspect ratio
  • External registration - baseline, orientation of each camera
  • Based on Tsai's algorithms [Tsai 1991]
• Rectification
  • Bilinear interpolation using the calibration parameters
• Disparity computation
  • Laplacian of Gaussian image filter
  • Correlation: sum of absolute differences over a square window
  • Correlation window sides from 5 to 21 pixels
  • Disparity search from 8 to 128 pixels
  • Subpixel interpolation to 1/16 pixel
• Post-filtering
  • Low-texture confidence check
  • Uniqueness check
• 3D Reconstruction
  • Transform routines to generate 3D points from image point and disparity

System Description

The SVS is a set of algorithms implemented as a software library.  There are routines for: 

• Calibration of stereo heads using a simple planar target
• Capturing video streams using standard frame grabbers
• Computing dense stereo range images at video rates
• Displaying video images and range information

Host Requirements

The Stereo Engine code is written in optimized MMX assembly code for Pentium-based PCs running Linux or MS Windows. The recommended hardware configuration for best performance is a Pentium III/IV processor and a PCI bus, and a display card with at least 8 MB of video memory. 

If you have your own cameras, then you must use frame grabbers to digitize the stereo video stream and place it in main memory, where the Stereo Engine can process it.  You must write your own code to do this; SVS provides function calls to take images from memory and process them.

Videre Design has developed several stereo head assemblies that have direct interfaces to SVS. 

• STH-MDCS3 and STH-DCSG (fixed and variable baseline models)
  Digital devices with resolutions up to 1280 x 960, and a full range of user controls including sub-sampling and exposure control. They connect to a host system via the digital IEEE 1394 (Firewire) bus.

Performance

The SVS algorithms are optimized for Pentium processors with MMX instructions.  Frame rates are a function of frame size (number of pixels) times the number of disparities (search range).  Here are some timings on Pentium M and IV processors.

Correlation window: 15
Texture filter
Uniqueness filter
Input: rectified grayscale images
 

Processor	Speed	OS	Memory	Resolution	Disparities	FPS	FOM (Mp*d/s)
Pentium M	1.4GHz	MSW	500MB	512x384	48	28	264
Pentium M	1.4GHz	MSW	500MB	640x480	64	15	295
Pentium M	2GHz	Linux	1GB	512x384	48	43	405
Pentium M	2GHz	Linux	1GB	640x480	64	22	432
Pentium 4	2.5GHz	Linux	500MB	640x480	64	15	295

For demanding stereo applications, the recommended PC configuration is a Pentium M.  These processors, besides being power-efficient, are better than Pentium IV's at executing integer and MMX/SSE instructions, which are used heavily by the algorithm.

The Figure of Merit (FOM) is the best direct comparison of the efficiency of the algorithm on different systems.  It gives the number of pixel-disparities processed per second.

Because the stereo algorithms are storage-efficient, performance scales linearly with increasing frame sizes.  The algorithms execute almost entirely from L1 cache, so that future increases in processor speed will translate directly  to higher frame rates.  The diagram below, which normalizes different frame sizes and disparity ranges to a common scale based on the pixel-disparity law, shows how the amount of processing needed per pixel-disparity stays relatively constant across frame sizes and disparity ranges.

Additional Technical Information

• Small Vision System User's Manual - Version 4.x
• Sample Images
• Calibration
• SRI Tech Pages
• SVS on SunSync

None