Computer Vision and Computational Neuroscience Laboratory

Space Variant Image Processing

The design of an effective data processing system depends critically on both the data structure and the algorithms employed. Machine vision has traditionally relied on a data structure composed of a uniform resolution matrix coupled with convolution based algorithms. For various reasons, many researchers have in the past been interested in freeing themselves of the constraints of a uniformly sampled, Cartesian data representation. Motivations include a dependence on non-uniform sensors, data reduction, or a desire to process even in the absence of some samples (or regions).

In contrast to traditional machine vision, biological vision systems sample visual space in a highly non-uniform fashion that is not shift invariant (i.e., does not permit the usual convolution). The vast superiority of biological vision systems to any current machine vision system in terms of speed and flexibility implies that there are lessons to be learned from this architecture. Traditional machine vision also suffers from an increasing overabundance of data that outstrips even modern computers' ability to process with anything but the most basic algorithms. These problems with standard, uniformly sampled, machine vision have promoted research into algorithms and techniques that operate in a space-variant setting. A good space-variant machine vision algorithm is not only unhampered by the non-uniformity, but actually takes advantage of the sampling structure and exploits the fact that many fewer samples are required in order to perform sophisticated processing.


Generalized Image Processing

[Foveal mesh] [Lena on mesh]

The Generalized Image Processing (GIP) project has focused on the development of a new data structure for machine vision along with an outline of algorithms to be used in this regime. Since this data structure and consequent algorithms are designed to operate for an arbitrary sampling, both the data structure and the processing techniques will apply to the traditional setting as a special case. Therefore, the approach taken in this project may be viewed as a generalized form of standard data processing. Although the data structure and many of the algorithms are directly applicable to problems other than machine vision, the main focus of this research has been in the context of machine vision.

The main reason for employing a space-variant architecture is to process with dramatically lower bandwidth while retaining a high resolution in part of the visual scene. However, the difference in visual sampling across species suggests that there is a relationship between features of the sampling regime and the animal's visual ecology. An example of such a relationship is the belief that the ``horizontal streak'' seen in many animals (e.g., the rabbit) is helpful to species that live in open (i.e., non-occlusive) visual environments. This belief has been supported by the strong correlation between species with less occlusive visual ecologies and those possessing a visual streak. Uncovering relationships of this nature help the designer of a machine vision system engineer an architecture that is optimized to match the design constraints for the ``visual ecology'' of the artificial system. In order for the designer to be free to craft the visual sampling to the purpose of the system it is necessary that a data structure exists with sampling-independent (i.e., generalized) machine vision algorithms.

Our approach has been to employ a graph-based structure with an explicit topology as the representation of visual data on a non-uniformly sampled structure. The visual data may be thought of as electric potentials associated with each node in a circuit. Many of the algorithms developed for the GIP setting exploit the network analogy. Network algorithms are especially useful both because of the physical intuition afforded and due to the straightforward transition from an implementation on a digital computer to physically realizable systems whose computational engine is nature.

Contact: Leo Grady