The evolution of perceptual organization in biological vision, and its necessity in advanced computer vision systems, arises from the characteristic that perception, the extraction of meaning from sensory input, is an intelligent process. This is particularly so for high order organisms and, analogically, for more sophisticated computational models. By perceptual organization we refer to the ability of a vision system to organize detected features in images based on viewpoint consistency and other Gestaltic perceptual phenomena. This imparts robustness, efficiency, and a qualitative and holistic nature to vision.

Our computational paradigm aims to organize features into highly plausible sets of higher level geometric features which are present in images of objects belonging to a large number of domains. Our organizational philosophy is hierarchical, with complex organizations being formed from simpler ones. Each level of the hierarchy is constructed using voting methods, graph operations, and knowledge based reasoning in a new extension of the Bayesian network we call the Perceptual Inference Network. Analogous to theories in human vision, our strategy divides broadly into two parts: detecting regularities and similarities in the tokens (preattentive vision) and reasoning, based on a knowledge base built from past experience, to enable one to go beyond the information provided (attentive vision). The voting method provides organizations based on Gestalt principles and the network reasons on those organizations to extract geometric features. The two steps of voting and evidential reasoning are repeated.

Previous approaches to perceptual organization have mostly been purely bottom up, without any top down knowledge base influence and therefore entirely dependent on the inputs, which may be imperfect. The knowledge base, besides coping with such input imperfections, also allows us to integrate multiple sources of information and to form a composite organization hypothesis.