RPU Technology

Technical: Image Sampling

Visual information is originally obtained, in all vision systems, by discrete Nyquist sampling of a continuous input optical image. The discrete Nyquist samples capture all of the information in the continuous image and are fully equivalent to it. In existing physical vision systems the visual information used is derived directly by high density Nyquist sampling of the input optical image.

However in human vision almost all of the visual information is derived by SD sampling. Although the input optical image is initially Nyquist sampled by the photoreceptors, in the periphery (99.99% of the retina) the result is then re-sampled (within the retina) by low-density SD sampling6 . It is only over the very small fovea (0.01% of the retina) that the visual information is derived directly by high-density Nyquist sampling7.

Nyquist Sampling: The sampling density is high and each pixel sample taken is a measure of the mean value of the input image over a small sample area. The result is a discrete Nyquist image. With Nyquist sampling, high-resolution information (essential for recognition by any means) is captured by the high sampling density.

SD Sampling: The sampling density is lower and each pixel sample taken is a measure of the standard deviation (SD) of the image over a correspondingly larger sample area. The SD sample is a measure of the total spatial frequency power in the image within the sample area. (Parseval's theorem). High-resolution information (essential for recognition) is thereby captured by SD sampling in spite of its low sample density. SD sampling is performed on a discrete Nyquist image. The result is a discrete SD image in which the sample density is SF times lower than in the Nyquist image from which it was derived, where SF (sampling factor) is a key SD-sampling parameter chosen according to nature of the current visual task.

Normalized SD Sampling: In this case the sample taken is the ratio of the standard deviation to the mean of the image over the sample area. The normalized SD sample is thereby dimensionless and thus nominally independent of the scene illumination.

A characteristic feature of a Nyquist image is that its sensitivity to high spatial frequencies is directly proportional to its ability to resolve patterns (such as a test letter). A very important characteristic of an SD image is its disproportional sensitivity to high spatial frequencies.

______________________________

  1. The sampling performed by the peripheral retina cannot be modeled by Nyquist sampling because of its disproportionate sensitivity to high spatial frequencies. However it can, to first order at least, be modeled by SD sampling.
  2. The extremely small size of the fovea can be made evident by fixing the eye on one letter of text and then trying to resolve adjacent letters without moving the eye. It will be observed that the fixated letter can be resolved but even immediately adjacent letters are not clear. However, although not clear they are not blurred, indicating that they are being seen via a different (non-Nyquist) sampling protocol. The area of the retina where Nyquist sampling is performed corresponds to about one letter of text (< 0.01% of the visual field).