# RPU Technology

# Technical: Image Sampling

Visual information is originally obtained, in all vision systems, by
*discrete* Nyquist sampling of a *continuous* input optical image. The discrete Nyquist
samples capture all of the information in the continuous image and are fully equivalent to it. In
existing physical vision systems the visual information used is derived *directly* by high
density Nyquist sampling of the input optical image.

However in human vision almost all of the visual information is derived by SD sampling. Although
the input optical image is initially Nyquist sampled by the photoreceptors, in the *periphery*
(99.99% of the retina) the result is then re-sampled (within the retina) by *low-density SD
sampling*^{6} . It is only over the very small fovea (0.01% of
the retina) that the visual information is derived directly by *high-density Nyquist
sampling*^{7}.

Nyquist Sampling: The sampling density is high and each pixel sample taken is a measure of the mean value of the input image over a small sample area. The result is a discrete Nyquist image. With Nyquist sampling, high-resolution information (essential for recognition by any means) is captured by the high sampling density.

SD Sampling: The sampling density is lower and each pixel sample taken is a measure of the standard deviation (SD) of the image over a correspondingly larger sample area. The SD sample is a measure of thetotal spatial frequency powerin the image within the sample area. (Parseval's theorem). High-resolution information (essential for recognition) is thereby captured by SD sampling in spite of its low sample density. SD sampling is performed on a discrete Nyquist image. The result is a discrete SD image in which the sample density is SF times lower than in the Nyquist image from which it was derived, where SF (sampling factor) is a key SD-sampling parameter chosen according to nature of the current visual task.

Normalized SD Sampling: In this case the sample taken is the ratio of the standard deviation to the mean of the image over the sample area. The normalized SD sample is thereby dimensionless and thus nominally independent of the scene illumination.

A characteristic feature of a Nyquist image is that its sensitivity to high spatial frequencies
is *directly proportional* to its ability to resolve patterns (such as a test letter). A very
important characteristic of an SD image is its *disproportional* sensitivity to high spatial
frequencies.

______________________________

- The sampling performed by the peripheral retina cannot be modeled by Nyquist sampling because of its disproportionate sensitivity to high spatial frequencies. However it can, to first order at least, be modeled by SD sampling.
- The extremely small size of the fovea can be made
evident by fixing the eye on one letter of text and then trying to resolve
adjacent letters without moving the eye. It will be observed that the fixated
letter can be resolved but
**even immediately adjacent**letters are not clear. However, although not clear they are not blurred, indicating that they are being seen via a different (non-Nyquist) sampling protocol. The area of the retina where Nyquist sampling is performed corresponds to about one letter of text (< 0.01% of the visual field).