|Welcome to The Neuromorphic Engineer|
Applications » Imaging
Capture and display in wide-dynamic-range imaging
PDF version | Permalink
It is commonly accepted1 that the dynamic range (DR) of an image, i.e. the image color definition, is equal to the image maximum signal-to-noise ratio (max SNR). We also know that the maximum amount of image information is given by the product of resolution (cross-image definition) and color definition. Therefore, increasing the DR of images is by no mean less important than increasing their resolution. It is worth remembering however that DR and resolution cannot possibly be traded-off against each other, since they live in orthogonal sub-spaces of the image. Below, we analyze the two most commonly-used methods to increase dynamic range.
Purely logarithmic photo-response
Working with a purely log-response practically eliminates saturation altogether, thereby making the maximum signal unlimited. It can be shown however2 that to first order, the DR at the output of any 1-1 analytic mapping equals the DR at its input.
To realize this in the log case we recall that the small-signal gain of the log function is inversely proportional to the average input signal. This implies that, at this sensor's output, a local contrast whose magnitude is usually small relative to the average luminance, maintains a pretty-much constant SNR for all levels of average luminance. At the same time, however, such a local contrast becomes progressively ‘washed-out’ as the average luminance increases: a familiar and most undesirable property of the log transmission.
The multiple-exposure method
Multi-exposure (ME) means to apply shorter exposure times to originally saturated pixels, such that saturation is altogether avoided. This is generically illustrated in Figure 1, where the exposure periods are determined according to the simplest normalized series of
The RMS of the camera shot noise, however, is proportional to the square root of the exposure time. Therefore the RMS of the camera output noise in the continuous transmission arrangement above is elevated by a factor of
One can see that there is not much point in having the shortest exposure period less than a 1/3 of the default, since e.g., in 1/4-default the output noise is twice that of the default, thereby amplifying the minimum detectable local contrast by a factor of 2 and causing an already-significant 'noise-washout’ effect. This limits the added DR of the ME arrangement, in practice, to just a single bit.
The role of display DR
As it turns out,3 in normal indoor lighting we cannot see more than 5 to 6 bits per color (bpc) with most cathode-ray-tube, liquid-crystal, and plasma displays. The effect of this bottleneck is demonstrated by the upper half of Figure 2, an 8bpc RGB picture Room (courtesy of Vincent Laforet, Photographer in Residence at the New-York Times) where the maximum amount of lost visual content due to the display is about nine bits per pixel (24 minus 15).
To get around the display bottleneck, we consider a similar situation that takes place inside our retina.4 The neural channels that carry all sensory information to the cortex cannot possibly support more than 7 bits of DR. Nevertheless, the DR we actually perceive can easily exceed 30bpc. The mechanism that makes this possible has been analytically modeled5 and is called retinal dynamic-range compression (DRC).
Reconstruction of retinal DRC
The basis of retinal DRC is the spatial feedback automatic-gain-control (fb-AGC), described in Figure 3. Here, the acquired image Ei is multiplied by the scalar forward gain K and fed into the first input of an image pixel-wise multiplier. The output is then fed back into the multiplier's second input after having passed through a linear spatial low pass filter (LPF, to average) and being subtracted from unity.
The average transmission (DC) of this model5 is4) is known as Weber's law.
The DR compression ratio (CR) is defined as the output/input ratio for Ei = 1/K (the knee-point). We then have: CR = K/2. The compression ratio is thus readily controlled through the parameter K. The fb-AGC gain for variations of low spatial frequencies (the ‘DC-gain’) is given by
The fb-AGC gain for local contrast variations (the ‘AC-gain’) is obtained by assuming that, for such variations, the LPF output output remains constant. Denoting this constant by Ē Eo we have
Taking the quotient (GAC /GDC) as a measure of the detail enhancement of the retinal DRC, we see that the amount of this detail enhancement, or the ‘effective visual acuity’ increases linearly with the average luminance of the viewed scene: a well known property of human and animal vision. The lower half of Figure 2 is Room after retinal DRC to 5bpc.
On the capture side of wide-dynamic-range imaging, we have demonstrated that dealing with saturation alone cannot significantly increase the DR of image sensors. We therefore conclude that this can only be achieved via significant noise reduction. On the display side of the problem, Figure 2 above demonstrates how retinal DRC can increase the potential number of perceived distinguished colors—by a factor of 512 in the current example—and shows how this affects our watching experience.–
Tell us what to cover!
If you'd like to write an article or know of someone else who is doing relevant and interesting stuff, let us know. E-mail the editor and suggest the subject for the article and, if you're suggesting someone else's work, tell us their name, affiliation, and e-mail.