High Dynamic Range with Stereoscopic Depth Cameras

If you have ever taken a photograph of a person in a dark environment using flash photography, you may have had the experience of disappointment with the resulting image – either the person Is over-exposed, or the background is under-exposed. Any situation where you are trying to capture a scene that includes both very dark and very bright areas can be challenging for sensors.

When it comes to depth sensors, commonly used in fields like robotics, this disparity between bright and dark areas of a scene can cause problems in understanding the environment properly. For example, when moving from a brightly lit doorway to a dark street, it’s important that a robot can ‘see’ its way clearly.

Intel RealSense stereo depth cameras are not immune to this problem – they rely on processing depth from two good quality left and right images. The cameras include an IR pattern projector to illuminate dark scenes. In situations where there are very bright regions, in comparison the lower level of light from the projected patterns at a distance falls off quickly, leading to a very large brightness difference between near and far objects.

What is HDR?

In standard photography, High Dynamic Range (HDR) imaging describes techniques that are designed to overcome the limitations of standard sensors by merging two or more frames into a single higher dynamic range synthetic image.

Big Ben and Westminster Bridge at sunset, HDR image

For example, in this image of Big Ben and Westminster Bridge at sunset, the sun is shining directly at the sensor. In a normal photograph, the buildings and details like the underside of the bridge would be dark, mostly in shadow and obscured. By taking multiple images over a short period of time with different settings for gain and exposure, the photographer can capture details both of the well lit sky, and the darker areas of the buildings and bridge, and then later composite the images together for something both colorful and detailed.

In the context of depth cameras, higher dynamic range can mean higher depth fill-rate due to similar access to more and better features, improving performance in areas like collision avoidance with changing lighting conditions.

To learn more and implement these techniques, read the whitepaper.