Computer Camera's Depth Detection: Unlocking The Secret

Computer vision is an important concept in artificial intelligence and image processing that enables machines to perceive the three-dimensional world around us, much like the human eye. Depth is a critical component of computer vision, providing information about the distance of objects from the camera. This depth information is used in applications such as video games, self-driving cars, robotics, and augmented reality. One way to obtain depth information is through stereo vision, which uses two cameras positioned side by side to capture images of the same scene. By analysing the disparity between the positions of objects in the two images, the computer can calculate the depth or distance of the objects. This mimics how humans use binocular vision to perceive depth, with each eye seeing a slightly different image.

Stereo vision

The basic principle behind stereo vision is triangulation. When two cameras (or "stereo cameras") capture images of the same scene from slightly different viewpoints, the resulting pair of images, called stereo pairs, contain disparities or differences in the positions of corresponding points in the two images.

By analysing these disparities, a computer vision system can calculate the depth information of objects in the scene. Objects that are closer to the cameras will have larger disparities, while objects farther away will have smaller disparities.

The depth from stereo algorithm finds disparity by matching blocks in left and right images. The Sum of Squared Differences (SSD) block-matching algorithm is a naive implementation of this idea. This algorithm calculates the SSD between blocks of pixels in the left and right images and selects the blocks with the minimum SSD as the matching blocks.

The depth from stereo algorithm relies on two parallel view-ports and calculates depth by estimating disparities between matching key-points in the left and right images. The following steps are involved in the process:

• Calibration: Determining camera parameters such as intrinsic matrix, distortion coefficients, and extrinsic parameters to ensure accurate depth computation.
• Rectification: Applying a geometric transformation to align corresponding features along epipolar lines, simplifying the stereo matching process.
• Stereo Matching: Finding correspondences or matching points between the left and right images to calculate disparities.
• Disparity Map: Creating a grayscale image where each pixel's intensity value corresponds to the disparity or depth at that point in the scene.
• Depth Map: Calculating the depth of each pixel in real-world units (e.g. meters) based on the disparities obtained from the stereo images and camera parameters.

Structured light

A structured-light 3D scanner is a device that measures the three-dimensional shape of an object using projected light patterns and a camera system. The scanner projects a series of parallel patterns of light onto the target object. When the light hits the object's surface, the patterns become distorted. The scanner's cameras capture these distorted patterns and send the images to 3D scanning software for processing.

There are two major methods of stripe pattern generation: laser interference and projection. The laser interference method uses two wide planar laser beam fronts, which interfere to produce regular, equidistant line patterns. The projection method, on the other hand, uses incoherent light, similar to a video projector. Patterns are generated by passing light through a digital spatial light modulator, typically using one of three digital projection technologies: transmissive liquid crystal, reflective liquid crystal on silicon (LCOS), or digital light processing (DLP).

Time-of-Flight

ToF cameras can use either light or sound as a signal, with light being the more common option. Light signals can be continuous waves or short pulses, and both can be used to calculate the distance to an object. When using continuous waves, the sensor detects the phase shift in the reflected light to determine the distance. With short pulses, the sensor measures the time it takes for the light to be emitted and return to the sensor.

ToF cameras have several advantages over other 3D depth mapping technologies. They are relatively cheap to produce and use, and they don't require much processing power. They can also provide precise and fast measurements and have a longer range than other technologies such as ultrasound. Additionally, ToF cameras are compact, as the sensor and illumination can be placed together.

ToF cameras have a variety of applications, including robot navigation, vehicle monitoring, people counting, object detection, and 3D printing. They are also used in smartphones as "depth" cameras to aid in producing higher-quality portrait mode photos.

LiDAR

Additionally, LiDAR is used in mapping and surveying to create high-resolution 3D models of terrain. By mounting LiDAR equipment on an aircraft, detailed maps of the landscape can be created, including digital elevation models (DEMs) that show the height and shape of the terrain. This technology can be particularly useful in areas with dense vegetation, as LiDAR can penetrate forest cover to reveal the ground beneath.

Overall, LiDAR is a versatile technology that has enhanced our ability to perceive and understand the world around us, with applications in a wide range of fields.

Monocular cues

Relative Size

Objects that are closer appear larger than distant objects. Relative size is a monocular cue that helps us understand the distance between objects and ourselves, as well as the size of objects in relation to each other. For example, when we see a person standing in front of a building, the person appears larger because they are closer to us. However, relative size alone is not always reliable as the size of an object can be influenced by other factors such as its actual size, distance, and viewing angle.

Interposition

Interposition, or overlap, refers to when one object partially blocks or overlaps another. The overlapped object is perceived as being farther away. Interposition provides important information about the relative positions of objects, especially when there are limited depth cues available, such as in a two-dimensional image.

Texture Gradient

Texture gradient refers to the change in the appearance of a texture as it extends into the distance. Objects up close display fine details and textures clearly, but as they move away, the texture appears smoother and less distinct. This monocular cue is particularly useful in natural scenes, such as a field of grass, where the individual blades of grass closer to the observer are clearly visible, but they blend together in the distance.

Linear Perspective

Linear perspective is the phenomenon where parallel lines, such as a road or railway tracks, appear to converge as they extend into the distance. This convergence creates a sense of depth and distance, allowing us to perceive objects as being farther away. This cue is especially useful in two-dimensional images or paintings, giving them a sense of three-dimensionality.

Aerial Perspective

Aerial perspective, or "distance fog", refers to how distant objects appear hazy, lighter in colour, and less detailed due to the scattering of light by the atmosphere. This is why distant mountains often appear lighter in shade and colour.

Monocular Motion Parallax

When we move our heads from side to side, objects at different distances move at different relative velocities. Objects that are closer move in the opposite direction to head movement, while farther objects move with the head movement.

Frequently asked questions