Kinect Lasers: How Cameras Capture And Interpret Depth Information

The Kinect is a motion-sensing input device produced by Microsoft, first released in 2010. It contains an RGB camera, an infrared projector, and a multi-array microphone. The RGB camera detects red, green, and blue colour components, as well as body type and facial features. The infrared projector helps create 3D imagery by transmitting invisible near-infrared light and measuring its time of flight after reflecting off objects. The Kinect's hardware works in conjunction with its software to enable advanced gesture recognition, facial recognition, and voice recognition. The Kinect's software technology uses artificial intelligence to perform segmentation of shapes to identify specific body parts and construct a skeleton for motion tracking.

The Kinect's IR laser projector

The Kinect sensor's most prominent physical feature is its relatively sophisticated optical system, which includes an IR laser projector, an RGB camera, and an IR camera. The IR laser projector is used to project a pattern of infrared light onto the scene, which is then captured by the IR camera to create a 3D depth image. This depth image is what allows the Kinect to perceive depth and perform tasks such as gesture recognition, skeletal detection, and body tracking.

The IR laser projector in the Kinect uses a single laser diode and a lens with a drilled pattern to create the pattern of infrared dots projected onto the scene. The pattern is pseudo-random and hardwired into the sensor, and it is designed to be spherical to match the curve of the projection area. The Kinect sensor constantly projects these dots, and they can be seen using a video camera or camcorder with night vision mode, as the camera in night vision mode is sensitive to the infrared light spectrum that the Kinect uses.

The IR projector works in conjunction with the IR camera to create the 3D depth image. The IR camera captures the reflected light pattern, which is deformed by the relative depth of objects in the scene. By analysing the deformation of the pattern, the Kinect can estimate the depth of objects in the scene. This process is called structured light sensing or structured light IR projection, and it allows the Kinect to create a 3D map of the environment.

The RGB camera

The Kinect's RGB camera plays a crucial role in applications such as computer graphics, image processing, computer vision, and human-machine interaction. It has been used in various industries, including gaming, robotics, theatre, and natural interfaces. For example, in robotics, the Kinect's RGB camera, together with depth sensing, allows robots to navigate and manoeuvre around obstacles.

The IR camera

The Kinect's IR camera can be used for various projects, such as imaging, object detection, and gesture recognition. It provides the locations of objects in an image relative to itself, allowing for a wide range of applications.

The Kinect's IR camera technology has been applied in robotics, medicine, and interactive entertainment, offering a cost-effective and versatile solution for depth-sensing and object recognition.

The Kinect's optical system

The Kinect optical system is comprised of an IR laser projector, an RGB camera, and an IR camera. The IR laser projector uses a structured light IR projector and sensor system, which is widely used in industrial manufacturing and inspection. The structured light sensing principle involves projecting a specific angle between the emitter and the sensor, and depth is then recovered through triangulation. The Kinect expands on this by using a predictable structure, where the corresponding image shift directly relates to depth.

The IR laser projector works in tandem with a diffraction grating, which is a carefully engineered computer-generated hologram with a specific periodic structure. The IR laser shines through the diffraction grating, producing a specific periodic structure of IR light. This structure is made up of square cells that diffract light into a periodic structure. The Kinect's IR-sensitive CMOS detector then images this projected pattern, which enables the device to calculate depth through triangulation.

The RGB camera has a pixel resolution of 640x480 and a frame rate of 30 fps, which aids in facial recognition and body recognition. The IR camera has a similar resolution and frame rate, and it captures the IR light reflected off objects in the environment. This allows the Kinect to create 3D imagery and map the environment.

The IR laser's wavelength

The Kinect sensor's depth sensor consists of an infrared laser projector combined with a monochrome CMOS sensor, which captures video data in 3D under any ambient light conditions. The sensing range of the depth sensor is adjustable, and Kinect software is capable of automatically calibrating the sensor based on gameplay and the player's physical environment, accommodating for the presence of furniture or other obstacles.

The infrared laser projector emits light within the red spectral region, defined as being between 750 nm and 1400 nm. This is also referred to as the near-infrared (NIR) region.

NIR lasers emit electromagnetic radiation (laser light) and are capable of extremely high-power outputs, making them ideal for applications such as welding and cladding. They can also be used in medical procedures such as laser surgery, laser therapy, and diagnostic imaging.

The Kinect sensor's infrared laser projector operates within the NIR region, and is therefore considered safe for the human eye, as the bulk of the energy emitted by the laser is absorbed by the eye's cornea and lens, protecting the retina.

Frequently asked questions