Cars' Overhead Camera: A Bird's-Eye View Advantage

Many modern cars now feature a 360-degree camera, also known as a bird's-eye view or surround-view camera. This technology uses several cameras placed around the vehicle to provide a top-down view of the car and its surroundings, making it easier and safer to park and manoeuvre. The cameras are usually positioned in the front grille, exterior mirrors, and at the rear, with some systems featuring additional side-view cameras. The video signals from these cameras are fed into an image-processing program, which stitches the images together to create a seamless, composite overhead view. This view is then displayed on the vehicle's infotainment screen, often with a split-screen showing both the top-down and rearview perspectives. While this technology was initially offered only on luxury vehicles, it is now available from many automakers, although it often remains an expensive add-on.

Camera placement

• Number of Cameras: A typical setup for a surround-view camera system involves four cameras. One camera is positioned at the front of the vehicle, often in the grille. Two wide-angle cameras are mounted on the exterior rear-view mirrors, providing a view of the sides of the car. The fourth camera is placed at the rear, usually serving as the backup camera. More advanced systems may utilise six cameras, with additional side-view cameras positioned ahead of the front wheels, enabling drivers to see what is on the other side of obstacles.
• Clear Field of View: It is essential to ensure that the cameras have a clear and unobstructed view of the vehicle's perimeter. This means placing them in locations that do not block each other's field of view and avoiding areas that may be prone to dirt or debris buildup, which could hinder the camera's visibility.
• Aesthetic Integration: Camera placement should also consider the vehicle's aesthetics. Designers aim to integrate the cameras seamlessly into the body panel without disrupting the car's overall design and appearance.
• Simulation and Analysis: Before finalising camera placement, engineers simulate CAD models of the vehicle to analyse the field of view and blind zones from different angles. This process is carried out under multiple vehicle load configurations to ensure optimal camera positioning.
• Camera Calibration: It is important to note that no two cameras produce identical outputs. Even if placed in the exact same location, cameras can create images with slight differences due to factors such as lens placement, aging, and thermal expansion. Therefore, proper camera calibration is crucial to compensate for these mismatches and ensure the images are accurately aligned and coloured.

By carefully considering these factors, automakers can optimise camera placement to capture the necessary data and provide drivers with an accurate and useful bird's-eye view of their surroundings. This technology enhances both safety and convenience, particularly when navigating tight parking spaces or manoeuvring in challenging environments.

Image stitching

Overhead car cameras, also known as bird's-eye view or surround-view cameras, are an impressive piece of automotive technology. They provide drivers with a real-time, composite view of their vehicle and its surroundings, making parking and manoeuvring much easier. This composite view is achieved through image stitching, a process that combines images from multiple cameras positioned around the vehicle.

Components of an Overhead Car Camera System

At its core, an overhead car camera system is a combination of hardware and software working together to provide drivers with a comprehensive view of their surroundings. The main components include:

• Surround-view cameras: Typically, 4-6 cameras with wide-angle lenses are integrated into the vehicle's body panel. These cameras are usually placed in the front grille, under the side-view mirrors, and at the rear of the vehicle.
• Proximity sensors: Ultrasonic or electromagnetic devices that aid in evaluating the distance to nearby objects by sending out signals and measuring the time it takes for them to reflect back.
• Image processing software: This software takes the individual images from the cameras and stitches them together to create a cohesive, real-time view of the vehicle and its surroundings.
• Display hardware: The final stitched image is projected onto an HMI (Human Machine Interface), typically the vehicle's infotainment system screen.

The Image Stitching Process

• Image registration: Detecting key points on the images and assigning them to a common ground plane. Custom pattern registration is commonly used in 360-degree vision systems.
• Warping: Deforming the undistorted image to match certain defined key points. Techniques such as homography, polynomial deformation, or moving least squares are applied.
• Blending: Merging the individual images to obtain the final stitched image. Algorithms such as multiband blending, feathering, or 50% blending are commonly used.

Benefits of Overhead Car Cameras

The bird's-eye view provided by these camera systems offers significant advantages to drivers:

• Improved parking: The top-down view makes it easier to navigate into tight parking spots and avoid nearby obstacles.
• Enhanced safety: By providing a comprehensive view of the vehicle's surroundings, these systems help reduce blind spots and improve situational awareness.
• Versatility: In addition to the overhead view, drivers can also select different camera angles to check the sides, front, or rear of the vehicle, aiding in various driving scenarios.
• Guidelines and alerts: Many systems superimpose guidelines onto the image, showing the vehicle's orientation and expected direction of travel. They also provide audible or haptic warnings when obstacles are nearby.

Proximity sensors

There are two main types of proximity or parking sensors used in cars: electromagnetic and ultrasonic. Electromagnetic proximity sensors use electromagnetic frequencies to detect nearby obstructions and offer a wider range of detection than ultrasonic sensors. They can be combined with cameras to enhance the driver's view of their surroundings. Electromagnetic sensors create an electromagnetic field around the bumper and trigger an alert when objects enter the field.

Ultrasonic proximity sensors, on the other hand, use high-frequency sound waves to identify obstructions. These sensors emit acoustic pulses and measure the return interval of reflected signals to calculate the distance to nearby objects. Similar to bats, they can determine the distance to obstacles by analysing the echo durations of sound waves bouncing off objects.

Both types of sensors typically notify the driver through a series of beeps, with the frequency of beeps indicating the proximity to the object. More recent models may also include digital displays that visually represent the distance to nearby objects.

In addition to parking assistance, proximity sensors have various other applications in cars. They can be used for security systems, detecting attempts of theft or tampering. Proximity sensors also enable keyless entry, unlocking car doors automatically when the owner is nearby. Furthermore, they can aid in finding parking spots and assist in automatic parking by sensing available spaces.

Image processing

The image processing module is a crucial component of the 360-degree camera system, also known as a Bird's Eye View or Surround View camera system. This module takes the individual images captured by the cameras positioned around the vehicle and stitches them together to create a cohesive, seamless, and positionally accurate top-down view.

The image-processing software receives inputs from multiple cameras, typically including a rearview camera, side-view cameras mounted on the outboard side mirrors, and a front grille or fascia camera. The software then applies various algorithms to create a composite image. This process involves geometric alignment, photometric alignment, and composite view synthesis.

Geometric alignment includes correcting lens distortion and transforming the perspective of each image. Photometric alignment ensures that the brightness and colour of the individual camera views are matched, creating a seamless final image as if it were captured by a single camera. The composite view synthesis step is where the actual stitching of images takes place.

Image registration, warping, and blending are the key activities involved in the image stitching process. Image registration involves detecting key points on each image and assigning them to a common ground plane. Warping involves deforming the undistorted image to match certain defined key points, using techniques such as homography or polynomial deformation. Finally, blending is the step where the individual images are merged to obtain the final stitched image, using algorithms such as multiband blending, feathering, or 50% blending.

The result is a lifelike, synthetic image that provides a bird's-eye view of the vehicle and its surroundings, making it easier for drivers to navigate and park their vehicles while avoiding nearby obstacles.

Display

The display of a bird's-eye view camera system in cars is an invaluable feature when manoeuvring a vehicle into a tight space, such as when parallel parking. The system utilises a combination of cameras and software to provide a top-down view of the car and its surroundings, often with a split-screen displaying the rearview and surround-view images. This allows drivers to easily see any objects behind them and correct their course if necessary.

The display screen, typically the infotainment system, shows a stitched-together image from multiple camera angles, giving a seamless, coherent overhead perspective. The image is processed using software that performs geometric alignment, photometric alignment, and composite view synthesis. Geometric alignment includes lens distortion correction and transformation of perspective, while photometric alignment matches the brightness and colour of individual camera views for a seamless final image. The composite view synthesis is where the image stitching occurs, using techniques such as multiband blending, feathering, and 50% blending to create a high-resolution, cohesive image.

In addition to the visual display, feedback mechanisms such as audio alerts are often used to notify the driver of nearby objects. This enhances the overall effectiveness of the bird's-eye view system in improving safety and manoeuvrability in tight or challenging situations.

The bird's-eye view display provides drivers with a clear and comprehensive understanding of their vehicle's position and surroundings, making parking and navigation in confined spaces much more manageable and reducing the stress associated with such tasks.

Frequently asked questions