KR20100060528A - Display system for driving vehicle - Google Patents

Abstract

PURPOSE: A driving assist display system for a vehicle is provided to calculate the moving distance of a vehicle based on the moving distances of feature points before and after movement, thereby easily grasping the position relation between peripheral obstacles and vehicles. CONSTITUTION: A driving assist display system for a vehicle comprises a vehicle detection unit(12), a feature point extracting unit(17), an image conversion unit, and an image synthesizing unit. The vehicle detection unit detects the state of a vehicle. The feature point extracting unit processes video and extracts feature points. The image conversion unit converts overlapped images using the feature points of the images according to the state of the vehicle. The image synthesizing unit synthesizes an created image with the real image, wherein the created image is created based on three-dimensional position relation by the feature point extracting unit and the real image is obtained by capturing the surrounding of the vehicle using a camera(14).

Description

Display system for driving vehicle {Display system for driving vehicle}

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to a vehicle driving assistance display system, and more particularly, to display a vehicle surrounding image photographed by a camera mounted on a vehicle as an image during low speed driving, such as an area that becomes a blind spot or a distance relationship between a host vehicle and a parking space. The present invention relates to a vehicle driving assistance display system capable of presenting to a driver.

Background Art Conventionally, a vehicle system for displaying a background image around a vehicle on a display has been known to support a parking operation when the vehicle is retracted. The system receives a video signal from a camera mounted on a vehicle, and outputs a peripheral image based on the video signal to a display installed near the driver's seat.

In such a system, there is a system that calculates an expected trajectory line based on the steering angle at that time, and superimposes the surrounding image together with the host vehicle position, the distance reference line, and the vehicle width extension line.

Related patent documents include Japanese Patent Laid-Open No. 2008-55958, Japanese Patent Laid-Open No. 2008-132882, and Japanese Patent Laid-Open No. 2008-120123.

Conventionally, a display system for supporting driving when parking is a system that supports driving by superimposing the converted image according to vehicle information such as blind spots.

In addition, Japanese Patent Laid-Open No. 2003-132349 calculates the distance to an obstacle using an image, detects the existence area of the three-dimensional object around the vehicle, calculates the existence area of the obstacle similarly to the plane, and then calculates the distance to the calculated plane. Disclosed is a system for projecting an image taken with a camera to create and display a composite image.

Therefore, when an error occurs in the calculation process of the distance from the car to the obstacle, there is a case where the driver is discomfort because a contradiction occurs in the synthesized image with respect to the actual landscape.

In addition, a system that detects the amount of movement of the vehicle and supports parking must synthesize a high-precision image according to the motion information of the own vehicle, and perform dynamic composite display for low-speed vehicle movement caused by the vehicle retreat. Because of the lack of the positional accuracy, there is a case where accurate image synthesis is difficult due to a misalignment between the position of the video and the synthesized image.

In addition, in the case where the garage is greatly changed compared with the initial state, when the angle of view of the image detecting means is wide, the difference between the actual position and the synthesized position is particularly large, resulting in an unnatural image.

The present invention has been made in view of the above-mentioned points, and a feature point extraction process is performed on an image taken around a vehicle to synthesize an image of an object existing in an actual space and an object present on an image. Vehicle driving assistance display that allows the user to easily grasp the positional relationship between the vehicle and the obstacle or the target point by making the synthesized image appear as if it is present and fixed to the obstacle or the target point. The purpose is to provide a system.

The above object is to take a video of the surroundings of the vehicle as a video, extract the singularity present in the captured image, estimate the positional relationship of the three-dimensional space, and make the path and space necessary for the driver to move the vehicle to the target position. In a vehicle driving assistance display system that supports driving by displaying imagination,

Vehicle detecting means for detecting a vehicle state; Singularity extracting means for extracting singularity using video processing; Means for changing the superimposed image using the singularity of the image according to the vehicle state; A vehicle synthesizing display comprising: an image synthesizing means for synthesizing an actual image photographed around a vehicle using a camera provided in the vehicle and an image generated based on the three-dimensional positional relationship estimated by the singularity extraction means; Achieved by the system.

Preferably, the vehicle state detecting means calculates a distance from an object present around the vehicle, and detects a space required around the vehicle based on the calculated distance and the moving direction of the vehicle.

The vehicle state detecting means may include means for detecting a vehicle speed in a space around the vehicle, means for detecting a shift position, and means for detecting a blind spot, and for controlling an image by receiving a signal from the detection means. It features.

The singular point extracting means is characterized by using a three-dimensional shape reconstruction process based on one-eye ranging using an amount of movement of an edge or particle.

The image synthesizing means is implemented on the basis of virtual reality and augmented reality technology for presenting a vision as if an object exists in place.

Accordingly, according to the vehicle driving assistance display system according to the present invention, by using the camera installed in the vehicle, the scenery of the parking space is photographed, the feature point is extracted from the captured image, and when the own vehicle moves, the scenery of the parking space is again displayed. Feature points are extracted from the captured images, and the amount of movement of the own vehicle is calculated based on the amount of movement of the feature points in the frame before the movement and the feature points in the frame after the movement, so that the obstacles around the vehicle and the positional relationship between the vehicles can be easily identified. do.

In addition, in order for the moving route of the vehicle to be synthesized on the image of the real space, it is possible to change the setting of the target position after the start of the movement or to visually identify the danger points existing in the process of parking by visually showing the moving route. do.

Hereinafter, preferred embodiments of the present invention will be described in detail with reference to the accompanying drawings.

1A is a block diagram illustrating a vehicle driving assistance display system according to an exemplary embodiment of the present invention. FIG. 1B is a block diagram illustrating a processing method of the planar region extraction unit 16 of FIG. 1A, and FIG. 1A is a block diagram showing a processing method in the feature point tracking unit 17. As shown in FIG.

The present invention provides a vehicle peripheral camera 14 for photographing the surrounding environment of the vehicle, a vehicle state detection unit 12 for detecting the state of movement of the vehicle, means for controlling an image according to the vehicle state, and a signal from the outside. A system for displaying a controlled image via means for controlling the image.

The vehicle driving assistance system according to the present invention includes a vehicle information acquisition unit 10, a vehicle state detection unit 12, an obstacle detection unit 11, an obstacle distance detection unit 13, a vehicle peripheral camera 14, and a space detection in a predicted path. It is composed of a calculating section 15, a planar region extracting section 16, a feature point tracking section 17, and an image display section 18.

First, the vehicle information acquisition unit 10 receives signals from the shift position sensor, the TPS, the steering angle sensor, and acquires information such as the position of the transmission gear, the throttle opening amount, the steering steering angle, and the like.

Then, the vehicle state detection unit 12 is based on the respective information, such as the position of the transmission gear, the throttle opening amount, the steering steering angle, etc. acquired by the vehicle information acquisition unit 10, the traveling direction, movement speed, left and right directions of the own vehicle. Ask for information such as instructions.

As a result, when the driver wants to start parking, the vehicle state detection unit 12 detects this and uses the system of the present invention. If you do not intend to start parking, the system of the present invention is not used.

The camera around the vehicle 14 is installed on the front, rear, left and right of the vehicle and performs a role of photographing the surrounding environment of the vehicle, and always photographs the external environment.

For example, when the vehicle is retracted as shown in FIG. 2 and the vehicle is moved within the parking range, the image captured by the vehicle peripheral camera 14 installed at the rear of the vehicle is as shown in FIG. 3, and the image is displayed on the image display unit 18. Mark on.

In addition, the vehicle information acquisition unit 10 acquires vehicle information, and simultaneously detects an obstacle in the external environment by the obstacle detection unit 11 (obstacle detection sensor) in parallel with imaging the vehicle's external environment with the camera 14. Doing.

When the obstacle is detected by the obstacle detection unit 11, the obstacle distance detection unit 13 measures the distance between the obstacle and the host vehicle.

The predicted intra-path space detection calculation unit 15 receives the signals of the obstacle detection unit 11 and the obstacle distance detection unit 13, and indicates that there is no obstacle in the image of FIG. 3. The position (near the parking space to be stopped) is recognized through the position information of the nearby white line and the like, and the parking range for parking so as to be parallel to the white line is displayed. If there are no nearby white lines, temporarily set the place where it is determined that no obstacle exists.

The parking range is displayed such that a frame (setting range) corresponding to the size of the vehicle is displayed on the image as shown in FIG. Then, the driver resets the edge position to the optimum position. The image after resetting is shown in FIG.

After the position of the parking range is determined by the driver as shown in FIG. 5, the planar region extracting unit 16 uses the shade of the image represented by the edge extraction and corner detection processing as shown in FIG. 6. The tone information of the image existing around the edge set by the extraction method is collected (S10).

At this time, if a group of straight lines (white line) or the like having the same shape as the setting range exists around the setting range on the image in Fig. 6, the position is specified. If a white line does not exist, such as a gravel vacant lot, a singular point that can be tracked during the movement of the vehicle is used as a substitute for the white line.

The method of determining the singularity requires that a plurality of traceable characteristic points such as white lines, three-dimensional objects, and unevenness of the road surface are determined from the image to be photographed (what may be photographed), and the characteristic and unique information is determined among them. , Denoted as 'special point' (S11).

Among a plurality of singular points selected from an image, several points are randomly extracted (for example, by random sample agreement (RANSAC)), and the data is used to evaluate the reliability of the measured value (for example, the least squares method). ). The plane with the highest reliability in it is set as the plane in which a setting range exists (S12).

When the straight line group that is considered to be the white line of the parking lot is detected, it is calculated based on the difference between the setting range and the position of the straight line group, so that the setting range overlaps with the straight line group, and the optimum positional relationship is required. Then, it is assumed that the desired optimum position (position where the set range and the straight line group overlap) is the ground position when the vehicle is calculated based on the host vehicle (S13).

If a white line does not exist, it is difficult for the operator to determine the parking target point at a time, so the position of the setting range is used as it is.

Using the above assumed positional relationship, as shown in FIG. 7, the horizontal plane area xy parallel to the ground (x, y, z = 0,0,0) where the host vehicle can exist is 3 Estimate by operation of the dimensional parameter. Then, the estimated positional relationship between the own vehicle and the ground, the angle, and the like are used as initial information (S14).

Based on this initial information (the relationship between the current position of the own vehicle and the position of the parking space to be stopped), the moving direction of the vehicle moving by the driver's operation is estimated by image processing from now on.

The image processing and estimation are performed by the singularity tracking unit 17.

First, a feature point extraction such as a particle filter is performed on an image obtained by the vehicle peripheral camera 14, and a singular point is further selected among them (S20).

Then, the positional relationship between the singularity in the video of the previous frame and the singularity in the video of the current frame is compared. Then, the extent of movement of the singularity is calculated (S21).

In addition, when a white line or the like used to request initial information becomes invisible due to the movement of the own vehicle (for example, when an obstruction enters between the vehicle peripheral camera 14 and the white line of the own vehicle, the white line is invisible). In addition, by calculating the small amount of movement in time series, it is possible to estimate the three-dimensional position of the current point of time by integrating the movement amount of the current point with the movement amount immediately before being obscured by the obstacle (S21).

At this time, in order to consider an area where the occlusion may occur by the intra prediction space detection calculation unit 15, it is determined whether the cause of the invisible white line is hidden or due to a misrecognition.

If it is occluded, the singularity existing on the road surface is newly selected and used for the subsequent calculation of the moving amount (S22).

In addition, in order to accurately calculate the amount of movement of the feature point described above, by using the three-dimensional parameter requested as the initial information, the reliability of the feature point is improved (S23), and a transformation in which a contradiction clearly occurs depending on the amount of movement of the feature point (e.g., For example, the positive and negative conversions on the z axis are not reversed.

By changing the three-dimensional parameters in this order, the relative vehicle behavior can be determined with a constant mind, and the relative position of the vehicle can be calculated very precisely (S24). For example, the relative position can be calculated to a higher degree than when the relative speed is calculated using the vehicle speed information.

Based on the calculated parameters, a virtual pole or the like is set as an obstacle at the end of the white line in the image, and then the user is presented with a course for bypassing the obstacle (S9) to facilitate the driver's space recognition. You can park safely while avoiding obstacles.

Here, FIG. 9 shows an image at the time of synthesis. According to the amount of movement of the vehicle, it presents an image combined with the extended reality that the object of the CG exists in place. The composite image of FIG. 9 and FIG. 10 can also be applied to an overhead view image which has undergone viewpoint transformation.

In addition, the present invention will be described below in comparison with the prior art.

In the prior art, first, the distance between a white line and a host vehicle is measured (for example, a stereo camera, a motion stereo method, etc.). When the host vehicle moves, the distance between the white line and the host vehicle is further measured. Then, the movement amount of the host vehicle is calculated based on the amount of change in the distance between the white line and the host vehicle before the movement and the distance between the white line and the host vehicle after the movement.

However, the present invention does not measure the distance between the white line and the host vehicle. In the present invention, first, the vehicle peripheral camera 14 installed in the host vehicle captures a landscape near the parking space, and extracts feature points from the captured image (for example, white line). Then, when the host vehicle moves, it captures the scenery near the parking space and extracts the feature points from the captured images.

Then, the movement amount of the host vehicle is calculated based on the movement amounts of the feature points in the frame before the movement and the feature points in the frame after the movement.

1A is a block diagram showing a vehicle driving assistance display system according to an embodiment of the present invention;

FIG. 1B is a block diagram showing a processing method in the planar region extraction unit of FIG. 1A;

1C is a block diagram illustrating a processing method in the feature point tracking unit of FIG. 1A;

2 is an explanatory diagram showing a state in which the vehicle is moved back within the parking range by retreating the vehicle;

3 is an explanatory diagram showing an image photographed by a camera installed at the rear of the vehicle;

4 is an explanatory diagram showing a parking range corresponding to the size of a vehicle on an image;

5 is an explanatory diagram showing a state in which the driver resets the parking range of FIG. 4 to an optimal position;

6 is an explanatory diagram showing a feature point extraction method using image shades;

7 is a diagram for explaining the estimation by the calculation of the three-dimensional parameters;

8 to 11 are explanatory diagrams showing a composite image by virtual reality and augmented reality;

12 is a conceptual diagram illustrating a method of calculating a relative movement amount of a vehicle based on a distance from a specific object in the prior art;

FIG. 13 is a conceptual view illustrating a method of calculating a relative movement amount based on a three-dimensional change around a vehicle in the present invention.

<Description of the symbols for the main parts of the drawings>

10: vehicle information acquisition unit 11: obstacle detection unit

12: vehicle state detection unit 13: obstacle distance detection unit

14: camera around the vehicle 15: sympathetic detection operation unit in the prediction path

16: plane area extraction unit 17: feature point tracking unit

18: image display unit 19: vehicle

Claims (5)

Take a video of the surroundings of the vehicle as a video, extract the singularity present in the captured image, estimate the positional relationship of the three-dimensional space, and display the path and space necessary for the driver to move the vehicle to the target position. In a vehicle driving assistance display system supporting driving, Vehicle detecting means for detecting a vehicle state; Singularity extracting means for extracting singularity using video processing; Means for changing the superimposed image using the singularity of the image according to the vehicle state; A vehicle synthesizing display comprising: an image synthesizing means for synthesizing an actual image photographed around a vehicle using a camera provided in the vehicle and an image generated based on the three-dimensional positional relationship estimated by the singularity extraction means; system. The method according to claim 1, And the vehicle state detecting means calculates a distance from an object present around the vehicle, and detects a space required around the vehicle based on the calculated distance and the moving direction of the vehicle. The method according to claim 1, The vehicle state detecting means includes means for detecting a vehicle speed in a space around the vehicle, means for detecting a shift position, and means for detecting a blind spot, and controlling an image by receiving a signal from the detection means. Vehicle driving assistance display system. The method according to claim 1, And said singular point extracting means uses a three-dimensional shape restoration process based on one-eye ranging using an amount of movement of an edge or particle. The method according to claim 1, And said image synthesizing means is implemented on the basis of virtual reality and augmented reality technology for presenting a vision as if an object exists in place.

Images