WO2018097595A1

WO2018097595A1 - Method and device for providing driving information by using camera image

Info

Publication number: WO2018097595A1
Application number: PCT/KR2017/013347
Authority: WO
Inventors: 기석철; 이신재; 윤형석; 박태형
Original assignee: 충북대학교 산학협력단
Priority date: 2016-11-28
Filing date: 2017-11-22
Publication date: 2018-05-31
Also published as: CN109070882B; KR101749873B1; CN109070882A

Abstract

According to the present invention, a method for providing driving information of a vehicle having one or more cameras mounted therein comprises the steps of: detecting an obstacle by processing, into images in real time, vehicular external image data received from the cameras; predicting a collision risk on the basis of the distance between a position of the detected obstacle and the vehicle itself; detecting, by using a free space detection (FSD) algorithm, a drivable area by processing the vehicular external image data received from the cameras; and providing drivability information through the collision risk and the drivable area. According to the present invention, a single color and a simple pattern of a wall or a pillar, which are difficult to be classified as obstacles, can be recognized as obstacles.

Description

[Revision 30.11.2017 by Rule 26] Method and device for providing driving information using camera image

The present invention relates to a technology for providing driving information in a vehicle, and more particularly, to a driving information providing technology for recognizing an obstacle and detecting a possible obstacle area using an image through a camera mounted on a vehicle.

Various types of obstacles exist around the vehicle, and obstacle detection is attempted in order to be protected from such a potential hazard.

In general, the vehicle approach obstacle detecting apparatus or system is configured to detect an obstacle at a very close distance to the vehicle and inform the driver through a sensor such as an ultrasonic wave or a laser.

However, in order to detect an obstacle approaching the vehicle using an image captured by a camera installed in the vehicle instead of the ultrasonic or laser sensor, a method different from the sensor is required.

Camera-based obstacle detection, unlike ultrasonic or laser sensors, can provide detailed information about obstacles as well as detection of the presence of obstacles. In particular, when the camera is an around view monitoring (AVM) system, it is common to use a wide-angle camera that can secure a wide field of view. However, in the wide-angle camera, radiation distortion, in which distortion is determined by the refractive index of the convex lens, inevitably occurs, and thus may cause serious errors in image recognition of the image processing apparatus as well as visual distortion of the display image.

In general, an around-view monitoring system consists of four cameras, one installed at the front and rear of the vehicle and one under the left and right side mirrors. The 4-channel AVM (Around View Monitoring) system, which is used for car parking and narrow road driving, provides the driver with the function of simply showing the synthesized image.However, by analyzing the video of each input channel, an obstacle that is expected to crash while driving Recognizing this, it can provide a new safety function.

Recently, automakers developing advanced driver assistance systems (ADAS) have been trying to commercialize AVM systems that have the ability to recognize and alert obstacles with a collision risk (Mod).

The existing MOD function development method uses a method of detecting a motion region in an image, but it is difficult to recognize a single color, a simple pattern of a wall or a pillar, which is difficult to detect as a motion region, as an obstacle.

On the contrary, there is a problem that a recognition error occurs because an object such as a lane or a manhole cover existing in a free space can be mistaken for a moving object.

The present invention has been made to solve the above problems, it is possible to recognize a wall or pillar of a single color, a simple pattern that is difficult to classify as an obstacle as an obstacle, an object such as a lane, a manhole cover existing in the driving area It is an object of the present invention to provide a method and apparatus for providing driving information using a camera image, which improves a problem of causing a recognition error that is misinterpreted as a moving object.

The object of the present invention is not limited to the above-mentioned object, and other objects not mentioned will be clearly understood by those skilled in the art from the following description.

In order to achieve the above object, the present invention provides a driving information providing method in a vehicle equipped with one or more cameras, the method comprising: detecting an obstacle by real-time image processing of external image data of a vehicle received from the camera, Predicting the risk of collision based on the detected position of the obstacle and the distance between the vehicle itself, and using the Free Space Detection (FSD) algorithm, processing the external image data of the vehicle received from the camera to detect the driving range And providing information about whether the vehicle is capable of traveling through the collision risk and the driving region.

In the detecting of the driveable area, a result of performing the FSD algorithm on the external image data detects an object having no danger of collision in the obtained free space by using an image recognition method and includes it in the driveable area. Can be. In this case, the image recognition method may detect an object including a lane and a manhole cover in the free area as an object without a collision risk.

In the detecting of the obstacle, the motion region is detected based on a motion history image (MHI) algorithm for the external image data of the vehicle received from the camera, and the obstacle is detected by real-time image processing on the detected motion region. Can be detected.

In the predicting of the collision risk, the position of the detected obstacle and the coordinates of the vehicle itself may be obtained using a distance transformation matrix, and the distance between the obstacle and the vehicle itself may be calculated using the obtained coordinate values.

The vehicle may be a vehicle to which an around view monitoring system or a camera mirror system is applied.

According to an aspect of the present invention, there is provided a driving information providing apparatus in a vehicle equipped with one or more cameras. Predicts the risk of collision based on the distance between the vehicle and the vehicle, and detects a driving range by processing external image data of the vehicle received from the camera using a free space detection (FSD) algorithm. And a control unit for providing driving availability information, a display unit for displaying driving availability information under the control of the controller, and an alarm unit for alarming the risk of collision under the control of the controller.

As a result of performing the FSD algorithm on the external image data, the controller may detect an object having no danger of collision in the obtained free space by using an image recognition method and include the same in the driving range. In this case, the controller may detect an object including a lane and a manhole cover in the free area as an object without a collision risk by using the image recognition method.

The controller may detect a motion region based on a motion history image (MHI) algorithm with respect to the external image data of the vehicle received from the camera, and detect an obstacle by processing an image in real time on the detected motion region.

The controller may acquire the position of the detected obstacle and the coordinate value of the vehicle itself using the distance transformation matrix, and calculate the distance between the obstacle and the vehicle itself using the obtained coordinate value.

According to the present invention, there is an effect that can recognize a wall or pillar of a single color, a simple pattern that is difficult to classify as an obstacle.

In addition, according to the present invention, there is an effect of improving the problem that a recognition error is generated because an object such as a lane or a manhole cover existing in a driving area is mistaken for a moving object.

In addition, according to the present invention, an image processing speed is improved based on a motion history image (MHI) algorithm, thereby enabling real-time processing of 30 frames per second (FPS) or more.

Applying the present invention, it is expected that the obstacle detection function can be implemented in a camera mirror system that is expected to be commercialized in the future.

1 is a block diagram illustrating an internal configuration of a driving information providing apparatus in a vehicle according to an embodiment of the present invention.

2 is a flowchart illustrating a driving information providing method in a vehicle according to an exemplary embodiment of the present invention.

3 is a diagram illustrating an image for explaining an obstacle detection process based on an MHI algorithm according to an embodiment of the present invention.

4 is a view showing an image for explaining a collision risk according to an embodiment of the present invention.

FIG. 5 is a diagram illustrating an image for explaining a process of detecting a driving possible region using an FSD algorithm according to an embodiment of the present invention.

According to an aspect of the present invention, there is provided a driving information providing method for a vehicle equipped with one or more cameras. Estimating the risk of collision based on the distance with the user; detecting external driving data by processing external image data of the vehicle received from the camera using a free space detection (FSD) algorithm; And providing information about whether the vehicle can be driven through the driving area.

As the present invention allows for various changes and numerous embodiments, particular embodiments will be illustrated in the drawings and described in detail in the written description. However, this is not intended to limit the present invention to specific embodiments, it should be understood to include all modifications, equivalents, and substitutes included in the spirit and scope of the present invention.

The terminology used herein is for the purpose of describing particular example embodiments only and is not intended to be limiting of the present invention. Singular expressions include plural expressions unless the context clearly indicates otherwise. In this application, the terms "comprise" or "have" are intended to indicate that there is a feature, number, step, operation, component, part, or combination thereof described in the specification, and one or more other features. It is to be understood that the present invention does not exclude the possibility of the presence or the addition of numbers, steps, operations, components, components, or a combination thereof.

Unless defined otherwise, all terms used herein, including technical or scientific terms, have the same meaning as commonly understood by one of ordinary skill in the art. Terms such as those defined in the commonly used dictionaries should be construed to have meanings consistent with the meanings in the context of the related art, and shall not be construed in ideal or excessively formal meanings unless expressly defined in this application. Do not.

In addition, in the description with reference to the accompanying drawings, the same components regardless of reference numerals will be given the same reference numerals and duplicate description thereof will be omitted. In the following description of the present invention, if it is determined that the detailed description of the related known technology may unnecessarily obscure the subject matter of the present invention, the detailed description thereof will be omitted.

The present invention relates to an apparatus and method for providing driving information in a vehicle equipped with one or more cameras. The present invention may be implemented for a vehicle to which an around view monitoring system or a camera mirror system is applied.

Referring to FIG. 1, a driving information providing apparatus in a vehicle according to an embodiment of the present invention includes one or more cameras 110, a control unit 120, a display unit 130, and an alarm unit 140.

The controller 120 performs image processing on the external image data of the vehicle received from the camera 110 in real time to detect an obstacle. Then, the collision risk is predicted based on the detected position of the obstacle and the distance between the vehicle itself. In addition, by using the Free Space Detection (FSD) algorithm, the external image data of the vehicle received from the camera 110 is processed to detect a driveable area, and provide driving possibility information through a collision risk and a driveable area. .

The display unit 130 serves to display driving availability information under the control of the controller 120.

The alarm unit 140 serves to alert the risk of collision under the control of the controller 130.

In the present invention, as a result of performing the FSD algorithm on the external image data, the controller 120 detects an object having no danger of collision in the obtained free space by using an image recognition method and includes it in the driving range. For example, the controller 120 may detect an object including a lane and a manhole cover as a non-collision risk object in a free area by an image recognition method.

The controller 120 detects a motion region of the vehicle's external image data received from the camera 110 based on a motion history image (MHI) algorithm, and processes an image in real time on the detected motion region to detect an obstacle. do.

In the present invention, the control unit 120 may obtain the position of the detected obstacle and the coordinate value of the vehicle itself using the distance transformation matrix, and calculate the distance between the obstacle and the vehicle itself using the obtained coordinate value.

Referring to FIG. 2, the controller 120 performs image processing on the external image data of the vehicle received from the camera 110 in real time to detect an obstacle (S210).

Then, the collision risk is predicted based on the detected position of the obstacle and the distance between the vehicle itself (S220).

In addition, the controller 120 detects a driving possible region by processing external image data of the vehicle received from the camera 110 using a free space detection (FSD) algorithm (S230).

In addition, information on whether the vehicle is capable of driving through the collision risk and the driving possible region is provided (S240).

In operation S230, as a result of performing the FSD algorithm on the external image data, an object having no danger of collision is detected in the obtained free space using an image recognition method and included in the driving region. . For example, an image recognition method may detect an object including a lane and a manhole cover in a free area as an object without a collision risk.

In the detecting of the obstacle (S210), the motion region is detected based on a motion history image (MHI) algorithm for the external image data of the vehicle received from the camera 110, and the image is detected in real time with respect to the detected motion region. The obstacle can be detected by processing.

In operation S220, the collision risk may be predicted by using the distance transformation matrix to obtain the position of the detected obstacle and the coordinates of the vehicle and calculate the distance between the obstacle and the vehicle using the obtained coordinate value. Can be.

In FIG. 3, the area marked in red is a motion area in which motion is detected. In the present invention, the obstacle is detected by performing a high speed image processing on a near obstacle such as a moving parking vehicle in real time.

In the present invention, using the camera mounted to the front of the vehicle to calculate the distance between the position of the obstacle and the vehicle itself, thereby predicting the risk of collision.

A distance transformation matrix is illustrated in FIG. 4 (a). As described above, in the present invention, a coordinate value of a distance between the red region shown in FIG. 4 (b) and the vehicle itself is obtained using the distance transformation matrix.

Referring to FIG. 5, a MeanShift algorithm is applied to bind regions having similar color pixel values in an original image obtained from the camera 110.

In the case of the fisheye lens, since the vehicle body is visible, it is assumed that the patch area is set in the area immediately before the vehicle body, and this part includes the road area.

After applying the MeanShift algorithm, the Flood Fill algorithm is applied to cluster color values similar to the patch area. At this time, only the patch area is displayed in the Flood Fill algorithm.

In this case, since only the Flood Fill algorithm is applied, noise remains, and the binarization is performed using a difference image from the original image, and then internal noise is removed by using a morphology operation.

As shown in FIG. 5, when the noise is removed, only the free space portion remains, and the portion is labeled to acquire and display various information such as the size and center point (blue) of the region.

In the present invention, the existing problem can be solved by performing the FSD algorithm and detecting an object without a collision risk, such as a lane and a manhole cover, existing in the FSD region through an image recognition method. For example, the manhole cover can be implemented by applying a learning-based algorithm.

In the present invention, when the obstacle detection method using the MHI algorithm and the driving area detection method using the FSD algorithm are combined, a wall of a simple color or a simple pattern that is not recognized as an obstacle having a collision risk may be recognized as an obstacle.

In addition, in the present invention, when the obstacle recognition using the MHI-based algorithm and the driving region using the FSD algorithm are used, it is possible to determine whether driving is possible by real-time processing the video received by the front camera when driving forward.

While the invention has been described using some preferred embodiments, these embodiments are illustrative and not restrictive. Those skilled in the art will appreciate that various changes and modifications can be made without departing from the spirit of the invention and the scope of the rights set forth in the appended claims.

Claims

In the driving information providing method in a vehicle equipped with one or more cameras,

Detecting an obstacle by image processing external image data of the vehicle received from the camera in real time;

Predicting a collision risk based on the detected position of the obstacle and the distance between the vehicle and the vehicle;

Using the Free Space Detection (FSD) algorithm, processing external image data of the vehicle received from the camera to detect a driving range; And

And providing information about driving possibility through the collision risk and the driving area.
The method according to claim 1,

In the detecting of the driveable area,

And as a result of performing the FSD algorithm on the external image data, an object having no danger of collision in the obtained free space is detected by an image recognition method and included in the driving range.
The method according to claim 2,

In the detecting of the driveable area,

And detecting an object including a lane and a manhole cover in the free area as an object without a collision risk by using the image recognition method.
The method according to claim 1,

Detecting the obstacle,

The driving information is detected by detecting a motion region based on a motion history image (MHI) algorithm on the external image data of the vehicle received from the camera and processing an image in real time on the detected motion region. How to Provide.
The method according to claim 1,

In predicting the collision risk,

And obtaining the position of the detected obstacle and the coordinates of the vehicle itself by using the distance transformation matrix, and calculating the distance between the obstacle and the vehicle itself using the obtained coordinate values.
The method according to claim 1,

And the vehicle is a vehicle to which an around view monitoring system or a camera mirror system is applied.
A driving information providing apparatus in a vehicle equipped with one or more cameras,

Image processing of the external image data of the vehicle received from the camera in real time to detect the obstacle, predict the risk of collision based on the location of the detected obstacle and the vehicle itself, FSD (Free Space Detection) algorithm A controller configured to process an external image data of the vehicle received from the camera to detect a driving range, and to provide driving possibility information through the collision risk and the driving range;

A display unit for displaying driving availability information under control of the controller; And

And a warning unit for alerting the risk of collision under the control of the controller.
The method according to claim 7,

As a result of performing the FSD algorithm on the external image data, the controller detects an object having no danger of collision in the obtained free space by using an image recognition method and provides the driving information to the driving region. Device.
The method according to claim 8,

And the controller detects an object including a lane and a manhole cover in the free area as an object without a collision risk by the image recognition method.
The method according to claim 7,

The controller detects a motion region based on a motion history image (MHI) algorithm with respect to the external image data of the vehicle received from the camera, and detects an obstacle by processing the detected motion region in real time. Driving information providing device.
The method according to claim 7,

The control unit obtains the detected position of the obstacle and the coordinates of the vehicle itself using the distance conversion matrix, and calculates the distance between the obstacle and the vehicle itself using the obtained coordinate value.
The method according to claim 7,

And the vehicle is a vehicle to which an around view monitoring system or a camera mirror system is applied.