KR20190067578A - Collision warning device and method using heterogeneous cameras having overlapped capture area - Google Patents

Abstract

Disclosed are a collision warning apparatus using heterogeneous cameras having an overlapped photographing area and a method thereof. According to the present invention, the collision warning apparatus using the heterogeneous cameras having an overlapped photographing area comprises: a wide-angle camera unit placed in a vehicle to generate a wide-angle image, including one or more wide-angle cameras; a narrow-angle camera unit placed in the vehicle to generate a narrow-angle image, including one or more narrow-angle cameras; and a control unit which is able to detect a lane in one or more of the wide-angle image and narrow-angle image, to calculate a lane curvature value of the detected lane, to, when the lane curvature value is the same as or lower than a pre-designated first threshold value, select the wide-angle image as a default image for generating collision expectation information, and to, when the lane curvature value is higher than the first threshold value, select the narrow-angle image as the default image. The present invention aims to provide a collision warning apparatus using heterogeneous cameras having an overlapped photographing area and the method thereof, which are able to use an optimal image for a driving situation of a vehicle.

Description

[0001] The present invention relates to a collision warning apparatus and method using a heterogeneous camera having an overlap shooting region,

The present invention relates to a collision warning apparatus and method using a heterogeneous camera having an overlap photographing region.

Generally, the driver's watch on the inside of the vehicle is mainly directed to the front, and the left and right side watches and the rear side watches of the driver are largely covered by the vehicle body.

In order to solve such a problem, a clock assist means such as a side mirror is used, and in recent years, techniques including a camera means for photographing an external image of a vehicle and providing the image to a driver have been applied to vehicles.

Among them, there is an Around View Monitoring system (hereinafter referred to as AVM system) in which a plurality of cameras are installed in a vehicle to display 360-degree forward images around the vehicle.

The AVM system combines the images of the surroundings of the vehicle photographed by the cameras provided at the respective positions of the vehicle to provide a top view image in which the driver looks at the vehicle in the sky to display an obstacle around the vehicle, It is possible to solve the problem.

In addition, in recent years, a narrow-angle camera system is additionally provided so that the driver can effectively recognize the far-off road situation ahead of the vehicle.

However, the AVM system, the narrow-angle camera system, and the like provided in the vehicle are provided only for performing individual operations, and are limited in that they do not complement each other in order to recognize the possibility of collision by sensing an object existing in the running direction there was.

Korean Patent No. 10-1739394 (Object Distance Estimation Apparatus Using a Stereo Camera of Different Viewing Angles and Its Method) U.S. Patent No. 8,633,810 (Rear-view multi-functional camera system) US Patent Application No. 2017/0113611 (Method for Stereo Map Generation with Novel Optical Resolutions)

B-spline-based road model for 3D lane recognition (IEEE conference on intelligent transportation system October 2010) Curve Template Matching Based Lane and Curvature Detection Algorithm Using Top View Image (Proceedings of SPIE Vol. 47, SP Issue 6, November 2010)

In the present invention, by selecting an appropriate one of the wide angle image and the narrow angle image as the basic image in consideration of the curvature of the path and the curvature of the lane of the vehicle in operation, And to provide a collision warning apparatus and method using the same.

The present invention can increase the amount of information included in the overlap region by considering images taken by different types of cameras having different ranges of angle of view, And to provide a collision warning apparatus and method using the same.

Other objects of the present invention will become readily apparent from the following description.

According to an aspect of the present invention, there is provided a camera system comprising: a wide-angle camera unit provided in a vehicle to generate a wide-angle image including at least one wide-angle camera; A coarse camera unit provided in the vehicle to generate a narrow-angle image including at least one narrow-angle camera; And calculating a lane curvature value of the detected lane if the lane curvature value is equal to or less than a predetermined first threshold value, detecting a lane at at least one of the wide angle image and the narrow angle image, And a control unit for selecting a narrow-angle image as the basic image when the lane curvature value is greater than the first threshold value.

Wherein the control unit selects one image for which a lane curvature value to be compared with the first threshold value is to be calculated according to a predetermined condition, wherein the predetermined condition is predetermined in one of a wide-angle image and a narrow-angle image, Angle image and the narrow-angle image, or may be changed to the narrow-angle image only when the curvature value of the lane detected in the wide-angle image is greater than or equal to a predetermined limit value, .

Wherein the control unit generates a depth map for an overlap photographing area of the wide angle camera and the narrow angle camera using the wide angle image and the narrow angle image and calculates a depth map of the object in the temporally successive depth map The path curvature value corresponding to the movement trajectory of the vehicle can be calculated.

Wherein the control unit selects a wide angle image as a basic image for generating collision prediction information when the path curvature value is not greater than a predetermined second threshold value when no lane is detected in the wide angle image and the narrow angle image, And when the lane curvature value is larger than the second threshold value, the narrow-angle image can be selected as the basic image.

Wherein at least one of the wide angle camera and the at least one narrow angle camera is installed such that an overlap photographing area exists in each photographing area and the control unit generates the collision prediction information for an object existing in an overlap photographing area and another photographing area of the basic image And the overlap shooting region may generate collision prediction information using an overlap region image generated by synthesizing an auxiliary image, which is one of a wide angle image and a narrow angle image, which are not selected as a basic image, and the basic image.

The overlap region image may be generated as a depth map.

Wherein the collision prediction information is generated so as to include at least one of a separation distance from the object and a remaining time until the collision with the object, and the control unit controls, when the collision prediction information is equal to or less than a predetermined third threshold, So that the alarm can be processed.

The control unit may determine whether there is an object capable of colliding with at least one of the wide angle image and the narrow angle image, and may calculate the lane curvature value and select the basic image only when a collision object exists.

The wide angle camera may be a camera for the surround view monitoring system.

According to another aspect of the present invention, there is provided a collision warning method performed in a collision warning apparatus, the method comprising: (a) detecting a lane at one or more of a wide angle image generated by a wide angle camera and a narrow angle image generated by a narrow angle camera And calculating a lane curvature value of the detected lane; (b) comparing the calculated lane curvature value with a predetermined first threshold value; And (c) if the lane curvature value is less than or equal to the first threshold value, selecting the wide angle image as a basic image for generating collision prediction information, and if the lane curvature value is larger than the first threshold value, And selecting as the basic image a collision warning method.

Wherein the step (b) is performed for a lane curvature value of an image selected in accordance with a predetermined condition, wherein the predetermined condition is specified in advance either in a wide angle image or in a narrow angle image, And the narrow-angle image, or may be changed to the narrow-angle image only when the curvature value of the lane detected in the wide-angle image is equal to or larger than a predetermined limit value.

The collision warning method includes: generating a depth map of an overlap photographing area of the wide angle camera and the narrow angle camera using the wide angle image and the narrow angle image; And calculating a path curvature value corresponding to the movement trajectory of the vehicle using the change in depth of the object in the depth map continuously generated in time. If a lane is not detected in both the wide angle image and the narrow angle image in the step (a), the wide angle image is selected as the basic image when the path curvature value is equal to or less than a predetermined second threshold value, If the value is larger than the second threshold value, the narrow-angle image may be preset to be selected as the basic image.

(A) to (c) may be performed only when it is determined that an object existing in at least one of the wide-angle image and the narrow-angle image is a collision-capable object according to a predetermined determination method.

The collision warning method includes generating an overlap region image for an overlapping shooting region of the base image and the auxiliary image by combining the base image and the auxiliary image that is either a wide angle image or a narrow angle image that is not selected as a base image, ; And generating collision prediction information for an overlap region image and an object existing in another shooting region of the base image. Here, the collision prediction information may be generated to include at least one of a separation distance from the object and a remaining time until the object collides with the object. The overlap region image may be generated as a depth map.

Wherein the collision warning method comprises: comparing the collision prediction information with a predetermined third threshold; And if the collision prediction information is equal to or less than the third threshold, performing an alarm through the output unit.

Other aspects, features, and advantages will become apparent from the following drawings, claims, and detailed description of the invention.

According to the embodiment of the present invention, it is possible to utilize the optimal image for the driving situation of the vehicle by selecting an appropriate image from the wide angle image and the narrow angle image as the basic image in consideration of the curvature of the path and the curvature of the lane have.

Also, considering images taken by different types of cameras having different angle of view ranges, the amount of information included in the overlap area can be increased, thereby improving object detection accuracy.

1 is a block diagram of a collision warning apparatus according to an embodiment of the present invention;
BACKGROUND OF THE INVENTION 1. Field of the Invention [0001]
3 and 4 are views for explaining a method of combining a wide angle image and a narrow angle image according to an embodiment of the present invention.
5 is a view for explaining a path curvature calculating method using an overlap region image according to an embodiment of the present invention.
6 and 7 are diagrams for explaining a lane curvature calculating method according to an embodiment of the present invention.
8 is a view for explaining a basic image and an auxiliary image determination technique according to various conditions according to an embodiment of the present invention.
9 to 11 are flowcharts showing a collision warning method according to an embodiment of the present invention.

While the invention is susceptible to various modifications and alternative forms, specific embodiments thereof are shown by way of example in the drawings and will herein be described in detail. It is to be understood, however, that the invention is not to be limited to the specific embodiments, but includes all modifications, equivalents, and alternatives falling within the spirit and scope of the invention.

It is to be understood that when an element is referred to as being "connected" or "connected" to another element, it may be directly connected or connected to the other element, . On the other hand, when an element is referred to as being "directly connected" or "directly connected" to another element, it should be understood that there are no other elements in between.

The terms first, second, etc. may be used to describe various components, but the components should not be limited by the terms. The terms are used only for the purpose of distinguishing one component from another. For example, terms such as a first threshold value, a second threshold value, and the like which will be described later may be previously designated with threshold values that are substantially different from each other or some of which are the same value, Because there is room, the terms such as the first and the second are to be mentioned for convenience of division.

The terminology used herein is for the purpose of describing particular embodiments only and is not intended to be limiting of the invention. The singular expressions include plural expressions unless the context clearly dictates otherwise. In this specification, the terms "comprises" or "having" and the like refer to the presence of stated features, integers, steps, operations, elements, components, or combinations thereof, But do not preclude the presence or addition of one or more other features, integers, steps, operations, elements, components, or combinations thereof.

It is to be understood that the components of the embodiments described with reference to the drawings are not limited to the embodiments and may be embodied in other embodiments without departing from the spirit of the invention. It is to be understood that although the description is omitted, multiple embodiments may be implemented again in one integrated embodiment.

DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS Reference will now be made in detail to the preferred embodiments of the present invention, examples of which are illustrated in the accompanying drawings, wherein like reference numerals refer to the like elements throughout. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS Hereinafter, exemplary embodiments of the present invention will be described in detail with reference to the accompanying drawings. In the following description, well-known functions or constructions are not described in detail since they would obscure the invention in unnecessary detail.

FIG. 1 is a block diagram of a collision warning apparatus according to an embodiment of the present invention. FIG. 2 is a view for explaining an image captured by a wide-angle camera unit and a coarse-angle camera unit according to an embodiment of the present invention. 3 and 4 are views for explaining a method of composing a wide-angle image and a narrow-angle image according to an embodiment of the present invention. FIG. 5 is a view for explaining a path curvature using an overlap region image according to an embodiment of the present invention. ) Calculation method. 6 and 7 are views for explaining a lane curvature calculating method according to an embodiment of the present invention. FIG. 8 is a view for explaining a method of calculating a lane curvature according to an embodiment of the present invention, FIG. 9 through FIG. 11 are flowcharts illustrating a collision warning method according to an embodiment of the present invention.

Referring to FIG. 1, the collision warning apparatus 100 may include a wide angle camera unit 110, a narrow angle camera unit 120, a control unit 130, and an output unit 140.

Each of the wide angle camera unit 110 and the narrow angle camera unit 120 can be controlled by the control unit 134 of the control unit 130 to generate a wide angle image or a narrow angle image in a predetermined condition and / or in real time.

The wide-angle camera unit 110 may include one or more wide-angle cameras and a processing unit for generating a captured image (i.e., a wide-angle image) according to the photographing operation of each wide-angle camera. The wide-angle image generated by the processing unit can be stored in a storage unit (not shown) so that it can be used in the control unit 130 or the like.

The wide angle camera is, for example, a camera for an AVM system, and is a wide angle camera (211, 213, 215, 217) installed at a plurality of positions (for example, a position designated to photograph the front, rear, left, It is possible.

The wide angle cameras 211, 213, 215, and 217 may be, for example, a wide angle lens having an angle of view of about 180 degrees or so A fisheye lens) is used. The wide-angle image generated by the wide-angle camera is illustrated in Figs. 2 (a) and 2 (b), respectively.

The coarse camera unit 120 may include one or more coarse angle cameras and a processing unit for generating a captured image (i.e., a coarse angle image) according to the photographing operation of each coarse angle camera. The narrow-angle image generated by the processing unit can be stored in a storage unit (not shown) so that it can be used in the control unit 130 or the like.

The narrow angle camera 240 is a camera in which a narrow angle lens having an angle of view of, for example, 30 to 50 degrees is used, as shown in FIG. 2 (d) to be. The photographed image generated by the coarse camera is illustrated in Fig. 2 (c).

Although FIG. 2 (d) illustrates one narrow-angle camera 240 for photographing the front of the vehicle, the narrow-angle camera 240 may be provided with one or more additional cameras, such as the rear of the vehicle, Of course.

The control unit 130 may include an image processing unit 132 and a control unit 134.

The image processor 132 determines whether or not there is an object capable of colliding with the photographed object in the wide-angle image and the narrow-angle image, and generates a depth map using the wide-angle image and the narrow- , The path curvature value is calculated using the generated depth map.

Also, the image processor 132 recognizes lanes around the vehicle using the wide-angle image and / or the narrow-angle image, calculates the curvature value of the lane, and selects the primary image and the secondary image.

Also, the image processor 132 generates collision prediction information between an overlap region between the basic image and the auxiliary image, and an object existing in another region of the basic image. The collision prediction information may include at least one of a distance to an object, a remaining time to collision, etc., as will be described later.

Hereinafter, the operation of the image processing unit 132 will be described in detail.

First, the image processing unit 132 recognizes an object photographed by the wide-angle image generated by the wide-angle camera and the narrow-angle image generated by the narrow-angle camera, and determines whether there is a possibility of collision with the recognized object.

For example, the image processing unit 132 determines whether the captured object is an object having a change in size that increases when referring to a wide-angle image and / or a narrow-angle image generated in real time, Whether the object is a collision object or not, whether or not the object exists in the same lane as the own vehicle (for example, lane, parking lot, etc.).

Herein, lanes can be detected by an image processing technique such as edge detection in a wide-angle image and / or a narrow-angle image, and these will be obvious to those skilled in the art, and a description thereof will be omitted. However, in the case of a wide-angle image, processing for conversion to a top view image may be performed in order to enable accurate lane detection and object detection.

For example, the image processor 132 may determine that there is a possibility of collision if an object existing in the wide-angle image and / or the narrow-angle image is changed so as to be continuously increased in the direction toward the subject vehicle. However, if the size of the object gradually decreases or the size thereof increases, but the size of the object increases in the direction of moving the vehicle, it can be judged that there is no possibility of collision.

If it is determined that the object is a collision object, the image processing unit 132 may generate a depth map using the wide angle image and the narrow angle image, and calculate the path curvature value using the generated depth map.

As illustrated in FIGS. 3A and 3B, the wide-angle image 310 is generated so as to facilitate detection of an object within a relatively short range, and the narrow-angle image 320 is generated from an object at a relatively long distance As shown in FIG. At this time, a wide-angle camera and a narrow-angle camera are installed in the vehicle such that the overlap photographing region 260 exists between the wide-angle image 310 and the narrow-angle image 320 (see FIG.

The image processing unit 132 generates a depth map using a stereo mapping method using the wide angle camera and the narrow angle camera as a stereo vision. That is, the image processing unit 132 uses a wide-angle image 310 and a narrow-angle image 320 generated by the wide-angle camera and the narrow-angle camera to generate a depth map ) 330 (see Fig. 4).

Since the shapes of the wide-angle image 310 and the narrow-angle image 320 are different from each other, the wide-angle image 310 may be transformed into the narrow-angle image 320 by using a pixel mapping method using a lookup table (LUT) ) In the same or similar format (for example, a flat image format). A detailed method of generating the depth map 330 is described in detail in U.S. Patent No. 8,633,810, U.S. Patent Application Publication No. 2017/0113611, and so on.

In addition, there are a variety of methods for mapping the pixels of the wide-angle image 310 and the narrow-angle image 320 and performing synthesis processing. For example, a synthesis process for generating the depth map 330 is performed using the following Equations 1 to 3 It is possible.

The equation ( ₁ ) is for calculating the H matrix (i.e., H ₁ ) of the wide angle camera to obtain the relationship between the plane floor of the real world and the wide angle camera. Equation ( ₂₎ ) To calculate the relationship between the plane floor of the real world and the narrow-angle camera.

Here, x and y are pixel coordinates in the photographed image, x 'and y' represent real world coordinates (measured coordinates) for distances, positions, etc., separated from the vehicle, and w is a predetermined weight. H ₁ and H ₂ , which are unknown matrices, can be calculated by Equations 1 and 2, respectively.

In addition, as calculated by reference to the calculated H ₁ and H _2, the same subject (i. E., Flat bottom), the wide-angle image and the narrow-angle image due to the narrow-angle camera according to the wide-angle camera is to have the relationship of Equation (3).

Accordingly, an arbitrary pixel position (P _Narrow ) of the _narrow- angle image can be projected to one pixel position (P _Wide ) of the wide-angle image according to Equation (3), and the image processing unit It is possible to generate an overlap region image obtained by combining the overlap region 260 of the image. Likewise, it is natural that one pixel position of the wide-angle image may be projected to one pixel position of the narrow-angle image. At this time, the process of correcting the wide-angle image into the shape of the image photographed by the normal lens using the predetermined look-up table (LUT) may be preceded and then mapped to each pixel of the narrow-angle image.

The image processing unit 132 calculates the path curvature value according to the driving of the vehicle using the generated depth map 330. [

As illustrated in Fig. 5, the vehicle can be driven in the directions of forward straight ahead, forward right turn, forward left turn, backward forward, reverse right turn, and reverse right turn.

In each case, the depth map 330 used by the image processing unit 132 to calculate the path curvature value according to the driving of the vehicle is a composite image generated to correspond to the forward or backward movement. That is, the depth map 330 is a depth map of the wide angle image and the narrow angle image taken in the front of the vehicle when the vehicle is running in the forward direction. When the vehicle is running in the backward direction, Depth map.

At this time, the information on whether the vehicle is moving forward or backward can be recognized, for example, by the control unit 130 being provided from at least one of a vehicle control system (for example, an ECU), a GPS system, a geomagnetic sensor, The processing unit 132 may generate the depth map 330 using the appropriate image.

The wide angle image 310 and the narrow angle image 320 are generated in real time while the vehicle is moving in an arbitrary direction, and the image processing unit 132 generates the depth map 330 in real time using them. The image processing unit 132 may calculate the path curvature value according to the driving of the vehicle by using the depth value change (i.e., the change of the relative distance) of the object in the depth map 330 generated in real time.

For example, if the vehicle moves in the direction A in the state in which the depth map 330 shown in FIG. 5 is generated, the image processing unit 132 determines whether there is a difference within the error range with respect to the objects 1, 2, Will recognize changes in depth that recognize a common approach.

However, if the vehicle moves in the b &thetas;, the image processing unit 132 determines that the depth of the object 1 ' in the front most deepens (i. E. ) Direction of each object.

In other words, the image processing unit 132 can recognize the movement and the trajectory of the vehicle using the depth map of the continuously generated depth map 330 and the positional change of each object, It is possible to calculate the curvature value of the path representing the flow of the movement direction of the vehicle itself. The calculation criterion for calculating the path curvature value for each successive motion size can be specified in advance.

Here, the path curvature value is a value indicating whether the rotational motion amount of the vehicle is large or small, and the smaller the path curvature value, the more the vehicle has a sudden turning motion.

In calculating the path curvature value according to the movement of the vehicle using the depth change and the position change of each object in the depth map 330, a large change amount due to the movement of the specific object included in the depth map 330 The path curvature value is calculated except for the amount of change of the object in which the amount of change of each object included in the depth map 330 largely changes to a value larger than a predetermined reference value in comparison with the surrounding object in order to prevent the path curvature value from being distorted. .

Next, the image processing unit 132 calculates a curvature value of the lane detected in the wide angle image 310 and / or the narrow angle image 320. [ Here, the lane includes all the display information shown on the floor surface to induce the movement of the vehicle. For example, the lane includes a lane (see Fig. 6A) of the road on which the vehicle travels, a parking lot partition line 6 (b)).

The image processing unit 132 can detect the lane in the narrow-angle image 320 and calculate the curvature value of the detected lane, as illustrated in Fig. 7A. The curvature value of the lane can be obtained by, for example, extending a clothoid model by a commonly used B-spline curve in the lateral direction, and by using two B-spline curves, And can be calculated using a 3D model such as modeling a profile.

A clothoid model assuming a planar road can be expressed by the following equation (4).

For reference, the position of the center of the lane in the X (L) is the length L in Equation 4, x _offset is the distance to the lane of the subject vehicle (ego vehicle), Δψ is a yaw angle (yaw angle), c ₀ is the next best Curvature, and c ₁ denotes a curvature change amount (closode parameter), wherein the width of the lane is predetermined. The method of calculating the curvature value (c ₀ ) of the recognized lane using the clause model is described in detail in the related paper (B-spline-based road model for 3D lane recognition, IEEE conference on intelligent transportation system October 2010) A description thereof will be omitted.

7B, the image processing unit 132 may convert the wide-angle image 310 into a top-view image, detect a lane based on the top-view image, and calculate a curvature value of the detected lane have. In the case of the curved lane, the method of detecting the curvature through the matching process of the curve template is a method of detecting the curved line based on the related paper (the curve-template matching based lane and the curvature Detection Algorithm - November 2010, vol. 47, pp. SP, No. 6), and the description thereof will be omitted.

The image processing unit 132 may calculate the curvature value of the lane based on both the wide angle image 310 and the narrow angle image 320 as described above. However, the image processing unit 132 may calculate the curvature value of the lane It may be designated in advance to select an object to be calculated as either the wide-angle image 310 or the narrow-angle image 320. [

For example, when the vehicle travels at a predetermined speed such as a highway traveling, the curvature value of the lane is calculated with respect to the narrow-angle image 320. When the vehicle travels at a predetermined speed or less such as moving in the parking lot It may be specified in advance to calculate the curvature value of the lane on the wide-angle image 310. [ The moving speed of the vehicle may be recognized, for example, by the control unit 130 receiving relevant information from a vehicle control system (e.g., an ECU), a GPS system, or the like.

As another example, after calculating the curvature values of the lanes with respect to both the wide-angle image 310 and the narrow-angle image 320, if the curvature value of the lane calculated by the wide-angle image 310 is equal to or larger than a predetermined limit value When the curvature value of the lane calculated in the wide angle image 310 is less than a predetermined limit value, the curvature value of the lane calculated in the wide angle image 310 is determined as the curvature value of the lane calculated in the wide angle image 310 May be specified in advance to be applied.

Thereafter, the image processor 132 generates collision prediction information with respect to an object positioned in the vehicle traveling direction by using either the wide-angle image 310 or the narrow-angle image 320 with reference to the calculated path curvature value and / or the lane curvature value And generates collision prediction information for the object using the primary image and the secondary image.

In detail, when the basic image is selected by referring to the motion of the vehicle or the curvature of the peripheral lane, the image processor 132 determines whether the curvature is smaller than a predetermined threshold value (i.e., (See Figs. 8 (a) and 8 (c)). At this time, the lane curvature value is compared with a predetermined first threshold value, and the path curvature value can be specified in advance so as to be compared with a predetermined second threshold value. Here, the first threshold value and the second threshold value may be designated as the same value or different values, respectively.

This is because, when the curvature is small, the vehicle rotation is required at a sudden angle, so the information around the vehicle may be more important. Therefore, the collision with an object in the running direction, which will be described later, This is because it is effective to generate information.

However, if the curvature is larger than the threshold value (i.e., the degree of bending is small), the image processing unit 132 selects a narrow-angle image among the wide-angle image and the narrow-angle image as a basic image (see Fig.

This is because, in the case of a large curvature, it is necessary to straight-line the vehicle or turn the vehicle at a gentle angle, so that the information of far distance is more important than the surroundings of the vehicle. Therefore, So that the collision prediction information can be generated.

However, it may be specified in advance to apply the lane curvature value to the path curvature value as information for selecting the basic image. This is because it is common for a vehicle to move along a displayed lane in order to derive the moving direction of the vehicle.

Accordingly, when the lane is detected using the wide angle image 310 and / or the narrow angle image 320 and the lane curvature value is calculated, the image processing unit 132 selects the basic image by comparing the lane curvature value with a predetermined threshold value. However, since the lane is not recognized when the vehicle is traveling on an empty road where the lane is not displayed, the path curvature value will be selected in comparison with the corresponding threshold value.

The image processor 132 divides the basic image selected by the wide-angle image 310 and the narrow-angle image 320 into an overlap region image and another region image, detects an object in the overlap region image and the other region image, Creates collision prediction information for an object.

The overlap region image is an image corresponding to the overlap photographing region 260 shown in (d) of FIG. 2, and the other region image is an image for the photographing region (i.e., other region) excluding the overlap region image. In other words, for example, if the basic image is a wide-angle image, it corresponds to the shooting area corresponding to (1) and (2) of FIG. 2 (d) .

The image processing unit 132 may generate an overlap region image corresponding to the overlap shooting region 260 in the form of the depth map 330 as described above.

The image processing unit 132 attempts to detect an object in the overlap area image, and calculates a spatial separation distance from the detected object. The separation distance from the object may be detected in real time by using an overlap region image generated using a wide angle image and a narrow angle image generated in real time.

Also, the image processor 132 may calculate the remaining time required to collide with the object using the size change amount of the collision object existing in the overlap area and / or the distance change amount with respect to the object.

As described above, the image processor 132 generates the overlap region images using the images (i.e., the wide-angle image and the narrow-angle image) shot by the different types of cameras having different angles of view angles, The amount of information can be increased, and the accuracy of object detection can be improved.

The image processing unit 132 also tries to detect an object in other areas than the overlap shooting area 260 in the basic image. The other regions corresponding to the other regions differ as described above depending on whether the basic image is selected as the wide-angle image or the narrow-angle image.

If a collision object is detected in the other region image corresponding to the other region, the remaining time required until collision with the object is calculated using the size change amount of the object and / or the change amount of the distance to the object as described above Can be calculated.

The control unit 134 can control the user to perform the alarm process when the collision prediction information generated by the image processing unit 132 (i.e., the remaining time and / or the separation distance) is smaller than a predetermined third threshold value

Here, the alarm processing can be performed in a visual or / and audible manner and can be processed through an output unit 140, for example a display device, a speaker device, and the like.

9 to 11 are flowcharts illustrating a collision warning method according to an embodiment of the present invention.

9 to 11, in step 910, the control unit 130 of the collision warning apparatus 100 is provided with the wide angle image and the narrow angle image respectively generated by the wide angle camera and the narrow angle camera provided in the vehicle.

In step 915, the control unit 130 determines whether an object is detected in the provided wide-angle image and / or the narrow-angle image. A method of detecting an object such as a vehicle or a traffic light from an image is obvious to those skilled in the art, and a description thereof will be omitted.

If an object is detected in the wide-angle image and / or the narrow-angle image, the control unit 130 determines in step 920 whether the object is a collision object with the subject vehicle. For example, it can be determined that a collision is possible if the object is located in the same lane as the lane on which the vehicle is traveling, and the object has a change in the direction of increasing the size of the wide-angle image or the narrow-angle image generated in real time.

If it is determined that the detected object is a collision object, the control unit 130 generates a depth map using the wide-angle image and the narrow-angle image in step 930, and calculates a path curvature value .

Here, the depth map is created by using a wide-angle camera and a narrow-angle camera as a stereo vision and using a stereo mapping method, and has a feature of acquiring information on the depth of each object in the image.

Accordingly, when the subject vehicle is moved in an arbitrary direction, the depth values of the objects included in the depth map are changed, and the direction of the subject vehicle in which direction and the orbit is being moved A corresponding path curvature value can be calculated.

In step 935, the control unit 130 determines whether a lane is detected in the wide-angle image 310 and / or the narrow-angle image 320. The lane may be, for example, a lane on the road on which the vehicle is traveling, a parking lot line in the parking lot, and the like.

 If a lane has been detected, then in step 940 the control unit 130 calculates the curvature value of the detected lane. The curvature of the lane can be calculated, for example, using a clause model or a matching process of a curve template, as described above.

In step 945, the control unit 130 determines whether the calculated lane curvature value is equal to or less than a predetermined first threshold value. Here, the lane curvature value is a value indicating the degree of bending of the lane, for example, and the smaller the curvature value, the more rapidly the lane is bent.

If the lane curvature value is equal to or less than the first threshold value, the control unit 130 selects the wide angle image and the wide angle image among the input wide angle image and the narrow angle image as a basic image in step 950, An overlap region image for the photographing region 260 is generated. Here, the overlap region image can be generated as a depth map as described above.

The control unit 130 attempts to detect an object in the generated overlap region image and generates collision prediction information corresponding to the object.

The collision prediction information may include, for example, a distance to an object, a time remaining until collision with the object, and the like. That is, the control unit 130 calculates the spatial separation distance from the object detected in the overlap area image. The calculated separation distance can be utilized as information for calculating the remaining time required for the collision with the object using the size change amount of the object and / or the change amount of the separation distance from the object.

In step 955, the control unit 130 additionally tries to detect an object in other areas (see 1 and 2 in Fig. 2) than the overlap shooting area in the wide angle image, which is the basic image. If a collision object is detected in the other region image, the control unit 130 generates collision prediction information corresponding to the object as described above.

However, if it is determined in step 945 that the lane curvature value is equal to or greater than the first threshold value, in step 960, the control unit 130 selects the narrow-angle image and the narrow-angle image as the basic image, Thereby generating an overlap region image for the overlap photographing region 260 of the wide angle image and the narrow angle image.

The control unit 130 also attempts to detect an object in the generated overlap region image and generates collision prediction information for the detected object.

Subsequently, in step 965, the control unit 130 additionally tries to detect an object in another area (see 3 in Fig. 2) other than the overlap area in the narrow-angle image, which is the basic image, Collision prediction information corresponding to the object as described above.

As described above, when a lane is detected in the wide-angle image and / or the narrow-angle image, the lane curvature value is calculated as described in steps 940 to 965, and the lane curvature value is compared with the predetermined first threshold value, And the collision prediction information for the detected object can be generated by using them.

However, if the lane is not detected in step 935, the control unit 130 determines in step 980 whether or not the calculated path curvature value is equal to or less than a predetermined second threshold value.

If the path curvature value is less than or equal to the second threshold value, the control unit 130 performs steps 950 and 955 described above to generate collision prediction information.

However, if the path curvature value is greater than the second threshold, the control unit 130 performs steps 960 and 965 described above to generate collision prediction information.

In step 970, the control unit 130 determines whether the collision prediction information (e.g., the separation distance and / or the remaining time to collision) for the collision object calculated in steps 950 to 965 is less than a predetermined third threshold value . It will be appreciated that the third threshold value will be specified in advance, such as a distance value and / or a time value.

If the collision prediction information is less than or equal to the third threshold value, the control unit 130 processes the alarm through the output unit 140 to ventilate the driver in a visual and / or audible manner (step 975).

It is a matter of course that the above-mentioned collision warning method may be performed by an automated procedure in a time-series sequence by a built-in or installed program in the digital processing apparatus. The codes and code segments that make up the program can be easily deduced by a computer programmer in the field. In addition, the program is stored in a computer readable medium readable by the digital processing apparatus, and is read and executed by the digital processing apparatus to implement the method.

It will be apparent to those skilled in the art that various modifications and variations can be made in the present invention without departing from the spirit or scope of the invention as defined in the appended claims. It will be understood that the present invention can be changed.

100: crash warning device 110: wide angle camera unit
120: coarse camera unit 130: control unit
132: image processor 134:
140: output unit 211, 213, 215, 217: wide angle camera
240: coarse camera 260: overlap photographing area
310: wide angle image 320: narrow angle image
330: depth map

Claims (17)

A wide angle camera unit provided in the vehicle to generate a wide angle image including at least one wide angle camera;
A coarse camera unit provided in the vehicle to generate a narrow-angle image including at least one narrow-angle camera; And
The method comprising: detecting a lane at one or more of the wide angle image and the narrow angle image, calculating a lane curvature value of the detected lane, and when the lane curvature value is equal to or less than a predetermined first threshold value, And a control unit for selecting a narrow-angle image as the basic image when the lane curvature value is greater than the first threshold value.
The method according to claim 1,
Wherein the control unit selects one image to calculate a lane curvature value to be compared with the first threshold value according to a predetermined condition,
The predetermined condition may be specified either in a wide angle image or in a narrow angle image, or may be specified as one of the wide angle image and the narrow angle image corresponding to the traveling speed of the vehicle, or may be specified as the wide angle image, And changes to the narrow-angle image only when the curvature value of the lane is equal to or larger than a predetermined limit value.
The method according to claim 1,
The control unit includes:
Angle camera and the coarse-angle camera using the wide-angle image and the narrow-angle image, and generating a depth map for the overlap photographing region of the wide-angle camera and the narrow- And calculates a path curvature value corresponding to the movement locus.
The method of claim 3,
The control unit includes:
When the lane is not detected in the wide angle image and the narrow angle image, if the path curvature value is equal to or less than a predetermined second threshold value, the wide angle image is selected as a basic image for generating collision prediction information, And selects a narrow-angle image as the basic image if the second threshold is greater than the second threshold.
5. The method of claim 4,
Wherein at least one wide angle camera and at least one narrow angle camera are installed such that an overlap shooting area exists in each shooting area,
Wherein the control unit generates the collision prediction information for an object existing in an overlap shooting area of the basic image and in another shooting area,
Wherein the collision prediction unit generates the collision prediction information using the overlap region image generated by synthesizing the basic image and the auxiliary image which is one of the wide angle image and the narrow angle image that are not selected as the basic image, .
6. The method of claim 5,
Wherein the overlap region image is generated as a depth map.
6. The method of claim 5,
Wherein the collision prediction information is generated so as to include at least one of a separation distance from the object and a remaining time until the object collides with the object,
Wherein the control unit causes the alarm to be issued through the output unit when the collision prediction information is equal to or less than a predetermined third threshold value.
The method according to claim 1,
Wherein the control unit determines whether there is an object capable of colliding with at least one of the wide angle image and the narrow angle image and performs calculation of the lane curvature value and selection of the basic image only when there is a collision object. Warning device.
The method according to claim 1,
Wherein the wide angle camera is a camera for an ambient view monitoring system.
A collision warning method performed by a collision warning device,
(a) detecting a lane at one or more of a wide angle image generated by the wide angle camera and a narrow angle image among the narrow angle image generated by the narrow angle camera, and calculating a lane curvature value of the detected lane;
(b) comparing the calculated lane curvature value with a predetermined first threshold value; And
(c) if the lane curvature value is equal to or less than the first threshold value, the wide angle image is selected as a basic image for generating the collision prediction information, and when the lane curvature value is larger than the first threshold value, And selecting as the primary image.
11. The method of claim 10,
Wherein the step (b) is performed on a lane curvature value of an image selected according to a predetermined condition,
The predetermined condition may be specified either in a wide angle image or in a narrow angle image, or may be specified as one of the wide angle image and the narrow angle image corresponding to the traveling speed of the vehicle, or may be specified as the wide angle image, And changes to the narrow-angle image only when the curvature value of the lane is equal to or greater than a predetermined limit value.
11. The method of claim 10,
Generating a depth map for an overlap shooting region of the wide angle camera and the narrow angle camera using the wide angle image and the narrow angle image; And
Further comprising the step of calculating a path curvature value corresponding to the movement trajectory of the vehicle by using the depth change of the object in the depth map continuously generated in time,
If a lane is not detected in both the wide angle image and the narrow angle image in step (a), the wide angle image is selected as the basic image when the path curvature value is equal to or less than a predetermined second threshold value, And if the second threshold value is greater than the second threshold value, the narrow-angle image is selected as the basic image.
11. The method of claim 10,
Wherein the steps (a) to (c) are performed only when it is determined that an object existing in at least one of the wide-angle image and the narrow-angle image is a collision-capable object according to a predetermined determination method.
13. The method of claim 12,
Generating an overlap region image of the overlapping image region of the basic image and the auxiliary image by synthesizing the basic image and the auxiliary image which is one of the wide angle image and the narrow angle image not selected as the basic image; And
Further comprising the step of generating collision prediction information for the overlap region image and an object existing in another imaging region of the basic image,
Wherein the collision prediction information is generated to include at least one of a separation distance from the object and a remaining time until the object collides with the object.
15. The method of claim 14,
Comparing the collision prediction information with a predetermined third threshold value; And
And if the collision prediction information is equal to or less than the third threshold value, processing to cause an alarm through the output unit to be performed.
15. The method of claim 14,
Wherein the overlap region image is generated as a depth map.
A computer program stored in a recording medium for executing the collision warning method according to any one of claims 10 to 16.

Images