KR102174728B1 - Method for detecting critical area based on monocular camera and system therefor - Google Patents

Abstract

A method of detecting a dangerous area based on a monocular camera is disclosed. The method includes receiving an image from a front camera of the vehicle, setting an ROI of the vehicle from the image, generating a side normal image for the set ROI, and a tire from the generated side normal image. And generating a vehicle candidate based on and detecting a dangerous area based on the generated vehicle candidate.

Description

Monocular camera-based hazardous area detection method and system {METHOD FOR DETECTING CRITICAL AREA BASED ON MONOCULAR CAMERA AND SYSTEM THEREFOR}

An embodiment according to the concept of the present invention relates to a method of detecting a danger area from a single image acquired from a front camera of a vehicle.

In recent years, convergence technology to actively protect pedestrians from automobile accidents by combining information technology and sensor technology has been actively studied.

In particular, in order to minimize false alarms and real-time of the active pedestrian protection system, a new approach to searching for pedestrians in a limited area is proposed instead of the existing method of searching for pedestrians in the entire image acquired from the front camera of the vehicle. For example, a method of first detecting a high-risk dangerous area and detecting a pedestrian only in that area is proposed in consideration of a situation in which the possibility of an accident is very high.

Registered Patent Publication No. 10-1188584 (hereinafter referred to as'prior technical document 1') relates to an object identification device in front of a vehicle using a camera and a laser scanner, and uses the position coordinates of the laser scanner to determine the object from the camera image. A configuration for detecting an area is disclosed.

The prior art document 1 has an advantage of reducing the amount of time and computation required to detect an object area. However, the combination of heterogeneous sensors complicates the structure of the system somewhat, and the laser scanner has a disadvantage in that the cost is high, durability is weak, and the appearance of the vehicle must be severely modified when the sensor is mounted.

Registered Patent Publication No. 10-1452628 (hereinafter referred to as'prior technical document 2') relates to a stereo vision-based hazardous area detection method and device, based on a change in distance to an obstacle acquired through a stereo camera. A configuration for detecting a dangerous area where it is difficult for pedestrians to recognize each other is disclosed.

The prior art document 2 shows accurate performance when detecting a dangerous area, but stereo vision requires a camera calibration process and a stereo matching process to obtain a displacement map, and such stereo matching requires a relatively large amount of computation. There is this.

Registered Patent Publication No. 10-1188584 Registered Patent Publication No. 10-1452628

The present invention has been devised to solve the above problems, and an object of the present invention is to detect a dangerous area with a single image acquired from a front camera of a vehicle.

The second object of the present invention is to generate vehicle candidates based on a single tire.

In order to achieve the above object, the method for detecting a dangerous area based on a monocular camera of the present invention includes receiving an image from a front camera of a vehicle, setting an ROI of the vehicle from the image, and And generating a side normal image, generating a vehicle candidate based on a tire from the generated side normal image, and detecting a danger area based on the generated vehicle candidate.

The generating of the vehicle candidate includes generating a tire candidate in the region of interest from the side normal image, classifying the two generated tire candidates into front and rear wheels, and a center point of each of the front and rear wheels, And generating the vehicle candidate using the ground point and the size.

The vehicle candidate is generated using the angle between the center point of the front wheel and the center point of the rear wheel, the physical distance between the ground point of the front wheel and the ground point of the rear wheel, and the ratio between the size of the front wheel and the size of the rear wheel. do.

The lateral length of the vehicle candidate is set based on a distance between the front and rear wheels, and the lateral length of the vehicle candidate is set to a size of a larger tire among the sizes of the front and rear wheels.

The generating of the vehicle candidate may include generating a tire candidate within the region of interest from the side normal image, and the one tire using the ground point position of the one tire from the generated one tire candidate. And generating a vehicle candidate when the tire is a front wheel and a vehicle candidate when the one tire is a rear wheel.

The monocular camera-based dangerous area detection system according to an embodiment of the present invention includes a camera that acquires an image in front of a vehicle, an ROI setting unit that sets an ROI of the vehicle from the image, and a side normalization for the set ROI. A side normal image generator for generating an image, a vehicle detection unit for generating a tire candidate within the region of interest from the generated side normal image and generating a vehicle candidate based on one or more tires, and the generated vehicle candidate It includes a danger area detection unit for detecting a danger area.

The vehicle detection unit generates a vehicle candidate selected from a geometric relationship between two tires constituting one vehicle, a vehicle candidate when one tire is a front wheel, and a vehicle candidate when the one tire is a rear wheel.

As described above, the method for detecting a dangerous area based on a monocular camera of the present invention detects a dangerous area with a single image acquired from a front camera of a vehicle, so that the system structure is simple and the amount of computation can be minimized.

In addition, the method for detecting a dangerous area based on a monocular camera of the present invention as described above has an effect of increasing a detection rate of a dangerous area because a vehicle candidate is generated with one tire.

1 is a view for explaining a danger area.
2 is a diagram for describing a region of interest.
3 is a diagram for explaining a side normal image.
4 to 7 are views for explaining a part-based vehicle detection method according to an embodiment of the present invention.
8 and 9 are views for explaining a part-based vehicle detection method according to another embodiment of the present invention.
10 and 11 are diagrams for explaining a method of generating a vehicle candidate based on a tire according to an embodiment of the present invention.
12 shows a system for detecting a dangerous area based on a monocular camera according to an embodiment of the present invention.

The present invention relates to a method of detecting a dangerous area with a single image acquired from a front camera of a vehicle, and in particular, a vehicle is detected through a part-based vehicle detection method after generating a side normal image with a minimal change in the side appearance of the vehicle. After that, the area in front of the vehicle is detected as a danger area.

Hereinafter, the present invention will be described in more detail with reference to embodiments and drawings according to the present invention.

1 is a view for explaining a danger area. Referring to FIG. 1, among pedestrian crossing accidents that account for most of pedestrian accidents, it shows a situation of a vehicle-to-pedestrian collision accident that occurs when a pedestrian suddenly appears after being covered by an obstacle such as a parked vehicle and the driver and the pedestrian cannot recognize each other. Here, the danger area is an area in front of the parked and stopped vehicles or between the parked and stopped vehicles.

Even if a pedestrian suddenly appears behind the obstacle on the left side of the vehicle, the distance between the vehicle and the pedestrian is relatively long, so it is a condition that the driver and the pedestrian can recognize each other.However, if a pedestrian suddenly appears behind the obstacle on the right side of the vehicle, there is a risk. Is higher.

The present invention relates to a method of detecting a danger area from a single image acquired from a front camera of a vehicle. In order to detect a danger area between vehicles parked and stopped on the right side of the road by using a monocular camera, the location of the parked vehicle must first be identified.

However, in the single image itself, perspective distortion, which is a change in the shape of the vehicle according to the longitudinal distance, is largely issued, so it is not easy to detect a vehicle based on a single image. In order to solve this problem, we introduce a side normal image.

2 is a diagram for describing a region of interest. In order to generate a lateral normal image, as shown in FIG. 2, a region of interest is first set on the right side of the host vehicle, and a lateral normal image is generated around the set region of interest.

For example, the longitudinal starting distance from the center of the vehicle is 6m, the maximum longitudinal distance is 22.5m, the right transverse starting distance is 1.2m, and the maximum transverse distance is 4.5m.

Since the camera image does not appear within 6m from the center of the vehicle due to the height of the front camera and the camera angle of view, the vertical starting distance can be set to 6m. In addition, assuming that the maximum speed of the vehicle is 30 to 40 km/h and the time for the driver to recognize and react to the pedestrian is 2 seconds, the minimum distance at which the risk of a pedestrian collision may appear at the maximum speed is 20 m ahead. At this time, if the length of the car is considered to be 5m, the maximum longitudinal distance can be set to 22.5m.

The side normal image is generated by projecting the image acquired from the front camera with a virtual camera having an optical axis perpendicular to the longitudinal axis of the vehicle. At this time, since the side surface of the counterpart vehicle is a plane at a certain distance from the virtual camera as shown in FIG. 2, it is projected in the same shape and size on the side normal image regardless of the longitudinal distance from the host vehicle.

One point p _r in the image acquired from the front camera is transformed into one point p _c in the side normal image by the following [Equation 1].

[Equation 1]

Here, Kc and Rc correspond to a rotation matrix of an intrinsic parameter and an extrinsic parameter of the front camera. Kr and Rr represent the rotation matrix of the internal and external parameters of the virtual camera.

In Rr, the values of roll, pitch, and yaw are 0°, 0°, and ±90°, respectively, and when the direction of the difference is 0°, when the rotation angle becomes ±90° The optical axis of the virtual camera is perpendicular to the longitudinal axis of the vehicle. In the side normal image, right and left images can be generated based on an own vehicle. In this specification, since the region of interest is included only on the right side, the value of the yaw is set to 90°.

3 is a diagram for explaining a side normal image. 3 shows a front camera image and a side normal image generated by [Equation 1]. In each drawing, the red line represents the region of interest represented in FIG. 2.

2 and 3, if a vehicle parked and stopped parallel to the host vehicle is located at a similar distance in the horizontal direction from the center of the host vehicle, the tire size is similar regardless of the lengthwise distance. Also, in the same vehicle, the distance between the two tires is obtained relatively constant.

The appearance of the side of the vehicle may vary depending on the vehicle type, but all vehicles have similar tires. Using this point, a vehicle parked or stopped in the region of interest is detected using a part-based vehicle detection method. That is, a tire, which is a characteristic characteristic of a vehicle, is first detected, and a vehicle in the region of interest is detected by using a geometric relationship of the detected tire.

4 to 7 are diagrams for describing a part-based vehicle detection method. 4 and 5, a side normal image is generated from the front camera image (S110). As shown in (a) of FIG. 5, a tire candidate is generated from the generated side normal image using a Viola-Jones object detection algorithm (S120), and only necessary tire candidates are left by filtering the tire candidate (S130). Although it has been described that the Viola-Jones object detection algorithm is used when generating the tire candidate, it is not limited thereto, and any object detection algorithm capable of detecting a tire may be applied.

Since only vehicles existing in the region of interest need to be detected, it is checked whether the ground point of the tire exists in the region of interest set in advance, and only tire candidates in the region of interest are left. In addition, the size of the tire is large because the tire at the bottom of the image is located in a lateral direction from the center of the vehicle, and the tire at the top of the image is located at a distance, so the size of the tire appears small. Different filtering is performed on the tire size through statistical analysis of the tire size according to the position in the image from the training data.

As shown in FIG. 5B, a tire candidate is verified using a histogram of oriented gradients-support vector machine (HOG-SVM) (S140). As shown in (c) of FIG. 5, a vehicle candidate is generated by using the geometrical relationship between the verified tires (S150).

6 shows tire information used to generate a vehicle candidate. Information of each of c1, c2, g1, g2, h1 and h2 is obtained from the tire verified in step S140.

c1 and c2 are the center points of each of the two tires, g1 and g2 are the ground points of each of the two tires, and h1 and h2 are the sizes of each of the two tires. Using the above information, three geometric relations that can be determined as vehicle candidates are obtained as follows.

1) the angle between c1 and c2

2) physical distance between g1 and g2

3) ratio between h1 and h2

A range of values for generating vehicle candidates for each of the relational expressions may be calculated by statistical analysis of ground-truth data in the training data. Among the combinations of the verified tires, only those that meet the above relationship conditions are generated as vehicle candidates. A vehicle area is set for the generated vehicle candidate.

7 shows a vehicle area made based on two tires. w is the distance between the front and rear wheels, Hmax is the size of a large tire, and Hmin is the size of a small tire. At this time, the lateral length of the vehicle region is 1.2 times w, and the vertical length of the vehicle region is the size of a large tire.

The reason why the vertical length is the same as the size of the tire is to maintain the optimal shape for verifying the car because the car's appearance varies depending on the car type as the top of the tire increases.

Referring back to FIG. 4 and FIG. 5D, the generated vehicle candidates are again verified by HOG-SVM (S160). In the present specification, after first detecting a vehicle in the region of interest, a partial area in front of the detected vehicle or an area between the detected vehicles is set as a danger area.

From the perspective of pedestrian protection, it is more practical to correctly detect as many dangerous areas as possible even if a dangerous area where pedestrians may appear is erroneously detected. This is because even if there are more or less erroneous detections in the danger area, the final erroneous detection may not increase because most of them can be removed by the pedestrian detection/verifier.

8 and 9 are diagrams for explaining a part-based vehicle detection method according to an embodiment of the present invention. The embodiment of FIG. 8 shows a simplified structure of detecting a vehicle candidate based on the detected tire candidate, omitting the step of verifying a candidate for a vehicle and a tire having a relatively large amount of computation and a lot of potential to remove the correctly detected candidate. Accordingly, it is possible to reduce the computational complexity and improve the speed, and by detecting a vehicle candidate that may not be detected in the verification step as it is, there is an effect of detecting all of the dangerous areas. In addition, the possibility of positive detection is maximized by generating a vehicle candidate with only one tire.

8 and 9, a side normal image is generated from the front camera image (S210). As shown in (a) of FIG. 9, a tire candidate is generated from the generated side normal image using a Viola-Jones object detection algorithm (S220), and only necessary tire candidates are left by filtering the tire candidate (S230).

In the step of generating a vehicle candidate from the detected tire candidates (S250), a tire-based vehicle candidate generation method is added to the two tire-based vehicle candidate generation method.

As shown and described in FIG. 6, since the method for generating two tire-based vehicle candidates is a method of selecting a vehicle candidate from the geometrical relationship between two tires constituting a vehicle, when only one of the two tires constituting the vehicle is detected, the vehicle Can never be detected. Therefore, in order to compensate for this point, a tire-based vehicle candidate generation method is added that can generate a vehicle candidate even if only one tire is detected.

One tire generation method assumes that one detected tire is a front wheel and a rear wheel, and generates a vehicle candidate that can be generated in the case of the front wheel and a vehicle candidate that can be generated in the case of the rear wheel.

10 and 11 are diagrams for explaining a method of generating a vehicle candidate based on Hantire according to an embodiment of the present invention. 10 shows a distribution of a vehicle's horizontal length and a vertical length with respect to a tire ground point from gound-truth data as a single log function.

As shown in FIG. 10, the vehicle candidate region is set by a method of determining the width and length of the vehicle candidate according to the position of the ground point of the tire among the detected tire information. 10A and 10B show the distribution of the lateral length and the vertical length of the vehicle candidate according to the tire ground point position assuming that the detected tire is a front wheel, and FIGS. 10C and 10D Assuming that the detected tire is a rear wheel, the distribution of the horizontal length and the vertical length of the vehicle candidate according to the position of the tire ground point is shown.

FIG. 11 generates two vehicle candidates for each detected tire by using the four functions obtained in FIG. 10. In FIG. 11, a solid line indicates a result of generating a vehicle candidate by assuming one detected tire as a rear wheel, and a dotted line indicates a result of generating a vehicle candidate by assuming one detected tire as a front wheel. Since two vehicle candidates are generated from one tire, the one-tire-based method degrades the accuracy of the system, but can increase the detection rate of the vehicle.

Referring back to FIGS. 8 and 9, two tire-based vehicle candidates are generated as shown in (b) of FIG. 9, and one tire-based vehicle candidate is generated as shown in (c) of FIG. 9 (S250). Both the vehicle candidate generated based on two tires and the vehicle candidate generated based on one tire are used as vehicle candidates.

For the vehicle candidate generated in step S250, the detection result is displayed on the front side of the detected vehicle as shown in FIG. 9D.

12 shows a system for detecting a dangerous area based on a monocular camera according to an embodiment of the present invention. Referring to FIG. 12, a monocular camera-based dangerous area detection system 10 includes a front camera 100 that acquires an image in front of the vehicle, an ROI setting unit 200 that sets an ROI of the vehicle from the image, and a set interest. A side normal image generator 300 that generates a side normal image for an area, a vehicle detection unit 400 that generates a tire candidate within the region of interest from the generated side normal image, and generates a vehicle candidate based on one or more tires. ), and a dangerous area detection unit 500 for detecting a dangerous area based on the generated vehicle candidate.

Since the monocular camera-based dangerous area detection system 10 detects a dangerous area with a single image acquired from a front camera of a vehicle, the system structure is simple and the amount of computation can be minimized.

The vehicle detection unit 400 generates a vehicle candidate selected from a geometrical relationship between two tires constituting one vehicle, a vehicle candidate when one tire is a front wheel, and a vehicle candidate when the one tire is a rear wheel.

The monocular camera-based dangerous area detection system 10 has an effect of increasing the detection rate of a dangerous area because a vehicle candidate is generated with one tire.

Although the present invention has been described with reference to the exemplary embodiment shown in the drawings, this is only exemplary, and those of ordinary skill in the art will appreciate that various modifications and other equivalent embodiments are possible therefrom. Therefore, the true technical protection scope of the present invention should be determined by the technical idea of the attached registration claims.

10; Monocular camera-based hazardous area detection system
100; Front camera
200; Area of interest setting unit
300; Side normal image generator
400; Vehicle detection unit
500; Hazardous area detection unit

Claims (6)

Receiving a single image from a front camera of the vehicle;
Setting an ROI of the vehicle in the image;
Generating a side normal image by calculating the following [Equation] matrix on the set region of interest;
Generating a vehicle candidate based on the tire from the generated side normal image; And
Including; detecting a danger area based on the generated vehicle candidate,
Generating the vehicle candidate,
Generating a tire candidate within the ROI from the side normal image; And
Classifying the generated tire candidate into a front wheel and a rear wheel from the geometrical relationship between two tires constituting a vehicle, or assuming a front wheel and a rear wheel using the ground point position of the one tire from the generated tire candidate ;
Generating a vehicle candidate using a center point, a ground point, and a size of each of the front and rear wheels, or generating a vehicle candidate when the one tire is a front wheel and a vehicle candidate when the one tire is a rear wheel Monocular camera-based hazardous area detection method comprising a.
[Equation]

Here, Pr is a point in the image acquired from the front camera, Pc is a point in the side normal image, and Kc and Rc are the rotation matrix of the intrinsic parameter and the external parameter of the front camera. matrix). Kr and Rr represent the rotation matrix of the internal and external parameters of the virtual camera.
delete The method of claim 1,
The vehicle candidate is generated using the angle between the center point of the front wheel and the center point of the rear wheel, the physical distance between the ground point of the front wheel and the ground point of the rear wheel, and the ratio between the size of the front wheel and the size of the rear wheel. Monocular camera-based dangerous area detection method, characterized in that.
The method of claim 3,
The lateral length of the vehicle candidate is set based on the distance between the front and rear wheels, and the lateral length of the vehicle candidate is set to a size of a larger tire among the sizes of the front and rear wheels. Hazardous area detection method.
delete A monocular camera that acquires an image in front of the vehicle;
An ROI setting unit for setting an ROI of the vehicle in the image;
A side normal image generator for generating a side normal image by calculating the following [Equation] matrix on the set region of interest;
A vehicle detection unit that generates a tire candidate in the region of interest from the generated side normal image and generates a vehicle candidate by using a geometric relationship of one or more tires; And
Including; a danger zone detection unit for detecting a danger zone based on the generated vehicle candidate,
The vehicle detection unit,
It is classified as a front wheel and a rear wheel from the geometrical relationship between two tires constituting a vehicle, or assumes a front wheel and a rear wheel using the ground point position of the one tire from the generated one tire candidate,
Generating a vehicle candidate using a center point, a ground point, and a size of each of the front and rear wheels, or generating a vehicle candidate when the one tire is a front wheel, and a vehicle candidate when the one tire is a rear wheel Monocular camera-based hazardous area detection system, characterized in that.
[Equation]

Here, Pr is a point in the image acquired from the front camera, Pc is a point in the side normal image, and Kc and Rc are the rotation matrix of the intrinsic parameter and the external parameter of the front camera. matrix). Kr and Rr represent the rotation matrix of the internal and external parameters of the virtual camera.

Images