KR20130020151A - Vehicle detection device and method - Google Patents

Abstract

PURPOSE: A vehicle detection device and a method are provided to perform image processing within a concerned region based on an object sensed in a radar sensor, thereby rapidly detecting a vehicle. CONSTITUTION: An image sensor(110) photographs a surrounding user vehicle. A radar sensor(120) senses an object of the surrounding user vehicle. A concerned region generation unit(130) sets up a concerned region in image data based on location information of the detected object. A vehicle extraction unit(140) extracts the vehicle from the image data. [Reference numerals] (110) Image sensor; (120) Radar sensor; (121) Transmitting antenna unit; (122) Receiving antenna unit; (123) Location extraction unit; (124) Speed extraction unit; (130) Concerned region generation unit; (140) Vehicle extraction unit; (150) Vehicle tracking unit

Description

Vehicle Detection Device and Method

The present invention relates to a vehicle detection apparatus and method, and more particularly, to a vehicle detection apparatus and method for detecting a vehicle by processing an image based on radar recognition results.

When a vehicle located in front of the own vehicle is close to the own vehicle and there is a danger of collision, a system for preventing a collision by providing a warning to a driver or automatically controlling the speed and steering of the own vehicle is provided. An image sensor or a radar sensor may be used to detect a possibility of collision with the vehicle.

In the case of using an image sensor, it is necessary to perform image processing on all regions in the image in order to detect a vehicle from the image data captured by the image sensor. In this case, the time required to perform image processing for vehicle detection becomes large, and it may be difficult to process in real time.

When the radar sensor is used, a moving object is identified as a vehicle, and a stationary object such as a guard rail, a lane change bar, a sign, and the like is used as a vehicle. In this case, the stopped vehicle is not identified as a vehicle and may be excluded from subsequent vehicle tracking.

SUMMARY OF THE INVENTION The present invention has been made to overcome the above-described problems, and an object thereof is to provide a vehicle detecting apparatus and method capable of accurately detecting a vehicle within a short time.

In order to achieve the above object, an embodiment of the present invention, the image sensor for photographing the surrounding vehicle; A radar sensor detecting an object around the host vehicle; A region of interest generator configured to set a region of interest (ROI) on image data captured by the image sensor based on position information of the object detected by the radar sensor; And a vehicle extracting unit extracting a vehicle from the image data in the ROI.

In addition, another embodiment of the present invention, the step of photographing the surrounding vehicle using an image sensor; Detecting an object around the host vehicle using a radar sensor; Setting a region of interest (ROI) on image data captured by the image sensor based on position information of the object detected by the radar sensor; And extracting a vehicle from image data in the ROI.

According to the present invention described above, the vehicle can be detected in a short time by performing image processing only within the ROI set based on the object detected by the radar sensor. In addition, the vehicle can be detected accurately by using both the results of the radar sensor and the results of the image sensor.

1 is a block diagram of a vehicle detection system according to an embodiment of the present invention.
2 is a flowchart of a vehicle detection method according to an embodiment of the present invention.
3 is a diagram illustrating position information of an object detected by the radar sensor.
4 is a diagram illustrating an ROI set in image data.
5A and 5B illustrate a region of interest and a bottom line of a vehicle detected in the region of interest.
6A and 6B illustrate a region of interest and a side line of a vehicle detected in the region of interest.
7 is a diagram illustrating a center line of a vehicle detected in the ROI and the ROI.
8A and 8B are diagrams illustrating an ROI and a vehicle detection result set in image data.

Hereinafter, some embodiments of the present invention will be described in detail with reference to exemplary drawings. In adding reference numerals to the components of each drawing, it should be noted that the same reference numerals are assigned to the same components as much as possible even though they are shown in different drawings. In the following description of the present invention, a detailed description of known functions and configurations incorporated herein will be omitted when it may make the subject matter of the present invention rather unclear.

In addition, in describing the component of this invention, terms, such as 1st, 2nd, A, B, (a), (b), can be used. These terms are intended to distinguish the constituent elements from other constituent elements, and the terms do not limit the nature, order or order of the constituent elements. When a component is described as being "connected", "coupled", or "connected" to another component, the component may be directly connected to or connected to the other component, It should be understood that an element may be "connected," "coupled," or "connected."

1 is a block diagram of a vehicle detection system 100 according to an embodiment of the present invention. The vehicle detection system 100 shown in FIG. 1 may be a system located in a host vehicle. The vehicle detection system 100 is a system for detecting a vehicle around a host vehicle. The vehicle detection system 100 is a system for preventing a collision of a vehicle or a system for notifying a driver of a collision risk of a vehicle, for example, collision damage mitigation (CDM), SCC ( It may be part of a Smart Cruise Control (FCW), Forward Collision Warning (FCW) system.

Referring to FIG. 1, the vehicle detection system 100 includes an image sensor 110, a radar sensor 120, a region of interest generator 130, a vehicle extractor 140, and a vehicle tracker 150.

The image sensor 110 is a camera device capable of capturing an image around the host vehicle. The image sensor 110 outputs image data by capturing the surrounding vehicle.

The radar sensor 120 is a device that detects an object around the host vehicle using the radar. The area detected by the radar sensor 120 may be similar to the area photographed by the image sensor 110. The radar sensor 120 includes a transmission antenna unit 121 for transmitting radio waves, a reception antenna unit 122 for receiving radio waves reflected by an object, and a beam direction or reception antenna of radio waves transmitted from the transmission antenna unit 121. The position extraction unit 123 which extracts the position of the object based on the beam direction of the radio wave received by the unit 122 and the time difference between the transmission radio wave and the reception radio wave, and the relative of the object based on the frequency difference between the transmission radio wave and the reception radio wave. It may include a speed extractor 124 for extracting the speed.

The transmitting antenna unit 121 and the receiving antenna unit 122 each include one or more antennas. At least one of the transmitting antenna unit 121 and the receiving antenna unit 122 is a directional antenna device, and has beam resolution in horizontal and vertical directions. For example, the transmitting antenna unit 121 is a directional antenna device and the receiving antenna unit 122 is an omni-directional antenna device, the object by the beam direction of the radio wave transmitted from the transmitting antenna unit 121 The horizontal and vertical directions in which the is located can be seen. For another example, the transmitting antenna unit 121 is a non-directional antenna device and the receiving antenna unit 122 is a directional antenna device, the horizontal and vertical in which the object is located by the beam direction of the radio wave received by the receiving antenna 122 The direction is known.

The transmitting antenna unit 121 transmits a pulse, and the receiving antenna unit 122 receives a pulse reflected by an object. When the distance from the host vehicle to the object is 'R', the difference T between the time when the pulse is transmitted by the transmitting antenna unit 121 and the time when the pulse is received by the receiving antenna unit 122 is T = 2R / c ( c is the speed of light). Therefore, the distance to the object can be known from the difference between the time at which the pulse is transmitted at the transmitting antenna unit 121 and the time at which the pulse is received at the receiving antenna unit 122.

The position extractor 123 extracts the position of the object based on the horizontal and vertical directions in which the object is located and the distance to the object. The position of the object may be expressed in three-dimensional coordinates based on the position of the host vehicle. Alternatively, the position of the object may be expressed by the horizontal and vertical directions of the object and the distance to the object.

The frequency of the radio wave transmitted from the transmitting antenna unit 121 and the frequency of the radio wave received from the receiving antenna unit 122 have a difference according to the relative speed of the object due to the Doppler effect. The speed extractor 124 extracts a relative velocity of the object with respect to the host vehicle based on the frequency difference between the frequency of the radio wave transmitted by the transmitting antenna unit 121 and the radio wave received by the receiving antenna unit 122.

The ROI generator 130 sets a region of interest (ROI) on the image data photographed by the image sensor 110 based on the position information of the object extracted by the radar sensor 120.

The radar sensor 120 may know the three-dimensional coordinates of the object based on the host vehicle through the direction in which the object is located and the distance of the object. The pixel coordinates of the object may be obtained from the image data generated by the image sensor 110 using the coordinates of the object. The ROI may be set based on the pixel coordinates of the object.

The region of interest may comprise at least the rear shape of the vehicle. The absolute width of the area occupied by the rear shape of the vehicle may be a preset value. The area of the rear shape of the vehicle in the image data is inversely proportional to the square of the distance from the host vehicle to the vehicle. Therefore, the region of interest may be set to an area inversely proportional to the distance from the host vehicle to the object based on the pixel coordinates of the object described above.

Although setting the ROI from the three-dimensional coordinates of the object has been described above, the pixel coordinates of the object are extracted in the image data from the direction of the object measured by the radar sensor 120, and the image is measured from the distance to the measured object. It is also possible to extract the size of the region of interest in the data.

The vehicle extractor 140 extracts a vehicle in the ROI of the image data. In the present embodiment, the vehicle extraction unit 140 does not extract the vehicle by performing image processing on the entire image data, but performs image processing within the ROI generated by the ROI generator 130 of the image data. To extract the vehicle. That is, the vehicle is extracted by image processing based on the object extracted by the radar sensor 120. Details of the method of extracting the vehicle from the vehicle extraction unit 140 will be described later.

The vehicle tracking unit 150 tracks the vehicle recognized by the vehicle extraction unit 140. As an example, the vehicle tracking unit 150 may track a vehicle by applying mean shift tracking (MST) and Kalman filtering (KF) to the recognized vehicle.

2 is a flowchart of a vehicle detection method according to an embodiment of the present invention executed in the vehicle detection system 100 of FIG.

Referring to FIG. 2, the image sensor 110 generates image data by capturing the surrounding vehicle (S210). The radar sensor 120 detects an object around the host vehicle using the radar (S220). Step S210 in which the image sensor 110 generates image data and step S220 in which the radar sensor 120 detects an object may be performed in the order of S210, S220, or may be executed in the order of S220, S210, or simultaneously. Can be. The radar sensor 120 detects the coordinates of the object around the host vehicle, or the direction and distance of the object. 3 illustrates plane coordinates based on the host vehicle with respect to an object detected by the radar sensor 120.

The ROI generator 130 sets an ROI on the image data generated by the image sensor 110 based on the recognition result of the object as shown in FIG. 3 (S230). FIG. 4 illustrates a region of interest in image data set from coordinates of the object detected in FIG. 3. The ROI generator 130 extracts the location of the ROI in the image data by converting the coordinates of the object into pixel data in the image data. Also, the ROI generator 130 extracts an area of the ROI based on the distance from the host vehicle to the object. The width of the region of interest is set to include, for example, the shape of the back of the vehicle. The width of the region of interest may be inversely proportional to the square of the distance from the host vehicle to the object. The ROI may be set to have an area extracted from the distance to the object based on the location of the ROI extracted from the coordinates of the object.

The vehicle extractor 140 extracts a vehicle in the ROI. The vehicle extracting unit 140 extracts the vehicle only within the set ROI, not the entire image data, thereby reducing the time required for detecting the vehicle.

The vehicle extractor 140 may process a plurality of image processing processes in parallel to extract a vehicle.

The vehicle extractor 140 may detect a bottom line of the vehicle using a boundary detection algorithm to extract the vehicle (S240).

FIG. 5A shows an example of the region of interest in the image data, and FIG. 5B shows an example of the lower line of the vehicle detected in the region of interest. For example, a boundary detection algorithm may be applied in the vertical direction in the ROI 501 illustrated in FIG. 5A to detect the lower line 502 of the vehicle illustrated in FIG. 5B. For example, among pixels arranged in the vertical direction in the region of interest 501, a pixel in which the pixel value rapidly changes may be detected as a boundary. When the pixels detected as the boundary are horizontally connected, the pixels may be detected as the lower line 502 of the vehicle. A numerical value score1 that can predict the accuracy of the lower line detection while detecting the lower line 502 of the vehicle can be calculated. For example, the score score1 may be edge strength.

In addition, the vehicle extraction unit 140 may detect a side line of the vehicle using a boundary detection algorithm to extract the vehicle (S250).

6A shows an example of a region of interest in the image data, and FIG. 6B shows an example of a side line of the vehicle detected in the region of interest. For example, a boundary detection algorithm may be applied in the horizontal direction in the ROI 601 illustrated in FIG. 6A to detect side lines 602 and 603 of the vehicle illustrated in FIG. 6B. For example, among pixels arranged in the horizontal direction in the region of interest 601, a pixel in which the pixel value rapidly changes may be detected as a boundary. When the pixels detected as the boundary are vertically connected, these pixels may be detected by the side lines 602 and 603 of the vehicle. While detecting the side lines 602 and 603 of the vehicle, a numerical value score2 that can predict the accuracy of the side line detection can be calculated. For example, the value score2 may be borderline intensity.

In addition, the vehicle extractor 150 may detect a center line of the vehicle by using a symmetric detection algorithm to extract the vehicle (S260).

7 shows a region of interest and a center line detected within the region of interest. For example, the center line 702 of the vehicle may be detected by applying a symmetry detection algorithm in the horizontal direction in the ROI 701 illustrated in FIG. 7. For example, from the distribution of pixel values of the pixels arranged in the horizontal direction in the region of interest 701, the center pixel to which they are symmetric can be detected. The line in which the center pixels are vertically connected may be detected as the center line 702 of the vehicle. A value score3 that can predict the accuracy of the center line detection while detecting the center line 702 of the vehicle can be calculated. For example, the numerical score3 may be a value representing the symmetry of pixel values within the region of interest.

In addition, the vehicle extractor 150 may detect a vehicle similarity through a support vector machine (SVM) classifier using histogram of gradient (HoG) information to extract a vehicle (S270).

HoG can, for example, divide a target image into a plurality of regions, calculate the gradient intensity and the gradient direction of luminance for each region, and can be created based on the gradient intensity and the gradient direction thereof. The SVM classifier learns the distribution of the vector and makes a decision (determines whether it is a vehicle) based on the probability distribution obtained in the learning process. In the present embodiment, the SVM classifier using HoG information is described, but vehicle similarity may be detected by using a value other than HoG or by using an algorithm other than the SVM classifier. A statistical value score4 representing vehicle similarity may be calculated while detecting vehicle similarity.

In the above example, the lower line detection S240, the side line detection S250, the center line detection S260, and the vehicle similarity detection S270 are executed in parallel, and the results of each detection affect the results of other detections. You may not receive it. The execution order of the lower line detection S240, the side line detection S250, the center line detection S260, and the vehicle similarity detection S270 may be possible in any case.

In the above-described example, the lower line detection (S240), the side line detection (S250), the center line detection (S260), and the vehicle similarity detection (S270) have been described, but various algorithms that can determine whether the vehicle is from the image data This may be further included.

In the above example, it has been described that all of the lower line detection (S240), the side line detection (S250), the center line detection (S260), and the vehicle similarity detection (S270) are executed to detect the vehicle, but one or more of these are described. Vehicle can be detected.

Next, as in Equation 1, each of the aforementioned values score1 to score4 is multiplied by the weights 1 to 4, and the sum is calculated to calculate a final score (S280).

[Equation 1]

final score = score1 * weight1 + score2 * weight2 + score3 * weight3 + score4 * weight4

Each weight (weight1 to weight4) detects the vehicle accurately and without error, how much each vehicle detection method (lower line detection (S240), side line detection (S250), center line detection (S260), and vehicle similarity detection (S270)). It depends on what you can do. For example, if the vehicle similarity detection method is used, the vehicle can be detected more accurately than other methods, and if there are few errors, the weight4 multiplied by the score4 calculated by the vehicle similarity detection method is different from the other detection methods. It may have a larger value than the weights (weight1 to weight3) multiplied by the calculated values score1 to score3.

The final score obtained is compared with a predetermined threshold to finally determine whether an object in each ROI is a vehicle (S290).

The vehicle tracking unit 150 tracks the vehicle by applying Mean Shift Tracking (MST) and Kalman Filtering (KF) to the detected vehicle (S300). And the result is output. The output result includes information of vehicles located in the vicinity of the host vehicle, and based on the system, the system for preventing a collision of the vehicle mounted on the host vehicle may control the vehicle or alert the driver when a collision of the vehicle is expected. .

FIG. 8A illustrates an example in which a region of interest is set in image data output from the image sensor 110 based on information from the radar sensor 120. 8B illustrates an example of a result of detecting a vehicle through the above-described image processing in the ROI. 8A and 8B, it can be seen that only the vehicle is detected without the object other than the vehicle through the above-described image processing in the plurality of ROIs.

In the above-described example, it is determined whether the vehicle is based on the results of the plurality of image processing processes. Therefore, even if an error occurs in one image processing process and a vehicle is not detected, it can be finally detected as a vehicle. Alternatively, when an error occurs in one image processing process and an object other than the vehicle is detected as the vehicle, it may be finally determined that the vehicle is not the vehicle.

On the other hand, since the apparatus using only the radar includes not only the stationary vehicle but also a guardrail, a lane change bar, a sign, and the like, the stationary object does not all have a problem of being recognized as a vehicle. However, in the above-described example, the vehicle may be classified by image processing instead of whether the object is stationary, and thus the vehicle may be detected.

It is also to be understood that the terms such as " comprises, "" comprising," or "having ", as used herein, mean that a component can be implanted unless specifically stated to the contrary. But should be construed as including other elements. All terms, including technical and scientific terms, have the same meaning as commonly understood by one of ordinary skill in the art to which this invention belongs, unless otherwise defined. Commonly used terms, such as predefined terms, should be interpreted to be consistent with the contextual meanings of the related art, and are not to be construed as ideal or overly formal, unless expressly defined to the contrary.

The above description is merely illustrative of the technical idea of the present invention, and those skilled in the art to which the present invention pertains may make various modifications and changes without departing from the essential characteristics of the present invention. Therefore, the embodiments disclosed in the present invention are intended to illustrate rather than limit the scope of the present invention, and the scope of the technical idea of the present invention is not limited by these embodiments. The scope of protection of the present invention should be construed according to the following claims, and all technical ideas falling within the scope of the same shall be construed as falling within the scope of the present invention.

Claims (10)

An image sensor for photographing the surrounding vehicle;
A radar sensor detecting an object around the host vehicle;
A region of interest generator configured to set a region of interest (ROI) on image data captured by the image sensor based on position information of the object detected by the radar sensor; And
And a vehicle extracting unit extracting a vehicle from the image data in the ROI. The method of claim 1,
And the ROI generator generates a ROI by converting position information of the object detected by the radar sensor into pixel information of the image data. The method of claim 1,
The vehicle extractor may include a lower line of the vehicle detected through boundary detection in the image data in the ROI, a side line of the vehicle detected through boundary detection in the image data in the ROI, and symmetry detection in the image data in the ROI. Based on at least one of a center line of the vehicle detected through a support vector and a vehicle similarity detected through a support vector machine (SVM) using gradient histogram (HG) information in the image data in the ROI. And detecting the vehicle. The method of claim 3, wherein
The vehicle extracting unit may compare a sum and a threshold value of a value obtained by multiplying a weight by at least one of a detection result of a lower line of the vehicle, a detection result of the side line, a detection result of the center line, and the vehicle similarity. Vehicle detection apparatus, characterized in that for extracting the vehicle. The method of claim 1,
And a vehicle tracking unit for tracking a vehicle extracted from the vehicle extraction unit by applying mean shift tracking (MST) and Kalman filtering (KF). Photographing the surrounding vehicle using an image sensor;
Detecting an object around the host vehicle using a radar sensor;
Setting a region of interest (ROI) on image data captured by the image sensor based on position information of the object detected by the radar sensor; And
And extracting a vehicle from image data in the region of interest. The method according to claim 6,
The setting of the ROI may include converting the position information of the object detected by the radar sensor into pixel information of the image data and setting the ROI. The method according to claim 6,
The extracting of the vehicle may include a lower line of the vehicle detected through boundary detection in the image data in the ROI, a side line of the vehicle detected through boundary detection in the image data in the ROI, and symmetry in the image data in the ROI. At least one of a center line of the vehicle detected through the detection and a vehicle similarity detected through a support vector machine (SVM) using gradient histogram (HG) information in the image data in the ROI. And a vehicle is extracted on the basis of the vehicle. The method of claim 8,
The extracting of the vehicle may include comparing a sum and a threshold value of a value obtained by multiplying a weight by at least one of a detection result of the lower line of the vehicle, a detection result of the side line, a detection result of the center line, and the vehicle similarity. Vehicle detection method characterized in that for extracting the vehicle. The method according to claim 6,
And applying a mean shift tracking (MST) and Kalman filtering (KF) to track the vehicle extracted by the vehicle extracting unit.

Images