JP4697063B2 - Approaching vehicle detection device - Google Patents

Description

  The present invention relates to an apparatus for detecting a vehicle approaching a host vehicle based on a captured image.

  2. Description of the Related Art Conventionally, a technique for obtaining an optical flow by template matching based on a plurality of images captured around a vehicle and detecting other vehicles on the captured image is known (see Patent Document 1).

JP 2004-173048 A

  However, the method for obtaining the optical flow has a problem that the amount of calculation required to detect the approaching vehicle increases.

An approaching vehicle detection device according to the present invention includes an imaging unit that captures an image of the surroundings of a vehicle, and a frequency separation that separates a time-varying component of luminance of each pixel on a video imaged by the imaging unit (hereinafter referred to as a captured image) for each frequency. An approaching vehicle detection means for detecting an approaching vehicle approaching the host vehicle from the captured image based on data obtained by separating the time-varying component of the captured image for each frequency by the frequency separation means; Image data synthesizing means for reconstructing an image by collecting data of pixel regions whose luminance change over time is a predetermined frequency or more from the data separated by the frequency separation means, the approaching vehicle detection means, based on the images that are reconstructed by the image synthesizing means, by performing a template matching process, detecting an approaching vehicle on the captured image And performing.

  According to the approaching vehicle detection device of the present invention, the time fluctuation component of the video around the vehicle is separated for each frequency, and the vehicle approaching the host vehicle is detected from the captured video based on the separated data. An approaching vehicle can be detected effectively.

  FIG. 1 is a diagram illustrating a configuration of an approaching vehicle detection device according to an embodiment. The approaching vehicle detection device in one embodiment includes an in-vehicle camera 10 and a processing device 20. FIG. 2 is a diagram illustrating an example of an attachment position of the in-vehicle camera 10. As shown in FIG. 2, the in-vehicle camera 10 is attached to the center of the rear part of the vehicle 100 and images the rear of the host vehicle.

  The processing device 20 includes a processor, and includes a frequency resolving unit 21, a storage unit 22, a frequency synthesizing unit 23, a noise removing unit 24, and an approaching vehicle detection unit 25 in terms of processing functions performed internally. . The frequency resolving unit 21 performs processing for decomposing time-varying components of the video for each frequency domain based on the video captured by the in-vehicle camera 10. The storage unit 22 stores data decomposed for each frequency domain by the frequency decomposition unit 21.

  The frequency synthesizing unit 23 obtains frequency domain data necessary for each imaging pixel from the data stored in the storage unit 22 according to the mounting position of the in-vehicle camera 10, the imaging direction, and the distortion (distortion) of the lens. Collecting and synthesizing (reconstructing) video. The noise removing unit 24 performs processing for removing noise from the video synthesized by the frequency synthesizing unit 23. The approaching vehicle detection unit 25 detects a vehicle approaching the host vehicle based on the image from which noise has been removed by the noise removal unit 24. The detailed processing content performed in each part 21-25 is demonstrated below.

  FIG. 3 is a diagram illustrating an example of an image captured by the in-vehicle camera 10. The captured image shown in FIG. 3 shows a vehicle 200 traveling in the adjacent lane and a vehicle 300 traveling directly behind the host vehicle. In addition to the vehicles 200 and 300, a center line 31, road end lines 32 and 33, a steel tower 35, a side wall 36, and the like are shown. Here, it is assumed that the vehicle 200 traveling in the adjacent lane is overtaking the host vehicle, and the vehicle 300 traveling directly behind the host vehicle is traveling at substantially the same speed as the host vehicle. .

  The frequency resolving unit 21 performs processing for decomposing time-varying components of the video for each frequency domain based on the video captured by the in-vehicle camera 10. Here, the process of decomposing time-varying components of video for each frequency domain is called time-frequency division.

  FIG. 4 is a diagram illustrating a temporal change in luminance of one pixel (x, y) on the captured image. The frequency resolving unit 21 divides this temporal change in luminance into frequency components. For example, based on a plurality of video frames taken at different times, an average value of luminance values is calculated for each pixel, and a difference between the calculated luminance average value and the luminance value of the current video is obtained. The processing for classifying by frequency domain is performed. When shooting at 30 frames per second, it can be classified into bandpass components of 15 Hz or less. Data decomposed for each frequency region by the frequency resolving unit 21 is stored in the storage unit 22.

  When the time-frequency division process is performed, pixels with a small change in luminance are classified as low-frequency components, and pixels with a large change in luminance are classified as high-frequency components. Therefore, the pixels in which the sky and the ground (road) are reflected are extracted as low-frequency components because the luminance does not change with time. On the other hand, pixels that move on the image as the host vehicle travels, such as pixels in which an approaching vehicle is reflected or a steel tower, are extracted as high-frequency components because the luminance changes with time. In the video shown in FIG. 3, the vehicle 200 trying to overtake the host vehicle has a high frequency component, and the vehicle 300 traveling at substantially the same speed as the host vehicle has a low frequency component. In addition, the steel tower 35 moves on the video as the host vehicle travels, and thus becomes a high-frequency component.

  In order to detect a vehicle approaching the host vehicle, the frequency synthesizer 23 collects high-frequency component data of a predetermined frequency or higher from the data stored in the storage unit 22 and synthesizes (re-creates) the video. Process). By collecting high-frequency component data of a predetermined frequency or higher, low-frequency component data such as the vehicle 300 traveling at substantially the same speed as the host vehicle can be excluded as described above.

  Here, even when a vehicle approaching at a constant speed is shown on the captured image, the speed appears differently depending on the location where it is shown. FIG. 5 is a diagram showing the optical flow of the vehicle 200 traveling at a constant speed by arrows Y1 and Y2. As shown in FIG. 5, the moving amount on the image is larger at the position outside the screen corresponding to the lens periphery of the in-vehicle camera 10 than in the center of the screen corresponding to the lens center of the in-vehicle camera 10.

  Therefore, the frequency synthesizing unit 23 collects frequency domain data corresponding to the imaging pixels according to the mounting position of the in-vehicle camera 10, the imaging direction, and the distortion (distortion) of the lens, and synthesizes (reconstructs) the video. I do. This makes it possible to appropriately perform video composition (reconstruction).

  The noise removing unit 24 performs processing for removing noise from the video synthesized by the frequency synthesizing unit 23. Here, the binarization process is performed on the luminance value of each pixel of the video synthesized by the frequency synthesis unit 23, and then the noise removal process using the median filter is performed.

  FIG. 6 is a diagram showing an image synthesized by the frequency synthesis unit 23 and subjected to noise removal processing by the noise removal unit 24. As shown in FIG. 6, the low frequency components such as the vehicle 300 that was traveling at the same speed as the host vehicle are deleted, and there are portions that change on the image, such as the approaching vehicle 200 and the steel tower 35. In particular, in the case of vehicles, information that is continuous in the horizontal direction (horizontal direction), such as the boundary between the bumper and the ground, the boundary between the bumper and the air intake, the boundary between the hood and the window, the boundary between the roof and the sky (line) ) Is likely to remain.

  The noise removing unit 24 also erases information from an area other than the area where the vehicle is considered to exist. For example, a background such as the sky is often reflected in the upper part of the video, and it is considered that there is no vehicle. Therefore, by deleting the information shown in the upper part of the video, unnecessary data for detecting the approaching vehicle can be deleted. FIG. 7 is a diagram in which the steel tower 35 shown in the upper part of the video is deleted from the video shown in FIG.

  The approaching vehicle detection unit 25 performs processing for detecting a vehicle approaching the host vehicle based on the image from which noise is removed (see FIG. 7). The approaching vehicle detection unit 25 first performs a process of enhancing the horizontal edge on the video using a spatial filter such as a horizontal line detection filter. FIG. 8 is a diagram after the processing shown in FIG. 7 for detecting the horizontal edge using the horizontal line detection filter. As shown in FIG. 8, the vertical line component disappears and the horizontal (horizontal) component is emphasized.

  Next, the approaching vehicle detection unit 25 detects an approaching vehicle by performing a template matching process. FIG. 9A and FIG. 9B are diagrams showing an example of a template representing the characteristics of the vehicle, which is held by the approaching vehicle detection unit 25. In passenger cars and trucks, the edge component at the lower end of the bumper tends to remain, so a horizontal line template as shown in FIGS. 9A and 9B is used. In the image after noise removal, the edge component of the vehicle may not be continuously connected in the horizontal direction, so use a discontinuous horizontal line template as shown in FIG. 9B, or Even if the edge component is discontinuous, the template shown in FIG. 9A is used as a continuous straight line.

  By performing the template matching process described above, a lateral edge representing the characteristics of the vehicle is detected. When a lateral edge representing the characteristics of the vehicle is detected, a vehicle detection area for detecting an approaching vehicle is set. Here, a rectangular area having the detected horizontal edge as one side and the detected horizontal edge as the bottom is set as the vehicle detection area. FIG. 10 is a diagram illustrating an example of the set rectangular area 50. The approaching vehicle detection unit 25 detects the vehicle based on the video component existing in the set vehicle detection area. For example, if the video component (lateral edge component) existing in the set vehicle detection area is a predetermined amount or more, the vehicle detection area is determined to be an area in which an approaching vehicle is reflected. The lateral edge detection process and the vehicle detection process within the set vehicle detection area are performed for all areas on the captured image.

  The vehicle detection process is performed over several frames. Even if an approaching vehicle is detected in a certain frame, but the vehicle cannot be detected in the vicinity of the area where the vehicle was detected in the previous frame in the next frame, the vehicle non-detection is not issued for the following several frames. If a vehicle is detected in the vicinity again in the next several frames, it is determined that the vehicle is continuously detected. Thereby, even when an approaching vehicle exists, even when an approaching vehicle is not detected during image processing, it is possible to continuously notify the user that an approaching vehicle exists.

  When the approaching vehicle detection unit 25 detects an approaching vehicle, information on the approaching vehicle can be displayed on a monitor (not shown), or information on the approaching vehicle can be notified to the driver from a speaker (not shown) by voice.

  FIG. 11 is a flowchart showing a flow of processing performed by each unit 21 to 25 in the processing device 20 described above. In step S10, the frequency resolving unit 21 acquires a plurality of frames of images captured by the in-vehicle camera 10, and proceeds to step S20. In step S20, the temporal variation component of the video is separated for each frequency domain, and the process proceeds to step S30.

  In step S30, the frequency synthesizer 23 collects high-frequency component data of a predetermined frequency or higher from the data obtained by separating the time-varying components of the video for each frequency domain, and synthesizes (reconstructs) the video. Do. In step S40 following step S30, the noise removing unit 24 performs a process of removing noise from the video synthesized by the frequency synthesizing unit 23.

  In step S50 following step S40, the approaching vehicle detection unit 25 performs a process of enhancing the horizontal edge on the image from which the noise has been removed, and proceeds to step S60. In step S60, a template matching process is performed using a template that represents the characteristics of the vehicle to detect a lateral edge that represents the characteristics of the vehicle. In step S70 following step S60, a vehicle detection region is set around the detected lateral edge, and the process proceeds to step S80. In step S80, an approaching vehicle is detected based on a video component existing in the set vehicle detection area.

  According to the approaching vehicle detection device in one embodiment, the temporal variation component of the image around the vehicle imaged by the in-vehicle camera 10 is separated for each frequency, and the approaching vehicle is approached from the captured image based on the separated data. Since an incoming vehicle is detected, an approaching vehicle can be detected effectively. For example, an approaching vehicle can be detected with a small amount of computation even when a large number of vehicles are shown on the captured image.

  In particular, we reconstructed and reconstructed the video by collecting the data of the pixel area whose temporal change in luminance is greater than or equal to the predetermined frequency from the data obtained by separating the temporal change in luminance of each pixel of the captured video for each frequency. Since template matching processing is performed based on the video, an approaching vehicle on the captured video can be detected with high accuracy.

  Further, according to the approaching vehicle detection device in one embodiment, by performing template matching processing, a predetermined vehicle detection region is set around the pixel region that matches the template representing the characteristics of the vehicle, and the set vehicle Since the approaching vehicle on the captured image is detected based on the data amount of the lateral edge in the detection area, the approaching vehicle can be detected with higher accuracy.

  Furthermore, according to the approaching vehicle detection device in one embodiment, the template matching process is performed after the process of removing noise from the reconstructed video, so that the detection accuracy of the approaching vehicle can be further improved.

  The present invention is not limited to the embodiment described above. For example, when the approaching vehicle detection unit 25 detects a lateral edge representing the characteristics of the vehicle by performing template matching processing, the approaching vehicle detection unit 25 sets a rectangular area (see FIG. 10) having the length of the detected lateral edge as one side. The process of detecting approaching vehicles was performed. However, a plurality of vehicle detection areas can be set above and below the detected horizontal edge. FIG. 12 is a diagram illustrating vehicle detection areas 60 to 63 that are respectively set two above and below the detected horizontal edge. Also in this case, it is determined that an area in which the video components (lateral edge components) existing in the set vehicle detection areas 60 to 63 are a predetermined amount or more is an area in which an approaching vehicle is reflected. Moreover, the place where the vehicle detection area is set is not limited to the upper and lower sides of the detected horizontal edge, and may be around the detected horizontal edge.

  The vehicle detection area is not limited to the rectangular area shown in FIGS. 10 and 12, and may be a square as shown in FIG. 13A, for example, or as shown in FIG. 13B. It may be trapezoidal. That is, if a vehicle detection area having a shape corresponding to the type of vehicle, such as a light vehicle or a sports car, is prepared in advance and a vehicle detection process is performed by applying a plurality of types of vehicle detection areas, various types of vehicles can be accurately detected. It can be detected well.

  An example of a template representing the characteristics of the vehicle is shown in FIGS. 9A and 9B. However, when detecting a two-wheeled vehicle, it is preferable to prepare a template for detecting the two-wheeled vehicle in advance. FIG. 14 is a diagram showing an example of a template for detecting a motorcycle. For motorcycles, the shape of the helmet or headlight is likely to remain on the image, so a template including a lateral edge and a round shape may be prepared.

  You may make it change the template of a vehicle according to environments, such as day and night, and rainfall. For example, when performing the process of detecting an approaching vehicle at night, the template of the vehicle can be shaped to include a headlamp. In this case, the detection accuracy of the approaching vehicle at night can be further improved.

  In the embodiment described above, by collecting high-frequency component data of a predetermined frequency or higher among the data obtained by separating the temporal change in luminance of each pixel for each frequency and synthesizing (reconstructing) the video, the vehicle An image including a vehicle approaching to is constructed. In the data in which the change in luminance over time is separated for each frequency, the lower the relative speed with respect to the host vehicle, the lower the frequency component, and the higher the relative speed, the higher the frequency component. Therefore, it is possible to detect an approaching vehicle having a specific relative speed by collecting data of a specific frequency and reconstructing the video.

  The spatial filter used for emphasizing the horizontal edge on the video is not limited to the horizontal line detection filter, and other spatial filters may be used.

  When detecting the horizontal edge representing the characteristics of the vehicle, the horizontal edge detection process was performed from the bottom of the video, and it was determined that the vehicle was reflected in the vehicle detection area set based on the detected horizontal edge. In this case, if the edge detection process is not performed in the vehicle detection area, the amount of calculation required for the vehicle detection process can be reduced.

  In the above-described embodiment, an example in which only one in-vehicle camera 10 is provided has been described. However, a plurality of in-vehicle cameras 10 may be provided.

  The correspondence between the constituent elements of the claims and the constituent elements of the embodiment is as follows. That is, the in-vehicle camera 10 constitutes an imaging unit, and the processing device 20 constitutes a frequency separation unit, an approaching vehicle detection unit, a video composition unit, and a noise removal unit. In addition, the above description is an example to the last, and when interpreting invention, it is not limited to the correspondence of the component of said embodiment and the component of this invention at all.

The figure which shows the structure of the approaching vehicle detection apparatus in one embodiment. The figure which shows the position example of the attachment position of the vehicle-mounted camera The figure which shows an example of the image imaged with the vehicle-mounted camera The figure which shows the time change of the brightness | luminance of one pixel (x, y) on a picked-up image. Diagram showing the optical flow of a vehicle traveling at a constant speed with arrows Y1 and Y2. The figure which shows the image synthesize | combined by the frequency synthetic | combination part, and the noise removal process performed by the noise removal part The figure which the steel tower reflected in the upper part of a picture was deleted from the picture shown in FIG. The figure after performing the process which emphasizes a horizontal edge using the horizontal line detection filter with respect to the image | video shown in FIG. FIG. 9A and FIG. 9B are diagrams showing an example of a template representing the characteristics of the vehicle. The figure which shows an example of the set rectangular area The flowchart which shows the flow of the process performed within a processing apparatus. The figure which shows the vehicle detection area set 2 each on the upper and lower sides of a horizontal edge FIG. 13A is a diagram when the vehicle detection area is a square, and FIG. 13B is a diagram when a trapezoid is formed. The figure which shows an example of the template for motorcycle detection

Explanation of symbols

DESCRIPTION OF SYMBOLS 10 ... Car-mounted camera, 20 ... Processing apparatus, 21 ... Frequency decomposition part, 22 ... Memory | storage part, 23 ... Frequency synthesis part, 24 ... Noise removal part, 25 ... Approaching vehicle detection part, 100 ... Own vehicle, 200 ... Approaching vehicle

Claims (8)

Imaging means for imaging the surroundings of the vehicle;
A frequency separation unit that separates a time-varying component of luminance of each pixel on a video imaged by the imaging unit (hereinafter referred to as a captured video image) for each frequency;
An approaching vehicle detection means for detecting an approaching vehicle approaching the host vehicle from the captured image, based on data obtained by separating the time-varying component of the captured image by frequency by the frequency separation means ;
From the data separated by the frequency separation means, it comprises video synthesis means for reconstructing a video by collecting data of pixel areas whose luminance change over time is a predetermined frequency or more,
The approaching vehicle detection means detects an approaching vehicle on the captured image by performing a template matching process based on the video reconstructed by the video composition means. apparatus. The approaching vehicle detection device according to claim 1 ,
The approaching vehicle detection device, wherein the approaching vehicle detection unit performs the template matching process after performing a process of detecting a lateral edge on the video reconstructed by the video composition unit. The approaching vehicle detection device according to claim 2 ,
The approaching vehicle detection device, wherein the approaching vehicle detection means performs the template matching process using a horizontal line template. In the approaching vehicle detection device according to claim 1 or 2 ,
The approaching vehicle detection means performs a template matching process to set a predetermined vehicle detection area around a pixel area that matches a template representing a vehicle characteristic, and to detect a lateral edge in the set vehicle detection area. An approaching vehicle detection device that detects an approaching vehicle on the captured image based on a data amount. The approaching vehicle detection device according to claim 4 ,
The approaching vehicle detection means sets a plurality of the vehicle detection areas around a pixel area that matches a template representing the characteristics of the vehicle, and sets the vehicle detection area with the largest amount of lateral edge data in the set vehicle detection area. An approaching vehicle detection device that judges that the approaching vehicle is reflected. In the approaching vehicle detection device according to claim 4 or 5 ,
The approaching vehicle detection device is configured to set a plurality of types of the vehicle detection area according to the type of the vehicle. In the approaching vehicle detection device according to any one of claims 1 to 6,
The approaching vehicle detection device is characterized in that the approaching vehicle detection means changes a template used when performing the template matching process between day and night. In the approaching vehicle detection device according to any one of claims 1 to 7,
Noise removing means for removing noise from the video reconstructed by the video synthesizing means;
The approaching vehicle detection device, wherein the approaching vehicle detection means performs the template matching process based on an image from which noise has been removed by the noise removal means.

Images