JP2009037542A - Adjacent vehicle detection apparatus and adjacent vehicle detection method - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To provide a new adjacent vehicle detection apparatus which can surely detect an adjacent vehicle running on the side of its own vehicle and persistently track the adjacent vehicle, and an adjacent vehicle detection method. <P>SOLUTION: The position of the adjacent vehicle is recognized by a head lamp detection method, a template is generated from image information around the vehicle lamp of the recognized adjacent vehicle and the adjacent vehicle is tracked by a template matching method using the generated template. Then, the relative position of the adjacent vehicle to the vehicle itself is specified on the basis of either one or both of positional information of the adjacent vehicle recognized by the head lamp detection method and the positional information of the adjacent vehicle tracked by the template matching method. Consequently, the adjacent vehicle running on the side of the vehicle itself can be surely detected and persistently tracked. <P>COPYRIGHT: (C)2009,JPO&INPIT

Description

  The present invention relates to an adjacent vehicle detection device for detecting another vehicle (hereinafter referred to as an “adjacent vehicle”) that travels adjacent to the side of the host vehicle, and is particularly suitable for detection of an adjacent vehicle during night driving. The present invention relates to an adjacent vehicle detection device and an adjacent vehicle detection method.

2. Description of the Related Art Conventionally, there is known a technique for detecting (sensing) other vehicles traveling near the host vehicle by installing sensors such as millimeter wave radars and laser radars at four corners of the vehicle.
However, in this detection method using millimeter wave radar, laser radar, etc., only the front and rear sides of the own vehicle can actually be detected due to restrictions such as the installation location of these sensors on the vehicle. Is difficult to detect as a blind spot.

Therefore, for example, as shown in the following Patent Document 1, a TV camera device for photographing the side of the vehicle is attached to the host vehicle, and a headlamp during night driving is performed by processing an image from the TV camera device. A method (headlamp detection method) of detecting an adjacent vehicle that travels to the side of the host vehicle by extracting the light of the vehicle and continuously detecting the light of the extracted headlamp is conceivable.
JP-A-5-189694

By the way, in order to detect an adjacent vehicle traveling on the side of the host vehicle based on the light of the headlamp as described above, it is necessary to always detect the headlamp of the adjacent vehicle by the television camera device.
However, when the adjacent vehicle is located beside the host vehicle, the headlamp of the adjacent vehicle is shielded by the outer shape of the vehicle body and cannot be detected by the TV camera device. It is difficult to detect adjacent vehicles.
Therefore, the present invention has been devised to effectively solve such a problem, and the object thereof is to reliably detect an adjacent vehicle running on the side of the host vehicle and to robustly track this. A novel adjacent vehicle detection device and an adjacent vehicle detection method are provided.

In order to solve the above problems, the present invention provides:
The side of the vehicle is photographed by the side camera, the vehicle lamp of the adjacent vehicle is detected based on the image information including the luminance information in the photographed image, and based on the detected position of the vehicle lamp of the adjacent vehicle Recognize the position of the adjacent vehicle. Further, a template is generated from the image information around the vehicle lamp of the adjacent vehicle recognized in this way, and the adjacent vehicle is tracked by a template matching method using the generated template.
The position information of the adjacent vehicle recognized based on the position of the vehicle lamp of the adjacent vehicle and the position information of the adjacent vehicle tracked by the template matching method are included in one or both of the position information. Based on this, the relative position of the adjacent vehicle with respect to the host vehicle is specified.

As described above, the present invention is based on the conventional technology for recognizing the position of the adjacent vehicle based on the vehicle lamp position of the adjacent vehicle (hereinafter, this technology is appropriately referred to as “headlamp detection method”) and this headlamp detection method. Adjacent vehicles traveling sideways of the vehicle even when it is not possible to detect the adjacent vehicle at night using the headlamp detection method because of the combined use of the adjacent vehicle tracking technology based on the template matching method based on the adjacent vehicle. Vehicles can be reliably detected and tracked robustly.
This makes it possible to realize an omnidirectional recognition system for the host vehicle by using it together with a vehicle detection technique using a sensor such as a conventional millimeter wave radar or laser radar.

Hereinafter, the best mode for carrying out the present invention will be described in detail with reference to the accompanying drawings.
(Constitution)
FIG. 1 shows an embodiment of an adjacent vehicle detection device 100 according to the present invention.
As shown in the figure, this adjacent vehicle detection device 100 includes a side camera 10, an image memory 12, a vehicle detection processing unit 14, a template creation unit 16, a template storage memory 18, a template update unit 20, a vehicle It mainly comprises a tracking processing unit 22, a detection result synthesis unit 24, a lamp position prediction unit 26, an exposure control signal calculation unit 28, and a result output unit 30.

  First, the side camera 10 is composed of a solid-state image sensor such as a color CCD sensor or a color CMOS sensor, for example. Then, for example, as shown in FIG. 2, the camera is attached to door mirror units 40 installed on both sides of the host vehicle, and continuously shoots the side of the host vehicle (video shooting: 30 frames / second, for example). The image data is sequentially output to the image memory 12 side. In the example of FIG. 2, it is attached to the front end of the door mirror unit on the left side of the host vehicle and photographs the left side of the host vehicle in a certain range.

  As the side camera 10, it is desirable to use a wide-angle lens so that the side of the host vehicle can be photographed in as wide a range as possible. The side camera 10 is not particularly limited in its mounting position and height as long as the side of the host vehicle can be photographed. In addition to the door mirror body, for example, the roof of the host vehicle, the door body, a bumper, etc. It may be attached so as to be embedded in.

The image memory 12 is composed of, for example, a semiconductor storage device such as SRAM or DRAM, and holds or records a color image taken by the side camera 10 temporarily or continuously as image information. .
The vehicle detection processing unit 14 converts the color image held / recorded in the image memory 12 into a monochrome image including only luminance information, and then performs image processing so that the vehicle lamp (particularly the headlamp) of the adjacent vehicle has a luminance. The adjacent vehicle is detected by extracting from the deviation information (headlamp detection method).

  The vehicle detection processing unit 14 stores the vehicle position (x coordinate, y coordinate) of the adjacent vehicle with respect to the own vehicle detected by such a headlamp detection method for a predetermined number of frames as a past history. The movement amount of the adjacent vehicle is calculated by subtracting the detected vehicle position and the position of the adjacent vehicle recorded as the past history, and the vehicle movement amount on the x-axis and the y-axis is within a predetermined value. The detected object is determined to be an adjacent vehicle, and the position of the adjacent vehicle is output. The vehicle detection processing unit 14 does not output a detection result when it is determined that the detected target is not an adjacent vehicle.

  For example, as shown in FIG. 3, the template creation unit 16 detects the vehicle of the adjacent vehicle detected from the image information recorded in the image memory 12 based on the position of the adjacent vehicle detected by the vehicle detection processing unit 14. A template of a predetermined size is created by cutting out an image area including the vehicle body outline around the lamp. FIG. 3 shows one of the fenders and tires in the vicinity of a headlamp on the right side (to the left side of the adjacent vehicle) of the adjacent vehicle recorded in the image memory 12 taken obliquely from the front right. The example which the template which cut out the partial area | region of a part and a front grille to the rectangular shape was produced is shown.

Similar to the image memory 12, the template storage memory 18 is composed of a semiconductor storage device such as SRAM or DRAM, and the template created by the template creation unit 16 and the template updated by the template update unit 20 are stored. It is designed to hold and store temporarily or continuously.
The template update unit 20 is a vehicle body contour around the vehicle lamp of the adjacent vehicle detected from the image information held and stored in the image memory 12 based on the adjacent vehicle detection result output from the detection result combining unit 24 described later. The template is updated by cutting out an image area including “”, and the updated template is output to the template storage memory 18. The template update unit 20 is configured to update the template for each frame of image information held and recorded one after another in the image memory 12.

  The vehicle tracking processing unit 22 performs template matching by a known template matching method on the image recorded in the image memory 12 using the template held and stored in the template storage memory 18, thereby adjacent vehicles in the image. Is located, and its neighboring vehicles are tracked. And this vehicle tracking processing part 22 preserve | saves the position of the adjacent vehicle detected by template matching (x coordinate of the template center point in a template matching applicable position, y coordinate) as a past log | history for a predetermined number of frames, When the movement amount of the adjacent vehicle is calculated by subtracting the detected position of the adjacent vehicle and the position of the adjacent vehicle recorded as the past history, and the vehicle movement amount on the x-axis and the y-axis is within a predetermined value Determines that the object being tracked is an adjacent vehicle and outputs the vehicle position. The vehicle tracking processing unit 22 does not output the vehicle position when it is not determined that the tracked target is an adjacent vehicle.

  The detection result synthesizing unit 24 determines the detection position of the adjacent vehicle based on the detection result of the adjacent vehicle by the vehicle detection processing unit 14 and the detection result of the adjacent vehicle by the vehicle tracking processing unit 22. If detection results are obtained from both the vehicle detection processing unit 14 and the vehicle tracking processing unit 22, the adjacent vehicle detection position is determined based on both output results. For example, it is possible to reliably determine the adjacent vehicle detection position by comparing both output results and adopting the more reliable output result. On the other hand, when only one of the output results can be obtained, the coordinates based on the output result are adopted as they are for the final determination.

  The lamp position prediction unit 26 is configured to predict the position of the other vehicle lamp when one vehicle lamp is detected by the vehicle detection processing unit 14. That is, when the headlamp is first detected, the position of the rear lamp on the image (screen), and when the rear lamp is detected for the first time, the position of the headlamp on the image (screen). It comes to predict.

  Specifically, as shown in FIG. 4, the headlamp H and the rear lamp R are at the same height based on the position of the vehicle lamp detected by the vehicle detection processing unit 14, and the side camera 10 has a wide angle. When a lens is used, the other vehicle lamp is detected in consideration of screen distortion caused by the lens. In the example of FIG. 4, if one vehicle lamp is detected, a lamp position prediction line L indicated by a broken line is provided at the height of this vehicle lamp, and it is predicted that the other vehicle lamp exists along this line L. However, the other vehicle lamp is detected around the predicted position.

  Based on the position of the other vehicle lamp predicted by the lamp position prediction unit 26, the exposure control signal calculation unit 28 obtains an optimal image when the vehicle detection processing unit 14 detects the other vehicle lamp around the predicted position. I try to get it. Specifically, the exposure control signal calculation unit 28 calculates an exposure control signal for performing exposure control, and performs exposure control of the side camera 10 based on the exposure control signal.

  That is, since the brightness (luminance) in the image of the headlamp H and the rear lamp R is greatly different, for example, in the exposure control when the headlamp H is detected, the rear lamp R cannot be detected due to underexposure. By performing exposure control (exposure aperture control) according to the brightness of the rear lamp R, the rear lamp R is reliably detected. It goes without saying that the exposure control when the vehicle lamp detected first is the rear lamp R is reversed.

  When an adjacent vehicle is detected by the detection result synthesizing unit 24, the result output unit 30 displays various notification means such as an alarm buzzer or HUD (Head-Up Display) to the driver of the vehicle, car navigation The presence of an adjacent vehicle on the side of the host vehicle is announced and warned by voice utterance from the system speaker, screen display from the monitor of the car navigation system, meter display, and the like.

  The image memory 12, vehicle detection processing unit 14, template creation unit 16, template storage memory 18, template update unit 20, vehicle tracking processing unit 22, detection result synthesis unit 24, lamp position prediction unit 26, exposure control signal calculation The unit 28 and the like are actually realized by an information processing system (microcomputer system: hardware resource) composed of arithmetic means such as a CPU and storage means such as RAM and ROM, and various dedicated programs (software). These hardware resources and software cooperate to specifically realize the functions and operations of the respective units.

(Operation)
Next, an adjacent vehicle detection method using the adjacent vehicle detection device 100 of the present invention having such a configuration will be described with reference mainly to the flowchart of FIG. The processing flow shown in FIG. 5 is a flowchart at the time of loop processing in the adjacent vehicle detection device 100 of the present invention, and this loop processing is performed while the adjacent vehicle detection device 100 is operating. Assume that you are doing it repeatedly.

(Step S201)
First, in the first step S201, an image taken by the side camera 10 is read from the image memory 12, and the process proceeds to the next step S202.
(Step S202)
In step S202, the template creation unit 16 or the template update unit 20 determines whether the template has already been created or updated at the time of the previous loop processing, and if it is determined that the template has already been created or updated (YES). The process proceeds to the step S203 side, but when it is determined that it is still created or updated (NO), the process proceeds to the step S204 side.

(Step S203)
In step S203, the vehicle tracking processing unit 22 performs vehicle tracking by the template matching method using a template that has already been created or updated. When the processing is completed, the process proceeds to step S205.
(Step S204)
On the other hand, in step S204, the vehicle detection processing unit 14 detects an adjacent vehicle by the headlamp detection method, and then proceeds to the next step S205.
(Step S205)
In step S205, the vehicle tracking result in step S203 and the vehicle detection result in step S204 are comprehensively determined, and the position of the adjacent vehicle is determined.

(Step S206)
In the next step S206, the detection result synthesis unit 106 determines whether or not the detection result of the adjacent vehicle is output, and when it is determined that one or more adjacent vehicle detection results are output (YES) However, if no adjacent vehicle detection result is output (NO), it is determined that the adjacent vehicle does not exist on the side of the host vehicle, and the process is terminated.
(Step S207)
In step S207, it is determined that the adjacent vehicle exists on the side of the host vehicle, and as a result, the output unit 30 warns the driver of the host vehicle of the presence of the adjacent vehicle by a warning sound or display.

(Step S208)
In step S208, the vehicle tracking processing unit 22 determines whether there is an adjacent vehicle being tracked. If it is determined that there is an adjacent vehicle being tracked (YES), the process proceeds to step S209 for tracking. When it is determined that there is no adjacent vehicle (NO), the process proceeds to step S212.
(Step S209)
In step S209, the template update unit 20 creates a template used for the next loop process based on the adjacent vehicle detection result output from the detection result combining unit 24, and the process proceeds to the next step S210.

(Step S210)
In step S210, the other vehicle lamp position of the adjacent vehicle is predicted based on the adjacent vehicle detection result output from the detection result combining unit 24, and the process proceeds to step S211.
(Step S210)
In step S211, based on the other vehicle lamp position predicted by the lamp position prediction unit 26, the exposure control is performed so that an optimal image can be obtained to reliably detect the other vehicle lamp around the predicted position. After completing the process. Specifically, as described above, when the headlamp H is first detected, exposure control in accordance with the luminance of the rear lamp R is performed.

(Step S212)
On the other hand, in step S212, the template creation unit 16 creates a template used for the next loop process based on the adjacent vehicle detection result output from the detection result synthesis unit 24, and the process ends.
Then, by repeating such processing for each frame, as shown in FIG. 6, when the adjacent vehicle approaches the host vehicle from the rear or the front and is positioned around the side of the host vehicle, the side camera 10 Even if the headlamp H and the rear lamp R are shielded by the outer shape of the vehicle body and disappear, the tracking can be continued robustly without losing sight of the adjacent vehicle.

FIG. 7 is an explanatory diagram showing a temporal relationship between detection of an adjacent vehicle by the headlamp detection method and vehicle tracking by the template matching method. In addition, this example shows the situation until the adjacent vehicle approaches and overtakes the own vehicle from behind.
As shown in the figure, the initial detection of the adjacent vehicle is first performed by the headlamp detection method, and vehicle tracking by the template matching method is started using this as a trigger. After that, vehicle detection by both detection methods is performed in parallel, but when the adjacent vehicle is positioned right next to the host vehicle, the headlamp H disappears due to the outer shape of the vehicle body and the headlamp detection method is used. It becomes impossible to detect the vehicle. However, since the vehicle tracking by the template matching method is continued at this time, there is no possibility of losing sight of the adjacent vehicle.

  When the adjacent vehicle further moves in front of the host vehicle, vehicle tracking by the template matching method becomes impossible. However, since the rear lamp R of the adjacent vehicle is detected immediately before that, the head based on the rear lamp R is detected. Vehicle detection by the ramp detection method is restored. Accordingly, even when vehicle detection by either one of the detection methods becomes impossible, detection by the other detection method becomes possible, so that the adjacent vehicle is not lost.

(effect)
As described above, according to the present invention, since the vehicle detection is performed by using the detection result by the headlamp detection method and the detection result by the template matching method in combination, the adjacent vehicle can be reliably detected, and the robust adjacent vehicle can always be detected. It becomes possible.
In addition, the template used in template matching is updated based on not only the position information of the adjacent vehicle by the template matching method but also image information of the area including the vehicle body shape around the vehicle lamp of the adjacent vehicle detected by the headlamp detection method. This makes it possible to improve the template identification accuracy.
As a result, the cutout size (area) of the template can be reduced, so that the capacity of the image memory 12 and the like can be reduced, and the load of subsequent image processing and the like can be reduced, so that the image processing speed can be further improved. it can.

In addition, after detecting one vehicle lamp, the position of the other vehicle lamp is predicted in consideration of the detection position, image distortion, etc., so the other vehicle lamp is detected easily and quickly. be able to.
Further, when the other vehicle lamp is predicted and detected in this way, the other vehicle lamp can be detected more reliably by changing image parameters such as changing the exposure of the image.

Further, when detecting the headlamp first and predicting and detecting the position of the rear lamp based on the position of the headlamp, since the rear lamp is usually red, not only the luminance information in the image but also red By detecting the amount of information (R signals of R, G, and B signals), the rear lamp R can be detected more quickly and reliably.
In particular, the brightness (luminance value) of the rear lamp R is significantly lower than that of the headlamp, and erroneous detection and non-detection are likely to occur due to the influence of street lamps and reflected light only by exposure control. In this way, it is desirable to detect the color information.

  The “photographing means” described in the column of problems to be solved by the invention and claims 1 to 6 (hereinafter abbreviated as “claims”) of the claims is the “side” of the present embodiment. Corresponding to “camera 10” and “image memory 12”, “vehicle lamp detecting means” and “adjacent vehicle recognizing means” recited in the claims also correspond to “vehicle detection processing unit 14” according to the present embodiment. Is.

Similarly, the “template generation unit” recited in the claims corresponds to the “template creation unit 16”, the “template storage memory 18”, and the “template update unit 20” according to the present embodiment. The “adjacent vehicle tracking means” corresponds to the “vehicle tracking processing unit 22” according to the present embodiment.
The “adjacent vehicle position specifying means” described in the claims corresponds to the “detection result combining unit 24” according to the present embodiment, and the “vehicle lamp detection means” in the claims is also described in the present embodiment. This corresponds to “lamp position prediction unit 26” and “exposure control signal calculation unit 28” and steps S210 and S211 in FIG.

In addition, the “shooting step” described in the column of the problem to be solved by the invention and in claim 7 of the claims (hereinafter abbreviated as “claim”) is the “side camera 10” of the present embodiment. 5 corresponds to the vehicle side photographing and the recording / holding of the photographed image data in the “image memory 12”, and the “vehicle lamp detection step” and the “adjacent vehicle recognition step” in the claims are the steps in FIG. This corresponds to S201, S202, S204, and the like.
Similarly, the “template generation step” in the claims corresponds to steps S209 and S212 in FIG. 5, and the “neighboring vehicle tracking step” in the claims also corresponds to step S203 in FIG.
Similarly, the “adjacent vehicle position specifying step” in the claims corresponds to steps S205 to S207 in FIG.

It is a functional block diagram which shows one Embodiment of the adjacent vehicle detection apparatus which concerns on this invention. It is a top view which shows the attachment position of the side camera with respect to the own vehicle, and its main imaging | photography range. It is a conceptual diagram which shows the example which cuts out a vehicle lamp periphery as a template from the adjacent vehicle whole image. It is a figure which shows the example of the image image | photographed with the side camera using a wide angle lens. It is a flowchart figure which shows the flow of the process which concerns on the adjacent vehicle detection method which concerns on this invention. It is a top view which shows the positional relationship of the own vehicle and the adjacent vehicle which drive | works the left side lane. It is explanatory drawing which showed the temporal relationship between the vehicle detection by a headlamp detection method, and the vehicle tracking by a template matching method.

Explanation of symbols

DESCRIPTION OF SYMBOLS 100 ... Adjacent vehicle detection apparatus 10 ... Side camera 12 ... Image memory 14 ... Vehicle detection process part 16 ... Template preparation part 18 ... Template storage memory 20 ... Template update part 22 ... Vehicle tracking process part 24 ... Detection result synthetic | combination part 26 ... Lamp position prediction unit 28 ... exposure control signal calculation unit 30 ... result output unit 40 ... door mirror unit H ... head lamp (vehicle lamp)
R ... Rear lamp (vehicle lamp)
L ... Ramp position prediction line

Claims (7)

An adjacent vehicle detection device that detects an adjacent vehicle that travels adjacent to the side of the host vehicle,
Photographing means for photographing the side of the vehicle;
Vehicle lamp detection means for detecting a vehicle lamp of an adjacent vehicle based on image information including luminance information in an image photographed by the photographing means;
Adjacent vehicle recognition means for recognizing the position of the adjacent vehicle based on the position of the vehicle lamp of the adjacent vehicle detected by the vehicle lamp detection means;
Template generating means for generating a template from image information around the vehicle lamp of the adjacent vehicle recognized by the adjacent vehicle recognizing means;
An adjacent vehicle tracking means for tracking the adjacent vehicle by a template matching method using a template generated by the template generation means;
Relative position of the adjacent vehicle with respect to the own vehicle based on position information of one or both of the position information of the adjacent vehicle by the adjacent vehicle recognition means and the position information of the adjacent vehicle by the adjacent vehicle tracking means An adjacent vehicle detection device comprising: an adjacent vehicle position specifying means for specifying The adjacent vehicle detection device according to claim 1,
The template generation means includes
An adjacent vehicle detection apparatus that creates a template based on position information of an adjacent vehicle by the adjacent vehicle tracking means in addition to image information around a vehicle lamp of the adjacent vehicle recognized by the adjacent vehicle recognition means. In the adjacent vehicle detection device according to claim 1 or 2,
The vehicle lamp detection means is
Determining whether the vehicle lamp of the adjacent vehicle detected based on the luminance information in the image captured by the imaging means is a headlamp or a rear lamp of the adjacent vehicle;
When it is determined that the detected vehicle lamp is a headlamp, the position of the rear lamp is predicted and detected based on the detection position of the headlamp, and
An adjacent vehicle detection device that detects and detects a position of a headlamp based on a detection position of the rear lamp when it is determined that the detected vehicle lamp is a rear lamp. In the adjacent vehicle detection device according to claim 3,
The vehicle lamp detection means is
When detecting the other vehicle lamp based on the position of one vehicle lamp, the image parameter when the first detected vehicle lamp is detected is changed according to the type of the other vehicle lamp. An adjacent vehicle detection device. In the adjacent vehicle detection device according to claim 4,
The adjacent vehicle detection apparatus characterized in that the image parameter changed by the vehicle lamp detection means is exposure information of an image photographed by the photographing means. In the adjacent vehicle detection device according to claim 3,
The vehicle lamp detection means is
When the image information of the image photographed by the photographing means includes color information together with luminance information, the adjacent vehicle detection device detects the vehicle lamp based on the color information. An adjacent vehicle detection device that detects an adjacent vehicle that travels adjacent to the side of the host vehicle,
A shooting step for shooting the side of the vehicle;
A vehicle lamp detection step for detecting a vehicle lamp of an adjacent vehicle based on image information including luminance information in the image photographed in the photographing step;
An adjacent vehicle recognition step for recognizing the position of the adjacent vehicle based on the position of the vehicle lamp of the adjacent vehicle detected in the vehicle lamp detection step;
A template generation step of generating a template from image information around the vehicle lamp of the adjacent vehicle recognized in the adjacent vehicle recognition step;
An adjacent vehicle tracking step of tracking the adjacent vehicle by a template matching method using the template generated in the template generation step;
Relative position of the adjacent vehicle with respect to the own vehicle based on position information of one or both of the position information of the adjacent vehicle in the adjacent vehicle recognition step and the position information of the adjacent vehicle in the adjacent vehicle tracking step And an adjacent vehicle position specifying step for specifying the adjacent vehicle.

Images