JP2002098509A - Vehicle detecting device - Google Patents

Abstract

PROBLEM TO BE SOLVED: To detect a vehicle with ease and high-accuracy utilizing an infrared image. SOLUTION: An image position of an exhaust duct section photographed with brightness higher than a determined brightness is specified from an image area section that is estimated as a vehicle in the infrared image. Then, the image position of the exhaust duct section is set as a reference, and a frame having a determined rate in size is set for the image position in the upward, downward, left and right directions, and a pattern matching process is carried out using the frame. The distance to the vehicle is estimated on the basis of a view angle of infrared camera, a corresponding relation between the distance and pixels composing an imaging device of the camera, and the number of pixels from the underside of the infrared image to the image position of the exhaust duct section.

Description

DETAILED DESCRIPTION OF THE INVENTION

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to a vehicle detection device, and more particularly, to a vehicle detection device suitable for application to a typical vehicle such as an automobile.

[0002]

2. Description of the Related Art Conventionally, in the field of automobiles, which are typical vehicles, a technique for assisting a driving operation to supplement a driver's vision according to a surrounding driving environment detected by using various sensors, or 2. Description of the Related Art A technique for automatically driving an automobile has been proposed.

As a sensing technique employed in such a technique, for example, Japanese Patent Application Laid-Open No. H10-255019 discloses an image captured by an infrared camera using a threshold calculated based on an image captured by a visible light camera. Has been proposed in which the position of a vehicle existing around the host vehicle is detected from the obtained binarized image.

As a technique for assisting a driver's vision in a traveling environment having poor visibility, Japanese Patent Application Laid-Open No. Sho 60-2 (1985) discloses a technique.
No. 31193 proposes a technique for displaying an image captured by an infrared camera on a display.

[0005]

According to the above-mentioned prior art, it is possible to efficiently assist a driver's vision in a traveling environment having poor visibility.

However, the detection method using infrared light is susceptible to changes in the temperature of the road surface and sunlight, and any object having a temperature is detected regardless of its type. (For example, an automobile), it is not easy to detect.

Accordingly, an object of the present invention is to provide a vehicle detection device that detects a vehicle easily and with high accuracy using an infrared image.

[0008]

In order to achieve the above object, a vehicle detecting device according to the present invention has the following features.

That is, an infrared camera for imaging a predetermined imaging range using infrared light, and an infrared camera included in the infrared image, which is included in the infrared image, based on the infrared image captured by the infrared camera. Detecting a partial image region estimated to be a vehicle to perform, and detecting a distance from the vehicle based on an image position of an exhaust pipe portion photographed with higher luminance than a predetermined luminance included in the detected partial image region. And a detection unit that performs the detection.

[0010] In a preferred embodiment, the detecting means sets a frame having a predetermined ratio of size vertically and horizontally with respect to the image position based on the image position of the exhaust pipe portion, and uses the frame. It is preferable to determine whether the partial image region is an image representing a vehicle by performing pattern matching on the partial image region.

[0011] Further, for example, the detecting means stores in advance a correspondence relationship regarding the distance between the imaging range and the pixels constituting the imaging device of the infrared camera. The distance to the vehicle may be detected based on the number of pixels from the lower side to the image position of the exhaust pipe portion and the correspondence relationship regarding the distance.

Further, for example, the detecting means includes a distance to a vehicle to be detected and a number of pixels in the infrared image corresponding to a standard vehicle width of the vehicle corresponding to the distance. The correspondence is stored in advance, and the detecting unit may estimate the vehicle width of the vehicle based on the distance detected by the infrared image and the correspondence related to the vehicle width.

[0013]

According to the present invention described above, it is possible to provide a vehicle detecting device that detects a vehicle easily and with high accuracy using an infrared image.

That is, according to the first aspect of the present invention, a portion of an exhaust pipe provided at a substantially similar position in a vehicle driven by an internal combustion engine is photographed with the highest luminance in an infrared image. By utilizing this, a vehicle can be easily and accurately detected with a relatively simple processing configuration.

In general, the relationship between the mounting position of an exhaust pipe in a vehicle and the overall cross-sectional shape of the vehicle is substantially the same. By storing in advance as the reference ratio, the vehicle can be detected easily and with high precision by pattern matching using an appropriate frame (template) regardless of the size of the partial image area.

According to the third aspect of the present invention, the distance to the vehicle can be easily and accurately detected using only the infrared image.

Further, according to the invention of claim 4, it is possible to easily and accurately estimate the width of the vehicle using only the infrared image.

[0018]

DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS Hereinafter, the present invention will be described in detail with reference to the drawings as an embodiment applied to a warning system for a vehicle such as an automobile.

FIG. 1 is a block diagram showing a device configuration of a vehicle equipped with an alarm system according to the present embodiment.

In the vehicle 100 shown in FIG. 1, reference numeral 1 denotes a distance sensor (hereinafter, LR) such as a laser radar or a millimeter-wave radar for detecting an inter-vehicle distance from another vehicle traveling ahead of the own vehicle. Reference numeral 2 denotes an image sensor (hereinafter, IS) that captures a predetermined imaging range in front of the vehicle using an imaging device (infrared camera) using infrared rays (including near infrared rays and far infrared rays).

Reference numeral 3 denotes a brake actuator for reducing the speed of the host vehicle. Reference numeral 4 denotes a throttle actuator for adjusting the opening of an engine throttle for adjusting the speed of the host vehicle. Note that an automatic transmission may be used together to adjust the speed of the vehicle 100.

Here, LR1 and IS2 are shown in FIG.
As shown in FIG. 2, the LR1 and the IS2 are substantially the same in the vehicle width direction of the vehicle 100, and are mounted near the bumper so as to be overlapped in the vehicle vertical direction. The coordinate system is such that a vertical upward direction from the mounting position is a + Y direction, a right lateral direction is a + X direction, and a forward direction of the vehicle is a + Z direction.

Further, in the present embodiment, IS2 corresponds to FIG.
The reason for mounting below the bumper as shown in (b) is to make it easier to photograph the periphery of the exhaust pipe located below the rear bumper of another vehicle to be detected by IS2.

The two types of sensors LR1 and LR1
As shown in FIG. 3, the detection range of S2 is set such that the detection range of IS2 includes the detection range of LR1. The detection range of LR1 (scanning range (propagation range) of a predetermined detection wave) is a three-dimensional region forming a predetermined elevation angle in the vertical direction, and the distance and direction to an object existing in the region range. Angle).

In the present embodiment, for convenience of explanation, LR
1 and 2 are arranged as shown in FIG. 1B. However, when mounting is actually shifted in the vehicle width direction due to the mounting space, the same amount of offset of the sensors is offset. It is preferable to adjust the coordinate system to the coordinate system shown in FIG.

Reference numeral 5 denotes a speaker that outputs an alarm sound at least relating to the inter-vehicle distance between the host vehicle and another vehicle ahead in the present embodiment. Reference numeral 6 denotes a wiper switch that allows a driver to set the operation of the wiper of the vehicle 100 on / off, the operation speed, and the operation cycle. It is estimated whether it is in stormy weather such as fog or dense fog.

Reference numeral 7 denotes a display device for displaying an image captured by IS2 (hereinafter, infrared image) and displaying various information. Reference numeral 8 denotes an illuminance sensor that detects whether or not the surrounding traveling environment is in a fine weather state by detecting the illuminance of sunlight on the vehicle 100.

Reference numeral 9 denotes a speed sensor for detecting the speed of the vehicle 100. Reference numeral 10 denotes a GPS sensor that detects the current position of the vehicle 100 based on a GPS (global positioning system) signal received from the outside. In the present embodiment, the GPS sensor 10 is also used to detect the current time.

Then, an ECU (electronic control unit) 11
In accordance with the information obtained by the various sensors described above, a warning regarding the following distance is notified to the driver from the speaker 5 and the brake actuator 3 and / or the throttle actuator 4 are controlled, so that the The inter-vehicle distance with another existing vehicle is maintained at a predetermined distance. A microcomputer (not shown) is mounted on the ECU 11, and the CPU operates according to programs and parameters stored in a memory in advance, thereby realizing an alarm / control process described below.

[Alarm / Control Process] FIG. 4 is a flowchart showing an alarm / control process executed by the ECU 11 in the present embodiment. For example, an ignition switch (not shown) is set to an ON state by a driver. Started by

In FIG. 3, step S1: the presence of a vehicle (preceding vehicle) existing in front of the vehicle 100 is recognized by sensor fusion based on the detection results of LR1 and IS2, and the vehicle and its own vehicle are connected. Acquire relative distance information (inter-vehicle distance, speed) (details will be described later).

Step S2: The own vehicle speed is detected based on the detection result of the vehicle speed sensor 9.

Steps S3 to S5: FIG.
Is stored in advance in a storage device such as a ROM as a look-up table (LUT), and a relative speed is calculated based on the distance information and the own vehicle speed obtained in steps S1 and S2. At the same time, by referring to the LUT in accordance with the calculated relative speed and the inter-vehicle distance obtained in step S1, a dangerous state in which the inter-vehicle distance control for driving the brake actuator 3 and / or the throttle actuator 4 should be executed. It is determined whether or not it is a dangerous state to output the alarm sound from the speaker 5 (step S3). Then, based on the result of the determination, when it is not dangerous, the process returns to step S1, and when an alarm is required, an alarm sound is output from the speaker 5 and guidance is displayed on the display device 7 (step S4). When it is necessary to execute the following distance control, the following distance is controlled by controlling the driving of the brake actuator 3 and / or the throttle actuator 4 (step S5), and thereafter, the process returns to step S1.

[Vehicle Recognition Processing] Hereinafter, the vehicle recognition processing performed in step S1 of FIG. 4 in the present embodiment will be described in detail.

FIG. 6 is a flowchart showing a vehicle recognition process executed by the ECU 11 in this embodiment. As shown in FIG. 6, in step S1 (FIG. 4), a vehicle qualification process using IS2 (step S11) is performed. ) And the vehicle recognition process using LR1 (step S12) are performed in parallel. In the vehicle recognition process in step S13, IS2 and L2 in steps S11 and S12 are used.
The final vehicle recognition result is calculated by sensor fusion based on the detection result of R1. And step S
In the present embodiment, the vehicle recognition result calculated in 13 is used for danger determination in step S3.

Hereinafter, a detailed processing procedure of each step shown in FIG. 6 will be described.

<Vehicle Certification Processing by IS2>
The vehicle certification process in S2 will be described with reference to FIGS.

First, an outline of the vehicle certification process by IS2 will be described. In the present embodiment, the vehicle qualification process based on IS2 is roughly divided into “vehicle image rough estimation process” performed on the entire infrared image captured by the infrared camera (IS2), and the rough estimation process. This is configured by “vehicle detection processing” individually performed on the estimated partial image area.

First, in the rough estimation process of the vehicle image, a portion which is included in the entirety of the infrared image (see the infrared image 21 illustrated in FIG. 7) and which is likely to be a vehicle (cross-sectional shape of the vehicle). Image area (partial image area 24 illustrated in FIG. 7)
Is estimated.

Next, in the vehicle detection processing, a general vehicle driven by an internal combustion engine is set as a detection target object, and a predetermined image included in the partial image area estimated by the rough estimation processing of the vehicle image is included in the area. A portion higher in brightness than the brightness is detected as an exhaust pipe attachment position (the exhaust pipe region image 23 illustrated in FIG. 7), and a frame of a predetermined size based on the detected high brightness portion is set in the partial image region. By performing the pattern matching process used for this, it is specified that the object represented by the partial image region is a vehicle, and the position and speed of the specified object are detected.

At this time, the ECU 11 previously stores in the ROM a value based on a standard amount of radiant heat in the area of the exhaust pipe when a general vehicle including the exhaust pipe is imaged by the infrared camera as the predetermined luminance. And characteristic information representing the relationship between the standard mounting position of the exhaust pipe in the vehicle and the cross-sectional shape of the vehicle is stored in advance in a memory such as a ROM.

At the time of the pattern matching process, by referring to the above-mentioned feature information, a frame having a predetermined ratio of size in the upper, lower, left, and right directions with respect to the mounting position of the exhaust pipe having a luminance higher than the predetermined luminance is noted. This is set for a partial image area that is present (details will be described later with reference to FIG. 11).

FIG. 8 is a flowchart showing a vehicle qualification process by the IS2 executed by the ECU 11 in this embodiment, and shows a detailed processing procedure of step S11 in FIG.

In FIG. 8, steps S21 to S23: The timing at which the captured image (infrared image) for one frame of the infrared camera is input (step S21), and the process of roughly estimating the vehicle image outlined above is performed. The count value of the counter C used for determination is reset to 0 (step S22), and the number of partial image areas detected from the one frame of the infrared image (the number of objects X) is reset to 0. (Step S23).

Since the operation of the vehicle to be detected is slower than the control cycle of the infrared camera and the ECU 11, the control cycle of the infrared camera is reduced in this embodiment in order to reduce the load of the arithmetic processing of the ECU 11. (In this embodiment, 1
In contrast to the execution frequency of the vehicle detection process performed on the infrared image input every 00 msec), the vehicle image rough estimation process is performed every 10 frames.

Steps S24 and S25: It is determined whether or not the count value of the counter C is 0 (step S2).
4) When the counter C = 0 in this determination, the portion that is estimated to be the cross-sectional shape of the vehicle is obtained by performing a rough estimation process of the vehicle image on the entire newly input infrared image for one frame. The position of the image area in the infrared image is specified, and the total quantity of the estimated partial image area is set to the number of objects X (step S25: details will be described later with reference to FIG. 9), and the process proceeds to step S26.

Steps S26 to S28: On the other hand,
If the counter C ≠ 0, the vehicle detection process outlined above is individually performed on the X partial image areas (object images) set in step S25 in step S2.
6 (the details will be described later with reference to FIG. 10)
Is repeated while the count value of the counter C is incremented by 1 each time the process of step S26 is completed (steps S27 and S28), and when the count value becomes 9, the process proceeds to step S21. Return to

(Schematic Estimation Processing of Vehicle Image) FIG.
9 is a flowchart showing details of a vehicle image rough estimation process (step S25 in FIG. 8) in the vehicle certification process according to No. 2;

In the figure, step S31: One frame of infrared image is input from the infrared camera as a new processing object.

Steps S32 and S33: To detect the features of the input infrared image, first, a differential image in the vertical direction (Y direction) (step S32) and a differential image in the horizontal direction (X direction) (step S32) S33) is generated. More specifically, in step S32, digital filtering in the vertical direction is performed on the entire infrared image using a 1-3 matrix of [1, 0, -1] ^T. Step S33
Then, the digital filter processing in the horizontal direction is performed on the entire infrared image using the 3-1 matrix of [1, 0, -1].

Steps S34 and S35: The logical product (A of the differential images in the two directions generated in the above two steps)
ND) (step S34), and accumulates the vertical differential values of the differential image obtained as a result. The calculated cumulative value is used as the partial image area of the vehicle from the infrared image in step S39 and step S40 described later. (Step S3)
5).

Steps S36 and S37: In Steps S39 and S40, which will be described later, in order to use as a density reference when removing an area other than the partial image area of the vehicle (ie, sky area and road area) from the infrared image, The density value of the predetermined pixel above the center of the input infrared image is sampled as the density value of the sky region in the infrared image (step S36), and the density value of the predetermined pixel below the center is calculated as
Sampling is performed as the density value of the road area in the infrared image (step S37).

In the present embodiment, the predetermined pixels that refer to the density value are set, for example, above and below the center of the infrared image. However, information about the road shape ahead of the host vehicle can be acquired from outside using a navigation device or the like. In such a case, the position of the pixel to be referred to may be appropriately changed according to the information.

Step S38: Step S39 to be described later
Then, a difference value between the left and right densities (left end and right end) of the infrared image to be used as the horizontal region determination threshold in step S40 is set.

Step S39: In this step, the image portion of the sky region is removed from the entire region of the infrared image. That is, under the condition that the density is equal to or less than the density value sampled in step S36 and equal to or less than the left and right density threshold value set in step S38, the infrared image input in step S1 is scanned from above the image and , Step S
The differential image calculated in step S34 is scanned from above the image under the condition that the threshold value is equal to or less than the threshold value of the differential cumulative value set in 35. Then, a predetermined density value (for example, black) is set in the image portion of the sky area removed by the scanning.

Step S40: In this step, the image portion of the road region is removed from the entire region of the infrared image.
That is, under the condition that the density is equal to or less than the density value sampled in step S37 and equal to or less than the threshold value of the left and right density set in step S38, the infrared image input in step S1 is scanned from below the image and Under the condition that the differential accumulated value is not more than the threshold value set in step S35, the differential image calculated in step S34 is scanned from below the image. Then, a predetermined density value (for example, black) is added to the image portion of the road area removed by the scanning.
Set.

Step S41: By referring to the density value of the road area sampled in step S37,
The density value B of the image portion of the vehicle included in the infrared image is
B = (sampling density value) × A (In the present embodiment, A
> T1 = 1.0).

Step S42: The infrared image input in step S1 is binarized using the density value B calculated in step S41. The binarized image calculated in this step is an image of a high density portion that is highly likely to be a vehicle.

Step S43: By taking the logical product (AND) of the two images from which the sky area and the road area have been removed in steps S339 and S40 and the binarized image calculated in step S42. A partial image area (edge image area) that is likely to be a contour image of the vehicle (the area 24 in FIG. 7) is obtained.

(Vehicle Detection Processing) FIG. 10 is a flowchart showing details of the vehicle detection processing (step S26 in FIG. 8) of the vehicle certification processing by IS2.

In the figure, step S51, step S52: An infrared image for one frame is input as a new processing object from the infrared camera (step S51), and the density value of the pixel below the center of the infrared image is determined. By sampling, the density value of the road area is obtained as in step S37 in FIG. 9 (step S52).

Step S53: By referring to the density value of the road area sampled in step S52,
The density value D of the image portion of the exhaust pipe region included in the infrared image is estimated by a calculation of D = (sampling density value) × F (F> T2 = 2.0> T1 in the present embodiment).

Step S54: Using the density value D calculated in step S53 as a threshold value, step S5
In step 1, the infrared image input is binarized. The binarized image calculated in this step is an image of a high-density portion (region 23 in FIG. 7) which is likely to be an exhaust pipe, and the position of this binarized image in the entire infrared image Is set as the image position of the exhaust pipe portion.

Here, the density value D used in this step is
Is not sufficiently large, the calculated binarized image becomes an image including the vicinity of the exhaust port of the vehicle (the area 22 in FIG. 7). Therefore, the present embodiment uses the image position of the exhaust pipe as a reference. Is not preferred.

Step S55: In this embodiment, E
The CU 11 stores in advance a correspondence relationship between an angle of view (see FIG. 3), which is an imaging range of the infrared camera, and a distance between pixels constituting an imaging device (for example, a CCD: Charge Coupled Device) of the infrared camera. . Therefore,
In this step, the number of pixels from the lower side of the infrared image to the image position of the exhaust pipe portion is determined based on the image position of the exhaust pipe portion (the image 26 of the exhaust pipe region in FIG. 11) set in step S54. Based on the above-mentioned correspondence relation with respect to the distance, the distance to the other vehicle existing in front of the own vehicle (that is, the position of the other vehicle: the distance ZIR in FIG. 11) is estimated, and this step in the previous control cycle is performed. The change rate per unit time is calculated as the relative speed between the own vehicle and the other vehicle by referring to the distance to the other vehicle estimated in.

Step S56: FIG.
As shown in the table, the correspondence between the distance to the vehicle to be detected and the number of pixels in the infrared image corresponding to the standard vehicle width of the vehicle according to the distance is stored in advance as a lookup table. In this step, by referring to the lookup table in FIG. 12 according to the distance ZIR estimated using the infrared image,
The vehicle width W (the number of pixels) of the vehicle whose exhaust pipe is photographed in the image 26 of the exhaust pipe area is estimated.

When the accuracy of the distance detection based on the infrared image by the IS2 is compared with the accuracy of the distance detection to the obstacle by the LR1, the detection accuracy by the LR1 is superior.

In step S56, the vehicle width W (the number of pixels) is estimated using the distance ZIR calculated based on the infrared image and the lookup table shown in FIG. This method is adopted when a vehicle is detected in an overlapping detection area (hereinafter, a second detection range) based on LR1 and IS2.

In the control cycle up to the last time, the final vehicle specification is performed by the sensor fusion described later (that is, the result of determination as to whether or not the vehicles individually detected by both methods are the same vehicle is obtained. If the distance ZLR to the corresponding vehicle (group) is detected based on the detection result of LR1, the detected vehicle is located in IS2 where the detection areas do not overlap from the second detection range. Single detection area (hereinafter, referred to as
When moving to the (first detection range), the vehicle width W (the number of pixels) is estimated with high accuracy by referring to the look-up table shown in FIG. 12 according to the distance ZLR. Alternatively, when the final vehicle identification by the sensor fusion is performed in the same manner, the grouping processing of the vehicle recognition processing by the LR1 described later (step S75 in FIG. 15).
When the two blocks are combined into one group, a standard vehicle width (the number of pixels) is obtained according to the distance between the blocks (the length corresponding to the distance between the reflectors at the left and right ends of the rear surface of the vehicle body). In advance, the value is set in step S5
6 may be adopted.

Step S57: In this step, the size of the frame (template) to be used for the partial image area calculated in the above-described rough estimation process of the vehicle image is determined.

That is, the ECU 11 provides, as size information relating to the frame to be determined, predetermined ratio information of up, down, left, and right with respect to the image position of the exhaust pipe region (1/2 in the horizontal direction and the vertical direction shown in FIG. 11). 2/7, 5/7) are stored in advance, and this ratio information is set based on a standard value of the exhaust pipe position in the cross-sectional shape of the detection target vehicle (vehicle type). Therefore, in step S57, by referring to the ratio information based on the vehicle width W estimated in step S56, the size of the predetermined ratio in the vertical and horizontal directions with respect to the image position of the exhaust pipe portion set in step S54. (A template 27 indicated by a square in FIG. 11) having

Therefore, according to the present embodiment, when determining the frame size, a large number of positions are determined in accordance with the position of the vehicle image included in the infrared image (that is, the distance to the vehicle) which will be detected in the preceding process. The frame information does not need to be stored in advance for each vehicle type to be detected, and the calculation of the frame size only needs to be performed once for each vehicle type based on the vehicle width W and the ratio information. The optimum frame size to be determined can be quickly determined, and the memory area required for the ECU 11 can be saved.

Step S58: By performing pattern matching using the frame set in step S57 on the partial image region calculated in the above-described vehicle image rough estimation processing, the partial image region of interest is It is determined whether the image is an image representing a vehicle. More specifically, the edge image of the focused partial image area is stored in step S5.
Take the logical product (AND) with the frame set in step 7,
As a result of the calculation, when the overlapping portion between the two images is larger than a predetermined value, it may be determined that the vehicle is a detection target vehicle.

When a plurality of vehicle types are to be detected by the ECU 11, the above ratio information is stored in advance for each vehicle type, and in step S57, one optimal frame size is determined for each vehicle type to be detected. Then, in step S58, pattern matching is sequentially performed on the partial image region of interest using the frame for each vehicle type, and the frame having the largest number of overlapping portions is selected to correspond to the selected frame. It can also be determined that a vehicle of the vehicle type exists in front of the own vehicle.

Steps S59 and S60: A weighting process is performed on the result of the pattern matching in step S59 (step S59), and based on the result, a partial image area of interest (in the current control cycle) is detected. It is determined whether or not this is the vehicle (step S60). Hereinafter, the details of step S59 and step S60 will be described.

In step S59, the vehicle using IS2
Point (weight value) W at time t of both detection results_IS ^(t)
Is the following equation (1): W_IS ^(t) = W_IS ^(t-1) + K_AR* W_AR ^(t) + K_EP* W_EP ^(t) + K_CER* W_CER ^(t) + K_TA* W_TA ^{(t, t-1)} + K_TEP* W_TEP ^{(t, t-1)} -K_IS* e^-1 ・ ・ ・ ・ ・ (1)

However, in the above equation, the score calculated at time t-1: W _IS ^(t-1) , the score related to the aspect ratio at time t: K _AR * W
_AR ^(t) , ・ Point related to position of exhaust pipe at time t: K _EP * W
_EP ^(t) , ・_Point related to ratio of image area of exhaust pipe area / image area of vehicle area: K _CER * W _CER ^{( t)} , ・_Point related to time change of image area of vehicle area: K _TA *
W _TA ^{(t, t-1)} , ・_Point related to time change of exhaust pipe position: K _TEP * W _TEP
^{(t, t-1)} , Coefficient for normal distribution: K _IS * e ^-1 ,

In the present embodiment, the aspect ratio is a value representing the reliability of the calculation result. The aspect ratio is closer to 1 as the reliability is higher, and is closer to 0 as the reliability is lower.

Here, the constant term W _AR ^(t) is a constant related to the aspect ratio of the calculation result at time t, as shown in FIG. 13A, and is the position of the partial image area in the entire infrared image area. It is set according to. When calculating the distance based on an infrared image captured by an infrared camera, this constant generally indicates that the reliability of the image deteriorates as the pixel is located closer to the edge of the infrared image (the pixel at the edge of the angle of view). This is a value for reflecting such characteristics.

The constant term W _EP ^(t) is a constant relating to the position of the exhaust pipe at time t shown in FIG. 13B, and is set according to the image position of the exhaust pipe in the entire infrared image area. Is done.

The constant term W _CER ^(t) relates to the ratio of the image area of the exhaust pipe area / the image area of the vehicle area as shown in FIG. A smaller value is selected as the area of the image portion at the position of the exhaust pipe greatly differs from a predetermined value (1/10 in this embodiment).

The constant term W _TA ^{(t, t-1)} is a constant relating to the time change of the image area of the vehicle area as shown in FIG. 14D, and is one control cycle of the infrared camera (this embodiment). In spite of a small time interval of t = 100 msec in the embodiment, a smaller value is selected as the area of the partial image area of interest largely differs.

The constant term W _TEP ^{(t, t-1)} is a constant relating to the time change of the position of the exhaust pipe as shown in FIG. 14 (e), and is one control cycle of the infrared camera (this embodiment). Then t
Despite the slight time interval of (= 100 msec), a smaller value is selected as the image position of the exhaust pipe region in the partial image region of interest largely differs.

The characteristics shown in FIGS. 13 and 14 are stored in the ECU 11 in advance as a look-up table.

The coefficient K _IS has the following relation: coefficient K _IS = K _AR + K _EP + K _CER + K _TA + K _TEP .

In the certification processing in step S60, when the value of the score (weighted value) W _IS ^(t) calculated in the processing in step S59 is 1 or more, attention is paid in the current control cycle. It is determined that the image of the partial image area represents the detection target vehicle. If it is determined in this step that the vehicle is a vehicle, the value of the score W _IS ^(t) is set to 1 for the convenience of the sensor fusion processing described later.
Set to.

According to the vehicle qualification process based on IS2 described above, the vehicle detection process (FIG. 10) is performed for each vehicle (edge image that is likely to be a vehicle) detected by the vehicle image rough estimation process (FIG. 9). ), The position (relative distance) and speed of those vehicles (step S55 in FIG. 10) and the value (FIG. 10) relating to the reliability of the calculation result.
Step S60) can be calculated with high accuracy.

<Vehicle Certification Processing by LR1>
Regarding the vehicle certification process by R1, FIGS.
This will be described with reference to FIG.

First, an outline of the vehicle recognition process by the LR1 in the present embodiment will be described. In the process of certifying the vehicle by LR1, the detected reflected waves are grouped into a block and a block, in which the reflected waves having a strength greater than a predetermined threshold and whose moving direction and speed are similar are grouped into one block. Of the plurality of blocks, a grouping process of grouping two blocks located in the vicinity as a detection target vehicle into one group, and a certification process of determining the presence and the moving state in units of the detected groups. Be composed.

As a premise of the present embodiment, reflectors (reflectors) are attached near both ends of the rear bumper of the vehicle to be detected, and the vehicle is detected by the LR1 from the reflector. This is performed based on the reflected wave of the detection wave. Hereinafter, the vehicle recognition processing by the LR1 will be described with reference to FIGS.

FIG. 15 shows the ECU 11 in this embodiment.
7 is a flowchart showing a process of certifying a vehicle by LR1 executed by the LR1, and shows a detailed processing procedure of step S12 in FIG.

In FIG. 15, step S71: LR1
3 is scanned, and as a result, based on the reflected wave returning from the obstacle existing in front of the vehicle 100, information on the distance to the obstacle and the existing direction (position) are obtained. Get. The information obtained in this step is a detection result of a state where a plurality of points are discrete in the propagation range of the detection wave shown in FIG. 3 as shown in “output data of laser radar” in FIG. .

Step S72: The distance (position) to the obstacle detected in step S71 and the distance (position) to the obstacle already blocked in step S74 described later in the previous control cycle. It is determined whether the obstacle detected in step S71 in the current control cycle has already been blocked.

Step S73: Since the block has been made in the judgment of step S72, a block whose intensity of the reflected wave from the target obstacle is larger than a predetermined value is selected and collected as one block.

Step S74: Since the block is not yet formed by the judgment in step S72, the moving direction and the speed of the obstacle detected in step S71 are changed from the reflected wave having the intensity larger than the predetermined threshold value. Blocking (block labeling) is performed to combine approximations into one block. Thus, the “laser radar output data” in FIG. 16A is grouped into a plurality of blocks as shown in “block labeling” in FIG. 16B. At this time, the standard for combining the output data of the laser radar as one block is set based on the size of a standard reflector.

Step S75: Of the plurality of blocks obtained in step S74 or the blocks obtained in step S73, two blocks having substantially equal relative speeds and located close to each other are regarded as one vehicle to be detected.
Perform grouping processing (group labeling) to combine into two groups. As a result, the blocks shown in “block labeling” in FIG. 16B are grouped into a plurality of groups as shown in “group labeling” in FIG. 16C. According to this step, for example, by calculating the average value of the detection results of the LR1 forming the two blocks referred to to form one group, the relative value to the own vehicle is determined for each group (vehicle). A great positional relationship (distance) is obtained.

When the vehicle detected in the single first detection range by IS2 moves to the second detection range overlapped by LR1 and IS2, the vehicle is already detected by IS2 in the previous control cycle. If the vehicle (group) is used, the blocking process described in step S75 for the vehicle can be simplified. That is, based on the vehicle position (distance) based on IS2 and the vehicle width data based on IS2,
It is sufficient to estimate an area where the reflected wave data from the reflector of the same vehicle existing in the detection range of LR1 should exist, and to combine two blocks existing in the estimated area into one group. However, this processing is based on IS2
Is not performed when the reliability of the vehicle width data is low (for example, when the road surface temperature is relatively high such as in the evening). In this case, the detection result by IS2 for the vehicle that has moved from the second detection range to the first detection range may be stored for a predetermined time. As a result, it is possible to quickly recognize the interrupted vehicle ahead of the own vehicle.

Step S76: For each detected group, a tracking process for detecting the moving state of the vehicle (group) detected by the grouping in step S75 is performed. That is, the speed (relative speed) vector V of the vehicle with respect to the vehicle 100 can be calculated from the position information of the preceding vehicle at time t and the position information of the vehicle at time t-1 (see FIG. 17). By calculating the X-direction component of the obtained speed vector V, the lateral movement speed of the vehicle is obtained.

That is, the lateral speed of another vehicle existing ahead of the own vehicle is as follows: Smoothing distance Xsk = Predicted distance Xpk + α (Detected distance Xk−Predicted distance Xpk) (2) Vsk = previous lateral movement speed Vsk−1 + β / sampling period T (detection distance Xk−predicted distance Xpk) (3), • predicted distance Xpk + 1 = smoothed distance Xsk + sampling period T × lateral movement Speed Vsk (4) can be represented by the following mathematical formula.

Here, in the above equations (2) to (4) relating to the speed of the other vehicle, α is a parameter indicating the ratio of the detected value and the predicted value to be combined with respect to the movement amount. β is a parameter indicating the ratio at which the images are combined to determine the next predicted distance.
In addition, the vertical speed can be calculated in the same manner by using an expression that replaces x with y in each of the above expressions.

Therefore, the output signal from LR1 is
Vehicles (groups) detected by grouping such tracking processes as long as they are input in time series to 1.
By performing each time, the behavior (position and speed) of those vehicles in the next control cycle can be accurately predicted. The result of the behavior prediction in the next control cycle is based on the fact that the vehicle specified in the sensor fusion processing described later moves from the second detection range to the first detection range, and the detection result by LR1 using the actually measured value is obtained. Even when it is not required, it is used when continuing sensor fusion (details will be described later).

Steps S77 and S78: A weighting process is performed on the result of the tracking process in step S76 for each detected group (step S77), and based on the result, attention is paid to the present (in the current control cycle). Each vehicle (one attention group)
Is recognized as a vehicle to be detected (step S
78). Hereinafter, the details of step S77 and step S78 will be described.

In step S77, the point (weight value) W _LR ^(t) at time t of the vehicle detection result by LR1 is obtained.
Is given by the following equation (5): W _LR ^(t) = W _LR ^(t-1) + K _BL * W _BL ^(t) + K _DB * W _DB ^(t) + K _TDR * W _TDR ^{(t, t _{-1) + K E PR * W}} EPR (t) - K BL - (K DB + K TDR + K EPR) * e -1 ····· (5), it can be determined by.

However, in the above formula, the score calculated at time t-1: W _LR ^(t-1) , the score related to the number of blocks detected for the group of interest: K _BL * W _BL ^(t) , Points related to the distance between the two blocks forming the group: K _DB * W _DB ^(t) , ・ Points related to the change in distance between the two blocks forming the group of interest: K _TDR * W _TDR ^{( t, t-1)} , Points related to the difference between the predicted position and the detected position of the target group: K _EPR * W _EPR ^(t) , ・ Coefficient for normal distribution: K _BL- (K _DB + K _TDR +
K _EPR ) * e ^-1 .

Here, the constant term W _BL ^(t) is a constant related to the number of blocks detected for the group of interest as shown in FIG.
Since it is assumed that two reflectors are attached, when the number of detected blocks (reflectors) is 2, 1 is selected as the constant.

The constant term W _DB ^(t) is a constant related to the distance between the two blocks forming the group of interest, as shown in FIG. 18B, and the distance between the blocks (reflectors) in the vehicle to be detected. The closer the distance to the standard distance between the two reflectors (1.2 m in this embodiment), the more the constant becomes 1.
Is selected.

The constant term W _TDR ^{(t, t-1)} is a constant related to a change in the distance between two blocks forming the group of interest, as shown in FIG. In spite of a small time interval, a smaller value is selected as the distance between the blocks (reflectors) greatly differs.

The constant term W _EPR ^(t) is a constant relating to the difference between the predicted position and the detected position of the target group, as shown in FIG. 19D, and is obtained by using the above equations (2) to (4). The predicted position calculated in step S76 and
As the difference from the actual detection position obtained at the time of grouping is larger, a smaller value is selected.

The characteristics shown in FIGS. 18 and 19 are stored in the ECU 11 in advance as a look-up table.

In the certification processing in step S78, when the value of the score (weighting value) W _LR ^(t) calculated in the processing in step S77 is 1 or more, the target group represents the vehicle to be detected. Certified. If it is determined that the vehicle is a vehicle in this step, the score W _{L R}
Set the value of ^(t) to 1.

According to the vehicle recognition processing by LR1 described above, the vehicle detection processing (FIG. 15) is performed, so that the position (relative distance) and speed (steps S75 and FIG. Step S76)
And a value relating to the reliability of the calculation result (step S78 in FIG. 15).

<Vehicle Recognition Processing by Sensor Fusion> In each of the above-described steps, the detection result of the vehicle recognition processing using LR1 and IS2 and the value related to its reliability have been obtained, and the sensor fusion based on those values has been performed. The vehicle recognition process will be described with reference to FIGS.

First, a vehicle recognition process by the sensor fusion according to the present embodiment will be outlined. In this recognition process, a value representing the correlation between the position and the speed of each object that has been recognized as a vehicle by the above-described vehicle recognition process using LR1 and IS2 is calculated, and a value representing the correlation and LR1 When the sum of the values of the position and speed detected by the vehicle certification process by IS and IS2 is larger than a predetermined threshold, those objects are formally recognized as one vehicle to be monitored. Perform processing.

(Sensor Fusion Based on LR1 and IS2 Detection Results) Hereinafter, a vehicle recognition process by sensor fusion will be described with reference to FIGS. 20 and 21. In the present embodiment, the evaluation of the result of the vehicle qualification process using LR1 and IS2 by sensor fusion is performed in the coordinate system of the infrared image.

However, since the distance estimation based on IS2 is a method using an optical system and an image pickup device, the reliability of the left and right ends is inferior to the central part of the infrared image, and the distance from the own vehicle in a direction farther away. It is generally known that the reliability in the (Z direction) gradually decreases after a certain distance. Further, when a part of the object to be detected deviates from the angle of view, accurate detection cannot be performed. Therefore, as shown in FIG. 20, with respect to IS2, the definition of reliability in the X direction (horizontal direction) (FIG. 20 (a)) and the definition of reliability in the Z direction (vertical direction) (FIG. 20 (b)) And ECU11
, The reliability of the distance estimation by IS2 is defined.

On the other hand, the distance estimation by LR1 has a longer detection range and higher reliability of the detection result as compared with the case of using IS2, but has a narrower detection range in the left-right direction. Therefore, as shown in FIG. 21, regarding LR1, the definition of reliability in the X direction (horizontal direction) (FIG. 21A) and the definition of reliability in the Z direction (vertical direction) (FIG. 21B) And ECU11
, The reliability of distance estimation by LR1 is defined.

Next, the sensor fusion algorithm will be described.

As shown with reference to FIG. 3, since the detection range of IS2 is set to include the detection range of LR1, the detection areas of the two types of detection methods overlap. There is an undetected first detection range and an overlapping second detection range. On the other hand, the basic concept of sensor fusion is based on the premise that there are a plurality of detection results for the same detection target. Therefore, in the present embodiment, basically, the actual measurement values by LR1 and IS2 are used. Sensor fusion is performed when there are two types of detection results based on.

(Sensor fusion of detection results by LR1 and IS2) When the detection results by LR1 and IS2 exist, in sensor fusion, whether the object detected by the two types of sensors is really a vehicle, or You must determine if it is due to some noise.

Here, when a vehicle is present in the second detection range, the reliability score (W _IS , W _LR ) detected by LR1 and IS2 is high. The vehicle detected by LR1 and IS2 There is a feature that the degree of correlation between the position and the speed is high, and that the sensor responds to both the LR1 and IS2 sensors for a relatively long time.

When noise is present in the second detection range, the reliability detected by LR1 and IS2 (W _IS ,
W _LR ) is low, the degree of correlation between the position and the speed of the vehicle detected by LR1 and IS2 is low, and the time for responding to both the LR1 and IS2 sensors is relatively short.

Therefore, in the present embodiment, the sensor fusion score S _F is obtained by formulating the sensor fusion formula based on the characteristics in the second detection range as the following formula (6). .

[0123]

(Equation 1)

The coefficients θ _IS () and θ _LR () will be described later with reference to equations (13) and (14).

[0125] Then, when the number S _F sensor fusion calculated by the above equation (6) is 1 or more is recognized as "a vehicle". At this time, the degree of correlation between the position and the speed calculated based on the time-series data,
Alternatively, if either the score W _IS or W _LR of the reliability of the detection result by LR1 or IS2 is a score close to a perfect score, even if the other value is a relatively low score, it is “vehicle”. Certified.

Here, in the relational expression representing the degree of correlation between the position and the velocity in the first term of the equation (6), f () represents the degree of correlation of the position, and g () represents the degree of correlation of the velocity. 0 ≦ f ()
≦ 1, 0 ≦ g () ≦ 1. Specific equations of these functions are shown in the following equations (7) and (8).

[0127]

(Equation 2)

Here, ω _X () and ω _Z () are functions of relative reliability in the X direction (lateral direction of the own vehicle) and the Z direction (forward direction of the own vehicle) described below. ω _X () + ω _Z () = 1 is satisfied. L _X () and L _Z () are limit deviations in the X and Z directions, and are both functions of the distances ZIS and ZLR. Furthermore,
T is a sampling time.

For example, when the vehicle (preceding vehicle) is farther from the host vehicle, the detection accuracy of IS2 is lower than that of LR1, and the deviation of the position in the Z direction detected by the two sensors becomes larger. . Therefore, in such a case,
Adjust so that the value of ω _Z () is lower than ω _X (). Further, when the vehicle (preceding vehicle) exists in the close distance of the own vehicle in the Z direction, the adjustment is performed so that ω _X ()> ω _Z (). As a result, changes in the surrounding environment can be reflected on the accuracy (reliability) of the recognition result.

Based on the same concept, when the vehicle (preceding vehicle) is present on the left and right with respect to the front of the own vehicle (in the X direction), adjustment is made so that ω _X () <ω _Z (). Also, since when the vehicle (preceding vehicle) is present in a relatively short distance position ahead of the host vehicle (on the Z axis) can be regarded to be the same reliability of the detection results by any of the sensors, omega _X ( ) = Ω _Z ().

In general, the result of detection by a laser radar is easily affected by weather such as rain or fog. Thus, in the present embodiment, when performing the sensor fusion as described above, the ECU 11 detects L when the bad weather is detected.
The reliability of the detection result by R1 is adjusted to be lower than that in fine weather. At this time, detection of the weather around the vehicle
The determination may be made based on the operation state of the wiper switch 6 (if the switch is on, it is assumed that the weather is bad) and the detection result of the illuminance sensor 8.

In general, the result of detection by an infrared camera is easily affected when the host vehicle travels in the direction of sunlight that is greatly inclined in summer or evening when the road surface temperature is high, or in the evening. In particular, in this embodiment, since the mounting position of the IS2 is lowered in order to facilitate the photographing of the exhaust pipe position of the vehicle existing in front, some countermeasures are required. Therefore, when performing sensor fusion as described above, the ECU 11 refers to, for example, time information obtained via the GPS sensor 10, traveling direction and calendar information obtained from a navigation unit (not shown), and detection results of the illuminance sensor 8. By doing so, it is estimated whether or not the driving environment is such, and if it is determined that the detection result is a driving environment that is easily affected, the reliability of the detection result by IS2 is compared with the case where it is not. Adjust lower. As a result, changes in the surrounding environment can be reflected on the accuracy (reliability) of the recognition result.

Reliability functions ω _X (), ω _Z (), L _X (), L _Z ()
Is formulated as in the following equations (9) to (12).

[0134]

(Equation 3)

Φ _f () and φ _g () are coefficients, which are functions that vary depending on the position of the vehicle (preceding vehicle), and 0 ≦ φ
_f () ≦ 1, 0 ≦ φ _g () ≦ 1
The relationship of φ _f () + φ _g () = 1 is satisfied. The specific adjustment tendency of the coefficients φ _f () and φ _g () by the ECU 11 is as follows: φ _f () <φ _g (): That is, the closer the vehicle (preceding vehicle) is to the own vehicle, the higher the speed becomes. Put specific gravity (IS2 is given priority). Φ _f ()> φ _g (): That is, the more the vehicle (preceding vehicle) is farther from the host vehicle, the greater the specific gravity at the position (LR1 is given priority).

Similarly, the actual parameters are LR
1 and IS2 reliability functions in the X and Z directions are used.

In the following, φ _f () and φ _g () are formulated.

[0138]

(Equation 4)

In the expression representing the reliability of LR1 and IS2 in the second term of equation (6), W _IS is the reliability score in IS2, W _LR is the reliability score in LR1,
Taking values in the range of 0 ≦ W _IS ≦ 1, 0 ≦ W _LR ≦ 1. Also,
θ _IS and θ _LR are coefficients that are functions that vary depending on the position of the vehicle, and take values in a range of 0 ≦ θ _IS ≦ 1, 0 ≦ θ _LR ≦ 1, and satisfy θ _IS + θ _LR = 1.

The specific adjustment tendency of the coefficients θ _IS and θ _{LR by} the ECU 11 is as follows: the coefficient θ _IS <θ _LR : that is, basically, LR1 is larger than IS2.
(LR1 is given priority). Coefficient θ _IS << θ _LR : However, as the vehicle (preceding vehicle) is farther from the host vehicle, a larger specific gravity is placed on LR1.

These actual parameters are similarly L
The X and Z reliability functions of R1 and IS2 are used.

The following formulas (13) and (14) are used to formulate θ _IR () and θ _LR ().

[0143]

(Equation 5)

According to the sensor fusion of the detection results by LR1 and IS2 in this embodiment, a more accurate vehicle position can be specified. However, LR1 and I
In the second detection range where the detection ranges by S2 overlap, actually, the position of the preceding vehicle existing in the front-rear direction (Z direction) of the own vehicle can be specified only by the detection result by LR1.
The position of the preceding vehicle existing in the direction can be determined by sensor fusion using the following equations (15) to (16).

X _SF = η _ISX () × XIS + η _LRX () × XLR (15) where, (15) is expressed by: (X coordinate (SF)) = (IS2 for LR1) Of X in the X direction) × (X coordinate (IS2)) + (L for IS2
(Reliability of R1 in X direction) × (X coordinate (LR1)).

Z _SF = η _ISZ () × ZIS + η _LRZ × ZLR (16) where Expression (16) is expressed as: (Z coordinate (SF)) = (Z of IS2 with respect to LR1 Reliability in direction) x (Z coordinate (IS2)) + (L for IS2
R1 reliability in the Z direction) × (Z coordinate (LR1)).

In equations (15) and (16), 0 ≦
η _ISX () ≦ 1,0 ≦ η _LRX () ≦ 1,0 ≦ η _ISZ () ≦
Take the value of 1,0 ≦ η _LRZ () ≦ 1, η _ISX () + η _LRX
() = 1, η _ISZ () + η _LRZ () = 1 are satisfied.

The specific adjustment tendency of these coefficients by the ECU 11 is as follows: η _ISX () = η _LRX (): In the X direction, basically, the same specific gravity is placed on LR1 and IS2. • η _ISX ()> η _LRX (): However, the closer the distance to the both ends of the infrared image is, the greater the specific gravity is given to IS2 (IS2 is given priority). • η _ISZ () <η _LRZ (): Basically I in the Z direction
A greater specific gravity is placed on LR1 than on S2 (LR1 has priority). Η _ISZ () << η _LRZ (): However, as the vehicle (preceding vehicle) is farther from the _host vehicle, the specific gravity is set higher in LR1.

The following formulas (17) and (20) are used to formulate η _ISX (), η _LRX (), η _ISZ (), and η _LRZ ().

[0150]

(Equation 6)

As described above, according to the above-described sensor fusion, in which the correlation is calculated based on the position and speed of the vehicle individually detected by the LR1 and IS2 and the reliability of the detection result, the object to be monitored is high. It can be recognized with high accuracy and precision.

<When other vehicle (preceding vehicle) exists in first detection range> The above-described procedure of sensor fusion is performed when there is a detection result based on the actually measured values by LR1 and IS2 as described above. Basic. For this reason, when another vehicle (preceding vehicle) exists in the first detection range, the detection by LR1 cannot be performed, and thus the above-described sensor fusion cannot be performed.

Therefore, in the present embodiment, when another vehicle (preceding vehicle) exists in the first detection range, the following two processes are performed.

(Sensor fusion based only on IS2 detection result) Sensor fusion based only on the IS2 detection result is performed using the evaluation function shown in equation (21).

[0155]

(Equation 7)

However, in equation (21), (point of SF)
= (Function of reliability of IS2 depending on position) x (point of IS2). As the parameters actually used by the ECU 11, the functions of the reliability of the IS2 in the X and Z directions shown in FIG. 20 are used.

The equation (22) indicates that the vehicle score is given by IS2.
By multiplying by a coefficient that varies depending on the position of the (preceding vehicle), the possibility that another vehicle exists in front of the own vehicle is formulated. The fifth parameter V in this equation (22)
Is a lateral (X direction) vector of the vehicle (preceding vehicle),
If this speed vector is directed substantially toward the center of the infrared screen for a predetermined period of time, by setting θ ′ _IR () = 1, it is recognized that the vehicle jumps out ahead of the host vehicle. This means that scores based on IS2 are purely evaluated.

As described above, when the detection result based on the actual measurement values by LR1 and IS2 cannot be obtained in the first detection range, the lateral vector of the vehicle (preceding vehicle) is displayed on the infrared screen for a predetermined time as described above. By performing sensor fusion based on only the detection result of IS2 described above only when the vehicle is facing substantially the center, it is possible to quickly detect a vehicle that has jumped ahead of the host vehicle at an early stage.

(Continuation of Sensor Fusion Using Vehicle Behavior Prediction Information by Tracking Process)
While performing the vehicle certification process 1 (FIG. 15), the second
The vehicle moves from the detection range to the first detection range, and L
If the measured data from R1 is no longer input,
The LR1 vehicle qualification process (FIG. 15) for a predetermined time (for example, about 1 second) using the vehicle behavior prediction information performed in the tracking process (step S76 in FIG. 15) when another vehicle is present in the second detection range. ) To continue.
Here, the reason why the continuation of the LR1 vehicle qualification process using the behavior prediction information is limited to the above-described predetermined time is that the longer the duration of the LR1 vehicle qualification process using the vehicle behavior prediction information is, the more reliable the calculation result becomes. This is because the sex becomes lower.

As described above, if the vehicle qualification process of LR1 (FIG. 15) is continued using the vehicle behavior prediction information, the information necessary for sensor fusion based on the detection results by LR1 and IS2 described above is obtained. Since they are aligned, such sensor fusion can be continued. Where L
Since the detection result by R1 is easily affected by the influence of raindrops scattered by the preceding vehicle as in the case of rainy weather, etc., the continuation processing using the behavior prediction information is not performed in bad weather. Better.

In the present embodiment described above, the result of vehicle recognition calculated based on the detection results of LR1 and IS2 is applied to the warning / control process (FIG. 4) for issuing an inter-vehicle distance warning as an example. The vehicle recognition processing by the sensor fusion in this embodiment (FIG. 6) is not limited to such a system configuration, and is widely used for recognition of another vehicle by another warning system, an automatic driving system, a driving support device, or the like. It is suitable.

Further, in the present embodiment, as an example, the case where another vehicle traveling in the same direction as the own vehicle, which is located ahead of the own vehicle, is detected. Although the reflected waves from the two reflectors attached to the vehicle were used, the present invention is not limited to this processing configuration. When a reflector is provided at the front of the vehicle, or when there is a portion having the same reflectance as the reflector, The oncoming vehicle may be detected by performing the same procedure as in the above-described embodiment using reflected waves from the oncoming vehicles.

[Brief description of the drawings]

FIG. 1 is a block diagram illustrating a device configuration of a vehicle equipped with an alarm system according to an embodiment.

FIG. 2 is a diagram illustrating the definition of XYZ coordinates in the present embodiment.

FIG. 3 is a diagram showing detection ranges of LR1 and IS2 in the present embodiment.

FIG. 4 is a diagram illustrating a flowchart of a warning / control process executed by an ECU 11 in the present embodiment.

FIG. 5 is a diagram exemplifying a criterion referred to by an ECU 11 in an alarm / control process.

FIG. 6 is a flowchart illustrating a vehicle recognition process executed by an ECU 11 according to the present embodiment.

FIG. 7 is a diagram illustrating an infrared image of a vehicle existing in front of the host vehicle;

FIG. 8 shows an IS executed by the ECU 11 in the embodiment.
6 is a flowchart showing a vehicle qualification process according to the second embodiment.

9 is a flowchart showing details of a vehicle image rough estimation process (step S25 in FIG. 8) in the vehicle certification process by IS2.

FIG. 10 is a flowchart showing details of a vehicle detection process (step S26 in FIG. 8) in the vehicle certification process by IS2.

FIG. 11 is a diagram illustrating a method of estimating a distance ZIS using an infrared image and a method of determining a frame size set based on an image of an exhaust pipe region.

FIG. 12 is a diagram showing a characteristic curve representing a relationship between a distance ZIS and a vehicle width W.

FIG. 13 is a diagram showing a characteristic curve referred to by the ECU 11 for determining a constant term in a weighting process of a detection result by IS2.

FIG. 14 is a diagram showing a characteristic curve referred to by the ECU 11 for determining a constant term in a weighting process of a detection result by IS2.

FIG. 15 illustrates an example of an L executed by the ECU 11 according to the embodiment.
It is a flowchart which shows the recognition process of the vehicle by R1.

FIG. 16 is a diagram illustrating a procedure up to grouping in a vehicle certification process by LR1.

FIG. 17 is a diagram illustrating a lateral movement speed of a vehicle in the tracking processing.

FIG. 18 is a diagram showing a characteristic curve referred to by the ECU 11 for determining a constant term in the weighting process of the detection result by the LR1.

FIG. 19 is a diagram showing a characteristic curve referred to by the ECU 11 for determining a constant term in the weighting process of the detection result by the LR1.

FIG. 20 is a diagram showing a definition of reliability of a detection result in the X direction and the Z direction of IS2.

FIG. 21 is a diagram illustrating a definition of reliability of a detection result of the LR1 in the X direction and the Z direction.

[Explanation of symbols]

 1: distance sensor (LR), 2: image sensor (IS), 3: brake actuator, 4: throttle actuator, 5: speaker, 6: wiper switch, 7: display device, 8: illuminance sensor, 9: vehicle speed sensor, 10: GPS sensor, 11: electronic control unit (ECU), 100: vehicle,

──────────────────────────────────────────────────続 き Continued on the front page (51) Int.Cl. ⁷ Identification symbol FI Theme coat ゛ (Reference) G01C 3/06 G01C 3/06 Z 5J084 G06T 7/00 300 G06T 7/00 300D 5L096 7/60 150 7 / 60 150J 180 180B G08G 1/16 G08G 1/16 CE // G01S 17/93 G01S 17/88 A (72) Inventor Takehiko Fujioka 7-3-1 Hongo, Bunkyo-ku, Tokyo Faculty of Engineering, The University of Tokyo F Term (reference) 2F065 AA04 AA06 AA22 BB05 BB15 CC11 DD04 DD11 FF01 FF04 FF09 JJ03 JJ08 JJ26 MM02 MM06 QQ04 QQ23 QQ25 QQ36 QQ39 QQ51 RR06 RR08 SS09 SS13 UU05 2F112 AD05 BA06 FA07 A04 FA32 A AC59 AE03 3G093 AA01 BA27 FA01 5H180 AA01 CC02 CC03 CC04 CC12 CC14 FF05 LL01 LL04 LL07 LL09 5J08 4 AA02 AA05 AB01 AC02 BA03 5L096 AA06 BA04 CA27 FA05 FA14 FA54 FA64 FA66 GA17 HA09 JA09

Claims (4)

[Claims] 1. An infrared camera for imaging a predetermined imaging range using infrared light, and an infrared camera included in the infrared image based on an infrared image captured by the infrared camera. Detecting a partial image region estimated to be a vehicle to perform, and detecting a distance from the vehicle based on an image position of an exhaust pipe portion photographed with higher luminance than a predetermined luminance included in the detected partial image region. A vehicle detection device comprising: 2. The image processing apparatus according to claim 1, wherein the detecting unit sets a frame having a predetermined ratio of size vertically, horizontally, and vertically with respect to the image position of the exhaust pipe portion, and uses the frame to set the partial image area. 2. The vehicle detection device according to claim 1, wherein pattern matching is performed on the partial image region to determine whether the partial image region is an image representing a vehicle. 3. The detecting means stores in advance a correspondence relationship between an angle of view as the imaging range and a pixel constituting an imaging device of the infrared camera. The vehicle detection device according to claim 1, wherein the distance to the vehicle is detected based on a number of pixels from a lower side of an outer image to an image position of the exhaust pipe portion and a correspondence relation regarding the distance. 4. The method according to claim 1, wherein the detecting means includes a correspondence between a distance to the vehicle to be detected and a number of pixels in the infrared image corresponding to a standard vehicle width of the vehicle according to the distance. The relationship is stored in advance, and the detection unit estimates the vehicle width of the vehicle based on the distance detected by the infrared image and the correspondence relationship regarding the vehicle width. The vehicle detection device according to any one of the preceding claims.