JP4871941B2 - Vehicle periphery monitoring device - Google Patents

Description

  The present invention relates to a vehicle periphery monitoring apparatus that extracts an object by binarization processing of an image photographed by an infrared camera.

Conventionally, an object that appears to be a pedestrian is extracted by binarizing an image of the periphery of the vehicle that is captured by an infrared camera provided on the own vehicle such as an automobile, and the extracted object and the own vehicle There is known a vehicle periphery monitoring device that notifies a driver of this information when there is a high possibility of collision.
In recent years, in a vehicle periphery monitoring apparatus that extracts pedestrians as objects using two infrared cameras, a search area is set above a high-brightness object such as a taillight of a vehicle and a high-brightness object. When the object having the same distance as is searched in the search area and the average brightness of the search area is smaller than the specified value, it is determined that the object in the search area is a windshield and the object is a vehicle. Have been proposed (see, for example, Patent Document 1).
JP 2003-230134 A

However, in the conventional vehicle periphery monitoring device described above, if there is a low-luminance object around the upper portion of the high-luminance object, there is a possibility that an object other than the vehicle is erroneously determined as a vehicle.
In addition, since the windshield of a vehicle is generally made of glass, it is easy to reflect infrared rays, and there may be a situation where there is no part with low brightness when reflecting the sky, clouds, surrounding scenery, etc. There exists a subject that there exists a possibility that a vehicle cannot be excluded from a target object.

  The present invention has been made in view of the above circumstances, and it is possible to more surely eliminate only the vehicle from the object included in the image captured by the infrared camera and further improve the accuracy of pedestrian recognition. A vehicle periphery monitoring device is provided.

In order to solve the above-described problem, the invention described in claim 1 is based on a first heat source that is considered to be a characteristic part of a vehicle from an image captured by an infrared camera (for example, the infrared cameras 2R and 2L in the embodiment). Feature extraction means (for example, steps S41 to S43 in the embodiment) for extracting as an object (for example, OBJ [1], OBJ [2] in the embodiment) and an area around the first object ( for example, a window candidate region a) in the embodiment, by comparing the distance between the first object, a second object distance between the first object to extract the second object falls within a predetermined range object extracting means (e.g., step S44~S46 in the embodiment) and, the amount of infrared change detecting means for detecting a temporal change in amount of infrared rays that are generated from the second object (e.g., the scan in the embodiment And-up S50), the amount of infrared rays temporal change that is generated from the second object detected by the infrared amount change detecting means, the amount of infrared radiation judgment judged whether or not the condition given in amount of infrared rays By means (for example, steps S51 and S52 in the embodiment) and the infrared amount determination means , it is determined that the temporal change in the amount of infrared rays generated from the second object satisfies the condition of a predetermined infrared amount . In this case, vehicle determining means (for example, step S53 in the embodiment) for determining that the object including the first object and the second object is a vehicle is provided.

According to a second aspect of the present invention, there is provided a determination unit that determines whether or not an external environment in which reflection in a vehicle window is likely to occur , and the infrared amount determination unit is moved into the vehicle window by the determination unit. The condition of the predetermined infrared light amount is changed between the case where it is determined that the external environment is likely to occur and the case where the determination means determines that the external environment is not likely to move into the window of the vehicle. Te, temporal change in amount of infrared rays that are generated from the second object is characterized that you determine whether or not the condition of the predetermined amount of infrared rays.
The invention described in claim 3 is characterized in that the external environment is a sky pattern.
The invention described in claim 4 is characterized in that the external environment is a situation of an object in the vicinity of a traveling path of a vehicle.

According to the first aspect of the present invention, the second object extraction means extracts the second object whose distance from the first object extracted by the feature extraction means is within a predetermined range, and the second object Since the second object is likely to be glass such as a windshield of a vehicle when the temporal change in the amount of infrared light of the object is a predetermined infrared light amount condition, the first object and the second object are It can be determined that the containing object is a vehicle.
Therefore, when the windshield whose distance from the first object is within a predetermined range reflects the surrounding scenery and the width of the temporal change in the amount of infrared rays increases, it is determined that the object is a vehicle or clear The object can be determined to be a vehicle when the sky of time or the sky of the clouds is reflected and there is no temporal change in the amount of infrared rays, so it is possible to determine that the object is a vehicle. There is an effect that can contribute to improvement of recognition accuracy.

The first embodiment of the present invention will be described below with reference to the drawings.
FIG. 1 is a block diagram showing the configuration of the vehicle periphery monitoring device according to the first embodiment of the present invention.
In FIG. 1, reference numeral 1 denotes an image processing unit including a CPU (central processing unit) that controls the vehicle periphery monitoring device according to the present embodiment, and includes two infrared cameras 2R and 2L capable of detecting far infrared rays. A yaw rate sensor 3 for detecting the yaw rate of the vehicle, a vehicle speed sensor 4 for detecting a traveling speed (vehicle speed) of the vehicle, and a brake sensor 5 for detecting a brake operation are connected. Thus, when the image processing unit 1 detects a moving object such as a pedestrian or an animal in front of the vehicle from an infrared image around the vehicle and a signal indicating the running state of the vehicle, and determines that the possibility of a collision is high Alarm.

  In addition, the image processing unit 1 displays a speaker 6 for issuing a warning by voice and images taken by the infrared cameras 2R and 2L, and makes the vehicle driver recognize an object having a high risk of collision. For example, a meter-integrated display integrated with a meter that expresses the running state of the host vehicle, a NAVID display installed on the console of the host vehicle, and information displayed at a position on the front window that does not obstruct the driver's front view An image display device 7 including a HUD (Head Up Display) 7a is connected.

  The image processing unit 1 includes an A / D conversion circuit that converts an input analog signal into a digital signal, an image memory that stores a digitized image signal, a CPU (central processing unit) that performs various arithmetic processes, RAM (Random Access Memory) used to store data, ROM (Read Only Memory) that stores programs and tables executed by the CPU, maps, etc., speaker 6 drive signals, display signals such as HUD7a, etc. The output signals of the infrared cameras 2R and 2L, the yaw rate sensor 3, the vehicle speed sensor 4 and the brake sensor 5 are converted into digital signals and input to the CPU.

As shown in FIG. 2, the infrared cameras 2R and 2L are disposed in the front part of the host vehicle 10 at positions that are substantially symmetrical with respect to the center of the host vehicle 10 in the vehicle width direction. The optical axes of 2R and 2L are parallel to each other and are fixed so that the heights from both road surfaces are equal. The infrared cameras 2R and 2L have a characteristic that the output signal level increases (the luminance increases) as the temperature of the object increases.
The HUD 7a is provided so that the display screen is displayed at a position that does not obstruct the driver's front view of the front window of the host vehicle 10.

Next, the operation of the present embodiment will be described with reference to the drawings.
FIG. 3 is a flowchart showing an object detection / alarm operation of a pedestrian or the like in the image processing unit 1 of the vehicle periphery monitoring device of the present embodiment.
First, the image processing unit 1 acquires infrared images, which are output signals of the infrared cameras 2R and 2L (step S1), performs A / D conversion (step S2), and stores a grayscale image in an image memory (step S1). S3). Here, the right image is obtained by the infrared camera 2R, and the left image is obtained by the infrared camera 2L. In addition, since the horizontal position of the same object on the display screen is shifted in the right image and the left image, the distance to the object can be calculated from this shift (parallax).

If a grayscale image is obtained in step S3, then the right image obtained by the infrared camera 2R is used as a reference image, and binarization processing of the image signal, that is, an area brighter than the luminance threshold value ITH is “1” ( White) and the dark area is set to “0” (black) (step S4).
FIG. 4A shows a grayscale image obtained by the infrared camera 2R, and binarization processing is performed on the grayscale image to obtain an image as shown in FIG. 4B. In FIG. 4B, for example, an object surrounded by a frame from P1 to P4 is an object displayed as white on the display screen (hereinafter referred to as “high luminance region”).
When the binarized image data is acquired from the infrared image, the binarized image data is converted into run-length data (step S5). The line represented by the run-length data is an area that is whitened by binarization at the pixel level, and has a width of one pixel in the y direction, and each has a width in the x direction. It has the length of the pixels constituting the run-length data.

Next, the target object is labeled from the image data converted into run-length data (step S6), thereby performing a process of extracting the target object (step S7). That is, of the lines converted into run length data, a line with a portion overlapping in the y direction is regarded as one object, so that, for example, the high luminance area shown in FIG. Things) will be grasped as 1 to 4.
When the extraction of the object is completed, next, the center of gravity G, the area S, and the aspect ratio ASPECT of the circumscribed rectangle of the extracted object are calculated (step S8).

Here, the area S is (x [i], y [i], run [i], a) (i = 0, 1, 2,... N−1) ), The length of the run length data (run [i] −1) is calculated for the same object (N pieces of run length data). The coordinates (xc, yc) of the center of gravity G of the object a are the length (run [i] -1) of each run-length data and the coordinates x [i] or y [i] of each run-length data. Are multiplied by the area S and calculated by multiplying them by the area S.
Further, the aspect ratio ASPECT is calculated as a ratio Dy / Dx between the length Dy in the vertical direction and the length Dx in the horizontal direction of the circumscribed rectangle of the object.
Since the run length data is indicated by the number of pixels (number of coordinates) (= run [i]), the actual length needs to be “−1” (= run [i] −1). Further, the position of the center of gravity G may be substituted by the position of the center of gravity of the circumscribed rectangle.

Once the center of gravity, area, and circumscribing aspect ratio of the object can be calculated, next, the object is tracked between times, that is, the same object is recognized for each sampling period (step S9). In the tracking between times, the time obtained by discretizing the time t as an analog quantity with the sampling period is set as k. For example, if the objects a and b are extracted at the time k, the objects c and d extracted at the time (k + 1) and the target The identity determination with the objects a and b is performed. When it is determined that the objects a and b and the objects c and d are the same, the objects c and d are changed to labels of the objects a and b, respectively, so that tracking is performed for the time.
Further, the position coordinates (center of gravity) of each object recognized in this way are stored in the memory as time-series position data and used for the subsequent calculation processing.

Note that the processes in steps S4 to S9 described above are performed on a binarized reference image (in this embodiment, a right image).
Next, the vehicle speed VCAR detected by the vehicle speed sensor 4 and the yaw rate YR detected by the yaw rate sensor 3 are read, and the turning angle θr of the host vehicle 10 is calculated by integrating the yaw rate YR over time (step S10).

On the other hand, in parallel with the processing of step S9 and step S10, in steps S11 to S13, processing for calculating the distance z between the object and the host vehicle 10 is performed. Since this calculation requires a longer time than steps S9 and S10, it is executed in a cycle longer than steps S9 and S11 (for example, a cycle that is about three times the execution cycle of steps S1 to S10).
First, by selecting one of the objects tracked by the binarized image of the reference image (right image), the search image R1 (here, the entire region surrounded by the circumscribed rectangle is searched for from the right image). Are extracted (step S11).

Next, a search area for searching for an image corresponding to the search image R1 (hereinafter referred to as “corresponding image”) from the left image is set, and a correlation operation is executed to extract the corresponding image (step S12). Specifically, a search area R2 is set in the left image according to each vertex coordinate of the search image R1, and a luminance difference total value C (a indicating the level of correlation with the search image R1 within the search area R2 , B), and a region where the total value C (a, b) is minimum is extracted as a corresponding image. This correlation calculation is performed using a grayscale image instead of a binarized image.
When there is past position data for the same object, an area R2a narrower than the search area R2 is set as a search area based on the position data.

The search image R1 is extracted from the reference image (right image) and the corresponding image R4 corresponding to the target object is extracted from the left image by the processing in step S12. Next, the search image R1 corresponds to the center of gravity position of the search image R1. The position of the center of gravity of the image R4 and the parallax Δd (number of pixels) are obtained, and the distance z between the vehicle 10 and the object is calculated from the position (step S13). An expression for obtaining the distance z will be described later.
Next, when the calculation of the turning angle θr in step S10 and the distance calculation with the object in step S13 are completed, the coordinates (x, y) and the distance z in the image are changed to real space coordinates (X, Y, Z). Conversion is performed (step S14).
Here, the real space coordinates (X, Y, Z) are, as shown in FIG. 2, with the origin O as the midpoint position (position fixed to the host vehicle 10) of the attachment position of the infrared cameras 2R, 2L. The coordinates in the image are determined as shown in the figure, with the center of the image as the origin and the horizontal direction as x and the vertical direction as y.

When the real space coordinates are obtained, the turning angle correction for correcting the positional deviation on the image due to the turning of the host vehicle 10 is performed (step S15). In the turning angle correction, when the host vehicle 10 turns, for example, to the left by the turning angle θr during the period from time k to (k + 1), the range of the image is shifted in the x direction by Δx on the image obtained by the camera. This is a process for correcting this.
In the following description, the coordinates after the turning angle correction are displayed as (X, Y, Z).

When the turning angle correction with respect to the real space coordinates is completed, N (for example, about N = 10) real space position data after the turning angle correction obtained during the monitoring period of ΔT for the same object, that is, From the time series data, an approximate straight line LMV corresponding to the relative movement vector between the object and the host vehicle 10 is obtained (step S16).
Next, the latest position coordinates P (0) = (X (0), Y (0), Z (0)) and (N-1) position coordinates P (N−1) before the sample (before time ΔT). = (X (N-1), Y (N-1), Z (N-1)) is corrected to a position on the approximate straight line LMV, and the corrected position coordinates Pv (0) = (Xv (0), Yv (0), Zv (0)) and Pv (N-1) = (Xv (N-1), Yv (N-1), Zv (N-1)) are obtained.

Thereby, a relative movement vector is obtained as a vector from the position coordinates Pv (N−1) to Pv (0).
In this way, by calculating an approximate straight line that approximates the relative movement locus of the object with respect to the host vehicle 10 from a plurality (N) of data within the monitoring period ΔT, the influence of the position detection error is reduced. Thus, the possibility of collision with the object can be predicted more accurately.

If the relative movement vector is obtained in step S16, an alarm determination process for determining the possibility of collision with the detected object is performed (step S17). Details of the alarm determination process will be described later.
If it is determined in step S17 that there is no possibility of collision between the host vehicle 10 and the detected object (NO in step S17), the process returns to step S1 and the above-described processing is repeated.
If it is determined in step S17 that there is a possibility of collision between the host vehicle 10 and the detected object (YES in step S17), the process proceeds to an alarm output determination process in step S18.

In step S18, it is determined whether or not the driver of the host vehicle 10 is performing a brake operation from the output BR of the brake sensor 5, thereby determining whether or not to perform alarm output determination processing, that is, whether or not alarm output is performed (step S18). Step S18).
If the driver of the host vehicle 10 is performing a braking operation, the acceleration Gs generated by the driver is calculated (the deceleration direction is positive). It is determined that the collision is avoided by the operation, the alarm output determination process is terminated (NO in step S18), the process returns to step S1, and the above process is repeated.
As a result, when an appropriate brake operation is performed, an alarm is not issued, so that the driver is not bothered excessively.

  Further, when the acceleration Gs is equal to or less than the predetermined threshold GTH, or when the driver of the host vehicle 10 is not performing a brake operation, the process immediately proceeds to the process of step S19 (YES in step S18), and may contact the object. Therefore, an alarm is issued by voice through the speaker 6 (step S19), and an image obtained by, for example, the infrared camera 2R is output to the image display device 7 so that the approaching object 10 Is displayed as an emphasized image for the driver (step S20).

  The predetermined threshold GTH is a value corresponding to a condition in which the host vehicle 10 stops at a travel distance equal to or less than the distance Zv (0) between the object and the host vehicle 10 when the acceleration Gs during the brake operation is maintained as it is. It is.

The above is the object detection / alarm operation in the image processing unit 1 of the vehicle periphery monitoring device of the present embodiment. Next, referring to the flowchart shown in FIG. 5, step S17 of the flowchart shown in FIG. The alarm determination process in will be described in more detail.
FIG. 5 is a flowchart showing the alarm determination processing operation of the present embodiment.
The warning determination process includes the object detected as the vehicle 10 by the following collision determination process, determination process whether or not the vehicle is in the approach determination area, approach collision determination process, pedestrian determination process, and artificial structure determination process. This is a process for determining the possibility of collision. Hereinafter, as shown in FIG. 6, a case where there is an object 20 traveling at a speed Vp from a direction of approximately 90 ° with respect to the traveling direction of the host vehicle 10 will be described as an example.

  In FIG. 5, first, the image processing unit 1 performs a collision determination process (step S31). In the collision determination process, in FIG. 6, when the object 20 approaches the distance Zv (0) from the distance Zv (N−1) during the time ΔT, the relative speed Vs in the Z direction with the host vehicle 10 is obtained. This is a process for determining whether or not they collide within the margin time T, assuming that both move within the height H while maintaining the relative speed Vs. Here, the margin time T is intended to determine the possibility of a collision by a time T before the predicted collision time. Accordingly, the margin time T is set to about 2 to 5 seconds, for example. H is a predetermined height that defines a range in the height direction, and is set to about twice the vehicle height of the host vehicle 10, for example.

  Next, in step S31, when there is a possibility that the host vehicle 10 and the target object collide within the margin time T (YES in step S31), the image processing unit 1 sets the target in order to further improve the reliability of the determination. A determination process is performed as to whether or not an object exists in the approach determination area (step S32). As shown in FIG. 7, if the area that can be monitored by the infrared cameras 2R and 2L is an outer triangular area AR0 indicated by a thick solid line, as shown in FIG. A region AR1 that is closer to the host vehicle 10 than Vs × T and corresponds to a range in which the object has a margin β (for example, about 50 to 100 cm) on both sides of the vehicle width α of the host vehicle 10, that is, the target This is a process for determining whether or not an object exists in the approach determination area AR1 where the possibility of a collision with the host vehicle 10 is extremely high if the object continues to exist. The approach determination area AR1 also has a predetermined height H.

Furthermore, when the target object does not exist in the approach determination area in step S32 (NO in step S32), the image processing unit 1 may enter the approach determination area and collide with the host vehicle 10. An approach collision determination process is performed to determine whether or not (step S33). In the approach collision determination process, the areas AR2 and AR3 in which the absolute value of the X coordinate is larger than the above-described approach determination area AR1 (outside in the lateral direction of the approach determination area) are called the entry determination areas, and the objects in this area are This is a process for determining whether or not the vehicle enters the approach determination area AR1 and collides with the host vehicle 10 by moving.
The entry determination areas AR2 and AR3 also have a predetermined height H.

On the other hand, when the target object is present in the approach determination area in step S32 (YES in step S32), the image processing unit 1 determines whether the target object is a pedestrian or not. Processing is performed (step S34). The pedestrian determination process is a process for determining whether or not the target object is a pedestrian from features such as the shape and size of the target object image and luminance dispersion on the grayscale image.
If it is determined in step S34 that the object is likely to be a pedestrian (YES in step S34), it is determined whether or not the object is an artificial structure in order to further increase the reliability of the determination. An artificial structure determination process is performed (step S35). The artificial structure determination process is a process of determining an object such as a vehicle as an artificial structure on a gray scale image and excluding it from an alarm target. The details of the artificial structure determination process will be described later.

  Therefore, in the above-described step S33, when there is a possibility that the target object enters the approach determination area and collides with the own vehicle 10 (YES in step S33), and in step S35, there is a possibility of a pedestrian. When the determined object is not an artificial structure (NO in step S35), the image processing unit 1 has a possibility of collision between the host vehicle 10 and the detected object (being an alarm target). A determination is made (step S36), the process proceeds to step S18 as YES in step S17 shown in FIG. 3, and an alarm output determination process (step S18) is performed.

  On the other hand, in step S31 described above, when there is no possibility that the host vehicle 10 and the target object collide within the margin time T (NO in step S31), or in step S33, the target object enters the approach determination area. If there is no possibility of colliding with the host vehicle 10 (NO in step S33), or if it is determined in step S34 that the object is not likely to be a pedestrian (NO in step S34), further step S35 is performed. When the object determined to be a pedestrian is an artificial structure (YES in step S35), the image processing unit 1 detects the host vehicle 10. It is determined that there is no possibility of collision with the object (not an alarm target) (step S37), and the process proceeds to step S1 as NO in step S17 shown in FIG. Ri, repeat the object detection and alarm operation such as a pedestrian.

Next, the artificial structure determination process in step S35 of the flowchart shown in FIG. 5 will be described in more detail with reference to the flowchart shown in FIG. FIG. 8 is a flowchart showing the artificial structure determination processing operation of the present embodiment.
In FIG. 8, first, the image processing unit 1 determines the center of gravity G (xc, yc) and area S of the binarized object calculated in step S8 of the flowchart shown in FIG. In addition to the ratio ASPECT and the distance z between the host vehicle 10 and the object calculated in step S13, the height hb and width wb of the circumscribed rectangle of the binarized object, and the coordinates of the circumscribed rectangle centroid (xb, yb) The binarized object shape feature amount indicating the shape characteristic of the binarized object in the real space is calculated using the value of (2). Note that the binarized object shape feature value to be obtained is a parallax calculated by correlation calculation of the base line length D [m], camera focal length f [m], pixel pitch p [m / pixel], and left and right images. It calculates using (DELTA) d [pixel].

Specifically, the distance z between the host vehicle 10 and the object is
z = (f × D) / (Δd × p) (1)
Therefore, the width ΔWb and height ΔHb of the binarized object in real space are
ΔWb = wb × z × p / f ΔHb = hb × z × p / f (2)
The upper end position coordinates (Xt, Yt, Zt) of the binarized object are
Xt = xb × z × p / fYt
= Yb * z * p / f- [Delta] Hb / 2Zt
= Z (3) can be calculated.

  Next, among the binarized objects extracted in step S7 of the flowchart shown in FIG. 3, among the binarized objects acquired in step S41, the binarized objects existing at symmetrical positions. Is searched (step S42). Specifically, in OBJ [I] (I is a number for distinguishing OBJ) as shown in FIG. 9, a binarized object that satisfies the conditions shown in the following expressions (4) to (6): Are determined to be binarized objects OBJ [X] and OBJ [Y] existing at symmetrical positions.

Condition 1: A binarized object that satisfies the equation (4) and has a distance difference smaller than the prescribed value TH1 and can be regarded as the same distance.
| Zt [X] −Zt [Y] | <TH1 (4)
Condition 2: A binarized object that satisfies the equation (5) and can be considered to exist at the same height with the difference in the upper end height position being smaller than the specified value TH2.
| Yt [X] −Yt [Y] | <TH2 (5)
Condition 3: The maximum width between the left and right edges (between the left edge of the binarized object located on the left side and the right edge of the binarized object located on the right side) satisfying equation (6) is defined. The binarized object that is larger than the vehicle width TH3 [m] and smaller than TH4 [m].
TH3 <| (Xt [X] + ΔWb [X] / 2) − (Xt [Y] −ΔWb [Y] / 2) | <TH4 (6)
However, Zt [X] and Zt [Y] indicate distances of OBJ [X] and OBJ [Y], respectively, and Yt [X] and Yt [Y] are OBJ [X] and OBJ [Y], respectively. Indicates the upper end height position. Also, equation (6) shows the case where OBJ (X) exists on the right side of OBJ (Y).

Then, the presence / absence of two binarized objects existing at symmetrical positions is determined (step S43), and two binarized objects existing at symmetrical positions satisfying the conditions shown in equations (4) to (6) If there is an object (YES in step S43), a plurality of search areas MASK [i] are set above the two binarized objects (step S44). The search areas MASK [i] are arranged in two upper and lower rows, and as shown in FIG. 9, the upper and lower widths L are set to be approximately the vertical width of the windshield of a general vehicle.
Here, for example, binarized objects OBJ [1] and OBJ [2] that exist in symmetrical positions that satisfy the conditions shown in equations (4) to (6) are vehicle lights (headlights, taillights, etc.). Therefore, the distance of the object captured in the search area MASK [i] can be regarded as substantially the same distance as the light (indicating the distance in the vicinity). If the binarized object described above is a pedestrian, the upper end of the head is a space, and no object is captured in the search area MASK [i], and the distance is indefinite.

Therefore, the distance MASK [i] _Z of the search area MASK [i] is calculated by correlation calculation (step S45), and the binarized objects OBJ [X] and OBJ existing at symmetrical positions satisfying the expression (7) are satisfied. The difference between the distances between any one of [Y] (OBJ [X] in this embodiment and X = 1 in FIG. 9) and the search area MASK [i] is smaller than the specified value TH5, and the search area MASK. It is determined whether or not the object captured in [i] is the same distance as the binarized object (step S46).
| MASK [i] _Z-Zt [X] | <TH5 (7)
However, the prescribed value TH5 is a value that takes into account the distance from the headlight or taillight to the windshield and the parallax accuracy.
FIG. 9 shows a case where the square search areas MASK [i] are set in two horizontal rows along the horizontal direction. However, the present invention is not limited to this configuration, and the search areas MASK [ The area and shape in which i] is set may be any as long as it sufficiently includes the windshield.

  Next, among the plurality of search areas MASK [i], an area consisting only of the search area MASK [i] that captures an object that can be regarded as being at substantially the same distance as the binarized objects OBJ [X] and OBJ [Y]. (Refer to FIGS. 10 and 11) is set as the window candidate area A in which the windshield is estimated to exist (step S47).

  Then, the horizontal edge and the vertical edge in the window candidate area A are calculated (step S48), and as shown in FIG. 11, in the area in the vehicle vertical direction inner side and the horizontal direction inner side than the horizontal edge and vertical edge. The maximum substantially rectangular area is set as the luminance dispersion calculation area B (step S49). As a result, the luminance dispersion calculation area B is set to a position where it is estimated that only the glass portion of the windshield not including the upper edge, the left and right edges, and the wiper of the windshield exists. In the example of FIG. 10, the search area MASK [i] constituting the window candidate area A is marked with a circle including a horizontal edge and a vertical edge. Further, as binarized objects OBJ [X] and OBJ [Y] that exist in symmetrical positions that can be characteristic parts of the vehicle, in addition to the above-mentioned vehicle lights, an exhaust pipe (for example, OBJ [3] ), Tires (for example, OBJ [4], [5]), pillars, and the like.

  Next, it is determined whether the temporal variation width W of the luminance dispersion in the luminance dispersion calculation area B is smaller than a predetermined threshold value TH6 (step S51). If the temporal variation width W of the luminance dispersion in the luminance dispersion calculation area B is larger than the predetermined threshold TH6 (NO in step S51), it is determined that the object is a vehicle (step S53).

Here, the temporal change width W is the luminance of the luminance dispersion calculation area B included in a predetermined number of infrared images Frame [t] (t = 1 to n) captured by the infrared cameras 2R and 2L from the present to the past. This is a change in the variance Var [t] (t = 1 to n), that is, the temporal change width W is a minimum luminance variance Min_Var that is the minimum value of luminance variance and a maximum luminance variance Max_Var that is the maximum value of luminance variance. It is calculated as a difference.
Max_Var−Min_Var <TH6 (8)
In general, when infrared rays of the surrounding landscape are reflected by the windshield of the traveling vehicle 10, the threshold TH6 is set to the surrounding landscape because the temporal change width W of the luminance dispersion of the windshield tends to increase. The luminance dispersion is set to a relatively large luminance dispersion that can be recognized as the temporal variation width W of the luminance dispersion due to the reflection of.

If it is determined in step S51 that the temporal change in luminance dispersion in the luminance dispersion calculation area B is smaller than the threshold value TH6 (YES in step S51), whether the temporal change in luminance dispersion in the luminance dispersion calculation area B is uniform or not. Is determined (step S52). When the temporal change of the luminance dispersion in the luminance dispersion calculation area B is uniform (YES in step S52), it is determined that the object is a vehicle (step S53), and when the object is not uniform (NO in step S52), It is determined that the object is not a vehicle (step S54). Here, the determination as to whether or not the temporal change of the luminance dispersion in the luminance dispersion calculation area B is uniform is, for example, in the infrared image Frame [t] (t = 1 to n), for all the luminance dispersion Var [ t] (t = 1 to n) is smaller than the predetermined threshold value TH7, it can be determined that the temporal change in the luminance dispersion in the luminance dispersion calculation area B is uniform.
TH7> Var [t] (9)

If there are two binarized objects that exist at symmetrical positions that satisfy the conditions shown in equations (4) to (6) in step S43 described above (YES in step S43), the search is performed in step S47. An object including a symmetric binarization target may be recognized as a vehicle without determining whether or not the area MASK1 includes a windshield.
Moreover, you may enable it to adjust arbitrarily the magnitude | size and position of a search area | region according to the shape of the vehicle to delete from a binarization target object.
Furthermore, an object having a low average luminance value such as a tire having a short continuous running time may be searched for at the positions OBJ [4] and OBJ [5] shown in FIG.

  In the present embodiment, the image processing unit 1 includes feature part extraction means and second object distance object extraction means. More specifically, S1 to S13 in FIG. 3 and S41 to S43 in FIG. 8 correspond to feature extraction means, S44 to S46 in FIG. 8 correspond to second object extraction means, and the steps of FIG. S50 corresponds to the infrared ray amount change detecting means. Further, steps S51 and S52 in FIG. 8 correspond to infrared amount determination means, and S53 in FIG. 8 corresponds to vehicle determination means.

  As described above, the vehicle periphery monitoring apparatus according to the present embodiment extracts the binarized object OBJ [i] by binarization processing from the grayscale images of the images photographed by the infrared cameras 2L and 2R. . If the binarized objects OBJ [X] and OBJ [Y] existing at the symmetric position are present in the acquired binarized objects, the two binarized objects existing at the symmetric positions are present. Based on the temporal change of the luminance dispersion in the luminance dispersion calculation area B set above the objects OBJ [X] and OBJ [Y], the surrounding landscape is reflected, the sky in clear weather and the sky of one side are also The presence of the reflected windshield is confirmed, and when this windshield exists, the object including the binarized objects OBJ [X] and OBJ [Y] is recognized as a vehicle.

  As a result, the infrared camera 2L or 2R was captured by a relatively simple procedure using a combination of an object that generates heat in the vehicle structure and an object that satisfies the condition of temporal variation in luminance dispersion of the windshield. Vehicles having characteristics that are clearly different from those of pedestrians can be excluded in advance from the object, and the accuracy of pedestrian recognition can be improved.

  Note that, in the vehicle periphery monitoring device according to the first embodiment described above, the binarized object that exists in a symmetric position with the search area MASK [i] based on the stereo image including the infrared cameras 2L and 2R. Although the case of calculating the relative distance between OBJ [X] and OBJ [Y] has been described, the present invention is not limited to this configuration. For example, as another aspect, a relative distance from a so-called monocular image using one infrared camera is used. May be calculated. This method of calculating a relative distance from a monocular image is performed by performing identity determination between a binarized object of a past image and a binarized object of a video after a predetermined time has elapsed, and binarization thereof. The relative positional relationship between the binarized object and the surroundings is calculated based on the change in the background around the object (for example, JP 2008-113296 A).

  In the first embodiment described above, an example of extracting an object that is estimated to be a windshield of a vehicle has been described. However, the object is present above a binarized object that exists at a symmetrical position such as a lamp. Examples of the object that easily reflects infrared rays include a tank lorry tank. In this case as well, the object can be determined as a vehicle.

Next, a second embodiment of the present invention will be described with reference to the flowchart of FIG. In the second embodiment, means for predicting whether or not the reflection to the windshield is likely to occur is added to the configuration of the first embodiment described above. A detailed description of the overlapping parts will be omitted.
Here, the reflection on the windshield is a situation in which some object is partially reflected on the glass of the windshield for a relatively short time, and in such a situation, the change in temporal luminance dispersion becomes large. . As an external environment in which this situation is likely to occur, objects such as utility poles, buildings, trees, etc. are sufficiently separated from each other as tall objects near the vehicle as objects near the road. There are situations where the sky is mottled and the clouds are not uniform. And as a case where said time change is large, there exists a case where a vehicle runs through the traveling path in a forest, for example.

First, similarly to the flowchart of FIG. 8, steps S41 to S49 are performed.
Next, the external environment of the vehicle is detected (step S60). Here, the external environment may be detected by a camera that captures the outside world of the vehicle or by acquiring weather information by road-to-vehicle communication or the like.
When the windshield exists in the luminance dispersion calculation area B, it is determined whether or not the external environment is highly likely to be reflected on the windshield (step S61).
When the external environment is highly likely to be reflected (YES in step S60), a temporal change in the luminance dispersion in the luminance dispersion calculation area B is detected (step S50). In addition, the process of step S60-S61 mentioned above is corresponded to the determination means which determines whether it is an external environment where the reflection to the window of a vehicle occurs easily.

  If it is determined in step S51 that the temporal change in luminance dispersion in the luminance dispersion calculation area B is smaller than the threshold value TH6 (YES in step S51), it is determined that the object is not a vehicle (step S54). If it is determined that the temporal change in luminance dispersion in the variance calculation area B is greater than or equal to the threshold TH6 (NO in step S51), that is, if it is determined that the temporal change in luminance dispersion is actually large, the object is a vehicle. It is determined that there is (step S53).

On the other hand, when the external environment is not highly likely to be reflected on the windshield (NO in step S61), for example, a relatively tall object is not detected in the vicinity of the traveling path, or the sky is uniform and clear. If the weather such as cloudiness or rain is detected, it is determined whether or not the temporal change of the luminance dispersion in the luminance dispersion calculation area B is actually uniform (step S52). When it is determined that the temporal change of the luminance dispersion in the luminance dispersion calculation area B is not uniform (NO in step S52), it is determined that the object is not a vehicle (step S54), and the luminance in the luminance dispersion calculation area B is determined. If it is determined that the temporal change in dispersion is uniform (YES in step S52), it is determined that the object is a vehicle (step S53). Note that whether or not the temporal change of the luminance dispersion calculation area B is uniform is determined based on the above-described equation (9).
Note that the determination time for detecting the temporal change in step S50 may be changed based on the correlation of the distance between the windshield, which is the second object, and the object near the travel path or the sky pattern. .

  Therefore, according to the second embodiment described above, the possibility of reflection on the windshield as the second object is determined based on the external environment, and when the possibility of reflection is high, the luminance dispersion calculation area Since the temporal change in the luminance dispersion of B is predicted to increase, it is determined whether the temporal change in the luminance dispersion is actually greater than or equal to the threshold value TH6, and if it is greater than or equal to the threshold value TH6, the object is determined to be a vehicle. When the possibility of contamination is low, it is predicted that the temporal change of the luminance dispersion in the luminance dispersion calculation area B will be uniform, so it is determined whether the temporal change of the luminance dispersion is actually uniform. Therefore, it is possible to determine the object as a vehicle, so that the determination accuracy is improved by determining the possibility of reflection rather than determining whether the object is a vehicle based only on the temporal change in luminance dispersion. Can do.

It is a block diagram which shows the structure of the vehicle periphery monitoring apparatus of the 1st Embodiment of this invention. It is a figure which shows the attachment position of the infrared camera, a sensor, a display, etc. in a vehicle. It is a flowchart which shows the target object detection and alarm operation | movement of the vehicle periphery monitoring apparatus of 1st Embodiment. It is a figure which shows the gray scale image obtained with an infrared camera, and its binarized image. It is a flowchart which shows the alarm determination processing operation | movement of 1st Embodiment. It is a figure which shows the case where a collision is easy to generate | occur | produce. It is a figure which shows the area | region division ahead of a vehicle. It is a flowchart which shows the artificial structure determination processing operation | movement of 1st Embodiment. It is a figure shown about the setting process of the search area | region of 1st Embodiment. It is a figure shown about the process of edge extraction of 1st Embodiment. It is a figure shown about the window candidate area | region and luminance dispersion | distribution calculation area | region B of 1st Embodiment. It is a flowchart which shows the artificial structure determination processing operation | movement of 2nd Embodiment.

Explanation of symbols

DESCRIPTION OF SYMBOLS 1 Image processing unit 2R, 2L Infrared camera 3 Yaw rate sensor 4 Vehicle speed sensor 5 Brake sensor 6 Speaker 7 Image display apparatus 10 Own vehicle S41-S43 Feature part extraction means S44-S46 2nd object extraction means S50 Infrared quantity change detection means S51 , S52 Infrared amount determination means S53 Vehicle determination means S60 to S61 Determination means

Claims (4)

A feature extraction means for extracting, as a first object, a heat source that seems to be a feature of the vehicle from an image captured by an infrared camera;
By comparing the distance between the peripheral region and the first object of the first object, a second object extraction means for extracting a second object distance between the first object is within a predetermined range and,
An infrared amount change detecting means for detecting a temporal change in amount of infrared rays that are generated from the second object,
An infrared amount determination means for determining whether or not a temporal change in the amount of infrared rays generated from the second object detected by the infrared amount change detection means satisfies a predetermined infrared amount condition;
When it is determined by the infrared light amount determination means that the temporal change in the amount of infrared light generated from the second object satisfies the condition of a predetermined infrared light amount, the first object and the second object A vehicle periphery monitoring device comprising: vehicle determination means for determining that an object including the vehicle is a vehicle. A determination means for determining whether the external environment is likely to be reflected in a vehicle window ;
The infrared amount determination means includes
When the determination means determines that the external environment is likely to move into the vehicle window, and when the determination means determines that the external environment is not likely to move into the vehicle window And change the condition of the predetermined amount of infrared rays,
The amount of infrared rays temporal change that is generated from the second object, a vehicle surroundings monitoring apparatus according to claim 1, characterized that you determine whether or not the condition of the predetermined amount of infrared rays.   The vehicle periphery monitoring device according to claim 2, wherein the external environment is in a sky pattern.   The vehicle periphery monitoring device according to claim 2, wherein the external environment is a situation of an object near a traveling path of the vehicle.

Images