JP2004135034A - Vehicle periphery monitoring unit - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To provide a vehicle periphery monitoring unit which stably extracts a walker by exactly deciding an unshaped binarized object extracted from an image photographed by a camera. <P>SOLUTION: An image processing unit calculates binarized object shape feature quantities in the real space by utilizing the center of gravity and the area of the object, the distance from its own vehicle, and further the aspect ratio, a height and a width of a circumscribed rectangular shape of the object and the coordinate values of the center of gravity (S41). Then when it does not rain around the vehicle, the unit obtains the height of the object in a gray-scale image (S42), and sets a plurality of mask regions in the gray-scale object and calculates a luminance mean value and a luminance change (dispersion) of each mask (S43). The unit decides whether the height, width, presence height, luminance means value, luminance dispersion, etc., of the object are values within suitable ranges as the walker (S44 to S48). If it is decided that any is not in the suitable range as a pedestrian, it is decided that the object is not a pedestrian (S49). <P>COPYRIGHT: (C)2004,JPO

Description

[0001]
TECHNICAL FIELD OF THE INVENTION
The present invention relates to a vehicle periphery monitoring device that extracts an object by binarizing an image captured by an infrared camera.
[0002]
[Prior art]
A conventional vehicle periphery monitoring device extracts an object such as a pedestrian having a possibility of collision with the own vehicle from an image of the surroundings of the own vehicle captured by an imaging unit such as an infrared camera. Provide to the driver of the vehicle. In this device, a portion having a high temperature in an image around the host vehicle taken by a pair of left and right stereo cameras is set as a target object, and a distance to the target object is calculated by obtaining a parallax of the target object. From the moving direction of the object and the position of the object, an object that is likely to affect the traveling of the host vehicle is detected and an alarm is output (for example, see Patent Document 1).
[0003]
[Patent Document 1]
JP 2001-6096 A
[0004]
[Problems to be solved by the invention]
By the way, when a pedestrian is extracted only by shape determination as in the conventional device, the pedestrian on the infrared image is divided into two values depending on the influence of a cap, clothing, and the presence environment of the pedestrian. The morphed shape is irregular, and in general, when the vehicle is running, there is a change in the shape of the road ahead and the effect of pitching of the vehicle, and pedestrians are also detected from children to adults with different heights from the original Therefore, the coordinates of the center of gravity of the object on the screen may not be fixed with respect to the distance, and only the pedestrian may not be stably extracted.
[0005]
Therefore, conventionally, the size of the object in the real space is calculated from the grayscale image, and only the pedestrian-like object is extracted based on the positional relationship where the binarized object exists. Although a method of extracting road structures and vehicles from objects and excluding them as objects other than pedestrians from warning objects has been proposed, if the vehicle is running in rain (rainy weather), the object Also, since the amount of infrared radiation changes due to the influence of rainfall, there is a problem that only pedestrians may not be stably extracted.
[0006]
That is, in a general environment, a heat storage body such as a signboard, a wall, a power pole, etc. that does not generate heat by itself and stores only heat given from outside does not appear on the infrared image because the temperature is lowered by rainfall (infrared camera) However, the heating elements that generate heat by themselves, such as vending machines, are detected by the infrared camera, but the infrared radiation part is reduced (infinitely equal), making it difficult to accurately determine the shape. There is.
In addition, a part (a head or the like) where a human is exposed is detected, but a part covered with clothes is not detected by the infrared camera due to wetness of clothes. In this way, in rainy weather and other times, even at the same point, the surroundings of the vehicle change, and the shapes of all objects on the grayscale image detected by the infrared camera change, With the conventional method, there was a possibility that only pedestrians could not be stably extracted.
Furthermore, in rainy weather, since rainfall adheres to the camera lens surface, the shape of the object on the image tends to be unclear, and there is a problem that it is difficult to determine the shape of the object.
[0007]
SUMMARY OF THE INVENTION The present invention has been made in view of the above problems, and has a vehicle surroundings monitoring device that accurately determines an indeterminate binarized object extracted from an image captured by a camera and extracts a stable pedestrian. The purpose is to provide.
[0008]
[Means for Solving the Problems]
In order to solve the above problem, a vehicle periphery monitoring device according to the invention of claim 1 is a vehicle periphery monitoring device that detects an object existing around a vehicle from an infrared image captured by an infrared camera, A weather detecting means for detecting the weather around the vehicle (for example, step S41-1 in the embodiment); and an object extracting means for extracting an object emitting infrared light from the infrared image (for example, step S1 to step in the embodiment) S13) and a heat storage element extracting means (for example, step S45 in the embodiment) for extracting, from the object extracted by the object extracting means, a heat storage element that does not generate heat and stores only heat given from outside. , S46, S47, S52, S53, S53-1, S60, S61, S62) and the weather detecting means determines that the weather around the vehicle is rainy If the pedestrian is detected, the pedestrian is recognized from among the objects, otherwise, the pedestrian is recognized from among the objects except the heat storage unit extracted by the heat storage unit extraction unit. It is characterized by comprising pedestrian recognition means (for example, steps S48 to S50, steps S54 to S59, steps S63 to S80 of the embodiment).
[0009]
In the vehicle periphery monitoring device having the above configuration, when the weather around the vehicle is determined to be rainy by the weather detection unit, the amount of infrared radiation emitted by the target extracted by the target extraction unit decreases. Therefore, the pedestrian recognition means directly recognizes the pedestrian from the extracted objects. On the other hand, if it is determined that the weather around the vehicle is other than that, the amount of infrared radiation emitted by the target object extracted by the target object extracting means does not generate heat itself but only heat given from outside. The difference between the heat storage element that extracts the heat storage element that stores heat and the heating element that emits heat by itself like a pedestrian appears in the luminance dispersion of the object image, so that the heat storage element is extracted from the object by the heat storage element extraction unit, The pedestrian recognition means can perform appropriate pedestrian recognition according to the situation around the vehicle by recognizing the pedestrian from objects other than the heat storage body.
[0010]
A vehicle periphery monitoring device according to a second aspect of the present invention is the vehicle periphery monitoring device according to the first aspect, wherein the pedestrian recognition unit determines a shape of the object (for example, S59 in the embodiment). , S73), wherein when the weather detecting means determines that the weather around the vehicle is rainy, the shape determining means is stopped.
The vehicle periphery monitoring device having the above configuration, when the weather around the vehicle is determined to be rainy by the weather detection means, stops the shape determination process, so that rain adheres to the camera lens surface. Therefore, it is possible to prevent the object shape on the image that has become unclear from being erroneously determined by the pedestrian recognition unit.
[0011]
BEST MODE FOR CARRYING OUT THE INVENTION
Hereinafter, embodiments of the present invention will be described with reference to the drawings.
FIG. 1 is a block diagram showing a configuration of a vehicle periphery monitoring device according to one embodiment of the present invention.
In FIG. 1, reference numeral 1 denotes an image processing unit provided with a CPU (Central Processing Unit) for controlling the vehicle periphery monitoring device according to the present embodiment, and includes two infrared cameras 2R and 2L capable of detecting far infrared rays. A yaw rate sensor 3 for detecting a yaw rate of the vehicle, a vehicle speed sensor 4 for detecting a traveling speed (vehicle speed) of the vehicle, and a brake sensor 5 for detecting an operation of a brake are connected. Thereby, the image processing unit 1 detects a pedestrian or an animal in front of the vehicle from the infrared image around the vehicle and the signal indicating the running state of the vehicle, and issues an alarm when it is determined that the possibility of collision is high. Emit.
[0012]
In addition, the image processing unit 1 displays a speaker 6 for issuing a warning by voice and images taken by the infrared cameras 2R and 2L, and allows the driver of the vehicle to recognize an object having a high risk of collision. For example, a meter-integrated display that is integrated with a meter that indicates the running state of the host vehicle by a number, a NAVI Display that is installed on the console of the host vehicle, and information is displayed at a position in the front window that does not obstruct the driver's front view. An image display device 7 including a HUD (Head Up Display) 7a is connected.
[0013]
The image processing unit 1 includes an A / D conversion circuit that converts an input analog signal into a digital signal, an image memory that stores a digitized image signal, a CPU (Central Processing Unit) that performs various types of arithmetic processing, (Random Access Memory) used to store the data of the CPU, ROM (Read Only Memory) storing programs, tables, maps, and the like executed by the CPU, a drive signal of the speaker 6, a display signal of the HUD 7a, and the like are output. The output signals of the infrared cameras 2R and 2L, the yaw rate sensor 3, the vehicle speed sensor 4, and the brake sensor 5 are converted into digital signals and input to the CPU.
[0014]
As shown in FIG. 2, the infrared cameras 2R and 2L are arranged at a front part of the host vehicle 10 at positions substantially symmetrical with respect to the center of the host vehicle 10 in the vehicle width direction. The optical axes of 2R and 2L are fixed so that the optical axes are parallel to each other and the heights from both roads are equal. Note that the infrared cameras 2R and 2L have a characteristic that the higher the temperature of the target, the higher the output signal level (the higher the luminance).
The HUD 7a is provided such that the display screen is displayed at a position on the front window of the host vehicle 10 where the front view of the driver is not obstructed.
[0015]
Next, the operation of the present embodiment will be described with reference to the drawings.
FIG. 3 is a flowchart showing an operation of detecting and warning an object such as a pedestrian in the image processing unit 1 of the vehicle periphery monitoring device of the present embodiment.
First, the image processing unit 1 acquires an infrared image which is an output signal of the infrared cameras 2R and 2L (step S1), performs A / D conversion (step S2), and stores a gray scale image in an image memory (step S1). S3). Here, a right image is obtained by the infrared camera 2R, and a left image is obtained by the infrared camera 2L. Further, in the right image and the left image, the horizontal position of the same object on the display screen is shifted, so that the distance (parallax) to the object can be calculated.
[0016]
After the grayscale image is obtained in step S3, the right image obtained by the infrared camera 2R is used as a reference image, and the image signal is binarized, that is, an area brighter than the luminance threshold ITH is set to "1" ( White) and the process of setting the dark area to “0” (black) (step S4).
FIG. 4A shows a grayscale image obtained by the infrared camera 2R. By performing a binarization process on the grayscale image, an image as shown in FIG. 4B is obtained. In FIG. 4B, for example, an object surrounded by a frame from P1 to P4 is an object displayed as white on the display screen (hereinafter, referred to as a “high luminance area”).
When the binarized image data is obtained from the infrared image, a process of converting the binarized image data into run-length data is performed (step S5). The line represented by the run-length data indicates, at a pixel level, an area that has been whitened by binarization, and each has a width of one pixel in the y direction and also has a width of one pixel in the x direction. It has the length of a pixel constituting the run-length data.
[0017]
Next, a process of extracting the target object is performed by labeling the target object from the image data converted to the run-length data (step S6) (step S7). That is, among the lines converted into run-length data, a line having a portion overlapping in the y-direction is regarded as one object, so that, for example, the high-luminance areas P1 to P4 shown in FIG. (Value object).
When the extraction of the object is completed, the center of gravity G, the area S, and the aspect ratio ASPECT of the circumscribed rectangle of the extracted object are calculated (step S8).
[0018]
Here, the area S is obtained by calculating the run length data of the target object of the label A by (x [i], y [i], run [i], A) (i = 0, 1, 2,..., N−1). ), The length (run [i] -1) of the run-length data is calculated by integrating the same object (N run-length data). The coordinates (xc, yc) of the center of gravity G of the object A are represented by the length (run [i] -1) of each run-length data and the coordinates x [i] or y [i] of each run-length data. Are multiplied by each other, and the sum of the multiplications for the same object is calculated by dividing by the area S.
Further, the aspect ratio ASPECT is calculated as a ratio Dy / Dx of the length Dy in the vertical direction and the length Dx in the horizontal direction of the circumscribed rectangle of the object.
Since the run length data is indicated by the number of pixels (the number of coordinates) (= run [i]), the actual length needs to be “−1” (subtract 1) (= run [i]). ] -1). In addition, the position of the center of gravity G may be replaced by the position of the center of gravity of a circumscribed rectangle.
[0019]
When the center of gravity, area, and aspect ratio of the circumscribed rectangle of the object can be calculated, next, tracking of the object between times, that is, recognition of the same object in each sampling cycle is performed (step S9). In time tracking, the time at which the time t as the analog quantity is discretized at the sampling cycle is k, and for example, if the objects A and B are extracted at time k, the objects C and D extracted at time (k + 1) The identity with the objects A and B is determined. When it is determined that the objects A and B are the same as the objects C and D, the inter-time tracking is performed by changing the objects C and D to the labels of the objects A and B, respectively.
Further, the position coordinates (of the center of gravity) of each object recognized in this way are stored in the memory as time-series position data, and are used in subsequent arithmetic processing.
[0020]
Note that the processes of steps S4 to S9 described above are performed on the binarized reference image (the right image in the present embodiment).
Next, the vehicle speed VCAR detected by the vehicle speed sensor 4 and the yaw rate YR detected by the yaw rate sensor 3 are read, and the turning angle θr of the host vehicle 10 is calculated by time-integrating the yaw rate YR (step S10).
[0021]
On the other hand, in parallel with the processing of steps S9 and S10, in steps S11 to S13, processing of calculating the distance z between the target object and the host vehicle 10 is performed. Since this calculation requires a longer time than steps S9 and S10, it is executed at a cycle longer than steps S9 and S11 (for example, about three times as long as the execution cycle of steps S1 to S10).
First, by selecting one of the objects tracked by the binarized image of the reference image (right image), the search image R1 (here, the entire area surrounded by the circumscribed rectangle is searched from the right image) Is extracted (step S11).
[0022]
Next, a search area for searching for an image corresponding to the search image R1 (hereinafter referred to as “corresponding image”) is set from the left image, and a correlation operation is executed to extract a corresponding image (step S12). More specifically, a search region R2 is set in the left image in accordance with each vertex coordinate of the search image R1, and the brightness difference sum value C (a) indicating the level of correlation with the search image R1 in the search region R2. , B) is calculated, and an area where the sum C (a, b) is minimum is extracted as a corresponding image. Note that this correlation operation is performed using a grayscale image instead of a binarized image.
When there is past position data for the same object, a region R2a narrower than the search region R2 is set as a search region based on the position data.
[0023]
By the process of step S12, the search image R1 is extracted in the reference image (right image) and the corresponding image R4 corresponding to this object is extracted in the left image. The position of the center of gravity of the image R4 and the parallax amount Δd (the number of pixels) are obtained, and the distance z between the host vehicle 10 and the target object is calculated from this (step S13).
Next, when the calculation of the turning angle θr in step S10 and the calculation of the distance to the target in step S13 are completed, the coordinates (x, y) and the distance z in the image are converted to the real space coordinates (X, Y, Z). Conversion is performed (step S14).
Here, as shown in FIG. 2, the real space coordinates (X, Y, Z) are obtained by setting the middle point of the mounting positions of the infrared cameras 2R, 2L (the position fixed to the host vehicle 10) as the origin O. The coordinates in the image are defined as x in the horizontal direction and y in the vertical direction with the center of the image as the origin.
[0024]
Further, when the real space coordinates are obtained, a turning angle correction for correcting a positional shift on the image due to the turning of the host vehicle 10 is performed (step S15). In the turning angle correction, if the host vehicle 10 turns, for example, to the left by a turning angle θr during the period from time k to (k + 1), the range of the image is shifted by Δx in the x direction on the image obtained by the camera. This is a process for correcting this.
In the following description, the coordinates after the turning angle correction are displayed as (X, Y, Z).
[0025]
When the turning angle correction for the real space coordinates is completed, N (for example, about N = 10) real space position data after turning angle correction, obtained for the same object during the monitoring period of ΔT, that is, From the time-series data, an approximate straight line LMV corresponding to a relative movement vector between the target object and the host vehicle 10 is obtained.
Next, the latest position coordinates P (0) = (X (0), Y (0), Z (0)) and the position coordinates P (N−1) before (N−1) samples (time ΔT before). = (X (N-1), Y (N-1), Z (N-1)) is corrected to a position on the approximate straight line LMV, and the corrected position coordinates Pv (0) = (Xv (0), Yv (0), Zv (0)) and Pv (N-1) = (Xv (N-1), Yv (N-1), Zv (N-1)).
[0026]
As a result, a relative movement vector is obtained as a vector from the position coordinates Pv (N-1) to Pv (0) (step S16).
In this way, the influence of the position detection error is reduced by calculating the approximate straight line that approximates the relative movement trajectory of the target object with respect to the vehicle 10 from the plurality (N) of data within the monitoring period ΔT. As a result, it is possible to more accurately predict the possibility of collision with the object.
[0027]
When the relative movement vector is obtained in step S16, next, an alarm determination process for determining the possibility of collision with the detected target is performed (step S17). The details of the alarm determination process will be described later.
If it is determined in step S17 that there is no possibility of collision between the host vehicle 10 and the detected object (NO in step S17), the process returns to step S1, and the above-described processing is repeated.
If it is determined in step S17 that there is a possibility of collision between the host vehicle 10 and the detected target (YES in step S17), the process proceeds to an alarm output determination process in step S18.
[0028]
In step S18, by determining from the output BR of the brake sensor 5 whether or not the driver of the host vehicle 10 is performing a brake operation, an alarm output determination process, that is, whether or not to output an alarm is performed (step S18). Step S18).
If the driver of the host vehicle 10 is performing a brake operation, the acceleration Gs generated thereby (the deceleration direction is assumed to be positive) is calculated. If the acceleration Gs is greater than a predetermined threshold GTH, the braking is performed. It is determined that the collision is avoided by the operation, and the alarm output determination processing ends (NO in step S18), and the process returns to step S1 to repeat the above processing.
Thereby, when an appropriate brake operation is being performed, it is possible to prevent a warning from being issued and not to add unnecessary trouble to the driver.
[0029]
Further, when the acceleration Gs is equal to or less than the predetermined threshold GTH, or when the driver of the host vehicle 10 does not perform the brake operation, the process immediately proceeds to the process of step S19 (YES in step S18), and the possibility of contact with the object is obtained. Is high, an audio warning is issued via the speaker 6 (step S19), and an image obtained by, for example, the infrared camera 2R is output to the image display device 7 so that the approaching target is recognized by the vehicle 10 (Step S20).
The predetermined threshold GTH is a value corresponding to a condition that the host vehicle 10 stops at a travel distance equal to or less than the distance Zv (0) between the target object and the host vehicle 10 when the acceleration Gs during the brake operation is maintained as it is. It is.
[0030]
The above is the object detection / warning operation in the image processing unit 1 of the vehicle periphery monitoring device of the present embodiment. Next, referring to the flowchart shown in FIG. 5, step S17 of the flowchart shown in FIG. Will be described in more detail.
FIG. 5 is a flowchart showing the alarm determination processing operation of the present embodiment.
The warning determination process includes a collision determination process described below, a determination process as to whether or not the vehicle is within the approach determination region, an approach collision determination process, a pedestrian determination process, and an artificial structure determination process. This is a process for determining the possibility of collision. Hereinafter, a case will be described as an example where there is an object 20 traveling at a speed Vp from a direction substantially 90 ° with respect to the traveling direction of the host vehicle 10 as shown in FIG.
[0031]
In FIG. 5, first, the image processing unit 1 performs a collision determination process (step S31). In the collision determination process, in FIG. 6, when the object 20 approaches the distance Zv (0) from the distance Zv (N−1) during the time ΔT, the relative velocity Vs in the Z direction with the host vehicle 10 is determined. It is a process of determining whether or not both collide within the margin time T, assuming that the two move while maintaining the relative speed Vs within the height H. Here, the margin time T is intended to determine the possibility of a collision by the time T before the predicted collision time. Therefore, the allowance time T is set to, for example, about 2 to 5 seconds. H is a predetermined height that defines a range in the height direction, and is set to, for example, about twice the height of the host vehicle 10.
[0032]
Next, in Step S31, when there is a possibility that the host vehicle 10 and the target object collide within the margin time T (YES in Step S31), the image processing unit 1 A determination process is performed to determine whether an object is present in the approach determination area (step S32). As shown in FIG. 7, assuming that the area that can be monitored by the infrared cameras 2R and 2L is an outer triangular area AR0 indicated by a thick solid line, Z1 = Z1 = A region AR1 that is closer to the host vehicle 10 than Vs × T and corresponds to a range where the target object has a margin β (for example, about 50 to 100 cm) added to both sides of the vehicle width α of the host vehicle 10, that is, the target This is a process of determining whether or not the object is present in the approach determination area AR1 where the possibility of collision with the host vehicle 10 is extremely high if the object continues to exist. Note that the approach determination area AR1 also has a predetermined height H.
[0033]
Further, in step S32, when the target object does not exist in the approach determination area (NO in step S32), the image processing unit 1 may cause the target to enter the approach determination area and collide with the host vehicle 10. An entry collision determination process is performed to determine whether or not the collision has occurred (step S33). In the approach collision determination process, areas AR2 and AR3 in which the absolute value of the X coordinate is larger than the above-described approach determination area AR1 (outside the approach determination area in the horizontal direction) are referred to as approach determination areas. This is a process of entering the approach determination area AR1 by moving and determining whether or not to collide with the vehicle 10.
The entry determination areas AR2 and AR3 also have a predetermined height H.
[0034]
On the other hand, in step S32, when the target is present in the approach determination area (YES in step S32), the image processing unit 1 determines whether the target is a pedestrian or not. The process is performed (Step S34). The details of the pedestrian determination process will be described later.
Also, in step S34, when it is determined that there is a possibility that the object is a pedestrian (YES in step S34), in order to further increase the reliability of the determination, it is determined whether the object is an artificial structure. An artificial structure determination process is performed (step S35). The artificial structure determination process is a process in which, when a feature that is impossible for a pedestrian such as the following is detected in the target object image, the target object is determined to be an artificial structure, and is excluded from a warning target. It is.
(1) When the image of the target object includes a portion indicating a straight edge.
(2) When the corner of the image of the object is a right angle.
(3) A case where a plurality of objects having the same shape are included in the image of the target object.
(4) When the image of the target object matches the shape of the artificial structure registered in advance.
[0035]
Therefore, in the above-mentioned step S33, when there is a possibility that the target enters the approach determination area and collides with the host vehicle 10 (YES in step S33), and in step S35, there is a possibility of a pedestrian. If the determined object is not an artificial structure (NO in step S35), the image processing unit 1 determines that there is a possibility of collision between the host vehicle 10 and the detected object (is a target of an alarm). A determination is made (step S36), and the process proceeds to step S18 as YES in step S17 shown in FIG. 3, and an alarm output determination process (step S18) is performed.
[0036]
On the other hand, in step S31 described above, when there is no possibility that the host vehicle 10 collides with the target object within the margin time T (NO in step S31), or in step S33, the target object enters the approach determination area. If there is no possibility of collision with the host vehicle 10 (NO in step S33), or if it is determined in step S34 that the object is not a pedestrian (NO in step S34), furthermore, step S35 In any of the cases, the image processing unit 1 has detected that the subject vehicle is the own vehicle 10 when the target object determined to be likely to be a pedestrian is an artificial structure (YES in step S35). It is determined that there is no possibility of collision with the target (not a target of alarm) (step S37), and the process proceeds to step S1 as NO in step S17 shown in FIG. Ri, repeat the object detection and alarm operation such as a pedestrian.
[0037]
Next, the pedestrian determination process in step S34 of the flowchart shown in FIG. 5 will be described in more detail with reference to flowcharts shown in FIGS. FIGS. 8 to 13 are flowcharts showing the pedestrian determination processing operation of the present embodiment.
8, first, the image processing unit 1 determines the center of gravity G (xc, yc) of the binarized object calculated in step S8 of the flowchart shown in FIG. 3 (the center of gravity of the binarized object shown in FIG. 14). G100), the area S (the binarized object area S101 shown in FIG. 14), the aspect ratio ASPECT of the circumscribed rectangle of the object, and the distance z between the host vehicle 10 and the object calculated in step S13. The height hb and the width wb of the circumscribed rectangle of the binarized object shown in FIG. 14, and the values of the circumscribed rectangle barycenter coordinates (xb, yb) (the circumscribed rectangle barycenter 102 shown in FIG. 14) are used in the real space. Then, a binarized object shape characteristic amount indicating the characteristic of the shape of the binarized object is calculated (step S41). Note that the binarized object shape feature quantity to be obtained is a camera base line length D [m], a camera focal length f [m], a pixel pitch p [m / pixel], and a parallax calculated by a correlation operation between left and right images. It is calculated using the quantity Δd [pixel].
[0038]
Specifically, the ratio Rate between the circumscribed rectangle and the object area is
Rate = S / (hb × wb) (1)
Asp representing the aspect ratio ASPECT of the circumscribed rectangle is:
Asp = hb / wb (2)
The distance z between the vehicle 10 and the object is
z = (f × D) / (Δd × p) (3)
Is expressed as
The width ΔWb and the height ΔHb of the binarized object in the real space are
ΔWb = wb × z × p / f
ΔHb = hb × z × p / f (4)
[0039]
The barycentric coordinates (Xc, Yc, Zc) of the binarized object are
Xc = xc × z × p / f
Yc = yc × z × p / f
Zc = z (5)
The coordinates of the center of gravity (Xb, Yb, Zb) of the circumscribed rectangle of the object are
Xb = xb × z × p / f
Yb = yb × z × p / f
Zb = z (6)
The upper end position coordinates (Xt, Yt, Zt) of the binarized object are
Xt = xb × z × p / f
Yt = yb × z × p / f−ΔHb / 2
Zt = z (7)
Can be calculated.
[0040]
After calculating the binarized object shape feature quantity, a weather detection process for detecting the weather around the host vehicle 10 is performed next (step S41-1).
The weather detection process is a process of obtaining a brightness histogram of a grayscale image obtained by, for example, the infrared camera 2R, and determining the weather around the host vehicle 10 based on the brightness histogram. This is a process for determining whether or not the weather is rainy. More specifically, for example, in a grayscale image in fine weather as shown in FIG. 17A and a grayscale image in rainy weather as shown in FIG. Since there is a difference in the amount of infrared rays to be obtained, when the respective luminance histograms are obtained, the result shown in FIG. 18 is obtained.
[0041]
FIG. 18 is a graph comparing a luminance histogram of a grayscale image in rainy weather with a luminance histogram of a grayscale image in fine weather. As shown in FIG. , The standard deviation σ of the luminance histogram of the entire image tends to be smaller (the interval of 2σ is narrower). Therefore, when the standard deviation σ of the luminance histogram of the entire image is smaller than the threshold TH24, the weather around the host vehicle 10 is determined to be rainy.
[0042]
The determination as to whether or not the weather around the host vehicle 10 is rainy is made by a raindrop sensor mounted on the host vehicle 10 for detecting rainfall, regardless of the brightness histogram of the grayscale image as described above. Or a signal for controlling the operation of the wiper for wiping the raindrops on the window glass (for example, an ON / OFF signal of the wiper).
[0043]
Then, if the weather around the host vehicle 10 can be determined in step S41-1, it is determined whether the weather around the host vehicle 10 is rainy (step S41-2).
If the weather around the host vehicle 10 is not rainy in step S41-2 (NO in step S41-2), the grayscale image acquired in step S3 of the flowchart shown in FIG. The height of the target on the grayscale image including the binarized target extracted in step S7 is determined using the same (step S42).
[0044]
The height of the object on the grayscale image is obtained by arranging a plurality of mask areas of a predetermined size from the upper end of the circumscribed rectangle of the binarized object on the grayscale image, and setting the luminance in the mask area. The change is large (the object and the background image are included), the degree of correlation of the mask region between the left and right images is high (two or more objects do not exist in the mask region), and the binary value is further increased. An area including a mask area at the same distance (same parallax) as the object to be converted is extracted as a grayscale object area.
Then, the height Height (pixel) of the region of the grayscale object on the image is calculated, and the height ΔHg of the grayscale object is calculated by Expression (8).
ΔHg = z × Height × p / f (8)
[0045]
Also, as shown in FIG. 15, the area of the grayscale object on the image is AREA0, and mask areas AREA1, AREA2, and AREA3 are set therein, and the average luminance value and luminance change (variance) of each mask are set. Is calculated (step S43). Here, the average luminance value of AREA1 is Ave_A1, the luminance dispersion of AREA2 is Var_A2, and the luminance dispersion of AREA3 is Var_A3. In the following processing, AREA1 is used for determining the presence of the head of the object, AREA2 is used for determining the presence of the body of the object, and AREA3 is used for determining the presence of a shape change from the head to the lower body. The AREA 3 is a heat storage body such as a wall that stores only heat given from the outside without generating heat by itself, and a part of an object showing a monotonous change in luminance is extracted by a binarization process. If this is done, it is also used to identify it as a pedestrian. FIG. 15 schematically illustrates a pedestrian captured by a camera, where a hatched area is a portion of the target captured by binarization, and an area surrounded by a dotted line is binarized. It represents a part of the target that has not been caught, but whose presence can be confirmed against the background in the grayscale image. The dimensions of each part shown in FIG. 15 are examples of the dimensions of each part in the real space.
[0046]
When the setting of the mask areas AREA1, AREA2, and AREA3 is completed, a pedestrian determination based on the shape of the binarized object and a pedestrian determination using the luminance variance of each mask area of the grayscale image are executed as described below.
First, the image processing unit 1 determines whether or not the height, width, existing height, average luminance, and luminance variance of the binarized object are within a range appropriate for a pedestrian.
Specifically, in order to target a pedestrian, it is determined whether or not the width ΔWb of the binarized object is equal to or more than the threshold value TH1 and equal to or less than TH2 (an appropriate value for the width of the pedestrian) (step S44).
[0047]
In step S41-2, if the weather around the host vehicle 10 is rainy (YES in step S41-2), the height calculation of the target on the grayscale image or the mask area in step S42 or step S43 is performed. AREA1, AREA2, and AREA3 are not set, and the process proceeds to step S44. In order to target a pedestrian, it is determined whether the width ΔWb of the binarized object is equal to or more than a threshold value TH1 and equal to or less than TH2 (step S44).
[0048]
If the width ΔWb of the binarization target object is equal to or greater than the threshold value TH1 and equal to or less than TH2 in step S44 (YES in step S44), it is determined whether the weather around the host vehicle 10 is rainy (step S44-). 1).
Then, in step S44-1, when the weather around the host vehicle 10 is not rainy (NO in step S44-1), the height ΔHb of the binarization target is set to the threshold TH3 (appropriate as the height of the pedestrian). It is determined whether the height ΔHg of the grayscale object is less than a threshold value TH4 (a value appropriate for the height of the pedestrian) (step S45).
[0049]
On the other hand, in step S44-1, when the weather around the host vehicle 10 is rainy (YES in step S44-1), the height ΔHb of the binarization target is set to the threshold TH3 (appropriate as the height of the pedestrian). Is determined (step S45-1).
Then, in step S45, when the height ΔHb of the binarized object is less than the threshold value TH3 and the height ΔHg of the grayscale object is less than the threshold value TH4 (YES in step S45), or step S45-1. In the case where the height ΔHb of the binarized object is less than the threshold value TH3 (YES in step S45-1), the height position Yt of the upper end of the object from the road surface is equal to the threshold value TH5 (appropriate as the height of the pedestrian). Is determined (step S46).
[0050]
Also, in step S46, when the upper end height position Yt of the object from the road surface is less than the threshold value TH5 (YES in step S46), it is determined whether the weather around the host vehicle 10 is rainy (step S46). -1).
Then, in step S46-1, when the weather around the host vehicle 10 is not rainy (NO in step S46-1), it is determined whether the luminance variance Var_A3 of the mask area AREA3 is larger than the threshold TH6 (step S46-1). S47). This processing will be described with reference to FIG. 16 which shows the luminance variance of the mask area AREA3 when the object is part or all of a pedestrian or when the object is a wall.
[0051]
Specifically, by setting the area width of the mask area AREA3 to the binarized object width, as shown in FIG. 16A, when only the pedestrian's head is extracted by the binarization processing, , A luminance difference from the lower body part occurs. Further, as shown in FIG. 16B, when at least the upper body or the whole body of the pedestrian is extracted by the binarization process, a luminance difference from the background region (image) occurs. On the other hand, as shown in FIG. 16C, in the case of an object such as a wall, in which the temperature difference of the entire object is small, the luminance difference between the binarized extraction part and the part other than that is small, and the object is AREA3. It is composed of linear portions as shown in FIG. Therefore, the luminance variance Var_A3 of AREA3 indicates a high value for a pedestrian and a low value for an object such as a wall.
Therefore, in step S47, it is determined whether or not the target is a pedestrian by determining whether or not the luminance variance Var_A3 of the mask area AREA3 is larger than the threshold value TH6.
[0052]
Further, in step S47, when the luminance variance Var_A3 of the mask area AREA3 is larger than the threshold value TH6 (YES in step S47), a pedestrian determination based on a temporal change in the shape of the object is performed.
Specifically, since the binarization target object of the pedestrian is targeted, it is considered that the binarization target shape does not significantly change over time. Therefore, it is determined whether or not the difference between the maximum value Max_Rate and the minimum value Min_Rate of the Rate, which is the area ratio of the circumscribed rectangle and the area of the binarized object, within the specified time is less than the threshold value TH7 (step S48).
[0053]
If the weather around the host vehicle 10 is rainy in step S46-1 (YES in step S46-1), the determination of the luminance variance Var_A3 of the mask area AREA3 in step S47 is not performed, and the process proceeds to step S48. Then, it is determined whether or not the difference between the maximum value Max_Rate and the minimum value Min_Rate of the rate, which is the area ratio of the circumscribed rectangle and the area of the binarized object, within the specified time period is less than the threshold value TH7 (step S48).
[0054]
On the other hand, in step S44, when the width ΔWb of the binarized object is smaller than the threshold TH1 or larger than TH2 (NO in step S44), or in step S45, the height ΔHb of the binarized object is set to the threshold TH3. If the height ΔHg of the grayscale object is equal to or greater than the threshold TH4 (NO in step S45), or the height ΔHb of the binarized object is equal to or greater than the threshold TH3 in step S45-1 In either case (NO in step S45-1) or in step S46, when the upper end height position Yt of the object from the road surface is equal to or greater than the threshold value TH5 (NO in step S46), the detection is performed. It is determined that the performed object is not a pedestrian (step S49), the pedestrian determination process ends, and the process proceeds to step S34 shown in FIG. O as the process proceeds to step S37 in FIG. 5, the object is determined not to be a warning target.
[0055]
Similarly, in step S47, when the luminance variance of the mask area AREA3 is equal to or smaller than the threshold value TH6 (NO in step S47), further, in step S48, the area of the circumscribed rectangle and the area of the binarized object within the specified time period If the difference between the maximum value Max_Rate of the ratio Rate and the minimum value Min_Rate (Max_Rate−Min_Rate) is equal to or larger than the threshold value TH7 (NO in step S48), the detected object is a pedestrian. (Step S49), the pedestrian determination process ends, and the process advances to Step S37 in FIG. 5 as NO in Step S34 shown in FIG. 5, and determines that the target object is not a warning target.
[0056]
Also, in step S48, when the difference between the maximum value Max_Rate and the minimum value Min_Rate of the Rate which is the area ratio of the circumscribed rectangle and the binarized object within the specified time is less than the threshold TH7 (YES in step S48). Next, the image processing unit 1 performs a pedestrian determination for each shape of the target object extracted in more detail.
Specifically, first, it is determined whether or not the upper end height position Yt of the target object from the road surface is greater than a threshold value TH8 (a value appropriate as a height capable of distinguishing the upper and lower bodies of a pedestrian) (step S50). .
In step S50, when the upper end height position Yt of the object from the road surface is equal to or smaller than the threshold TH8 (NO in step S50), the process proceeds to step S51 in FIG. It is determined whether the width ΔWb of the binarization target is equal to or smaller than a threshold TH9 (a value appropriate for the width of the pedestrian's body) (step S51).
[0057]
FIG. 9 shows a processing procedure for identifying whether a lower body has been extracted by the binarization processing or a pedestrian who is sitting. In step S51, the width ΔWb of the binarization target object is smaller than the threshold TH9. If there is (YES in step S51), it is determined whether the weather around the host vehicle 10 is rainy (step S51-1).
Then, in (Step S51-1), when the weather around the host vehicle 10 is not rainy (NO in Step S51-1), in order to determine whether or not the object is a sitting pedestrian, It is determined whether the height ΔHg of the grayscale object is less than a threshold value TH10 (a value appropriate for the height of the pedestrian) (step S52).
[0058]
In step S52, when the height ΔHg of the grayscale object is equal to or larger than the threshold value TH10 (NO in step S52), it is assumed that this object corresponds to the torso of the pedestrian or the lower body, and the head exists at the upper part. In order to determine whether or not to perform the determination, it is determined whether or not the average luminance value Ave_A1 of the upper mask area AREA1 shown in FIG. 15 is larger than the threshold value TH11 (step S53).
In step S53, when the average luminance value Ave_A1 of the mask area AREA1 is larger than the threshold value TH11 (YES in step S53), since the body part may not easily radiate heat due to the influence of clothes, the luminance on the grayscale image may be reduced. As an object having a pattern, it is determined whether or not the luminance variance Var_A2 of the mask area AREA2 is larger than a threshold TH18 (step S53-1).
[0059]
If the luminance variance Var_A2 of the mask area AREA2 is larger than the threshold value TH18 in step S53-1 (YES in step S53-1), it is determined that the detected object is a pedestrian (step S54). The person determination process ends, and the process proceeds to step S35 in FIG. 5 as YES in step S34 shown in FIG. 5 to perform artificial structure determination.
Further, in (Step S51-1), if the weather around the host vehicle 10 is rainy (YES in Step S51-1), the determination of the mask area in Steps S52 to S53-1 is not performed, and Step S54 is not performed. Then, it is determined that the detected object is a pedestrian (step S54), the pedestrian determination process ends, and the process advances to step S35 in FIG. 5 as YES in step S34 shown in FIG. I do.
[0060]
On the other hand, in step S51, when the width ΔWb of the binarization target is larger than the threshold TH9 (NO in step S51), or when the average luminance value Ave_A1 of the mask area AREA1 is equal to or smaller than the threshold TH11 in step S53 ( If the luminance variance Var_A2 of the mask area AREA2 is equal to or smaller than the threshold value TH18 (NO in step S53-1) or further in step S53-1, the detected target object Is determined not to be a pedestrian (step S55), the pedestrian determination process ends, and the process advances to step S37 in FIG. 5 as NO in step S34 shown in FIG. 5, and determines that the object is not a warning target.
[0061]
If the height ΔHg of the grayscale object is less than the threshold value TH10 in step S52 (YES in step S52), the object is regarded as a sitting pedestrian, and the binarized object is viewed from the road surface. It is determined whether or not the upper end height position Yt of the target is larger than a threshold value TH12 (an appropriate value as a height that can distinguish a sitting pedestrian and a standing pedestrian) (step S56).
In step S56, when the upper end height position Yt of the binarized object from the road surface is larger than the threshold value TH12 (YES in step S56), the aspect ratio ASPECT of the circumscribed rectangle of the binarized object is represented. It is determined whether Asp is greater than or equal to threshold TH13 and less than or equal to TH14 (a value appropriate for a pedestrian) (step S57).
[0062]
In step S57, when Asp representing the aspect ratio ASPECT of the circumscribed rectangle of the binarized object is equal to or more than the threshold value TH13 and equal to or less than TH14 (YES in step S57), the circumscribed rectangle centroids 102 and 2 represented by the expression (9) are obtained. It is determined whether or not the distance Dis_c in the real space of the object to be valued from the center of gravity G100 is less than a threshold value TH15 (a value appropriate for a pedestrian) (step S58).
Dis_c = SQRT ((Xb-Xc) ² + (Yb-Yc) ² (9) If the distance Dis_c is less than the threshold value TH15 in step S58 (YES in step S58), for example, a pedestrian may be placed on an object whose ΔWb is 1.0 m or less and ΔHg is less than 1.0 m. Since the target object other than the above, specifically, the front part of the vehicle, is included, it is determined whether or not there is a part having a high degree of correlation with the previously registered head pattern in the upper mask area AREA1 of the binarized target object. A determination is made (step S59).
[0063]
In step S59, when there is a portion having a high degree of correlation with the pre-registered head pattern in the upper mask area AREA1 of the binarized object (YES in step S59), it is determined that the detected object is a pedestrian. Then (step S54), the pedestrian determination processing ends, and the process advances to step S35 in FIG. 5 as YES in step S34 shown in FIG. 5 to perform artificial structure determination.
[0064]
On the other hand, in step S56, when the upper end height position Yt of the binarized object from the road surface is equal to or smaller than the threshold TH12 (NO in step S56), or in step S57, the circumscribing of the binarized object is performed. When Asp representing the aspect ratio ASPECT of the quadrangle is smaller than the threshold TH13 or larger than TH14 (NO in step S57), or when the distance Dis_c is equal to or larger than the threshold TH15 in step S58 (NO in step S58), In step S59, if there is no part having a high degree of correlation with the previously registered head pattern in the upper mask area AREA1 of the binarized object (NO in step S59), the detected object It is determined that the object is not a pedestrian (step S55), and the pedestrian determination process ends. Proceeds to step S37 in FIG. 5 as NO in step S34 shown in, the object is determined not to be a warning target.
[0065]
In addition, in step S50 of FIG. 9, when the upper end height position Yt of the object from the road surface is larger than the threshold value TH8 (an appropriate value as a height that can distinguish the upper body and the lower body of the pedestrian) (YES in step S50). Then, the process proceeds to step S50-1 in FIG. 10, and it is determined whether the weather around the host vehicle 10 is rainy (step S50-1).
Then, in (Step S50-1), if the weather around the host vehicle 10 is not rainy (NO in Step S50-1), the object is an object floating in the air (for example, an object such as a curved mirror). In order to determine whether or not the height ΔHg of the grayscale object is greater than the threshold value TH16 (the same value as the above-described threshold value TH8), it is determined (step S60).
[0066]
FIG. 10 illustrates a processing procedure for identifying a pedestrian whose head or upper body has been extracted by the binarization processing. In step S60, the height ΔHg of the grayscale object is greater than the threshold value TH16. In this case (YES in step S60), since the target object is not an object floating in the air, it is next determined whether the head exists at the upper end of the target object area (AREA0) or the torso exists. I do. Specifically, first, since the head is exposed, it is determined whether or not the average luminance value Ave_A1 of the mask area AREA1 is larger than the threshold value TH17 (step S61).
[0067]
In step S61, when the average luminance value Ave_A1 of the mask area AREA1 is larger than the threshold value TH17 (YES in step S61), the body part may not easily dissipate heat due to the influence of clothes, so that the luminance pattern may be displayed on the grayscale image. As an object, it is determined whether or not the luminance variance Var_A2 of the mask area AREA2 is larger than a threshold TH18 (step S62).
If the luminance variance Var_A2 of the mask area AREA2 is larger than the threshold value TH18 in step S62 (YES in step S62), first, in order to determine a pedestrian whose head or upper body has been extracted by the binarization processing, It is determined whether or not the width ΔWb of the binarized object is equal to or smaller than a threshold TH19 (a value appropriate as a width capable of distinguishing the pedestrian's head or upper body) (step S63).
[0068]
Further, in (Step S50-1), when the weather around the host vehicle 10 is rainy (YES in Step S50-1), the determination of the mask area in Step S60 to Step S62 is not performed, and the process proceeds to Step S63. In order to determine the pedestrian whose head or upper body has been extracted by the binarization process, the width ΔWb of the binarized object is set to the threshold TH19 (appropriate as a width that can distinguish the pedestrian's head or upper body). Value) is determined (step S63).
[0069]
Next, in step S63, if the width ΔWb of the binarization target is larger than the threshold value TH19 (NO in step S63), at least the pedestrian whose upper body or whole body is extracted by the binarization processing is determined. In order to do so, it is determined whether the width ΔWb of the binarization target is equal to or smaller than a threshold TH9 (a value appropriate for the width of the pedestrian's body) (step S64).
Further, when the width ΔWb of the binarization target object is larger than the threshold TH9 in step S64 (NO in step S64), the binarization is performed to determine whether or not a plurality of pedestrians are walking in parallel. It is determined whether or not the width ΔWb of the target object is equal to or smaller than a threshold value TH2 (a value appropriate for the width of the pedestrian's body) (step S65).
[0070]
In the above determination, in step S60, when the height ΔHg of the grayscale object is equal to or smaller than the threshold TH16 (NO in step S60), or in step S61, the average luminance value Ave_A1 of the mask area AREA1 is set to the threshold TH17. If it is less than or equal to (NO in step S61), or if the luminance variance Var_A2 of the mask area AREA2 is equal to or less than the threshold TH18 in step S62 (NO in step S62), furthermore, in step S65, the binarization target object Is greater than the threshold value TH2 (NO in step S65), it is determined that the detected object is not a pedestrian (step S66), and the pedestrian determination process ends. , NO in step S34 shown in FIG. 5 and step S37 in FIG. Then, it is determined that the target object is not a warning target.
[0071]
On the other hand, in step S63, when the width ΔWb of the binarized object is equal to or smaller than the threshold value TH19 (YES in step S63), the object is a pedestrian whose head or upper body has been extracted by the binarization process. Then, the process proceeds to step S67 in FIG. 11 to determine whether Asp representing the aspect ratio ASPECT of the circumscribed rectangle of the binarized object is equal to or more than the threshold value TH20 and equal to or less than TH21 (a value appropriate for the pedestrian's head and upper body). (Step S67).
[0072]
FIG. 11 shows a processing procedure for identifying a pedestrian whose head and upper body have been extracted by the binarization processing. In step S67, the aspect ratio ASPECT of the circumscribed rectangle of the binarization target object is shown. When Asp is equal to or more than the threshold TH20 and equal to or less than TH21 (YES in step S67), it is determined whether or not the distance Dis_c in the real space between the circumscribed rectangular center of gravity 102 and the center of gravity G100 of the binarized object is less than the threshold TH15. (Step S68).
If the distance Dis_c is less than the threshold value TH15 in step S68 (YES in step S68), the detected object is determined to be a pedestrian (step S69), and the pedestrian determination process ends, and FIG. As YES in step S34 shown, the process proceeds to step S35 in FIG. 5 to perform artificial structure determination.
[0073]
On the other hand, if Asp representing the aspect ratio ASPECT of the circumscribed rectangle of the binarized object is smaller than the threshold TH20 or larger than TH21 in step S67 (NO in step S67), or in step S68, the distance Dis_c is equal to the threshold TH15. If the above is the case (NO in step S68), it is determined that the detected object is not a pedestrian (step S70), the pedestrian determination process ends, and the NO in step S34 shown in FIG. Proceeding to step S37, it is determined that the object is not a warning target.
[0074]
If the width ΔWb of the binarized object is equal to or smaller than the threshold TH9 in step S64 of FIG. 10 (YES in step S64), the object is extracted by at least the pedestrian's upper body or the whole body by the binarization processing. It is determined that the pedestrian is a pedestrian, and the process proceeds to step S71 in FIG. It is determined whether or not this is the case (step S71).
[0075]
FIG. 12 shows a processing procedure for identifying a pedestrian whose upper body or whole body has been extracted by the binarization processing. In step S71, Asp representing the aspect ratio ASPECT of the circumscribed rectangle of the binarization target object is shown. Is greater than or equal to the threshold TH13 and less than or equal to TH21 (YES in step S71), it is determined whether or not the distance Dis_c in the real space between the circumscribed rectangular center of gravity 102 and the center of gravity G100 of the binarized object is less than the threshold TH15. (Step S72).
[0076]
If the distance Dis_c is less than the threshold value TH15 in step S72 (YES in step S72), it is determined whether the weather around the host vehicle 10 is rainy (step S72-1).
Then, in (Step S72-1), if the weather around the host vehicle 10 is not rainy (NO in Step S72-1), the target object is an object other than a pedestrian, for example, a front part of the vehicle. Therefore, in the upper mask area AREA1 of the binarization target, it is determined whether or not there is a part having a high degree of correlation with the previously registered head pattern (step S73).
[0077]
In step S73, when there is a part having a high degree of correlation with the pre-registered head pattern in the upper mask area AREA1 of the binarized object (YES in step S73), it is determined that the detected object is a pedestrian. Then (step S74), the pedestrian determination processing ends, and the process advances to step S35 in FIG. 5 as YES in step S34 shown in FIG. 5 to perform artificial structure determination.
Further, in (Step S72-1), if the weather around the host vehicle 10 is rainy (YES in Step S72-1), the mask area is not determined in Step S73, and the process proceeds to Step S74 to be detected. It is determined that the target object is a pedestrian (step S74), the pedestrian determination processing ends, and the process advances to step S35 in FIG. 5 as YES in step S34 shown in FIG. 5 to perform artificial structure determination.
[0078]
On the other hand, when Asp representing the aspect ratio ASPECT of the circumscribed rectangle of the binarized object is less than the threshold TH13 or greater than TH21 in step S71 (NO in step S71), or in step S72, the distance Dis_c is equal to the threshold. If TH15 or more (NO in step S72), and furthermore, in step S73, if there is no part having a high degree of correlation with the previously registered head pattern in the upper mask area AREA1 of the binarization target (step S73). NO), it is determined that the detected object is not a pedestrian (step S75), the pedestrian determination process ends, and the determination in step S34 shown in FIG. Proceeding to S37, it is determined that the object is not a warning target.
[0079]
If the width ΔWb of the binarized object is equal to or smaller than the threshold value TH2 in step S65 of FIG. 10 (YES in step S65), the object is a target object because a plurality of pedestrians are walking in parallel. It is determined that many background areas are included in the circumscribed rectangle of, and the process proceeds to step S76 in FIG. 13, in which the rate of the area of the circumscribed rectangle and the area of the binarized object within the specified time is less than the threshold value TH22. It is determined whether or not (Step S76).
[0080]
FIG. 13 shows a processing procedure when the object is a plurality of pedestrians walking in parallel. In step S76, the area ratio of the circumscribed rectangle and the area ratio of the binarized object within the specified time are shown. If a certain Rate is less than the threshold value TH22 (YES in step S76), Asp representing the aspect ratio ASPECT of the circumscribed rectangle of the binarized object is equal to or more than the threshold value TH23 and equal to or less than TH14. It is determined whether it is an appropriate value) (step S77).
[0081]
In step S77, when Asp representing the aspect ratio ASPECT of the circumscribed rectangle of the binarized object is not less than the threshold value TH23 and not more than TH14 (YES in step S77), the above-described circumscribed rectangle centroid 102 and the centroid of the binarized object are determined. It is determined whether the distance Dis_c from the G100 in the real space is less than the threshold TH15 (step S78).
In step S78, when the distance Dis_c is less than the threshold value TH15 (YES in step S78), the detected object is determined to be a pedestrian (step S79), and the pedestrian determination process ends, and FIG. As YES in step S34 shown, the process proceeds to step S35 in FIG. 5 to perform artificial structure determination.
[0082]
On the other hand, in step S76, when Rate, which is the ratio of the area of the circumscribed rectangle to the binarized object within the specified time, is equal to or larger than the threshold value TH22 (NO in step S76), or in step S77, the binarized object When Asp representing the aspect ratio ASPECT of the circumscribed rectangle of the object is smaller than the threshold TH23 or larger than TH14 (NO in step S77), and further, in step S78, the distance Dis_c is equal to or larger than the threshold TH15 (step S78). NO), it is determined that the detected object is not a pedestrian (step S80), the pedestrian determination process ends, and the determination in step S34 shown in FIG. Proceeding to S37, it is determined that the target is not a warning target.
[0083]
In the present embodiment, the image processing unit 1 includes a weather detecting unit, an object extracting unit, a heat storage unit extracting unit, a pedestrian recognizing unit, and a shape determining unit. More specifically, S41-1 in FIG. 8 corresponds to the weather detection means, S1 to S13 in FIG. 3 correspond to the object extraction means, and S45, S46, S47 in FIG. 8, and S52, S53 in FIG. , S53-1, and S60, S61, and S62 in FIG. 10 correspond to a heat storage body extraction unit.
[0084]
S48 and S49 in FIG. 8, S50 and S54 to S59 in FIG. 9, S63 to S66 in FIG. 10, S67 to S70 in FIG. 11, S71 to S75 in FIG. 12, and S76 to S80 in FIG. Is equivalent to In particular, S59 in FIG. 9 and S73 in FIG. 12 correspond to the shape determining means.
[0085]
As described above, the vehicle periphery monitoring device according to the present embodiment extracts an object such as a pedestrian from the grayscale image of the image captured by the infrared camera by binarization processing, and then extracts the surroundings of the vehicle 10. If the weather is not rainy, a grayscale object within a range including the binarized object is extracted from the grayscale image by a change in the brightness of the grayscale image, and a plurality of grayscale objects are extracted in the grayscale object area. A search area is set, and a pedestrian in the search area is recognized based on the shape of the search area and the luminance variance of the search area.
When the weather around the host vehicle 10 is rainy, only the existence condition of the binarized object is determined, and the binarized image is determined based on the height and size of the binarized object in the image. Recognize pedestrians.
[0086]
Accordingly, when the weather around the host vehicle 10 is not rainy, for example, when the width of the image of the object is unnatural as a pedestrian, or when the image of the object is unnatural as a pedestrian The features of the pedestrian that removes these objects from the image of the target object and satisfies them include a portion with high luminance variance and a portion corresponding to the head, or a portion with high luminance variance and a portion corresponding to the torso. Or, it is determined whether the luminance variance of a wall or the like is low, and an image of an object whose luminance variance is different from the image of the pedestrian is removed from the image of the target object, and the detection accuracy of the pedestrian is improved. The effect of being able to improve is obtained.
[0087]
Further, when the weather around the host vehicle 10 is rainy, the amount of infrared radiation emitted by the target object decreases, so that the determination based on the luminance dispersion is not performed. For example, the width of the image of the target object is unnatural as a pedestrian. If the height of the image of the target object is unnatural as a pedestrian, it only removes these objects from the image of the target object to prevent false detection of pedestrians by judgment using luminance variance However, there is an effect that the detection accuracy of the pedestrian can be maintained.
[0088]
【The invention's effect】
As described above, according to the vehicle periphery monitoring device of the first aspect, when the weather around the vehicle is determined to be rainy by the weather detection unit, the pedestrian recognition unit is configured to output the directly extracted target Recognize pedestrians from objects. On the other hand, when it is determined that the weather around the vehicle is other than that, the heat storage medium is extracted from the object by the heat storage medium extraction means, and the pedestrian recognition means walks from the object except the heat storage medium. By recognizing the pedestrian, appropriate pedestrian recognition can be performed according to the situation around the vehicle.
Therefore, if the weather around the vehicle is not rainy, an object that emits heat (infrared rays) equivalent to that of a pedestrian, such as a wall that receives sunlight from the sun, is characterized by its luminance dispersion characteristic. Therefore, the object can be removed from the target object based on the above, and the effect of detecting the pedestrian can be improved. In addition, when the weather around the vehicle is rainy, the determination based on the luminance variance is not performed, and erroneous detection of the pedestrian by the determination using the luminance variance is prevented, and the detection accuracy of the pedestrian is improved. The effect of being able to maintain is obtained.
[0089]
According to the vehicle periphery monitoring device of the second aspect, when the weather around the vehicle is determined to be rainy by the weather detection means, the shape determination process is stopped, and the shape of the object on the image is stopped. Can be prevented from being erroneously determined by the pedestrian recognition means.
Therefore, when the weather around the vehicle is rainy, erroneous detection of a pedestrian by determining the shape of the target object can be prevented, and the effect of maintaining pedestrian detection accuracy can be obtained.
[Brief description of the drawings]
FIG. 1 is a block diagram illustrating a configuration of a vehicle periphery monitoring device according to an embodiment of the present invention.
FIG. 2 is a diagram showing attachment positions of an infrared camera, a sensor, a display, and the like in a vehicle.
FIG. 3 is a flowchart showing an object detection / warning operation of the vehicle periphery monitoring device of the embodiment.
FIG. 4 is a diagram showing a grayscale image obtained by an infrared camera and its binary image.
FIG. 5 is a flowchart showing an alarm determination processing operation of the embodiment.
FIG. 6 is a diagram illustrating a case where a collision is likely to occur.
FIG. 7 is a diagram showing an area division in front of a vehicle.
FIG. 8 is a flowchart showing a pedestrian determination processing operation of the embodiment.
FIG. 9 is a flowchart showing a pedestrian determination processing operation of the embodiment.
FIG. 10 is a flowchart showing a pedestrian determination processing operation of the embodiment.
FIG. 11 is a flowchart showing a pedestrian determination processing operation of the embodiment.
FIG. 12 is a flowchart showing a pedestrian determination processing operation of the embodiment.
FIG. 13 is a flowchart showing a pedestrian determination processing operation of the embodiment.
FIG. 14 is a diagram showing a binarized object shape feature amount according to the embodiment;
FIG. 15 is a diagram showing setting of a mask area according to the embodiment.
FIG. 16 is a diagram showing a luminance variance of the mask area AREA3 when the object is a part or the whole of a pedestrian or when the object is a wall.
FIG. 17 is a diagram showing a grayscale image in fine weather and a grayscale image in rainy weather in the vehicle periphery monitoring device of the embodiment.
FIG. 18 is a graph comparing a luminance histogram of a grayscale image in rainy weather and a luminance histogram of a grayscale image in fine weather in the vehicle periphery monitoring device of the embodiment.
[Explanation of symbols]
1 Image processing unit
2R, 2L infrared camera
3 Yaw rate sensor
4 Vehicle speed sensor
5 Brake sensor
6 Speaker
7 Image display device
10 own vehicle
S1 to S13 Object extraction means
S41-1 Weather detection means
S45, S46, S47, S52, S53, S53-1, S60, S61, S62 Heat storage body extraction means
S48-S50, S54-S59, S63-S80 Pedestrian recognition means
S59, S73 Shape determination means

Claims (2)

A vehicle periphery monitoring device that detects an object existing around the vehicle from an infrared image captured by an infrared camera,
Weather detection means for detecting the weather around the vehicle,
Object extraction means for extracting an object that emits infrared light from the infrared image,
From the object extracted by the object extraction means, a heat storage body extraction means for extracting a heat storage body that stores only heat given from outside without generating heat itself,
When the weather around the vehicle is judged to be rainy by the weather detecting means, a pedestrian is recognized from the object, otherwise, the pedestrian is extracted by the heat storage body extracting means. And a pedestrian recognizing means for recognizing a pedestrian from among the objects other than the heat storage body. The pedestrian recognition means,
Including a shape determination means for determining the shape of the object,
2. The vehicle periphery monitoring device according to claim 1, wherein when the weather around the vehicle is determined to be rainy by the weather detection unit, the shape determination unit is stopped. 3.

Images