JP4734884B2 - Person detection apparatus and method - Google Patents

Description

  The present invention relates to a person detection apparatus and method for detecting a person existing in front of or behind a vehicle, for example, and presenting it to a driver or the like.

  Conventionally, for example, in order to detect a person as an obstacle present in the traveling direction of a vehicle, an infrared image obtained by changing the brightness of a result of measuring a value corresponding to a temperature is captured by an infrared camera. As techniques for specifying a person, techniques described in Patent Document 1 and Patent Document 2 below are known.

  In the technique described in Patent Document 1, first, an infrared image is binarized with pixels near the temperature corresponding to a person's head and other pixels. Next, assuming that the head is a person's head based on the binarized and extracted width of the target object, the periphery of the target object is cut into a rectangle and the variance of the luminance value in the rectangle is examined. And when the dispersion | distribution of the luminance value in a rectangle has the characteristic which shows a person, the target object in a rectangle is detected as a person.

In the technique described in Patent Document 2, first, a part corresponding to a person's head is detected based on temperature, and a part corresponding to a torso is detected. And when the site | part which has temperature equivalent to a trunk | body exists under the site | part which has temperature equivalent to a head, the position in the image of the said head and trunk vicinity is detected as a person.
JP 2003-284057 A Japanese Patent Laid-Open No. 11-328364

  However, in the above-described conventional technology, for example, the human body has a preset temperature characteristic in which the head has the highest temperature and the body temperature is lower than the head. Since it is assumed that it is detected, if a temperature distribution is not detected with the preset temperature feature, a person cannot be detected.

  In other words, conventionally, no consideration has been given to the temperature distribution when the environment is different from the characteristics of the preset temperature distribution or when the person to be detected is wearing clothes. Specifically, in winter when the temperature is low and the clothes are worn, the body part of the clothing surface often has a lower temperature than the surroundings, and a part having a temperature equivalent to the body may not be detected. The temperature distribution is not established. Even outside the winter, the temperature corresponding to the body changes frequently depending on the type of clothes, the temperature of the day, the sunshine condition, etc., so it is often difficult to accurately detect the person depending on the season. .

  Moreover, in the technique described in Patent Document 1 described above, the head is first detected and the characteristics of the temperature distribution including the head are determined, but at the same height as the head in the image, There may be an object that has substantially the same size and temperature as the head. In such a case, there are many errors in detecting the presence of a person in a place that is not on the person's head, but is often on the road of the vehicle. Furthermore, there are cases where street lamps existing on the road of the vehicle are at a temperature similar to that of a person, and erroneous detection that a person exists in such a place is detected.

  Furthermore, because the temperature distribution and texture below the street lights and part of the vehicle are different from the surrounding walls and background, like the person's clothing, both head and torso temperatures are detected. However, there is a problem that other objects are often erroneously detected as persons.

  Accordingly, the present invention has been proposed in view of the above-described circumstances, and an object thereof is to provide a person detection apparatus and method that can reliably detect a person regardless of the ambient temperature, background object, and the like. And

  In the present invention, a thermal image composed of pixels having a luminance value corresponding to the temperature of the object is captured, and the thermal image has a luminance value within the range between the upper threshold value and the lower threshold value corresponding to the temperature of the person. A binarized image in which a pixel and a pixel outside the range between the upper threshold value and the lower threshold value of the luminance value corresponding to the person's temperature are distinguished is created, and the person's Detects an area where the pixel group within the upper and lower thresholds of the luminance value corresponding to the temperature is greater than or equal to the predetermined area, and within the predetermined distance threshold among the detected areas By grouping one or a plurality of regions existing in the group as human candidate regions, and determining whether the object of the human candidate region is a person based on the area and shape of the grouped human candidate regions The above-mentioned problem is solved.

  According to the present invention, referring to the binarized image, a region of a person portion in which a pixel group within the range between the upper threshold and the lower threshold is equal to or larger than a predetermined area is detected, Among them, one or a plurality of regions existing within a predetermined distance threshold is grouped as a human candidate region, and based on the area and shape of the human candidate region, whether or not the object of the human candidate region is a person Therefore, even when the luminance value corresponding to the temperature of the person is separated into a plurality of values by the binarization process, the head and the leg are regarded as one object and whether or not the person is a person. And can detect people reliably regardless of the ambient temperature, background objects, etc.

  Hereinafter, embodiments of the present invention will be described with reference to the drawings.

  For example, as shown in FIG. 1, the present invention detects a person using a thermal image captured by a far-infrared camera 10 (imaging means) in which the forward direction of the host vehicle 1 is an imaging range. This is applied to the detection device 20. 1A shows the optical axis of the far-infrared camera 10 when the host vehicle 1 is viewed from the side, and FIG. 1B shows the far-infrared camera when the host vehicle 1 is viewed from above. 10 optical axes and a reference coordinate system are shown. In this coordinate system, the horizontal axis is X, the vertical axis is Y, and the coordinates indicating the pixel position in the image can be obtained with the upper left pixel as the origin.

  The far-infrared camera 10 is provided, for example, in the vicinity of an instrument panel of a vehicle or an interior mirror, and has an imaging range in front of the vehicle. The far-infrared camera 10 generates thermal image data having a higher luminance value as the temperature value of the object is higher, and outputs the thermal image data to the person detection device 20. Therefore, the luminance value of the thermal image data is a value corresponding to the temperature of an object existing in front of the vehicle.

  The person detection device 20 has an input I / F for inputting thermal image data from the far-infrared camera 10, an image memory, and a configuration for performing processing as shown in steps S 1 to S 9 as shown in FIG. It consists of a program that describes the contents of various processes to be described later, a CPU that executes the program, and the like.

  That is, the person detecting device 20 is an input I / F that inputs thermal image data from the far-infrared camera 10 in step S1, an image memory that stores thermal image data in step S2, and a person for detecting a person in step S3. A temperature threshold setting unit for setting a temperature threshold, a binarization processing unit (binarization processing unit) that performs binarization processing in step S4, and a labeling processing unit (region detection unit) that performs labeling processing in step S5 ), A grouping target region detecting unit for detecting a region to be grouped in step S6, a grouping processing unit (grouping means) for performing grouping in step S7, and a person determining unit for performing person determination in step S8 (Person determination means).

  First, when the human image data from the far-infrared camera 10 is input by the input I / F (step S1), the person detection device 20 stores the thermal image data in the image memory (step S2).

  Next, the person detection device 20 uses the temperature threshold setting unit to detect a person present in front of the host vehicle 1 according to the current season, weather, temperature, and environment around the host vehicle 1. Set the value. At this time, the temperature threshold value setting unit sets an upper limit threshold value that is an upper limit value of the temperature corresponding to the person and a lower limit threshold value that is a lower limit value of the temperature corresponding to the person. A luminance value of a pixel corresponding to the lower threshold is set. In the following, when simply called “upper threshold value” and “lower threshold value”, a luminance value corresponding to the upper limit value of the temperature corresponding to a person and a luminance value corresponding to the lower limit value of the temperature corresponding to a person Shall be pointed to.

  Next, the person detection device 20 uses the binarization processing unit to set pixels between the upper threshold and the lower threshold set by the temperature threshold setting unit, and the upper threshold and the lower threshold. Binarization is performed with pixels other than those between (step S4). As a result, the person detection device 20 sets the luminance value corresponding to the temperature corresponding to the person to “1” and sets the luminance value not corresponding to the temperature corresponding to the person to “0” as shown in FIG. 5C and FIG. A binary image is created.

  Here, the temperature threshold value setting unit sets upper and lower threshold values suitable for detecting a person based on the weather and temperature of the day when the image is taken. The luminance value of the thermal image data captured by the far infrared camera 10 corresponds to a temperature value within the imaging range of the far infrared camera 10. Further, the skin temperature, which is the temperature of the person, is usually around 36 ° C., and in the thermal image data, when the distance between the person and the vehicle 1 is small, the skin portion of the person has a luminance value close to 36 ° C., As the distance between the person and the vehicle 1 increases, the infrared potential for temperature detection decreases, and the luminance value is affected by the temperature around the person.

  For example, in thermal image data including the same person in the same clothes, since the temperature of both the person and the background is high on a day when the temperature is high, heat having a high luminance value as shown in FIG. Since the temperature of the person and the background is low on a day when the temperature is low, the image data becomes thermal image data having a generally low luminance value as shown in FIG. Here, in FIG. 3 and the thermal image data shown below, the higher the luminance value, the lighter the color, and the lower the luminance value, the darker the image.

  Further, pedestrians on the street wear various clothes, and the clothes part in the case of light clothing is close to the body temperature, but the clothes part in the case of thick clothes is close to the outside air temperature. The temperature distribution detected in the above varies depending on the material of the clothes. Therefore, on a day when the outside air temperature is low, when a person with clothes of different materials existing at the same distance is imaged, the surface of the thick clothes shows a person such as the trunk because there is a difference between the skin temperature and the outside air temperature. Even the portion is closer to the temperature of the outside air and surrounding objects than the skin temperature. Therefore, as shown in FIG. 4, a thin pedestrian A detects a head and limbs at a temperature close to body temperature, and a torso is lower than the body temperature, but with a high luminance value. It is detected with a lower luminance value than the body part of A.

  Accordingly, the temperature threshold value setting unit can separate the person and the background regardless of the temperature so that the upper threshold value and the lower threshold value are different depending on whether the temperature is low or high. A range (person detection range) between the threshold and the lower threshold is set.

  Specifically, as shown in FIG. 5 (a), when thermal image data having a high brightness value on average is acquired when the temperature is high as shown in FIG. Both the upper threshold and the lower threshold are increased, and the range between the upper threshold and the lower threshold is set narrow. As a result, as shown in FIG. 5A, the luminance value of the person portion corresponding to the range between the upper threshold value and the lower threshold value is set for the thermal image data having the luminance value close to the person portion and the background portion. By performing binarization with “1” and non-applicable luminance value “0”, a binarized image shown in FIG. 5C is created.

  Also, as shown in FIG. 5 (d), when thermal image data having a low luminance value on average is acquired when the temperature is low, as shown in FIG. 5 (e), the upper limit threshold is set. Both the value and the lower threshold are lowered, and the range between the upper threshold and the lower threshold is set wide. Thus, as shown in FIG. 5D, the luminance value of the person portion corresponding to the range between the upper threshold value and the lower threshold value for the thermal image data in which the luminance value differs greatly between the person portion and the background portion. The binarized image shown in FIG. 5F is created by performing binarization by setting “1” to “1” and “0” as the non-applicable luminance value.

  On the other hand, as shown in FIG. 6 (b), a temperature threshold value as shown in FIG. 5 (e) is set on a warm day where the thermal image data as shown in FIG. 6 (a) is obtained. In this case, since the temperature value of the person portion and the temperature value of the background portion are close to each other, the background portion also falls within the person detection range, and the overall luminance value is “1” as shown in FIG. Becomes a binarized image. Further, on a cold day with low temperature where thermal image data as shown in FIG. 6 (d) is obtained, as shown in FIG. 6 (e), the temperature of the entire person is included as in the warm day of light clothing. When a temperature threshold value that is a wider person detection range than (e) is set, the background portion also becomes the person detection range, and as shown in FIG. A binarized image in which the person and the wall cannot be separated is obtained.

  For such inconvenience, the person detection device 20 can detect a person even in a binarized image that does not include a person's torso part by a grouping process that will be described later. The threshold value is set so that a large area is not extracted from the background.

  Next, the person detection device 20 uses the binarized image created by the binarization processing unit by the labeling processing unit, and is an area having a luminance value of “1”, corresponding to a person in the image A labeling process for detecting (person candidate area) is performed (step S5). At this time, the labeling processing unit searches for pixels having a luminance value of “1” in order from the upper left pixel of the binarized image, for example, and sets adjacent pixels having a luminance value of “1” or within a predetermined distance to each other. One area (label).

  As a result, the labeling process is performed on the binarized image of FIG. 7A in which the head, torso, and limbs of a person obtained on a warm day with a high temperature are continuous, as shown in FIG. 7B. As described above, the label a and the label b in which the head, the trunk, and the limbs are continuous can be extracted. Then, as shown in FIG. 7C, the labeling processing unit performs fillet diameter (maximum and minimum coordinates in the X-axis direction (left-right direction) and maximum in the Y-axis direction (vertical direction) of the region including the same label. And a rectangle including a minimum coordinate) can be set to extract the label area a and the label area b.

  Further, by performing a labeling process on the binarized image of FIG. 7D in which the human head, hands, and feet obtained on a cold day with low temperature are separated, as shown in FIG. 7E. In addition, head labels a and b, hand labels c, d, f and h and foot labels e and g can be extracted. Then, as shown in FIG. 7F, the labeling processing unit can extract the label areas a to h by setting the fillet diameter of the area including the same label for each label.

  Next, the person detection device 20 excludes the label area smaller than the predetermined image area from the label areas detected by the labeling processing section by the grouping target area detection section, and only the label area larger than the predetermined image area. Is selected (step S6). Here, the predetermined image area is set so as to detect a person existing within a person detection target distance from the host vehicle 1, and a too small label area (for example, about 1 to 2 pixels) is excluded as noise. The label area is set so as to be away from the host vehicle 1 so that the driver of the host vehicle 1 need not be warned.

  That is, the person detection apparatus 20 removes the labels that are not the grouping target from the labels obtained by the labeling processing unit, and sets only the labels that are the grouping target. Thereby, as shown in FIGS. 7A and 7D, the person detection device 20 has a luminance value between the upper threshold value and the lower threshold value, and appears in the binarized image. By removing the noise in the background portion, an image of only the human portion is obtained except for the background portion as shown in FIGS.

  Further, the grouping target area detection unit may delete not only a label area that is too small for a person but also a label area that is too large for a person. As a result, when a wall or the like is included in the background and the temperature is about the level of a person and included in the binarized image, a large label area such as the wall can be removed.

  Next, the person detection device 20 detects the distance between the labels selected by the grouping target area detection unit by the grouping processing unit, and the distances within the predetermined distance threshold are set as the same group. The grouping to be set is performed (step S7). Here, the predetermined distance threshold value includes a distance threshold value thy in the Y-axis direction (vertical direction) and a distance threshold value thx in the X-axis direction (horizontal direction).

  Here, in the state where the detection target of the person detection device 20 is a person and the person is walking, the person in the thermal image is detected vertically, or as described above, on a cold day when the temperature is low. In the acquired thermal image, the torso portion of the person is often not extracted by the labeling process, and therefore, a distance threshold value thy in the Y-axis direction is set to about the distance between the head and the leg. On the other hand, the distance threshold thx in the X-axis direction is approximately the lateral width of the person shorter than the distance threshold value thy in the Y-axis direction because the person is detected in a vertically long position. The distance is set to about the width of.

  Thereby, as shown in FIG. 8, when there are three labels a, b, and c, the grouping processing unit moves the labels a to c within the range of the distance threshold value thy. When it is within the range of the threshold value thx, the labels a to c are grouped as a group of labels detected from the same object, and a rectangular area including the maximum and minimum X coordinates and the maximum and minimum Y coordinates of the labels a to c In this case, candidate human regions having distances xLn, yLn (n = 1, 2, 3...) Are set.

  Specifically, as shown in FIG. 9A, when the label a and the label b are detected in the thermal image after the labeling process detected on the day when the temperature is high, the grouping processing unit first Focusing on the label a, a label existing in the distance range from the label a to the distance thresholds thx and thy is searched. In this case, the thermal image data includes the label a and the label b, but since the label b does not exist within the distance threshold thx of the label a, only the label a is grouped as a single object. Only b can be grouped as a single object. As a result, the grouping processing unit can set the person candidate area A including the label a and the person candidate area B including the label b, as illustrated in FIG. 9B.

  In addition, in the case of the thermal image data after the labeling process detected on a cold day when the temperature is low as shown in FIG. 9C and the labels a to h are detected, the grouping processing unit first displays the label a When the labels existing in the distance range of the distance threshold values thx and thy are retrieved, the labels c, d, and e are detected in the range of the distance threshold value thy having substantially the same X coordinate. Then, the grouping processing unit detects that the labels c, d, and e are within the range of the distance threshold value thy in the vertical direction and the distance threshold value thx in the horizontal direction, respectively. Labels a, c, d, and e are grouped as the same object. Similarly, the person detecting device 20 similarly searches for labels in the distance range of the distance thresholds thx and thy for the label b, and groups the labels b, f, g, and h as the same object.

  As a result, the grouping processing unit, as shown in FIG. 9 (d), regardless of whether or not the human torso portion has been detected by the labeling process, has a human candidate area with the vertical distance yL1 and the horizontal distance xL1. It is possible to detect the person candidate region B having A, the vertical distance yL2, and the horizontal distance xL2 as a grouping result.

  Although not shown in the example of FIG. 9, for a label in which no other label exists in the distance range of the distance thresholds thx and thy, a single label is set as a person candidate area, The result of grouping.

  Next, the person detection device 20 obtains the size and aspect ratio of the person candidate region detected by the grouping processing unit by the person determination unit, and determines whether or not the object in the person candidate region is a person. (Step S8). At this time, the person determination unit determines whether or not the aspect ratio of the detected person candidate area, that is, yLn: xLn is about the person as shown in FIGS. 9B and 9D. Here, when the aspect ratio of the person candidate area is within the range of 2: 1 to 3: 2 in the length: width direction, it is determined that the label in the person candidate area constitutes a person. In this way, the aspect ratio for determining that a label is a person does not always detect the label from the person's head to the feet, depending on the temperature, and the limbs can swing laterally when walking. Therefore, in order to make it slightly thicker than the aspect ratio of the actual person, the range of vertical: horizontal = 2: 1 to 3: 2 is set. Further, the aspect ratio for determining that the person is a person is not limited to the case where the aspect ratio is set to 2: 1 to 3: 2, but in a time zone where there are many children pedestrians, for example, 1.5: 1 As in the above, it may be set slightly longer than 1: 1.

  Furthermore, since the person walking speed is usually about 4 to 5 km / h, the person determination unit uses a plurality of person candidate areas that are detection results that are temporally continuous, and from the movement of the person, the number of continuous detections, and the like. Alternatively, an object that is detected continuously a plurality of times, does not move rapidly, and has an aspect ratio of the order of a person may be determined as a person.

  Next, when the person determination unit determines that the object in the grouping region is a person, the person detection device 20 performs, for example, an alarm for presenting the presence of the person on a vehicle-mounted display, vehicle control, or the like ( Step S9). At this time, the person detection device 20 may perform an alarm or vehicle control when a person is continuously detected a plurality of times (for example, three times) in front of the host vehicle 1 based on the results of steps S1 to S8. good.

  Further, the person detection device 20 obtains the position on the X axis in the thermal image of the candidate person area where the object is determined to be a person, obtains the azimuth θ with respect to the host vehicle 1 based on the position, and generates an alarm or light Irradiate. At this time, as shown in FIG. 17 described later, the person detection device 20 is based on the orientation θ where the person exists, that is, based on the relationship between the position point P of the person in the real space and the position xp on the thermal image. The azimuth θ is obtained from Equation 4 described later and Equation 5 below.

tan θ = xp / f (xp: X-axis coordinate at which a person is detected in the thermal image) (Formula 5)
Thus, the person detection device 20 can detect a person regardless of the ambient temperature and clothes, can be applied to a system that notifies the driver of the position of the person, and can reliably notify the position of the person. In particular, a thermal image using infrared rays measures the ambient temperature, can detect a person even in a scene with poor visibility at night, and can improve the driver's visibility with respect to the person ahead.

  As described above in detail, according to the human detection device 20 to which the present invention is applied, the pixel group within the range between the upper threshold value and the lower threshold value is larger than a predetermined area with reference to the binarized image. The area of the person part is detected, and one or more areas within the predetermined distance threshold are grouped as person candidate areas, and the area and shape of the person candidate area are determined. Based on this, it is determined whether or not the object in the person candidate region is a person. Even when the luminance value corresponding to the temperature of the person is separated into a plurality of values by the binarization process, the head and legs The part can be regarded as one object, and it can be determined whether or not it is a person.

  Therefore, according to the person detection device 20, an object having an area equivalent to the person within a predetermined distance threshold value can be regarded as one object. For example, the person is thickly worn and the torso is outside air. If the temperature threshold is such that only the part close to the skin temperature whose temperature change is small due to the temperature is detected by binarization because the temperature is almost the same as Even if it is possible to group as a single object using only parts with small temperature changes such as the head, legs, and back of the hand, regardless of whether the body part with large temperature changes can be detected or not, Detection is possible. As a result, the person detection device 20 can stably detect a person regardless of the season or temperature.

  Next, another embodiment that is effective in addition to the person detection device 20 as described above will be described. In the following description, the same name is used for a portion having substantially the same function as the person detection device 20 described above.

“Another embodiment of binarization processing”
First, another embodiment of the binarization process will be described.

  As described above, in the binarization process of the person detection device 20, when appropriate upper and lower thresholds are set for the outside air temperature, as shown in FIGS. 5 (c) and 5 (f). A binarized image in which a person and a background are distinguished from each other can be created. However, when an inappropriate upper threshold and lower threshold are set for the outside air temperature, FIG. Thus, no person appears at all, or many background objects appear superimposed on the person as shown in FIG.

  On the other hand, in the labeling process and the noise removal process after the binarization process, the person detection device 20 leaves only the label extracted as a part of the person and can remove the label extracted as the background object. Set the threshold and lower threshold.

  For this reason, the upper and lower luminance values in the thermal image when observing the skin of a person at a certain temperature and the temperature of the background area that is wide on the thermal image such as the road surface and the sky area at the temperature are statistically calculated in advance. Keep track. Statistically, as shown in FIG. 10 (a), when a temperature region such as a road surface or sky and a person temperature region are partially overlapped with each other, a road surface or sky Set upper and lower thresholds that are ranges excluding the temperature range.

  By using such an upper limit threshold and a lower limit threshold, the person detection device 20 can create a binarized image excluding a temperature region such as a road surface or sky.

  Further, as an object to be imaged with a person, not only the sky and the road surface, but also when the own vehicle 1 is traveling on a narrow road such as an alley, the fence or wall of the house occupies most of the thermal image There is also. For such a scene, the human detection device 20 detects, by image processing, a wall, a road surface, and a sky having a larger area than an area corresponding to a person with few edge portions in the thermal image data. The background luminance value (background temperature) of the wall or the like is obtained. Then, as shown in FIG. 10B, the person detection device 20 removes the background brightness value from the person temperature area when the detected background brightness value overlaps with a preset person temperature area. Change the upper threshold and lower threshold.

  As a result, the person detection device 20 can set the upper and lower thresholds for detecting the luminance value corresponding to the person according to the actually detected background temperature, and the background can be reliably detected. An excluded binary image can be created.

  As described above, the person detection device 20 is a range of luminance values corresponding to the temperature range including the skin temperature of the person and the clothing surface temperature when the person is lightly worn, and has a luminance value corresponding to the ambient temperature. A binarized image can be created by setting an upper threshold and a lower threshold excluding the range. In addition, by setting the upper and lower thresholds excluding the ambient temperature, the range between the upper and lower thresholds can be narrowed, and other background objects can be It is possible to suppress the erroneous detection of detecting that the temperature is equal and reliably detect the temperature corresponding to the person.

  Note that if the range between the upper threshold and the lower threshold is narrowed, a torso portion with a wide range of temperatures that can be taken due to the difference in temperature does not appear in the binarized image. With the above-described grouping, the head and the leg can be regarded as the same object, and it can be detected that the object in the person candidate area is a person based on the aspect ratio of the person candidate area.

  Further, when the temperature close to the human torso is substantially the same as the background temperature and the range between the upper threshold value and the lower threshold value is widened, the labeling process as shown in FIG. People and the background are detected as the same area, and it is often impossible to perform the subsequent grouping process. By setting a narrow person temperature area excluding the background temperature, the shape of the person can also be accurately determined. It is possible to accurately detect a person.

“Other Embodiments of Grouping Processing”
Next, another embodiment of the grouping process will be described.

  In the above-described grouping process, a distance threshold value corresponding to a person included in the thermal image is set in advance, and the process of grouping a plurality of labels has been described. The smaller the distance from the host vehicle 1, the smaller is detected.

That is, as shown in FIG. 11A, the far-infrared camera 10 detects the temperature of the person on the imaging surface 10b via the camera lens 10a, and the distance z from the host vehicle 1 to the person and the focal length f Accordingly, the person height hc (= number of pixels) appearing on the imaging surface 10b differs from the person height H. Specifically, the person height hc in the image is
hc = f · H / z (Formula 1)
It is expressed by the following arithmetic expression.

  That is, from the above formula 1, as the distance z between the host vehicle 1 and the person increases, the person height hc in the image decreases, and is present near the host vehicle 1 as shown in FIG. The height hc1 in the image of the person who is far away from the host vehicle 1 is smaller than the height hc2 in the image of the person who performs.

In addition, the person detection device 20 calculates the distance z between the host vehicle 1 and the person in Expression 1 by performing the calculation of Expression 2 below. Here, as shown in FIG. 12A, the installation height L of the far-infrared camera 10 with respect to the road surface is known. From the installation height L of the far-infrared camera 10 and the focal length f of the far-infrared camera 10, The position yd in the image where the position of the person's foot is imaged is as shown in Equation 2,
yd = f · L / z (Formula 2)
It is expressed by the following arithmetic expression.

  That is, as shown in FIG. 12B, when the Y coordinate corresponding to the human foot detected in the image, that is, the lowest coordinate of the human temperature region is determined as the foot coordinate yd, the distance z is By substituting the distance z into Equation 1, the in-image height hc of the person existing at the coordinate yd can be obtained.

  As described above, the in-image height hc of the person according to the distance z corresponds to the distance between the head and the leg that should be imaged by the far-infrared camera 10 for the person existing at the distance z. The distance threshold value thy is set to the in-image height hc obtained by Expression 1. Further, the person detection device 20 sets the distance threshold thx so that the aspect ratio falls within a predetermined range (2: 1 to 3: 2) from the distance threshold thy obtained by Equation 1, thereby walking. A distance threshold thx corresponding to the length of a person's stride is set. For example, when the distance threshold value thy obtained by Expression 1 is 1 m, the distance threshold value thx is set to about 0.7 m from a person of a general system as the length of the stride. In the case of performing human detection during a time when there are many children pedestrians such as in the daytime or in front of school, the distance threshold value thy is set to about 50 cm, and the distance threshold value thx is set to about half of 0,35 cm. You may do it.

  As described above, the distance threshold values thy and thx used in the grouping process are determined by calculating the distance z from the actual focal length f and the installation height L of the far-infrared camera 10 and setting the distance z according to Equation 1, thereby Set the distance threshold thy in the vertical direction between the head and legs of the person or the head and legs of the person included in the image, and the distance threshold thx in the horizontal direction that is the width of the legs of the person can do.

  Therefore, according to this person detection device 20, in the grouping process, it is possible to set the person temperature region by grouping as a shape close to the size and shape of the person, and more accurately detect the person. This has the effect of preventing false detection of detecting an object having a shape different from that of a person as a person.

  In the case of repeatedly performing the processing of step S1 to step S9, when a person is detected after setting the optimum distance threshold values thy and thx using the expressions 1 and 2 in the grouping process, It is desirable to be able to detect the approximate height and stride from the person's label and to set the distance threshold values thy and thx again. Accordingly, it is possible to avoid erroneous detection that cannot be determined as a person in the next process even though a certain person is detected once.

  Further, in the person detection device 20 described above, the label detection stability is low and the area of the label is often observed on a cold day when the temperature is low and the person is wearing a thick coat. . In addition, for a person who has been once determined to be a person by the person determination unit, it can be assumed in advance that a person exists at the position of the person candidate area of the person.

  On the other hand, in the case where it is determined that the person is a person once, the grouping process determines whether or not each label is within the distance threshold thx, thy in the vicinity of the person candidate area where the person exists. In addition, distance threshold values thx and thy that are longer than the previous distance threshold values thx and thy are used. At this time, the grouping processing unit has the current distance threshold values thx and thy about 1.1 to 1.2 times longer than the previous distance threshold values thx and thy before being determined as a person. Set to.

  In addition, when the grouping processing unit continuously performs the processes of step S1 to step S9 and continuously detects a person, the area of the person in the thermal image gradually changes. On the other hand, the grouping processing unit gradually increases the distance thresholds thx and thy when the distance between the person and the vehicle 1 that gradually increases the candidate area when the person is determined to be a person approaches. However, when the distance between the person and the vehicle 1 such that the person candidate area when it is determined to be a person gradually decreases, the distance thresholds thx and thy are gradually shortened.

  According to such a grouping processing unit, in the case where the process of steps S1 to S9 is continuously performed to continuously detect a person, in the next grouping determined to be a person by the person determination unit In the vicinity where the person exists, the distance thresholds thx and thy are changed longer than a preset value. Therefore, even if the label is detected to be small due to the climate or the like, the grouping is surely performed. A person can be detected.

  Further, according to this grouping processing unit, the distance threshold thx, thy is changed longer when the person approaches the own vehicle 1, and the distance threshold thx, thy when the person moves away from the own vehicle 1. Therefore, even if some labels, such as limbs, cannot be detected due to changes in temperature or movement of the person after the person determination unit has detected that the person has been detected by the previous time, In addition to being able to detect, the distance thresholds thx and thy are changed according to the movement of the person with respect to the host vehicle 1, so that it is possible to prevent erroneous detection of grouping background objects of persons. Thereby, according to person detection device 20, a person can be detected stably and a person can be tracked to own vehicle 1.

“Other Embodiments of Person Determination Processing”
Next, another embodiment of the person determination process will be described.

  In this person determination process, the aspect ratio of the person candidate area for determining that the person is a person is within the range of the aspect ratio equivalent to the person in the preset thermal image, and the lowest end of the person candidate area When the position is a position corresponding to the road surface in the thermal image, it is determined that the object in the person candidate area is a person.

  Here, as shown in FIG. 12 (a), if the distance z from the infrared camera 10 where the person is present is known to some extent, the feet of the person walking on the road surface can be expressed by the above equation (2). When substituting for, the value is in the range that Equation 2 can take.

  For example, when the detection range of a person to be detected is set to a range of about 5 to 50 m from the front of the host vehicle 1, the feet of the person existing 50 m ahead, which is the farthest distance, are the highest in the thermal image The foot of the person in front of the vehicle 1 in front of the host vehicle 1 appears at a lower position in the thermal image. This is because, as shown in FIG. 12 (b), the foot of the person in front of 50m ahead of the host vehicle 1 from the in-image position yd where the position of the foot of the person based on Equation 2 above is imaged is L -It is a position below f / 50.

  In this way, the position of the person in the thermal image moves up and down depending on the distance between the person to be detected and the far-infrared camera 10, as shown in FIG. 13, in a scene where there is a street lamp above the person and the wall. When the above-described grouping process is performed, a person candidate area is also detected for the street lamp separately from the person candidate area of the person. From Equation 2, the lowest end position ye1 of the person candidate area and the lowest end position ye2 of the street lamp Is obtained.

  Further, the lowermost position yd (= L · f / 50) of the human candidate region at the longest detection distance (50 m) of the person set in the human detection device 20 and the in-image offset A are given to the lowermost position. The lowest end position yd ′ (= yd + A) is as shown in FIG. Here, the image offset A is a predetermined offset amount in the thermal image set in advance, and the head of the person when the feet of the person such as a person whose feet are cold or a person wearing boots is not detected is detected. This is an offset amount for enabling a person to be detected only by the part and the hand, and is set based on the average length of the leg part appearing in the thermal image in the Y-axis direction.

  Therefore, when the person detection distance range from the own vehicle 1 of the person detection device 20 is set, the lowermost position of the person candidate region appears below yd ′. That is, when the person candidate area is obtained by the grouping process, the person determination unit determines that the object in the person candidate area is a person only when the lowermost position ye1 in FIG. 13 is located below yd ′. It can be determined that there is. On the other hand, the person candidate area whose bottom end position is ye2 is positioned higher than yd ′, and therefore, a person farther than the preset person detection distance range of the person detection apparatus 20 or the person candidate area shown in FIG. Thus, it can be determined that the street lamp is located higher than the person.

  As described above, according to the person determination unit, when determining whether or not the object in the person candidate area as a result of the grouping is a person, even if the aspect ratio and size of the person candidate area are equivalent to a person, If the lowermost position of the candidate area is not close to the position corresponding to the road surface, it can be determined that the object in the person candidate area is not a person. This prevents, for example, erroneous detection that determines that a taxi lantern or gate light that is present above the head and whose temperature is often observed as the skin of a person is a person on the road. Can do.

  Next, still another embodiment of the person determination process will be described.

  This person determination process corrects the lowest end position of the candidate person area determined to be a person multiple times based on the average value of the lowest end positions of the plurality of person candidate areas or the plurality of candidate person areas. A lower end position is set, and it is determined whether or not the lowermost position is a position corresponding to a road surface in the thermal image. That is, this person determination process is not a person when the process of determining whether the object in the person candidate area is a person based on the lowermost position of the person candidate area described above is performed a plurality of times. It is characterized in that the lowermost position of the person candidate area for determining that is corrected is corrected.

  Here, on a cold day with a low ambient temperature, the person's legs are often covered with a thick cloth such as pants. Therefore, the surface temperature is lower than that of the head and hands. In many cases, the temperature range becomes unstable because the labeling process does not detect the label.

  That is, on the day when the temperature is low, as shown in FIG. 14A, at time t, a thermal image in which the luminance value of the leg is not significantly different from the luminance value of the head can be obtained. As a result of the conversion process and the labeling process, as shown in FIG. 14B, not only the head and the hand but also the leg label can be detected. On the other hand, on the day when the temperature is low, at time t + Δt, as shown in FIG. 14C, a thermal image in which the luminance value of the leg is significantly lower than the luminance value of the head becomes a binarization process and a labeling process. As a result, as shown in FIG. 14D, the head and hand labels can be detected, but the leg labels cannot be detected.

  In this way, when the labels of the respective parts of the person are detected or not detected, the lowermost position ye1 of the person candidate area is the same as in the case of FIG. This is different from the case of FIG. 14D in which no label is detected. In other words, at the time t + Δt, even if it can be determined that the object is a person because the lowermost position of the person candidate area is ye1 at time t and the person candidate area is lower than the above-described yd ′, The lowermost position ye1 of the region changes upward. As a result, as described above, the lowermost position ye1 of the human candidate area at time t + Δt is higher than the lowermost position yd ′ (= yd + A) at which it can be determined that the lowermost position of the human candidate area exists near the road surface. Then, it cannot be determined that the object in the person candidate area is a person.

  On the other hand, the person determination unit compares the label detection position at time t with the label detection position at time t + Δt. For example, since the detection part of the upper body is the same, the label detection position at time t If it is determined that the degree of overlap with the label detection position at time t + Δt is large, it can be determined that the person candidate area at time t and the object in the person candidate area at time t + Δt are the same person.

  Therefore, the person determination unit obtains an average value of the lowest end position ye1 of the person candidate area before time t + Δt and the lowest end position ye1 of the person candidate area at time t + Δt, and the person candidate area at time t + Δt By setting the lowermost position ye1, the lowermost position ye1 in FIG. 14 (d) can be corrected to the lowermost position ye1 of the person candidate region at time t + Δt as shown in FIG. 14 (e).

  In addition, when the lowermost position ye1 of the person candidate area changes rapidly, the person determination unit interpolates the lowermost position ye1 of the current person candidate area from the lowermost position ye1 of the person candidate area acquired in the past. Thus, the value of the lowest end position ye1 of the person candidate area is obtained from the past value.

  That is, even when the grouping result is a detection result as shown in FIG. 14D, applying the lowest end position ye1 of the past human candidate region as shown in FIG. 14B, or If the average value is about five times in the past, the lowermost position ye1 of the final person candidate area can be set to a position as shown in FIG.

  Therefore, according to this person determination unit, even when the label of the person being detected continuously becomes only the upper body, the same person can be reliably detected, and the day when the temperature is low or a pedestrian runs Even if it is impossible to detect a stable label, such as when the leg moves fast, etc., if a person can be detected continuously in the past, If the label is detected, even if the detection of the label of the leg becomes unstable, it can be determined that they are the same person.

  Therefore, even if the leg portion is not detected, if it is detected in the past, the lowermost position of the person candidate area is higher than the lowermost position yd ′ (= yd + A), so that it is erroneously determined as an overhead object. Can be prevented from being detected.

“Another embodiment of labeling process”
Next, still another embodiment of the labeling process will be described.

  This labeling processing unit is limited to the range below the position in the thermal image corresponding to the imaging position where the person's head is imaged in the imaging range in real space by the far-infrared camera 10. A label is detected in which a pixel group within a range between an upper limit threshold and a lower limit threshold of a luminance value corresponding to temperature is a predetermined area or more.

  Here, as shown in FIG. 15A, in a scene where a pedestrian is walking on a road where a background street lamp exists at a position higher than the head, the pedestrian person candidate area and the street lamp If the position of the person candidate area is different, the object in the person candidate area of the pedestrian can be determined to be a person based on the aspect ratio and size of the person candidate area. It can be determined that the person is not a person.

  Then, at time t + Δt, as shown in FIG. 15B, when the pedestrian moves to a position just below the streetlight and the X-axis coordinates are substantially the same, the pedestrian and the streetlight become one object by the grouping process. There is a risk of being determined as. Further, as shown in FIG. 15 (c), when the pedestrian moves to the left side of the streetlight, the person candidate area corresponding to the streetlight is a portion corresponding to the head of the person once at time t + Δt in FIG. 15 (b). Therefore, at time t + 2Δt, the bottom end position of the street candidate person candidate area is set to the person in FIG. 15B by the process of correcting the bottom end position of the person candidate area with reference to FIG. There is a possibility that the position of the lowermost position of the candidate area is set, and an erroneous determination is made that a person is also present in the person candidate area of the streetlight.

  Therefore, as illustrated in FIG. 16, the binarization processing unit and the labeling processing unit exclude pixels at a height that clearly does not capture a person from the binarization and labeling processing range. Specifically, as shown in FIG. 16, a processing restriction line is provided above the range in the thermal image in which a person appears. At this time, the binarization processing unit or the labeling processing unit designates four points in the thermal image, configures a processing limit line by a straight line connecting the four points, and excludes the lower part of the processing limit line from the processing target. limit.

  As a result, as shown in FIG. 16A, the binarization processing unit can exclude the street lamp portion above the person from being processed, and the labeling processing unit removes the label about the street lamp. Thus, it is possible to detect a label only for a person.

  Specifically, the person detection device 20 sets a distance range from the host vehicle 1 to detect a person present. Here, for example, it is usually a distance range of 5 to 50 m ahead where the driver of the host vehicle 1 needs attention and is on a road with a width of about 4 m ahead. In this case, if the height of the person's back is H, as shown in FIG. 16A, the position ys at which the person's head is imaged on the thermal image is the position of the person's foot calculated by Equation 2. Since the position is higher by the person height hc in the thermal image calculated by Expression 1 from the position yd in the imaged image, it is obtained by Expression 3 below.

ys = f · L / z−f · H / z (Formula 3)
Further, in the person detection device 20, when the person detection range is limited to only on the road on which the host vehicle 1 travels, there is no person that suddenly appears in the immediate distance in front of the host vehicle 1, and the person who appears in front is gradually closer in distance. Or a scene where a person jumps out from the side. Therefore, near the left and right edges of the thermal image, it is necessary to provide a processing limit line having a height that can detect the head of a person existing at a short distance (about 5 m). A processing limit line having a height for continuously detecting a person who has been detected once in time is provided as a detection target. In addition, the temperature of a person at a long distance from the host vehicle 1 is easily detected as a value close to the ambient temperature, but the temperature of a person in front of the host vehicle 1 at a short distance is detected at a temperature equivalent to the person at the trunk and legs. Therefore, even if the head is out of the detection range, detection is possible.

  Therefore, as shown in FIG. 16C, the coordinate yu1 indicating the heights of the left and right ends of the processing restriction line is, for example, about 5 m at the distance z in Equation 3 so that a person entering a short distance can be detected. By substituting the short distance value, the height ys at which the person's head is detected is determined. For the coordinate yu2 indicating the height of the processing limit line near the center of the image, a long distance value of, for example, about 50 m is substituted for the distance z in Equation 3 so that the head of a person existing at a long distance can be detected. Thus, the height ys at which the head of the person is detected is determined. Note that the left and right end heights yu1 and yu2 of the processing restriction line may be set to values slightly offset upward in consideration of the pitching motion of the host vehicle 1 and the like.

  Further, as shown in FIG. 17A, a point R at the road surface end (position where the distance from the optical axis is XR) at the front distance z from the camera lens 10a is as shown in FIG. Appears as coordinates xr on the thermal image. The coordinate xr on the thermal image of the road surface end can be calculated by the following equation 4.

xr = f · XR / z (Formula 4)
Then, in order to be able to detect a distant person existing on the road surface on which the host vehicle 1 travels in the vicinity of the center on the thermal image, the left and right XRs with the optical axis as the center, the focal length f Substituting the distance z of 50 m from the host vehicle 1, the left and right road surface edges xl and xr on the thermal image are obtained, and the X axis coordinates xl and xr with respect to the Y axis coordinate yu2 in FIG. it can.

  In this way, the person detection device 20 displays a processing limit line consisting of four points: two points of the Y-axis coordinates yu1 at the left and right ends on the thermal image and two points of the X-axis coordinates xl and xr with respect to the Y-axis coordinates yu2. It is possible to set the thermal image above the processing limit line in the binarization process to be excluded from the processing target, and to remove the label detected above the processing limit line in the labeling process from the processing target.

  Therefore, according to the person detection device 20, the position in the real space of the person to be detected in front of the host vehicle 1 is limited, and the binarization process is performed only on the area below the position where the person's face can be imaged. By limiting the labeling process, other than the position on the thermal image where the person's head can be imaged, for example, the third floor and the fourth floor of a distantly high building, a position close to an apparent overhead or empty area is excluded from the processing target. By doing so, it is possible to prevent erroneous detection of a temperature zone present at a distant high position as a human head.

  When a pedestrian is present at a close distance of the host vehicle 1, the person's head is imaged above the thermal image corresponding to a distant sky area. If the person including the torso is detected, the object in the human temperature region is detected using the fact that the body and legs that are likely to become unstable are easily detected. It can be determined that the person is a person, and erroneous detection that the person is a person can be prevented when a label only for the upper head is detected.

  Thereby, according to the person detection device 20, as shown in FIG. 15 described above, the detection stability of the label is low, whereas the size of the person temperature region is changed by the label of the head only. Therefore, it is possible to eliminate the determination of a streetlight or the like existing above a person as a person.

“Other Embodiments of Person Determination Processing”
Next, still another embodiment of the person determination process will be described.

  In this person determination process, the lowermost position of the person candidate area is lower than the position in the thermal image corresponding to the imaging position where the leg of the person is imaged in the imaging range in the real space by the far-infrared camera 10. In the case where the object is an object in the person candidate area, it is determined that the object is a person.

  In the above-described binarization process and labeling process, the person determination unit detects that the object in the person candidate region is a person, as shown in FIG. A processing limit line is provided for determining whether or not. This processing restriction line is provided to determine the position of the person's foot.

  Here, the person detection device 20 is set to detect a person 5 to 50 m ahead of the host vehicle 1, and there is a foot of a pedestrian walking on the road in consideration that the person exists within the road width. An area to be set is set in advance. Here, the position in the thermal image of the foot position of the person in the range of 5 to 50 m ahead of the host vehicle 1 is obtained by the above-described equation 2.

  That is, the left and right end heights ye1 of the processing limit line are obtained by substituting the shortest value (5 m) of the detection target into the distance z in Equation 2, and the coordinate ye2 indicating the height of the processing limit line near the center of the image is the distance It is obtained by substituting the longest detection target value (50 m) into Equation 2 for z. Note that the heights ye1 and ye2 of the processing limit lines may be set to values slightly offset in the upward direction in consideration of the pitching motion of the host vehicle 1 and the detection of the foot label.

  Further, as described with reference to FIG. 17, the coordinates xr and xl on the thermal image of the road surface end can be obtained as the X-axis coordinates xl and xr with respect to the Y-axis coordinate ye2 in FIG. .

  As described above, the person detection device 20 displays a processing limit line including four points, that is, two points of the Y axis coordinate ye1 at the left and right ends on the thermal image and two points of the X axis coordinates xl and xr with respect to the Y axis coordinate ye2. It is possible to determine that the person is a person when the label of the person's feet is detected below the processing limit line in the person determination process.

  Accordingly, as described with reference to FIG. 13 above, the position of the person's foot is set to the in-image position yd where the position of the same person's foot is imaged at all the X coordinates of the thermal image. 19 is compared with the lowermost position of the person candidate area, for example, a warm part corresponding to the person exists above the wall, such as the left wall shown in FIG. If it is located, it may be erroneously detected as a person. On the other hand, since a far-away person cannot be imaged at the left and right ends of the thermal image, the processing limit line is taken into consideration by lowering the position of the processing limit line at the left and right ends of the thermal image as shown in FIG. A line is set. Thereby, it is possible to correctly determine the foot position of the person on the road on which the host vehicle 1 travels, and more accurately distinguish between the person and an object other than the person.

  Therefore, according to such a person determination process, the position of the person to be detected in front is limited in real space, the position where the person's leg can be imaged within the limited range, and the leg is Compare the positional relationship between the lowest limit position of the candidate human regions of the grouping result and the processing limit line by providing a processing limit line at a position that has an offset on the thermal image by a predetermined height from the imaged position. When the lowermost position of the person candidate area is below the processing limit line, it can be determined that the object in the person candidate area is a person.

  In addition, when the far-infrared camera 10 is used so that the optical axis of the far-infrared camera 10 is parallel to the center of the road existing ahead, the left and right ends of the thermal image are captured on the roadside walls and the buildings on the left and right of the road Therefore, there is no possibility that the legs of the person are imaged above the left and right ends. On the other hand, since the vicinity of the center of the thermal image is the road surface in front of the host vehicle 1, a leg portion of a person who is far away may be captured near the center in the vertical direction of the thermal image. On the other hand, the person determination unit takes into account the region on the thermal image where the person is imaged at the left and right end portions of the thermal image and the approximate center position, and in the thermal image where the human leg can be imaged. Since the area is set, it is possible to accurately determine the person as compared with the case where the in-image position yd is simply set.

  In addition, as described above, this person detection device 20 performs processing so that a distant pedestrian can be detected near the approximate center in the thermal image, and a nearby pedestrian can be detected at both the left and right ends in the thermal image. Although the limit line is set, when the host vehicle 1 is traveling, it is limited to the region in the thermal image corresponding to the road surface on which the host vehicle 1 travels in the imaging range in the real space by the far infrared camera 10. Then, binarization processing, labeling processing, person determination processing, and the like may be performed.

  That is, for example, when the road width is narrow, the person detection device 20 sets the value of the distance XR from the optical axis to be substituted into Expression 4 to be small when obtaining the X-axis coordinates xl and xr based on the road surface width. When the road is wide, the distance XR is set large. Then, the person detection device 20 sets a process limit line for adjusting the distance XR to determine the upper end of the person candidate area and a process limit line for determining the lowermost position of the person candidate area.

  Thereby, when the host vehicle 1 is traveling on the road surface, a processing limit line corresponding to the road surface width can be set so that only pedestrians in the road surface area can be processed. It is possible to prevent erroneous detection that determines that a wall or the like that appears in the vicinity is a person.

  Furthermore, when the own vehicle 1 is traveling on a road having two or more lanes, when the other vehicle traveling in the opposite lane is excluded from the detection target, the person detection device 20 includes: The distance XR from the optical axis is set small so that the distance from the optical axis of the far-infrared camera 10 to the opposite lane is not reached. When the right side of the host vehicle 1 is an opposite lane, the distance XR from the right optical axis in Equation 4 may be set to approximately zero so that it is excluded from the detection target.

  Thereby, even if the temperature of the other vehicle traveling in the opposite lane of the host vehicle 1 is a temperature equivalent to a person, it is possible to prevent erroneous detection that determines that the other vehicle is a person. In particular, it is possible to reliably prevent erroneous detection on a traveling road with many oncoming vehicles.

  The above-described embodiment is an example of the present invention. For this reason, the present invention is not limited to the above-described embodiment, and various modifications can be made depending on the design and the like as long as the technical idea according to the present invention is not deviated from this embodiment. Of course, it is possible to change.

It is a figure which shows schematic structure of the own vehicle carrying the person detection apparatus to which this invention is applied, Comprising: (a) is a side view, (b) is a top view. It is a flowchart for demonstrating the process structure which detects the person ahead of the own vehicle with the person detection apparatus to which this invention is applied, and the functional structure of a person detection apparatus. It is a figure for demonstrating the difference of the thermal image imaged with a far-infrared camera, Comprising: (a) is a thermal image imaged on a warm day with high temperature, (b) is imaged on a cold day with low temperature. It is a thermal image. It is a figure for demonstrating that the person in the thermal image imaged with a far-infrared camera changes with clothes of a person. It is a figure for demonstrating the process which produces the thermal image which binarized the thermal image imaged with the far-infrared camera, Comprising: (a) is a thermal image imaged on the day with high air temperature, (b) is air temperature. (C) is a thermal image after binarization created on a day with a high temperature, (d) is a thermal image captured on a day with a low temperature, and (e) is a temperature. (F) is a thermal image after binarization created on a day when the temperature is low. It is a figure for demonstrating the binarization process using an improper upper threshold and a lower threshold, Comprising: (a) is a thermal image imaged on the day with high temperature, (b) is temperature (C) is a thermal image obtained by binarizing the image of (a) using the temperature threshold value of (b), (d) is a thermal image captured on a day when the temperature is low, (e ) Is a temperature threshold, and (f) is a thermal image after binarizing the image of (d) using the temperature threshold of (e). It is a figure for demonstrating the labeling process and noise removal process with respect to the thermal image after binarization, (a) is the image which binarized the thermal image imaged on the day with high temperature, (b) is ( An image after the labeling process and the noise removal process are performed on the image of a), (c) is an image in which a label area is set for the image of (b), and (d) is captured on a day when the temperature is low. An image obtained by binarizing the thermal image, (e) is an image after labeling processing and noise removal processing are performed on the image of (d), and (f) is a label area for the image of (e). This is the set image. It is a figure for demonstrating a grouping process. It is a figure for demonstrating a grouping process, (a) is the image after a labeling process, (b) is the image after grouping a process to the image of (a), (c) is a labeling process The later image, (d), is an image after the grouping process is performed on the image of (c). (A) is a figure for demonstrating using the temperature threshold value remove | excluding the road surface and the sky temperature area from the temperature area | region equivalent to a person, (b) is background temperature from a temperature area | region equivalent to a person. It is a figure for demonstrating using the temperature threshold value except the area | region. It is a figure for demonstrating the process which calculates | requires the person height which appears in a thermal image, Comprising: (a) is a principle figure, (b) demonstrates that the person height in a thermal image changes with the distance from the own vehicle. It is a figure for doing. It is a figure for demonstrating the process which calculates | requires the position of the person's step which appears in a thermal image, Comprising: (a) is a principle figure, (b) is the position of the person's step in a thermal image according to the distance from the own vehicle. It is a figure for demonstrating changing. It is a figure for demonstrating the process which determines a person and a background object by the lowest end position of a person candidate area | region. It is a figure for demonstrating the process which correct | amends the lowest end position of a person candidate area | region, Comprising: (a) is a thermal image, (b) is an image which performed the grouping process to the thermal image of (a), (c ) Is a thermal image, (d) is an image obtained by subjecting the thermal image of (d) to the grouping process, and (e) is an image after correcting the lowermost position of the person candidate region of (d). It is a figure for demonstrating the inconvenience which arises by correct | amending the lowest end position of a person candidate area | region, Comprising: (a) is a position where a person and a streetlight differ on an X-axis, (b) is a person and a streetlight. (C) is an image after correcting the lowermost position of the person candidate area for the streetlight. It is a figure for demonstrating the process which sets the process restriction line which makes it outside the process range of a binarization process and a labeling process above a thermal image, Comprising: (a) provided the process restriction line in the binarization process An image, (b) is an image after a binarization process and a labeling process are performed by providing a process restriction line, and (c) is a diagram for explaining a position at which the process limit line is provided. It is a figure for demonstrating the process which pinpoints the road surface which the own vehicle drive | works in a thermal image, Comprising: (a) is a principle figure, (b) is a figure which shows the relationship between a road surface edge part and a person presence position. . It is a figure which shows the positional relationship of the lowest end position of a person candidate area | region, and the process restriction line provided in the downward direction in a thermal image. It is a figure for demonstrating the inconvenience at the time of making the process restriction line provided in the downward direction in a thermal image into the fixed height over the right-and-left both ends of a thermal image.

Explanation of symbols

1 own vehicle 10 far-infrared camera 20 person detection device

Claims (12)

Imaging means for capturing a thermal image composed of pixels having luminance values corresponding to the temperature of the object;
Of the thermal image captured by the imaging means, pixels within the upper and lower threshold values of the luminance value corresponding to the person's temperature, and the upper limit threshold of the luminance value corresponding to the person's temperature. Binarization processing means for creating a binarized image in which pixels outside the range of the value and the lower threshold are distinguished;
An area detection unit that detects an area in which a pixel group within a range between an upper limit threshold value and a lower limit threshold value of a luminance value corresponding to a person's temperature is equal to or larger than a predetermined area with reference to the binarized image When,
Grouping means for grouping one or a plurality of areas existing within a predetermined distance threshold among the areas detected by the area detection means as person candidate areas;
Based on the area and shape of the person candidate area grouped by the grouping means, and a person determination means for determining whether the object of the person candidate area is a person ,
The distance threshold is a value set in advance in the vicinity of the person candidate area determined as a person in the previous process when grouping is performed in the process after the person determination unit determines that the person is a person. The person detection device characterized by being changed for a longer time . Imaging means for capturing a thermal image composed of pixels having luminance values corresponding to the temperature of the object;
Of the thermal image captured by the imaging means, pixels within the upper and lower threshold values of the luminance value corresponding to the person's temperature, and the upper limit threshold of the luminance value corresponding to the person's temperature. Binarization processing means for creating a binarized image in which pixels outside the range of the value and the lower threshold are distinguished;
An area detection unit that detects an area in which a pixel group within a range between an upper limit threshold value and a lower limit threshold value of a luminance value corresponding to a person's temperature is equal to or larger than a predetermined area with reference to the binarized image When,
Grouping means for grouping one or a plurality of areas existing within a predetermined distance threshold among the areas detected by the area detection means as person candidate areas;
Based on the area and shape of the person candidate area grouped by the grouping means, and a person determination means for determining whether the object of the person candidate area is a person ,
The person determination means includes
The aspect ratio of the person candidate area is within a preset range of an aspect ratio corresponding to a person in the thermal image, and the lowest end position of the person candidate area corresponds to a road surface in the thermal image. It is determined that the object of the person candidate area is a person,
The position corresponding to the road surface in the thermal image is set to a position corrected based on the average value of the lowest end positions of the plurality of person candidate areas or the lowest end position of the plurality of person candidate areas. Detection device. Imaging means for capturing a thermal image composed of pixels having luminance values corresponding to the temperature of the object;
Of the thermal image captured by the imaging means, pixels within the upper and lower threshold values of the luminance value corresponding to the person's temperature, and the upper limit threshold of the luminance value corresponding to the person's temperature. Binarization processing means for creating a binarized image in which pixels outside the range of the value and the lower threshold are distinguished;
An area detection unit that detects an area in which a pixel group within a range between an upper limit threshold value and a lower limit threshold value of a luminance value corresponding to a person's temperature is equal to or larger than a predetermined area with reference to the binarized image When,
Grouping means for grouping one or a plurality of areas existing within a predetermined distance threshold among the areas detected by the area detection means as person candidate areas;
Based on the area and shape of the person candidate area grouped by the grouping means, and a person determination means for determining whether the object of the person candidate area is a person,
The binarization processing means is a range of luminance values corresponding to a temperature range equivalent to a person, and sets an upper threshold value and a lower threshold value excluding a luminance value range corresponding to a background temperature, and human detection device characterized that you create a binary image. The binarization processing means sets both the upper threshold and the lower threshold when the temperature is high, and sets both the upper threshold and the lower threshold when the temperature is low. The person detection device according to claim 3. The binarization processing means narrows the range between the upper threshold and the lower threshold when the temperature is high, and sets the range between the upper threshold and the lower threshold when the temperature is low. The human detection apparatus according to claim 3, wherein the human detection apparatus is set widely. The distance threshold includes a vertical distance threshold that is a distance between a person's head and hand or a person's head and leg included in the thermal image, and a person's leg width. With a lateral distance threshold that is
It said grouping means to claim 3, wherein the vertical distances are within the threshold, and grouping the one or more regions present within the lateral distance threshold The person detection device described. The distance threshold is changed long when the distance between the person detected by the person determination unit and the imaging unit approaches, and the distance between the person detected by the person determination unit and the imaging unit decreases. The person detection apparatus according to claim 3 , wherein the person detection apparatus is changed to be shorter. The region detection means is limited to the range below the position in the thermal image corresponding to the imaging position where the head of the person is imaged in the imaging range in real space by the imaging means. The person detection apparatus according to claim 3 , wherein a process by an image processing unit or a process by the area detection unit is performed. The person determination unit has a lowermost position of the person candidate region that is lower than a position in a thermal image corresponding to an imaging position where a leg portion of the person is imaged in an imaging range in real space by the imaging unit. The person detection apparatus according to claim 3 , wherein the object in the person candidate area is determined to be a person. The imaging means has an imaging range around the host vehicle,
When the host vehicle is traveling, processing is limited to a region in a thermal image corresponding to a road surface on which the host vehicle travels in an imaging range in real space by the imaging unit. The person detection device according to claim 8 or 9. The apparatus according to any one of claims 3 to 4, further comprising: an alarm unit that obtains the presence direction of the person and presents it to the driver when the person determining unit determines that the object in the person candidate area is a person. The person detecting device according to any one of 10. A far-infrared camera capturing a thermal image composed of pixels having a luminance value corresponding to the temperature of the object;
A human detection device including a CPU includes pixels within the upper limit threshold and lower limit threshold of a luminance value corresponding to a person's temperature in the thermal image, and an upper limit of a luminance value corresponding to the person's temperature. Creating a binarized image in which pixels outside the range of the threshold and the lower threshold are distinguished;
Region in which the person detection device refers to the binarized image and the pixel group within the range between the upper threshold value and the lower threshold value of the luminance value corresponding to the temperature of the person is a predetermined area or more Detecting steps,
The person detecting device grouping one or a plurality of areas existing within a predetermined distance threshold among the detected areas as a person candidate area;
The person detection device has a step of determining whether or not the object of the person candidate region is a person based on the area and shape of the grouped person candidate region;
The step of creating the binarized image is a range of luminance values corresponding to a temperature range equivalent to a person, and sets an upper threshold and a lower threshold excluding the luminance value range corresponding to a background temperature. Te, human detection method characterized that you create the binarized image.