JP4701961B2 - Pedestrian detection device - Google Patents

Description

  The present invention relates to a pedestrian detection device, and more particularly to a pedestrian detection device for a vehicle that detects a pedestrian based on an infrared image.

  Conventionally, various devices using infrared cameras have been proposed as devices for detecting pedestrians at night.

  For example, in Patent Document 1, a laser radar and an infrared camera are used in combination, and a search range for detecting pedestrians in an infrared image is set based on detection information of pedestrian candidates obtained from the laser radar. An apparatus for detecting whether a pedestrian is present is disclosed.

  Further, in Patent Document 2, luminance separation is performed on a pixel group (such as a pedestrian's face) whose luminance value is greater than or equal to a threshold value and a pixel group (such as a background) whose luminance value is less than the threshold value among pixels of an image captured by an infrared camera. An apparatus is disclosed that performs different processing on the two separated pixel groups and displays the result of adding these and the original image of the infrared camera.

Further, Patent Document 3 discloses an apparatus that searches for a region where bright parts are concentrated based on an infrared image and determines this as the head of the person to be detected.
JP 2003-302470 A JP 11-243538 A Japanese Patent Laid-Open No. 11-328364

  However, when a pedestrian is detected by a laser radar as in the invention described in Patent Document 1, there is a problem that a reflected wave from the pedestrian is weak and difficult to detect. For this reason, if the detection level is lowered so that a pedestrian can be detected, other objects are erroneously detected. Further, when FFT (Fast Fourier Transform) is necessary for the detection process, there is a problem that the calculation load is heavy.

  Moreover, when detecting a pedestrian from the image image | photographed with the infrared camera, image processings, such as pattern recognition, are very complicated, and there exists a problem that a processing load becomes large.

  The present invention has been made in consideration of the above-described facts, and an object of the present invention is to provide a pedestrian detection device that can improve the detection accuracy of a pedestrian at night or the like.

In order to achieve the above object, the invention according to claim 1 is an infrared light emitting means for emitting infrared light, an infrared camera for photographing an infrared image, and a conversion for converting an infrared image photographed by the infrared camera into a luminance image. Means, first extraction means for extracting a first region having a luminance equal to or higher than a first predetermined value based on the luminance image, and the first extraction unit based on the extracted luminance image of the first region. A second extraction means for extracting two adjacent second regions having a luminance greater than a predetermined value greater than a predetermined value as pedestrian reflection point candidates , and the extracted two retinal reflection points Determination means for determining whether or not a pedestrian exists in the infrared image based on the position of the candidate.

According to the invention, it emits infrared, based on the infrared images taken by the infrared camera, issued extract the candidate of the retinal reflection point of the pedestrian, on the basis of the position of the candidate of the extracted retinal reflection point, infrared images Whether or not there is a pedestrian is determined by the determining means.

  In this way, since it is determined whether or not the pedestrian is based on the position of the candidate for the retinal reflection point, there is no need to perform complicated processing such as pattern matching precisely about the shape of the pedestrian as in the past, It becomes possible to detect a pedestrian easily and accurately.

Specifically, a converting means for converting the pre-Symbol infrared image into a luminance image, on the basis of the luminance image, first extracting means for extracting a first region having a first predetermined value or more brightness, Based on the extracted luminance image of the first region, two adjacent second regions having a luminance equal to or higher than a second predetermined value larger than the first predetermined value are extracted as candidates for the retinal reflection point. And a second extraction unit, wherein the determination unit can determine whether or not a pedestrian exists in the infrared image based on the extracted positions of the two candidate retinal reflection points.

  That is, in an infrared image, since an object having heat such as a human has high luminance, the portion is extracted as a first region by the first extraction means. In addition, since the portion reflected by the human retina is considered to have higher luminance than other body portions, the portion is extracted as the second region by the second extraction means.

  Then, the two extracted second regions are used as candidates for the retinal reflection point, and based on these positions, it is determined whether or not the person is a pedestrian.

  Thus, since the candidate of a pedestrian is extracted as a 1st area | region and the candidate of a retinal reflection point is further extracted from a 1st area | region, it can be determined more correctly whether it is a pedestrian.

In addition, as described in claim 2 , the determination unit may determine whether or not the person is a pedestrian based on a distance between the candidates for the two retinal reflection points. For example, when the distance between the candidates is within a range where it can be determined that the distance is between human eyes, it can be determined as a pedestrian.

In this case, as described in claim 3 , further comprising distance detection means for detecting a distance to the pedestrian, wherein the determination means includes a distance between the two retinal reflection point candidates and the distance to the pedestrian. You may make it determine whether it is a pedestrian based on distance. Thereby, it can be determined whether it is a pedestrian more accurately.

According to a fourth aspect of the present invention, the distance detecting means can be a millimeter wave radar.

In addition, as described in claim 5 , the determination unit determines whether the pedestrian is based on a distance from the position of at least one candidate of the two retinal reflection point candidates to the end of the first region. It may be determined whether or not. For example, when the ratio of the distance from the position of the retinal reflection point to the end of the first area with respect to the length in the predetermined direction of the first area is obtained, and this is the ratio corresponding to the position of the human eye A pedestrian can be determined. Thereby, it can be determined whether it is a pedestrian accurately regardless of the magnitude | size of the 1st area | region which is a pedestrian candidate.

Further, as described in claim 6 , it may be configured to further include an output means for outputting a determination result by the determination means.

Specifically, for example, as described in claim 7 , the output unit may be a display unit that highlights a detected pedestrian.

  By mounting such a pedestrian detection device in, for example, an automobile, the driver can easily recognize the presence of a pedestrian even at night or the like, and can easily avoid danger.

  As described above, according to the present invention, the detection accuracy of pedestrians at night can be improved.

  Hereinafter, an example of an embodiment of the present invention will be described in detail with reference to the drawings.

  FIG. 1 shows a schematic configuration of a pedestrian detection device 10 according to an embodiment of the present invention. As shown in FIG. 1, the pedestrian detection device 10 includes a control unit 12, an infrared light emitting unit 14, an infrared camera 16, a distance detection sensor 18, an operation unit 20, a display unit 22, an audio output unit 24, a memory 26, and the like. It is configured to include.

  The infrared light emitting unit 14 emits infrared light (infrared light) having a predetermined wavelength suitable for detection of a pedestrian according to an instruction from the control unit 12.

  The infrared camera 16 is a camera capable of detecting infrared rays using, for example, a CCD or C-MOS sensor as an imaging device. The infrared image captured by the infrared camera 16 is an image whose luminance changes according to the temperature, color, etc. of the captured object. For example, an object having a high temperature is an image having a high luminance, and an object having a low temperature is an image having a low luminance.

  As shown in FIG. 2, the infrared light emitting unit 14 is provided at a predetermined position where infrared light can be favorably projected onto a pedestrian 36 or the like existing in front of the vehicle 30. In addition, the infrared light emission part 14 is good also as a structure combined with the headlamp 32 by providing the filter which cuts the visible light of the high beam light irradiated from the headlamp 32 of the vehicle 30, for example, and infrared light is received. A structure in which a light emitting element is provided alone may be employed. In the case where the infrared light emitting unit 14 and the headlamp 32 are combined, it is possible to irradiate visible light in front of the vehicle 30 with a low beam and further irradiate infrared light to a region in front of the visible light.

  Further, as shown in FIG. 2, the infrared camera 16 is provided at a predetermined position where the front of the vehicle 30 can be photographed well, for example, on a rearview mirror or dashboard inside the front window of the vehicle 30. Note that the position is not limited to this as long as the position in front of the vehicle 30 can be satisfactorily photographed.

  The distance detection sensor 18 can be configured by, for example, a laser radar, and is attached to the front of the vehicle so that an electromagnetic wave signal for detection is output toward the front of the vehicle 30. Distance and relative speed can be detected.

  This radar can be constituted by, for example, a millimeter wave radar. The millimeter wave radar is a radar using the Doppler effect, and can simultaneously detect the distance to the object to be detected and the relative velocity. The radar detection range is, for example, a range of several degrees to several tens of degrees.

  Note that any sensor may be used as long as it can detect the distance to an object existing in front of the vehicle 30.

  The operation unit 20 includes buttons and switches for setting various functions of the vehicle 30, and also includes a night view switch for instructing operation of a night view function to be described later.

  In response to an instruction from the control unit 12, the display unit 22 displays, for example, an image itself captured by the infrared camera 16, an image obtained by performing predetermined image processing on the captured image, or the like. The display unit 22 can be configured by, for example, a liquid crystal display or the like, but may be configured by a so-called head-up display that projects an image on a front window. The display unit 22 is not limited to this as long as it can display an image.

  The voice output unit 24 outputs a designated sound or voice, such as a warning sound or a warning message, to an occupant in the vehicle interior according to an instruction from the control unit 12.

  The memory 26 stores a control program, various parameters, table data, and the like which will be described later. The control unit 12 controls each unit according to a control program stored in the memory 26.

  Next, as an operation of the present embodiment, night view control executed by the control unit 12 will be described with reference to a flowchart shown in FIG.

  Note that the control routine shown in FIG. 3 is executed, for example, when the night view switch of the operation unit 20 is turned on by the user. Alternatively, it may be executed when the headlamp 32 of the vehicle 30 is turned on. Further, brightness detection means for detecting brightness outside the vehicle 30 is provided, and when it is detected that the brightness detected by the brightness detection means is less than a predetermined value, that is, pedestrians are recognized. It may be executed when it is detected that the brightness is difficult.

  First, in step 100, the control unit 12 instructs the infrared light emitting unit 14 to project infrared light. As a result, infrared light is projected from the infrared light emitting unit 14 to the front of the vehicle 30.

  In step 102, the infrared camera 16 is instructed to start photographing. Thereby, the infrared camera 16 starts photographing in front of the vehicle 30 and outputs the photographed infrared image to the control unit 12 every predetermined time.

  In step 106, each pixel value of the infrared image output from the infrared camera 16 is converted into a luminance value of a predetermined gradation to generate a luminance image. In addition, you may convert into other physical quantities as long as it is a physical quantity which shows not only a brightness | luminance but brightness.

  In step 108, it is determined whether or not there is an area having a higher brightness than the surrounding area and having a certain size. Specifically, for example, a first region where the average luminance value is equal to or greater than a first predetermined value is extracted from the luminance image. Here, the first predetermined value is set to a value by which it can be determined that there is a possibility that the region is a pedestrian if the luminance value is equal to or greater than this value, and is stored in the memory 26 in advance. That is, in step 108, an area where there is a possibility of a pedestrian is detected.

  As shown in FIG. 4A, for example, when a pedestrian 36 is present on the road 34, an infrared image taken by the infrared camera 16 becomes an infrared image 40 as shown in FIG. That is, since the pedestrian 36 normally has a higher temperature than the road 34, the brightness is increased. Therefore, by detecting a region with high brightness, a region with a possibility of a pedestrian can be detected.

  If the first area with the possibility of a pedestrian can be extracted, the process proceeds to step 110. If the first area with the possibility of a pedestrian cannot be extracted, the process proceeds to step 118. Transition.

  In step 110, pedestrian candidates are identified from the extracted first region. Specifically, it is determined by using a known pattern matching method or the like whether or not the shape of the extracted first region is approximate to a human shape. Then, the first area determined to be approximate to the human shape is identified as a pedestrian candidate. In the case of infrared images, the area that is actually a human may not necessarily be close to the shape of a human, so it is not necessary to accurately determine whether it is close to the shape of a human or not. Any region having a certain size (area) may be specified as a pedestrian candidate.

  As shown in FIG. 4A, normally, infrared light is projected from the infrared light emitting unit 14 with the face of the pedestrian 36 facing the vehicle 30, and the light hits the pedestrian 36 and is reflected. The brightness of the light reflected from the retina of the eye 38 of the pedestrian 36 is higher than that of the light reflected from other parts. For this reason, in the infrared image 40 shown to the same figure (B), the brightness | luminance of the pedestrian's eyes 38 becomes higher than the brightness | luminance of another body part. Therefore, by detecting the position of the eyes 38 of the pedestrian 36, it can be accurately determined whether or not the person is a human.

  Therefore, in step 112, whether or not there is a region having a certain level of brightness in the first region of the identified pedestrian candidate that is higher in luminance than the other regions in the first region. Determine whether. Specifically, for example, the second region having an average luminance value equal to or higher than the second predetermined value and having the same size (area) as the pedestrian's eyes is relatively close. Whether there are two positions is determined based on the luminance image of the first region of the pedestrian candidate. Here, the second predetermined value is a value larger than the first predetermined value, and if the luminance value is greater than or equal to this value, the second predetermined value can be determined to be a possibility that the region is the pedestrian's eyes. It is set and stored in the memory 26 in advance. That is, in step 112, a second region that has the possibility of pedestrian eyes is detected.

  Then, if a region with the possibility of pedestrian eyes is detected, the process proceeds to step 114, and if not detected, the process proceeds to step 118.

  In step 114, the extracted second region is a human based on the positions of the two second regions (retinal reflection point candidates, hereinafter simply referred to as retinal reflection points) that may be detected by the pedestrian. It is determined whether or not. For example, it is determined whether or not the extracted second region is a human by determining whether or not the positions of the two retinal reflection points are within a predetermined range.

  Specifically, for example, it is first determined whether or not the vertical distance (for example, the number of pixels) on the image of two retinal reflection points is within a predetermined first predetermined distance. Here, the first predetermined distance is set to a value that can be determined to be human eyes when the distance in the vertical direction between the two retinal reflection points is within this distance. Since the human eyes are naturally arranged in the horizontal direction in the normal posture, the first predetermined distance is normally set to a value close to ‘0’ and stored in the memory 26 in advance.

  Next, it is first determined whether or not the distance in the lateral (left / right) direction on the image of the two retinal reflection points is within a predetermined second predetermined distance. Here, the second predetermined distance is set to a value that can be determined to be a human eye when the distance in the lateral direction of the two retinal reflection points is within this distance, and is stored in the memory 26 in advance.

  When the vertical distance between the two retinal reflection points is within the first predetermined distance and the lateral distance is within the second predetermined distance, the extracted first region of the pedestrian candidate is human. It is determined that there is, and otherwise it is determined that the person is not human.

  Note that the head may be detected from the extracted first region of the pedestrian candidate, and it may be determined that the person is a human only when there are two retinal reflection points that satisfy the above conditions within the head. .

  In addition, the distance in the vertical direction and the horizontal direction of the two retinal reflection points may differ greatly depending on whether the pedestrian is present at a position close to the vehicle 30 or at a position far from the vehicle 30. Therefore, for example, as shown in FIG. 5, the head 42 is detected from the extracted first region of the pedestrian candidate and the length L thereof is obtained, and from the upper end of the head 42 with respect to the length L of the head 42. It may be determined whether or not the ratio of the distance d1 of the retinal reflection point 44 is within a predetermined range (for example, within 0 to 10%). It is determined whether the ratio of the distance of the retinal reflection point 44 from the lower end of the head 42 to the length L of the head 42 is within a predetermined range (for example, within 90 to 100%). You may determine that you are human.

  Further, as shown in the figure, the head 42 is detected from the extracted first region of the pedestrian candidate and the width W thereof is obtained, and the retinal reflection point from the right end of the head with respect to the width W of the head 42 is obtained. It may be determined whether or not the ratio of the distance d2 of 44 is within a predetermined range. Note that it may be determined whether or not the ratio of the distance of the retinal reflection point 44 from the left end of the head to the width W of the head 42 is within a predetermined range.

  Thus, by determining whether or not the person is a human based on the ratio of the distance of the retinal reflection point 44 from the end of the head 42 to the length L or width W of the head 42, the vehicle 30 and the pedestrian 36 are determined. It is possible to determine whether or not the person is simple and accurate regardless of whether the distance to is near or far.

  Alternatively, the distance between the two retinal reflection points may be obtained using the distance to the pedestrian 36 detected by the distance detection sensor 18.

  For example, table data in which the correspondence between the distance to the pedestrian and the distance between the two retinal reflection points is predetermined is stored in the memory 26. Then, the distance between the retinal reflection points corresponding to the distance to the pedestrian detected by the distance detection sensor 18 is obtained from the table data, and this is compared with the distance between the retinal reflection points obtained from the photographed image. If the difference is within the allowable range, it is determined that the person is human, and if the difference is outside the allowable range, it is determined that the person is not human. Thereby, it can be determined whether it is a person appropriately according to the distance to a pedestrian.

  Further, for example, the vertical angle and the horizontal angle formed by two lines connecting the position of the vehicle 30 (infrared camera 16) and the two retinal reflection points are obtained, and whether or not the person is a person based on these angles is obtained. May be judged.

  For example, as shown in FIG. 6, a line 50A connecting the position of the infrared camera 16 and the detected position of the retinal reflection point 44A and a line 50B connecting the position of the infrared camera 16 and the position of the retinal reflection point 44B are formed. Find the approximate angle θ. For example, based on the distance D1 between the retinal reflection points on the image obtained from the photographed image and the distance D2 from the infrared camera 16 to the pedestrian determined in advance based on the photographing range (in-focus range) of the infrared camera 16 and the like. To obtain the angle θ.

  Then, the angle θ is obtained for each of the vertical direction and the horizontal direction, and it is determined whether or not the person is a person based on the obtained angle θ in each direction. This determination process may be a process in which the first predetermined distance described in step 114 is replaced with a first predetermined angle, and the second predetermined distance is replaced with a second predetermined angle.

  Further, for example, an actual distance between the retinal reflection points is obtained based on the angle θ obtained as described above and the distance to the pedestrian 36 detected by the distance detection sensor 18, and the above-mentioned is based on the actual distance. It may be determined whether or not the person is a human by the determination process.

  If it is determined that the person is a human by the above determination process, the process proceeds to step 116, and if it is determined that the person is not a human, the process proceeds to step 118.

  In step 116, predetermined response processing is performed. Specifically, for example, an image in which the detected pedestrian is emphasized, for example, an image in which the outline of the pedestrian is emphasized is generated and displayed on the display unit 22, or a warning message indicating that a pedestrian is present ahead Is displayed on the display unit 22, or a warning sound or warning message for warning that a pedestrian is present is output from the voice output unit 24. Thereby, the driver of the vehicle 30 can easily recognize that a pedestrian is present ahead, and can avoid danger.

  In step 118, for example, it is determined whether or not the night view switch of the operation unit 20 has been turned off by the user, that is, whether or not the end of the night view mode has been instructed. If termination is not instructed, the process returns to step 104 and the same processing as described above is repeated.

  When the night view mode is executed when the headlamp 32 of the vehicle 30 is turned on, this routine may be terminated when the headlamp 32 is turned off. Further, when the brightness of the outside of the vehicle 30 is detected and night view is executed when the detected brightness is equal to or lower than the predetermined value, the main brightness is detected when the detected brightness exceeds the predetermined value. The routine may be terminated.

  As described above, in this embodiment, since it is configured to determine whether or not a person is a human being by detecting a retinal reflection point as a human eye based on an infrared image, a pedestrian can be detected with high accuracy. it can.

  In the present embodiment, in step 114 in FIG. 3, it is determined whether or not the extracted first region of the pedestrian candidate is a human based on the positions of the two retinal reflection points. For example, the determination may be made as follows.

  For example, as shown in FIG. 7, the first region of the extracted pedestrian candidate, that is, the length L2 (length corresponding to the height) of the pedestrian 36 is obtained, and the retinal reflection point 44 from the upper end of the head 42 is obtained. Length d4 may be obtained, and if these satisfy the following relationship, it may be determined that the person is human.

K1 × L2 ≦ d4 ≦ K2 × L2 (1)
Here, K1 and K2 are constants, but considering that the length d4 from the upper end of the head 42 to the retinal reflection point 44 is generally about 10 to 30% of the height of a human being, For example, K1 is set to 0.1 and K2 is set to 0.3. Note that the values of K1 and K2 are not limited to this, and may be values that can be appropriately determined as humans.

  Further, as shown in FIG. 7, the width W2 of the body of the pedestrian 36 (for example, the length between both hands or the shoulder width) is obtained, and from one retina reflection point 44 to the end of the pedestrian 36 in the width direction. The length d5 may be obtained, and if these satisfy the following relationship, it may be determined that the person is human.

K3 × W2 ≦ d5 ≦ K4 × W2 (2)
Here, although K3 and K4 are constants, the length d5 from one retinal reflection point 44 to the end in the width direction of the pedestrian 36 may be approximately 10 to 50% of the human width. For example, K3 is set to 0.1 and K4 is set to 0.5. Note that the values of K3 and K4 are not limited to this, and may be values that can be appropriately determined as humans.

  In addition, as shown in FIG. 7, the width W2 of the body of the pedestrian 36 is obtained, and the length d6 between the two retinal reflection points is obtained. Also good.

K5 × W2 ≦ d6 ≦ K6 × W2 (3)
Here, K5 and K6 are constants, but considering that the length d6 between the three retinal reflection points is generally about 10 to 50% of the width of a human being, for example, K5 is 0. 1, K6 is set to 0.5. Note that the values of K5 and K6 are not limited to this, and may be values that can be appropriately determined as humans.

It is a block diagram which shows schematic structure of the pedestrian detection apparatus which concerns on this invention. It is an image figure which shows a vehicle and a pedestrian. It is a flowchart of the control routine performed by a control part. (A) is an image figure which shows the front actual image seen from the vehicle, (B) is an image figure which shows the infrared image of (A). It is a figure for demonstrating the detection method of a retinal reflection point. It is a figure for demonstrating the other detection method of a retinal reflection point. It is a figure for demonstrating the other detection method of a retinal reflection point.

Explanation of symbols

10 pedestrian detection device 12 control unit (extraction means, determination means)
14 Infrared light emitting part (Infrared light emitting means)
16 Infrared camera 18 Distance detection sensor (distance detection means)
20 operation part 22 display part (display means)
24 audio output unit 26 memory 30 vehicle 32 head lamp 36 pedestrian 40 infrared image 42 head 44 retinal reflection point

Claims (7)

An infrared light emitting means for emitting infrared light;
An infrared camera for taking infrared images;
Conversion means for converting an infrared image taken by the infrared camera into a luminance image;
First extraction means for extracting a first region having a luminance equal to or higher than a first predetermined value based on the luminance image;
Based on the extracted luminance image of the first region, two adjacent second regions having a luminance equal to or higher than a second predetermined value that is greater than the first predetermined value are used as retinal reflection point candidates for the pedestrian. Second extracting means for extracting;
Determination means for determining whether or not there is a pedestrian in the infrared image based on the positions of the extracted two candidate retinal reflection points;
A pedestrian detection device. It said determination means, based on the distance between the two retinal reflection point candidates, pedestrian detection device according to claim 1, wherein the determining whether the pedestrian. Further comprising distance detection means for detecting the distance to the pedestrian,
The pedestrian detection device according to claim 2 , wherein the determination unit determines whether or not the user is a pedestrian based on a distance between the candidates for the two retinal reflection points and a distance to the pedestrian. . The pedestrian detection device according to claim 3 , wherein the distance detection means is a millimeter wave radar. The determination unit determines whether or not the person is a pedestrian based on a distance from a position of at least one candidate of the two retinal reflection point candidates to an end of the first region. Item 1. A pedestrian detection device according to item 1 . The pedestrian detection device according to any one of claims 1 to 5 , further comprising output means for outputting a determination result by the determination means. The pedestrian detection apparatus according to claim 6 , wherein the output unit is a display unit that highlights the detected pedestrian.

Images