JP2005092448A - Device and method for image recognition - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To reduce cost necessary to calculate the distance from a vehicle to an object. <P>SOLUTION: An image processing unit 4 detects the height of an object and the height of the shadow by analyzing the infrared rays image photographed by one-set of infrared camera 3, the ratio of the height of the detected object and the height of the shadow and the angle of the shadow to the object are calculated, and the distance from the vehicle to the object is calculated based on the calculated ratio and angle. Thereby, two-sets of infrared camera need not be installed in a night-vision unit for vehicle, so that cost necessary to calculate the distance from the vehicle to the object can be reduced. <P>COPYRIGHT: (C)2005,JPO&NCIPI

Description

  The present invention relates to an image recognition device and an image recognition method for calculating a distance to an object existing in the vehicle traveling direction, for example, in front of the vehicle using an infrared camera, and more specifically, to a distance to an object existing in the vehicle traveling direction. The present invention relates to a technology that makes it possible to reduce the cost required to calculate the distance to an object that exists in the vehicle traveling direction by calculating with a single infrared camera.

Conventionally, in consideration of the size of the object on the image taken by two infrared cameras mounted on the vehicle, the presence or absence of overlapping of different objects is determined from the parallax, and the objects are overlapping An image recognition device that independently calculates the distance to each object is known (see, for example, Patent Document 1). According to such an image recognition device, it is possible to perform a warning operation to the driver, an automatic vehicle avoidance operation, and the like with high reliability according to the calculated distance.
Japanese Patent Laying-Open No. 2003-150938 (paragraphs [0037] to [0039], FIG. 13)

  However, since the conventional image recognition apparatus is configured to calculate the distance to an object existing in the vehicle traveling direction using two infrared cameras, it is difficult to manufacture at low cost, and the vehicle traveling It has been difficult to reduce the cost required to calculate the distance to an object present in the direction.

  The present invention has been made to solve the above-described problems, and an object thereof is an image recognition device capable of reducing the cost required to calculate the distance to an object existing in the vehicle traveling direction. And providing an image recognition method.

  In order to solve the above-described problems, the image recognition apparatus according to the present invention is characterized by detecting means for detecting the height of an object and the height of a shadow by analyzing an infrared image picked up by a single image pickup means. A ratio calculating means for calculating the ratio of the height of the object and the height of the shadow, a shadow angle calculating means for calculating the angle of the shadow with respect to the object, and the object from the vehicle based on the calculated ratio and angle. And a distance calculating means for calculating the distance up to.

  In order to solve the above-mentioned problems, the feature of the image recognition method according to the present invention is to detect the height of an object and the height of a shadow by analyzing an infrared image captured by a single imaging means. Calculating the ratio between the height of the detected object and the height of the shadow, calculating the angle of the shadow with respect to the object, and calculating the distance from the vehicle to the object based on the calculated ratio and angle It is in.

  According to the image recognition apparatus and the image recognition method of the present invention, since the distance from the vehicle to the object can be calculated by one image pickup means, it is not necessary to use two or more image pickup means, and the The cost required for calculating the distance to the object can be reduced.

  The image recognition apparatus according to the present invention, for example, as shown in FIG. 1, captures an image in the vehicle traveling direction, for example, the front of the vehicle using near infrared light, and displays the captured image on the display unit. The present invention can be applied to a night vision device for a vehicle that makes it easier for the driver to recognize the state in front of the vehicle even at night. Hereinafter, the configuration and operation of a night vision device for a vehicle according to an embodiment of the present invention will be described with reference to the drawings.

[Configuration of night vision device for vehicle]
First, with reference to FIG. 1 and FIG. 2, the structure of the night-vision device for vehicles used as one Embodiment of this invention is demonstrated.

  As shown in FIGS. 1 and 2, a vehicle night vision apparatus according to an embodiment of the present invention includes left and right headlamps 1a and 1b, two infrared projectors 2a and 2b, one infrared camera 3, and image processing. The unit 4 and a HUD (Head Up Display) 5 are provided as main components. The infrared camera 3 and the HUD 5 function as an imaging unit and a display unit according to the present invention, respectively, and the image processing unit 4 includes first and second detection units, a ratio calculation unit, and a shadow angle calculation unit according to the present invention. And function as a distance calculation means.

  The left and right headlamps 1a and 1b are arranged symmetrically in the front part of the vehicle and irradiate visible light in the vehicle traveling direction in accordance with the on / off operation of the light switch 6 (see FIG. 2) by the driver. The infrared projectors 2a and 2b are arranged symmetrically in the front part of the vehicle, like the left and right headlamps 1a and 1b, and irradiate near-infrared light in front of the vehicle according to the control of the image processing unit 4. The near-infrared light is light having a wavelength that cannot be seen by human eyes. Unlike the visible light emitted by the headlamps 1a and 1b, the near-infrared light causes trouble for oncoming vehicles due to irradiation. It is not multiplied.

  The infrared camera 3 is a near-infrared type infrared camera, and is provided in a place such as a rearview mirror that does not obstruct the driver's front view. The infrared camera 3 captures an infrared image in front of the vehicle using near-infrared light emitted by the infrared projectors 2 a and 2 b, and inputs the captured infrared image data to the image processing unit 4.

  The image processing unit 4 is mounted in a vehicle and is configured by an arithmetic processing device such as a microcomputer. In this embodiment, the image processing unit 4 has the vehicle ignition switch (not shown) and the night vision system operation switch 7 (see FIG. 2) turned on, and the illuminance sensor 8 (see FIG. 2) is at night. In response to detecting that the illuminance outside the vehicle is low, an object that exists in the vehicle traveling direction by executing position detection processing described later on the data of the infrared image input from the infrared camera 3 The distance to is calculated.

  The HUD 5 is provided at a position that does not obstruct the front view of the driver of the front window glass, and projects and displays an infrared image of the vehicle traveling direction captured by the infrared camera 3 on the front window glass under the control of the image processing unit 4.

  The vehicle night vision apparatus configured as described above calculates a distance to an object existing in front of the vehicle using an infrared image captured by the infrared camera 3 by executing a position detection process described below. Hereinafter, with reference to the flowchart shown in FIG. 3, the operation of the vehicle night vision apparatus when the position detection process is executed will be described.

[Position detection processing]
The flowchart shown in FIG. 3 starts in response to the operating conditions of the vehicle night vision apparatus being satisfied, and the position detection process proceeds to step S1. In this embodiment, the image processing unit 4 detects that the vehicle ignition switch and the night vision system operation switch 7 are turned on and that the illuminance sensor 8 is nighttime. Is determined to be satisfied.

  In the process of step S1, the image processing unit 4 irradiates near-infrared light ahead of the vehicle by controlling the power of the infrared projectors 2a and 2b to be on. Thereby, the process of step S1 is completed, and the position detection process proceeds from the process of step S1 to the process of step S2.

  In the process of step S2, the image processing unit 4 controls the infrared camera 3 so as to capture an infrared image in front of the vehicle using the near infrared light emitted by the infrared projectors 2a and 2b, and the captured infrared image. The HUD 5 is controlled so as to be projected and displayed on the front window glass. Thereby, the process of step S2 is completed, and the position detection process proceeds from the process of step S2 to the process of step S3.

  In the process of step S3, the image processing unit 4 determines whether or not the operating condition of the vehicle night vision apparatus is satisfied. When it is determined that the operation condition of the vehicle night vision device is not satisfied, the image processing unit 4 ends the position detection process. On the other hand, when it is determined that the operating condition of the vehicle night vision device is satisfied, the image processing unit 4 advances the position detection process from the process of step S3 to the process of step S4.

  In the process of step S4, the image processing unit 4 analyzes the data of the infrared image captured by the infrared camera 3, so that the object, that is, the obstacle is in the traveling direction of the vehicle and cannot be safely stopped. It is determined whether or not an object is present, and if there is an obstacle, the HUD 5 is controlled so that the distance to the obstacle and information regarding the obstacle are projected and displayed on the front window glass (hereinafter referred to as “this”). The process in step S4 is referred to as a detection measurement process). The details of this detection measurement process will be described later with reference to the flowchart shown in FIG. Thereby, the process of step S4 is completed, and the position detection process returns from the process of step S4 to the process of step S3.

[Detection measurement processing]
Next, the detection measurement process will be described with reference to the flowchart shown in FIG.

  The flowchart shown in FIG. 4 starts when the image processing unit 4 determines that the operating condition of the vehicle night vision apparatus is satisfied in the process of step S3. This detection measurement process is performed in step S11. Proceed to processing.

  In the process of step S11, the image processing unit 4 performs differential processing on the data of the infrared image captured by the infrared camera 3, thereby extracting a portion (edge portion) where the luminance change is large in the infrared image, and the infrared image. Is converted into a line image (so-called contour extraction method). Thereby, the process of step S11 is completed, and the detection measurement process proceeds from the process of step S11 to the process of step S12.

  In the process of step S12, the image processing unit 4 determines whether or not there is an uneven portion (a linear object that floats) in the line image. Then, as a result of the determination, when the uneven portion does not exist in the line image, the image processing unit 4 ends the detection measurement process. On the other hand, when the uneven portion exists in the line image, the image processing unit 4 advances the detection measurement process from the process of step S12 to the process of step S13.

  In the process of step S13, the image processing unit 4 extracts the uneven portion from the line image and stores the extraction position. Thereby, the process of step S13 is completed, and the detection measurement process proceeds from the process of step S13 to the process of step S14.

  In the process of step S14, the position of the extracted concavo-convex part is within the light distribution ranges R and L of the infrared projectors 2a and 2b based on the extraction position stored in the process of step S13 by the image processing unit 4 (FIG. 5). It is determined whether it exists in the reference). And when the position of the extracted uneven part does not exist in the light distribution ranges R and L of the infrared projectors 2a and 2b, the position of the uneven part cannot be calculated accurately, or the uneven part with respect to the vehicle Is low (= the possibility of becoming an obstacle is low), the image processing unit 4 ends the detection measurement process. On the other hand, when the position of the extracted uneven portion is within the light distribution ranges R and L of the infrared projectors 2a and 2b, the image processing unit 4 changes the detection measurement process from the process of step S14 to the process of step S15. Proceed.

  In the process of step S15, the image processing unit 4 uses a line that forms the concavo-convex part as the object that is lifted vertically (reference numerals 11a to 11c in FIG. 5), and a part that is inclined obliquely as a shadow of the object. (Reference numerals 12a to 12c, 12b ', 12c' in FIG. 5), or the object constituting the concavo-convex part and its shadow are separated by the contrast difference in the concavo-convex part. Here, in FIG. 5, the shadows of the objects 11a to 11c are respectively irradiated by only the left infrared projector 2a, by the left and right infrared projectors 2a and 2b, and by the left and right infrared projectors 2a and 2b. It shows what was formed in the case of irradiating but irradiating side by side. Note that the shadow of the object always has the same level of darkness regardless of the color of the object, so that the object and its shadow can be separated by the contrast difference in the concavo-convex portion.

  When the object and its shadow are separated, the image processing unit 4 calculates the length of the object and its shadow, as shown in FIGS. The ratio (shadow length / object length) is calculated. Here, in the example shown in FIGS. 6A to 6C, the ratio of the length of the object and its shadow is (a) 0.93 / 1, (b) 1.1 / 1, (c), respectively. It is calculated as 3/1, 3.2 / 1. Since the calculated ratio does not depend on the size of the object, if the object is at the same position, it can be calculated regardless of the size of the object. Thereby, the process of step S15 is completed, and the detection measurement process proceeds from the process of step S15 to the process of step S16.

  In step S16, the image processing unit 4 calculates the shadow angle θ with respect to the object, as shown in FIGS. Here, in the example shown in FIGS. 6A to 6C, the shadow angles θ with respect to the object are calculated as (a) 5 °, (b) 12 °, and (c) 15 °, respectively. Thereby, the process of step S16 is completed, and the detection measurement process proceeds from the process of step S16 to the process of step S17.

＜仮想条件（１）＞
長さ１［ｍ］の棒（１１ａ〜１１ｅ）を左ヘッドランプ１ａ正面に配置し、棒の長さと影（１２ａ〜１２ｅ）の長さが一致する地点を探した所、図７に示すように、（１）影と棒の長さが一致する地点は車体から２０［ｍ］離れた地点Ｂにあった。また、影の長さが±１０［％］変化した地点を探した所、（２）影の長さが＋１０［％］変化した地点は車体から２２［ｍ］離れた地点Ａ、（３）影の長さが−１０［％］変化した地点は車体から１８［ｍ］離れた地点Ｃにあった。これらの実験結果から、影と対象物の長さが同じになる地点は車体から２０［ｍ］先にあり、車体から対象物までの距離は、対象物と影の長さの比率が＋１０［％］及び−１０［％］変化する度毎に、それぞれ＋２［ｍ］及び−２［ｍ］増減されることが想定される。 In the process of step S17, the image processing unit 4 calculates the distance from the vehicle to the object by inputting the ratio and angle values calculated in a predetermined position detection calculation formula. In this embodiment, the position detection calculation formula is derived as shown in the following formula 1 based on virtual conditions (1) and (2) described later.

<Virtual condition (1)>
As shown in FIG. 7, a rod (11a to 11e) having a length of 1 [m] is arranged in front of the left headlamp 1a and a point where the length of the rod matches the length of the shadow (12a to 12e) is searched. In addition, (1) the point where the length of the shadow coincides with the length of the rod was at point B 20 [m] away from the vehicle body. Also, a place where the shadow length has changed by ± 10 [%] is searched, (2) a point where the shadow length has changed by +10 [%] is a point A which is 22 [m] away from the vehicle body, (3) The point where the shadow length changed by -10 [%] was at point C, which was 18 [m] away from the vehicle body. From these experimental results, the point where the length of the shadow and the object is the same is 20 [m] ahead of the vehicle body, and the distance from the vehicle body to the object is such that the ratio of the object and shadow length is +10 [ %] And −10 [%], it is assumed that +2 [m] and −2 [m] are increased or decreased, respectively.

<Virtual condition (2)>
Moved in the horizontal direction so that the shadow and the bar have the same length, and looked for a point where the shadow shifted 1 [°] from the bar. (4) The shadow shifted 1 [°] relative to the bar. The point was at point D, which was 19.9 [m] away from the vehicle body. (5) Under the same conditions, a place where the shadow was shifted by 2 [°] with respect to the rod was searched. The point where the shadow was shifted by 2 [°] with respect to the rod was 19.8 [m] away from the vehicle body. It was at point E. From these experimental results, it is assumed that the distance from the vehicle body to the object changes by 0.5 [%] as the shadow angle θ is inclined by 1 [°] (= virtual condition 2).

  Here, with reference to FIG. 6 again, the process of calculating the distance from the vehicle body to the object using the position detection calculation formula will be specifically described.

  In the case of FIG. 6A, the ratio of the length of the object 11a and its shadow 12a is 0.93 (= 0.93 [m] / 1 [m]), and the angle θ of the shadow 12a with respect to the object 11a is 5 Since it is [°], the image processing unit 4 can calculate the distance (= 18.14 [m]) from the vehicle body to the object 11a by substituting these values into the above equation. In this case, since the distance to the object 11a is short, the image processing unit 4 can calculate the distance from the vehicle body to the object 11a only with information about one shadow.

  In the case of FIG. 6B, the ratio of the lengths of the object 11b and the shadows 12b and 12b ′ thereof is 1.1 (= 1.1 [m] / 1 [m]), and the shadow 12b, Since the angle θ of 12b ′ is 12 [°], the image processing unit 4 substitutes these values into the above formula to obtain the distance (= 20.68 [m]) from the vehicle body to the object 11b. Can be calculated. In this case, the object 11b has two shadows. However, since the two shadows are located symmetrically with respect to the object 11b, the image processing unit 4 can only detect information about one shadow from the vehicle body 11b. The distance to can be calculated.

  In the case of FIG. 6C, the ratio of the length of the object 11c and its shadows 12c and 12c ′ is 3.2 (= 3.2 [m] / 1 [m]) and 3.0 (= 3.0 [m] / 1 [m]), and the angles θ1 and θ2 of the shadows 12c and 12c ′ with respect to the object 11c are 15 and 0 [°]. By substituting the value, the distance from the vehicle body to the object 11c (= 59.2 and 60 [m]) is calculated. In this case, the image processing unit 4 calculates the average value (59.6 [m]) of the two calculated distances as the distance from the vehicle body to the object 11c.

  Thereby, the process of step S17 is completed, and the detection measurement process proceeds from the process of step S17 to the process of step S18.

  In the processing of step S18, the image processing unit 4 determines whether or not the calculated distance is a reasonable value within a predetermined numerical range. If the calculated distance is not an appropriate value within the predetermined numerical range as a result of the determination, the image processing unit 4 ends the detection measurement process. On the other hand, when the calculated distance is a reasonable value within the predetermined numerical range, the image processing unit 4 advances the detection measurement process from the process of step S18 to the process of step S19.

  In the process of step S19, the image processing unit 4 determines whether or not an object exists in the traveling direction of the vehicle. If the object does not exist in the traveling direction of the vehicle as a result of the determination, the image processing unit 4 ends the detection measurement process. On the other hand, when an object exists in the traveling direction of the vehicle, the image processing unit 4 advances the detection measurement process from the process of step S19 to the process of step S20.

  In the process of step S20, the image processing unit 4 determines whether or not the distance to the object existing in the traveling direction of the vehicle is within a range where the vehicle can be safely stopped. As a result of the determination, when the distance to the object existing in the traveling direction of the vehicle is within a range where the vehicle can be safely stopped, the image processing unit 4 ends the detection measurement process. On the other hand, when the distance to the object existing in the traveling direction of the vehicle is not within the range in which the vehicle can be safely stopped, the image processing unit 4 advances the detection measurement process from the process of step S20 to the process of step S21. .

  In the process of step S21, the image processing unit 4 controls the HUD 5 so as to project and display the distance to the object existing in the traveling direction of the vehicle and the information related to the object on the front window glass. Thereby, the process of this step S21 is completed and a series of detection measurement processes are complete | finished.

  As is clear from the above description, according to the night vision device for a vehicle that is an embodiment of the present invention, the image processing unit 4 analyzes the infrared image captured by the single infrared camera 3 to thereby detect the object. The height and the height of the shadow are detected, the ratio of the height of the object to the height of the shadow and the angle of the shadow with respect to the object are calculated, and the distance from the vehicle to the object is calculated based on the calculated ratio and angle. That is, according to the night vision device for a vehicle that is an embodiment of the present invention, the distance from the vehicle to the object is calculated by one infrared camera 3, so two infrared cameras are provided in the night vision device for the vehicle. There is no need, and the cost required to calculate the distance from the vehicle to the object can be reduced.

  Moreover, according to the night-vision device for vehicles which becomes one Embodiment of this invention, two infrared projectors (2a, 2b) are provided, and the image processing unit 4 is shown in the case of FIG.6 (c), The distance from the vehicle to the object is calculated for each infrared projector, the average value of the calculated values is calculated, and the calculated average value is used as the distance from the vehicle to the object. Can be calculated with high accuracy.

  Furthermore, according to the night vision device for a vehicle according to an embodiment of the present invention, the distance from the vehicle to the object is calculated by one infrared camera 3, so that the layout of the infrared camera can be easily performed. Further, an object can be detected by simple aiming adjustment. Further, since the infrared camera can be laid out in the center of the vehicle, the image displayed on the HUD 5 is not shifted to the left and right, and the image at the center of the vehicle can be displayed.

  As mentioned above, although the embodiment to which the invention made by the present inventor is applied has been described, the present invention is not limited by the description and the drawings that form part of the disclosure of the present invention according to this embodiment. For example, in the above embodiment, when detecting the height and shadow of an object, the two infrared projectors 2a and 2b are always lit, but the present invention is not limited to this and the two infrared projectors 2a and 2b are not limited to this. The height and shadow of the object may be detected by alternately turning on the infrared projectors 2a and 2b. According to such a configuration, the consumption necessary for detecting the height and shadow of the object, compared with the case where the height and shadow of the object are detected by always turning on the two infrared projectors 2a and 2b. Electric power can be reduced. In addition, since the manner of shadows differs between when the two infrared projectors 2a and 2b are always lit and when they are alternately lit, for example, a V-hour type as shown by shadows 12b and 12b 'in FIG. A shadow and a symbol such as a V-shape on the road surface can be easily distinguished. Further, in the above embodiment, the distance from the vehicle to the object is calculated using the shadow generated by irradiating the near infrared light from the infrared projectors 2a and 2b, but visible light from the left and right headlamps 1a and 1b. Alternatively, the distance from the vehicle to the object may be calculated using a shadow formed by irradiating. Furthermore, although the front of the vehicle has been described in the above embodiment, the rear distance at the time of reverse may be calculated by providing an infrared camera and an infrared projector at the rear of the vehicle. As described above, it is a matter of course that all other embodiments, examples, operation techniques, and the like made by those skilled in the art based on the above embodiments are included in the scope of the present invention.

1 is a schematic diagram showing a configuration of a vehicle night vision apparatus according to an embodiment of the present invention. It is a block diagram which shows the internal structure of the night-vision device for vehicles used as one Embodiment of this invention. It is a flowchart figure which shows the flow of the position detection process used as one Embodiment of this invention. It is a flowchart figure which shows the flow of the detection measurement process shown in FIG. It is a figure for demonstrating the calculation method of the ratio of the length of an object and its shadow, and the angle of the shadow with respect to an object. It is a figure for demonstrating the calculation method of the ratio of the length of an object and its shadow, and the angle of the shadow with respect to an object. It is a figure for demonstrating one experimental result for deriving the position detection formula which calculates the distance from a vehicle body to an object.

Explanation of symbols

1a, 1b: Head lamps 2a, 2b: Infrared projector 3: Infrared camera 4: Image processing unit 5: HUD (Head Up Display)
6: Light switch 7: Night vision system operation switch 8: Illuminance sensors 11a to 11e: Objects 12a to 12e: Shadow

Claims (6)

An infrared projector that emits infrared rays in the traveling direction of the vehicle;
One image pickup means provided in the vehicle for picking up an infrared image in the traveling direction of the vehicle with infrared rays irradiated by the infrared projector;
First detection means for detecting a height of an object by analyzing an infrared image captured by the imaging means;
Second detection means for detecting a height of a shadow of the object by analyzing an infrared image captured by the imaging means;
A ratio calculating means for calculating a ratio between the height of the object detected by the first and second detecting means and the height of the shadow of the object;
A shadow angle calculating means for calculating a shadow angle with respect to the object based on the height of the object detected by the first and second detecting means and the height of the shadow of the object;
Image recognition comprising: distance calculation means for calculating a distance from the vehicle to the object based on the ratio calculated by the ratio calculation means and the angle calculated by the shadow angle calculation means apparatus. Two infrared projectors are provided apart in the left-right direction,
The distance calculation unit calculates the distance for each infrared projector, calculates an average value of the calculated values, and sets the calculated average value as the distance. Image recognition device.   The image recognition apparatus according to claim 1, further comprising a display unit that is provided at a position that is easily recognized by a driver of the vehicle and that displays an infrared image captured by the imaging unit.   The image recognition apparatus according to claim 1, wherein the infrared light is near infrared light. Irradiating infrared rays in the traveling direction of the vehicle;
Capturing an infrared image in the traveling direction of the vehicle using one imaging means;
Detecting the height of the object by analyzing the captured infrared image;
Detecting the shadow height of the object by analyzing the captured infrared image;
Calculating a ratio of the detected height of the object and the shadow height of the object;
Calculating a shadow angle with respect to the object based on the detected height of the object and the shadow height of the object;
An image recognition method comprising: calculating a distance from the vehicle to the object based on the calculated ratio and angle.   6. The image recognition method according to claim 5, further comprising a step of displaying a captured infrared image.

Images