JP2008146404A - In-vehicle image recognition device - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To provide an in-vehicle image recognition device for maintaining high object recognition precision even under circumstances in which a region in the periphery of a vehicle is irradiated with near infrared rays of light. <P>SOLUTION: This in-vehicle image recognition device is provided with a visible light irradiating means 32 for irradiating a first region in the periphery of a vehicle with visible rays of light; a near infrared light irradiating means 42 for irradiating a partial region of the first region or a second region including a region in the neighborhood of the first region with near infrared rays of light; and an image pickup means 12 for picking up the image of the region including at least a part of the first region. The device is configured to recognize the image of the object in the periphery of the vehicle on the basis of the image picked up by the image pickup means. Further, the device is provided with a brightness control means 20 for controlling the received light quantity of the image pickup means, and the brightness control means is characterized in changing the control configurations of the received light quantity according as whether or not the near infrared rays of light are emitted by the near infrared light irradiating means. <P>COPYRIGHT: (C)2008,JPO&INPIT

Description

  The present invention relates to an in-vehicle image recognition apparatus applied to a vehicle including both visible light irradiation means and near infrared light irradiation means.

Conventionally, an imaging unit that images the front of the vehicle, an area setting unit that sets an image processing area for detecting a front vehicle on an image captured by the imaging unit, and an image process that is set by the area setting unit An area correction unit that corrects the area according to the driving environment, a feature derivation unit that derives an image characteristic in the image processing area corrected by the area correction unit, and a front vehicle based on the feature derived by the feature derivation unit There is known a vehicle image processing apparatus including a vehicle determination unit that detects a vehicle (see, for example, Patent Document 1). In this vehicular image processing apparatus, the area correction means corrects the image processing area within a range in which the headlight is irradiated when the headlight is turned on.
JP 2005-148816 A

  In recent years, in order to assist the driver's vision at night, a night view system has been put into practical use in which, for example, an image of the periphery of a vehicle imaged by projecting near-infrared light to a far front area of the vehicle is displayed on a windshield. Yes. In addition, as described in Patent Document 1 described above, an image recognition system that captures a vehicle front area with a camera and recognizes various objects (for example, road marking lines (white lines) and other vehicles) from the captured image. It has been put into practical use. The white line recognition result by the image recognition system is used, for example, for control (lane keep assist) for assisting the vehicle to properly maintain the lane (lane).

  By the way, since the above-mentioned night view system and image recognition system are systems having different characteristics from each other, they can be used together. However, when these systems are used in combination, the following problems may occur. That is, when the near-infrared light irradiation region and the visible light irradiation region are close or partially overlap, only the visible light is irradiated when both the near-infrared light and the visible light are irradiated. Since the brightness of the image increases due to near-infrared light, the object recognition accuracy in the image recognition system may deteriorate.

  Accordingly, an object of the present invention is to provide an in-vehicle image recognition apparatus capable of maintaining high object recognition accuracy even under the condition where near infrared light is irradiated.

In order to achieve the above object, the first invention comprises a visible light irradiating means for irradiating a first area around the vehicle with visible light,
Near-infrared light irradiation means for irradiating near-infrared light to a second region including a partial region of the first region or a region in the vicinity of the first region;
An imaging means for imaging an area including at least a part of the first area;
An in-vehicle image recognition device for recognizing an object around a vehicle based on an image captured by the imaging unit,
Further comprising brightness control means for controlling the amount of light received by the imaging means;
The brightness control unit is configured to receive the light when the near-infrared light irradiation unit emits near-infrared light and when the near-infrared light irradiation unit does not emit near-infrared light. The control mode of the quantity is changed.

A second invention is an in-vehicle image recognition device according to the first invention,
The brightness control unit controls the amount of received light based on an average value of luminance values of pixels within a predetermined range in an image captured by the imaging unit,
The size or position of the predetermined range in the image is when the near infrared light irradiating means irradiates near infrared light and when the near infrared light irradiating means does not irradiate near infrared light. It is characterized by being changed with time.

A third invention is an in-vehicle image recognition device according to the first invention,
The brightness control unit controls the amount of received light so that an average value of luminance values of pixels within a predetermined range in an image captured by the imaging unit becomes a predetermined target value;
The predetermined target value is changed between when the near-infrared light irradiating means irradiates near-infrared light and when the near-infrared light irradiating means does not irradiate near-infrared light. It is characterized by that.

A fourth invention is the in-vehicle image recognition device according to the first invention,
When the near infrared light irradiating means irradiates near infrared light, the brightness control means alternately performs near infrared brightness control and visible light brightness control in time. It is characterized by performing.

  ADVANTAGE OF THE INVENTION According to this invention, the vehicle-mounted image recognition apparatus which can maintain a high object recognition precision also in the condition where near infrared light is irradiated is obtained.

  The best mode for carrying out the present invention will be described below with reference to the drawings.

  FIG. 1 is a diagram showing a main configuration of a system including an embodiment of an in-vehicle image recognition apparatus 10 according to the present invention. FIG. 2 is a diagram illustrating an example of the relationship between the headlight irradiation region and the near-red irradiation region.

  The in-vehicle image recognition apparatus 10 of the present embodiment is mainly configured by a brightness control ECU 20 that controls the brightness of the headlight 30. Like other ECUs, the brightness control ECU 20 is configured as a microcomputer including a CPU, a ROM, a RAM, and the like that are connected to each other via a bus (not shown). Various programs executed by the CPU are stored in the ROM. As shown in FIG. 1, the brightness control ECU 20 may be provided as a dedicated ECU. However, part or all of the functions of the brightness control ECU 20 are other ECUs (for example, a headlight control ECU (not shown)). It may be incorporated into. Further, the brightness control ECU 20 may incorporate some or all of the functions of the image processing apparatus 14 described later.

  The lane keep ECU 50 is a control (lane keep assist) that assists the vehicle to properly maintain the lane (lane) based on the forward environment information (particularly the road lane line recognition result) obtained from the image processing device 14 described later. Execute. The lane keep assist may be realized by controlling the steering torque via the EPS / ECU 52 that controls the power steering device. Lane keep assist is started, for example, when there is an instruction (switch operation) from the user. When such an instruction is input, the lane keep ECU 50 transmits a signal indicating the start of lane keep assist to the brightness control ECU 20 and transmits a lane recognition instruction to the image processing apparatus 14 described later.

  A headlight light ECU 30 that controls the lighting state of the headlight 32 is connected to the brightness control ECU 20. The headlight 32 is mounted on, for example, both sides of the front portion of the vehicle. The headlight 32 includes a low beam headlamp whose optical axis is adjusted so as to irradiate visible light toward the vehicle front area. Hereinafter, the headlamp 32 refers to a low beam headlamp unless otherwise specified. Further, the vehicle front area irradiated by the headlight 32 is referred to as a “headlight irradiation area”.

  The headlight light ECU 30 normally controls the ON / OFF state of the headlight 32 according to the operation of the driver. Alternatively, when the brightness control ECU 20 determines that the surrounding area is darkened by the conlight sensor (sunlight sensor) (that is, when it is determined that it is sunset), the headlight 32 is turned on regardless of the driver's operation. It may be turned on. In any case, the headlight light ECU 30 appropriately supplies a signal indicating the lighting state of the headlight 32 to the brightness control ECU 20.

  The brightness control ECU 20 is connected to a near red lighting ECU ECU 40 that controls the lighting state of the near infrared light irradiation means 42. The near infrared light irradiation means 42 is mounted, for example, on both sides of the front portion of the vehicle. The near-infrared light irradiation means 42 may be incorporated in a headlight assembly including the headlight 32 as schematically shown in FIG. The optical axis of the near-infrared light irradiation means 42 is adjusted so as to irradiate near-infrared light toward the vehicle front area. Hereinafter, the vehicle front area irradiated by the headlight 32 is referred to as a “near red irradiation area”. The near-infrared image which imaged the near-red irradiation area | region may be displayed on the windshield glass of a vehicle by a head up display (HUD), for example.

  For example, as shown in FIG. 2, the near-red irradiation area is set on the far side from the headlight irradiation area. This is because the near-red irradiation region is set to a region where the headlight 32 cannot be illuminated or the amount of illumination is insufficient, thereby monitoring a pedestrian or the like in a distance that is difficult for the driver to visually recognize. For example, the headlight irradiation area may be an area up to about 20 m ahead of the vehicle, and the near-red irradiation area may be an area ahead of the vehicle relative to the headlight irradiation area. The near-red irradiation area and the headlight irradiation area may be close to each other and partially overlap.

  The near-red lighting ECU ECU 40 controls the ON / OFF state of the near-infrared light irradiation means 42 according to the operation of the driver. Alternatively, the near-red lighting ECU ECU 40 may automatically control the ON / OFF state of the near-infrared light irradiation means 42 as necessary. In any case, the near red lighting ECU ECU 40 appropriately supplies a signal indicating the lighting state of the near infrared light irradiation means 42 to the brightness control ECU 20.

  Referring back to FIG. 1, the brightness control ECU 20 is connected to an image processing device 14 that executes various image processes on a captured image obtained from the camera 12.

  The camera 12 acquires a front environment image including a road surface in front of the vehicle by an imaging element such as a charge-coupled device (CCD) or a complementary metal oxide semiconductor (CMOS). The camera 12 is attached to, for example, the rear side (the front surface of the vehicle) of the room mirror. The vehicle front area imaged by the camera 12 includes all or at least most of the headlight irradiation area described above. Moreover, the vehicle front area imaged by the camera 12 may include a part of the near-red irradiation area described above. The camera 12 may acquire a front environment image in real time while the vehicle is running, and supply it to the image processing device 14 in a stream format with a predetermined frame period.

  The image processing device 14 performs image processing on the front environment image obtained from the camera 12 to generate front environment information regarding the environment ahead of the vehicle. The forward environment information is typically information such as the position of a road lane marking or the position of an obstacle. In this example, the image processing device 14 operates in response to, for example, the lane recognition instruction from the lane keep ECU 50 described above, and receives and processes the captured image of the camera 12, so that the roads on both sides of the vehicle on the traveling road of the vehicle. Detect and recognize lane markings. This type of road lane marking image recognition method may be diverse and any suitable method may be employed. For example, edge processing may be performed using a predetermined area of the captured image as an image processing target area, and a road marking line may be recognized using pattern matching or the like. The lane detection result (road lane marking recognition result) by the image processing device 14 is supplied to the lane keep ECU 50.

  The image processing device 14 also calculates the brightness of the front environment image obtained from the camera 12 and supplies a signal representing the calculation result to the brightness control ECU 20. In this example, the brightness of the front environment image is calculated as an average value of the luminance values of the pixels in the target area. In this case, the target area may be an area of a road portion of the front environment image. Alternatively, the target area may correspond to an image processing target area when performing image recognition of road lane markings. The target area may be an area defined by a set exposure window range, which will be described later.

  The brightness control ECU 20 controls the brightness of the front environment image, that is, the amount of light incident on the image sensor of the camera 12 via the lens (the amount of received light). The brightness control ECU 20 controls the amount of received light so that the average value of luminance values obtained from the image processing device 14 becomes the target luminance average value. The control of the amount of received light may be realized by controlling the gain value. The control of the amount of received light may be realized by adjusting a variable aperture instead of or in addition to the control of the gain value. When the camera 12 is equipped with an electronic shutter, the shutter speed or the shutter time is set. It may be realized by adjusting. The control of the amount of received light may be realized by setting a movable lens hood in the camera 12 and moving the lens hood.

  Next, on the premise of the above configuration, a preferable received light amount control (brightness control) method realized by the brightness control ECU 20 will be described. Since there are a plurality of preferable received light amount control methods, the following description will be given in order.

  FIG. 3 is a flowchart showing a first embodiment of the received light amount control method. The processing routine shown in FIG. 3 may be repeatedly executed at a predetermined processing cycle when the headlight 32 is in a lighting state.

  In step 100, the brightness control ECU 20 determines whether or not the near-infrared light irradiation means 42 is turned on based on information obtained from the near-red lighting ECU ECU 40. That is, it is determined whether or not near infrared light is irradiated by the near infrared light irradiation means 42. If the near-infrared light irradiation means 42 is on, the process proceeds to step 110. If the near-infrared light irradiation means 42 is not on, the process proceeds to step 120.

  In step 110, the brightness control ECU 20 maintains or changes the target luminance average value used in the received light amount control (brightness control) to a value higher than the normal value. In this case, control is performed in a direction in which the brightness of the front environment image is brighter than that in the normal state, that is, in a direction in which the amount of received light is increased.

  In step 120, the brightness control ECU 20 maintains or changes the target luminance average value to a normal value.

  By the way, as described above, when the near red irradiation area and the headlight irradiation area are set in a positional relationship close to each other, the front environment image is obtained in a situation where both the headlight 32 and the near infrared light irradiation means 42 are on. Brightness increases due to near-infrared light. That is, the average luminance value obtained from the image processing device 14 is increased. On the other hand, as usual, when the brightness of the front environment image becomes darker as the average value of the luminance values increases, the brightness of at least a part of the image range related to the headlight irradiation area is controlled. There is a risk that the recognition accuracy of an object to be recognized (in this example, a road lane marking) will deteriorate.

  On the other hand, according to the received light amount control method shown in FIG. 3, when both the headlight 32 and the near-infrared light irradiation means 42 are turned on, as described above, the target luminance average used in the brightness control is used. Since the value is maintained or changed to a value higher than the normal value, it is prevented that the brightness of the front environment image is darkened, and the brightness of the image range related to the headlight irradiation area is insufficient. It is prevented. Thereby, it is possible to prevent the recognition accuracy of the recognition target object from deteriorating in a situation where both the headlight 32 and the near-infrared light irradiation means 42 are on.

  Note that the received light amount control method shown in FIG. 3 is a headlight irradiation area in which the image processing target area in the front environment image, that is, the area processed by the image processing device 14 for object recognition does not include the near-red irradiation area. It is suitable for the case corresponding to. This is because, according to the received light amount control method shown in FIG. 3, in the headlight irradiation area, as described above, the appropriate brightness similar to that at the time of non-projection is maintained during near-infrared light projection. As a result, the luminance difference between the pixel related to the object to be recognized and the pixels other than the object appears appropriately, while in the near-red irradiation region, the target luminance average value is higher than the normal value when projecting near-infrared light. This is because the difference in luminance between the pixels related to the object to be recognized and the pixels other than the object is reduced (that is, no contrast is generated).

  FIG. 4 is a flowchart showing a second embodiment of the received light amount control method. The processing routine shown in FIG. 4 may be repeatedly executed at a predetermined processing cycle when the headlight 32 is in a lighting state. FIG. 5 is an explanatory diagram of FIG. 4 and shows each exposure window range in the front environment image.

  In step 200, the brightness control ECU 20 determines whether or not the near-infrared light irradiation means 42 is turned on based on information obtained from the near-red lighting ECU ECU 40. If the near infrared light irradiation means 42 is on, the process proceeds to step 210. If the near infrared light irradiation means 42 is not on, the process proceeds to step 220.

  In step 210, the brightness control ECU 20 outputs an instruction to the image processing device 14 so as to calculate the brightness of the front environment image within the reduced exposure window range. As shown in FIG. 5A, the reduced exposure window range is set so as to correspond to the headlight irradiation region that does not substantially include the near-red irradiation region. In this case, the image processing device 14 calculates the brightness of the front environment image in the reduced exposure window range, that is, in the image range related to the headlight irradiation region that does not substantially include the near-red irradiation region.

  In step 220, an instruction is output to the image processing apparatus 14 so as to calculate the brightness of the front environment image in the enlarged exposure window range (normal exposure window range). As shown in FIG. 5B, the enlarged exposure window range is set so as to correspond to the near-red irradiation area + the headlight irradiation area. That is, the expanded exposure window range is expanded by the amount including the near-red irradiation region, compared to the exposure window range shown in FIG. In this case, the image processing device 14 calculates the brightness of the front environment image in the enlarged exposure window range, that is, in the image range related to the headlight irradiation region substantially including the near red irradiation region.

  According to the received light amount control method shown in FIG. 4, when both the headlight 32 and the near-infrared light irradiation means 42 are turned on, as described above, the headlight irradiation that substantially does not include the near-red irradiation region. In the region, the brightness of the front environment image is evaluated. Thereby, even when the brightness of the front environment image increases due to near-infrared light, the brightness of the front environment image can be appropriately controlled without being affected by the increase in brightness. As a result, it is possible to prevent the recognition accuracy of the recognition target object from deteriorating in a situation where both the headlight 32 and the near infrared light irradiation means 42 are on.

  When the received light amount control method shown in FIG. 4 is used, the image processing target region in the front environment image is an image range related to a headlight irradiation region (or a reduced exposure window range) that does not include the near-red irradiation region. Alternatively, the image processing target area in the front environment image may be varied between when the near-infrared light irradiation means 42 is lit and when it is not lit, similarly to the exposure window range. For example, the image processing target area in the front environment image is set so as to correspond to the headlight irradiation area not including the near red irradiation area when the near infrared light irradiation means 42 is turned on, and the near infrared light irradiation means 42. At the time of non-lighting, it may be set so as to correspond to both the near-red irradiation region and the headlight irradiation region. In this case, when the near-infrared light irradiation means 42 is not lit, information obtained from the image range far from the vehicle can also be used, so that the object recognition accuracy is improved.

  FIG. 6 is a flowchart showing a third embodiment of the received light amount control method. The processing routine shown in FIG. 6 may be repeatedly executed at a predetermined processing cycle when the headlight 32 is in a lighting state.

  In step 300, the brightness control ECU 20 determines whether or not the near-infrared light irradiation means 42 is turned on based on information obtained from the near-red lighting ECU ECU 40. If the near infrared light irradiation means 42 is on, the process proceeds to step 320. If the near infrared light irradiation means 42 is not on, the process proceeds to step 310.

  In step 310, the brightness control ECU 20 maintains or changes the target luminance average value used in the received light amount control (brightness control) to a normal value.

  In step 320, the brightness control ECU 20 maintains or changes the target luminance average value to a value lower than the normal value. In this case, control is performed in a direction in which the brightness of the front environment image becomes darker than that in the normal state, that is, in a direction in which the amount of received light is reduced.

  By the way, as described above, when the near red irradiation area and the headlight irradiation area are set in a positional relationship close to each other, in the situation where both the headlight 32 and the near infrared light irradiation means 42 are turned on, the headlight irradiation is performed. Since the brightness of the image area closer to the near-red illumination area among the image areas related to the area increases due to near-infrared light, the object to be recognized (in this example, the road lane marking) ) And a pixel other than the object have a small luminance difference (that is, no contrast is produced). As a result, there is a possibility that the object recognition accuracy in the image recognition system deteriorates under the situation where visible light and near infrared light are irradiated.

  On the other hand, according to the received light amount control method shown in FIG. 6, when both the headlight 32 and the near-infrared light irradiation means 42 are turned on, as described above, the target luminance average used in the brightness control is used. Since the value is maintained or changed to a value lower than the normal value, the image range relating to the near red irradiation region and a part of the headlight irradiation region (the region closer to the near red irradiation region in the headlight irradiation region) The brightness that increases due to near-infrared light can be moderately reduced. Thereby, it is possible to prevent the recognition accuracy of the recognition target object from deteriorating in a situation where both the headlight 32 and the near-infrared light irradiation means 42 are on.

  Note that in the received light amount control method shown in FIG. 6, the image processing target area in the front environment image, that is, the area processed by the image processing device 14 for object recognition is one of the near-red irradiation area and the headlight irradiation area. It is suitable for the case of corresponding to a portion (a region closer to the near red irradiation region in the headlight irradiation region). According to the received light amount control method shown in FIG. 6, in the near-red irradiation region and the headlight irradiation region, the region closer to the near-red irradiation region, as described above, during near-infrared light projection, By maintaining an appropriate brightness similar to that at the time of non-projection, a luminance difference between a pixel related to the object to be recognized and a pixel other than the object appears appropriately, while the remaining part of the headlight irradiation area ( This is because the brightness of the front environment image may be insufficient in the near-infrared irradiation area) when the target luminance average value is set to a value lower than the normal value during near-infrared light projection. .

  FIG. 7 is a flowchart showing a fourth embodiment of the received light amount control method. The processing routine shown in FIG. 7 may be repeatedly executed at a predetermined processing cycle when the headlight 32 is in a lighting state.

  In step 400, the brightness control ECU 20 determines whether or not the near-infrared light irradiation means 42 is on based on information obtained from the near-red lighting ECU ECU 40. If the near infrared light irradiation means 42 is on, the process proceeds to step 420. If the near infrared light irradiation means 42 is not on, the process proceeds to step 410.

  In step 410, the brightness control ECU 20 outputs an instruction to the image processing device 14 so as to calculate the brightness of the front environment image within the reduced exposure window range (normal exposure window range). As shown in FIG. 5A, the reduced exposure window range is set so as to correspond to the headlight irradiation region that does not substantially include the near-red irradiation region. In this case, the image processing device 14 calculates the brightness of the front environment image in the reduced exposure window range, that is, in the image range related to the headlight irradiation region that does not substantially include the near-red irradiation region.

  In step 420, an instruction is output to the image processing apparatus 14 so as to calculate the brightness of the front environment image within the enlarged exposure window range. As shown in FIG. 5B, the enlarged exposure window range is set so as to correspond to the near-red irradiation area + the headlight irradiation area. That is, the expanded exposure window range is expanded by the amount including the near-red irradiation region, compared to the exposure window range shown in FIG. In this case, the image processing device 14 calculates the brightness of the front environment image in the enlarged exposure window range, that is, in the image range related to the region including both the near-red irradiation region and the headlight irradiation region.

  According to the received light amount control method shown in FIG. 7, when both the headlight 32 and the near-infrared light irradiation means 42 are turned on, as described above, the entire light comprising the near-red irradiation area and the headlight irradiation area. In the region, the brightness of the front environment image is evaluated. As a result, when the brightness of the front environment image increases due to near infrared light, the brightness of the front environment image is controlled to become darker, so a part of the near red irradiation region and the headlight irradiation region ( In the image range related to the near-red irradiation region in the headlight irradiation region, it is possible to moderately reduce the brightness that increases due to near-infrared light. That is, in such an image range, it is possible to prevent a luminance difference between a pixel related to the object to be recognized and a pixel other than the object from becoming small (that is, no contrast is generated). Thereby, it is possible to prevent the recognition accuracy of the recognition target object from deteriorating in a situation where both the headlight 32 and the near-infrared light irradiation means 42 are on.

  When the received light amount control method shown in FIG. 7 is used, the image processing target area in the front environment image is variable depending on whether the near-infrared light irradiation means 42 is lit or not as in the exposure window range. May be. For example, the image processing target area in the front environment image is set to a headlight irradiation area or a near red irradiation area + headlight irradiation area when the near infrared light irradiation means 42 is not lit, and the near infrared light irradiation means When 42 is turned on, it may be set to the near red irradiation region or a part of the near red irradiation region and the headlight irradiation region (the region of the headlight irradiation region closer to the near red irradiation region). When the near-infrared light irradiation means 42 is not turned on, if the image processing target area is set to the near red irradiation area + headlight irradiation area, the image range far from the vehicle (image range related to the near red irradiation area) Since the information obtained from can also be used, the object recognition accuracy is improved.

  FIG. 8 is a flowchart showing a fifth embodiment of the received light amount control method. The processing routine shown in FIG. 8 may be repeatedly executed at a predetermined processing cycle when the headlight 32 is in a lighting state. The predetermined processing period may correspond to the frame period of the front environment image captured by the camera 12.

  In step 500, the brightness control ECU 20 determines whether or not the near-infrared light irradiation means 42 is on based on information obtained from the near-red lighting ECU ECU 40. If the near infrared light irradiation means 42 is on, the process proceeds to step 510. If the near infrared light irradiation means 42 is not on, the process proceeds to step 520.

  In step 510, the brightness control ECU 20 determines whether or not the frame of the front environment image captured by the camera 12 in the current processing cycle is an even frame. If the frame of the front environment image in the current processing cycle is an even frame, the process proceeds to step 520. If the frame of the front environment image in the current processing cycle is an odd frame, the process proceeds to step 530.

  In step 520, the brightness control ECU 20 executes brightness control for the headlight irradiation area. The brightness control for the headlight irradiation area may be the same as the processing shown in step 110 of FIG. 3 and step 210 of FIG. 4 described above. That is, the brightness control ECU 20 is not controlled so that the brightness of the image range related to the headlight irradiation region (near red non-irradiation region) becomes dark due to the near infrared light projected in the vicinity thereof. Thus, brightness control is performed. Thereby, in the even-numbered frame, the brightness of the image range related to the headlight irradiation area that does not include the near-red irradiation area is maintained at an appropriate brightness similar to that when no near-red irradiation is performed. In this step 520, the brightness control for the camera 12 in the current processing cycle may be executed based on the brightness (luminance average value) of the front environment image calculated in the even frames before the previous cycle.

  In step 530, the brightness control ECU 20 executes brightness control for the near-red irradiation region. The brightness control for the near red irradiation region may be the same as the processing shown in step 320 of FIG. 6 and step 420 of FIG. That is, the brightness control ECU 20 performs brightness control so that the brightness of the image range related to the near-red irradiation region does not become excessively bright due to near-infrared light. Thereby, in the odd-numbered frame, the brightness of the image range related to the near-red irradiation region is maintained at an appropriate brightness. In step 530, the brightness control for the camera 12 in the current processing cycle may be executed based on the brightness (luminance average value) of the front environment image calculated in the odd frame before the previous cycle.

  In step 540, the brightness control ECU 20 recognizes an object to be recognized (a road lane line in this example) from the front environment image in different image recognition processing modes for odd frames and even frames with respect to the image processing device 14. Outputs instructions to Specifically, the brightness control ECU 20 makes the near-red irradiation region and a part of the headlight irradiation region in the front environment image (near red irradiation in the headlight irradiation region) in the odd number frame. The image recognition processing is executed with the image range relating to the region close to the region) as the image processing target region. On the other hand, the brightness control ECU 20 displays the image range related to the headlight irradiation area (near red non-irradiation area) that does not include the near red irradiation area in the front environment image in the even frame in the image processing device 14. Image recognition processing is executed as a processing target area.

  According to the received light amount control method shown in FIG. 8, the brightness control for the headlight irradiation area and the brightness control for the near red irradiation area are temporally alternated according to the imaging timing (frame) of the camera 12. By executing the above, it is possible to achieve two brightness controls having opposite characteristics. That is, when the brightness control for the headlight irradiation area is always executed, the lack of brightness in the headlight irradiation area can be prevented, but the brightness in the near red irradiation area is saturated and the near red The object recognition accuracy in the irradiation area deteriorates. On the other hand, when the brightness control for the near red illumination area is always executed, saturation of the brightness in the near red illumination area can be prevented, but the brightness in the headlight illumination area is insufficient and the headlight The object recognition accuracy in the irradiation area deteriorates. According to the received light amount control method shown in FIG. 8, high object recognition accuracy can be achieved in both the near-red irradiation area and the headlight irradiation area by appropriately combining two brightness controls having such contradictory characteristics. Can be maintained.

  In the received light amount control method shown in FIG. 8, the brightness control for the near red illumination area and the brightness control for the headlight illumination area are executed in the odd frame and the even frame, respectively. The brightness control for the headlight irradiation area and the brightness control for the near red irradiation area may be executed in the odd-numbered frame and the even-numbered frame, respectively. Alternatively, the brightness control for the near red irradiation area and the brightness control for the headlight irradiation area may be executed alternately in units of a plurality of frames.

  The preferred embodiments of the present invention have been described in detail above. However, the present invention is not limited to the above-described embodiments, and various modifications and substitutions can be made to the above-described embodiments without departing from the scope of the present invention. Can be added.

  For example, in the above-described embodiment, an example in which a front environment image is acquired by the camera 12 has been described. However, the present invention is similarly applied to brightness control of a camera that images the side of the vehicle and the rear of the vehicle. Is possible. In this case as well, the same problem occurs when near-red light is irradiated to the side of the vehicle or the rear of the vehicle.

  In the embodiment described above, the amount of received light (brightness) of the camera 12 may be increased by increasing the amount of light of the headlight 32. Increasing the amount of light of the headlight 32 may include turning on an auxiliary light such as a fog lamp in the lighting state of the low beam headlight.

  In the above-described embodiment, the object recognition result obtained from the image processing device 14 is used for lane keep assist. However, a pre-crash that predicts a collision with an obstacle and activates an occupant protection device. It may be used for control or the like.

It is a figure which shows the main structures of the system containing one Example of the vehicle-mounted image recognition apparatus 10 by this invention. It is a figure which shows an example of the relationship between a headlight irradiation area | region and a near-red irradiation area | region. 3 is a flowchart showing a first embodiment of a received light amount control method realized by a brightness control ECU 20; It is a flowchart which shows the 2nd Example of the light reception amount control method implement | achieved by brightness control ECU20. It is a figure which shows an example of the expansion / contraction method of the exposure window range implement | achieved by brightness control ECU20. It is a flowchart which shows the 3rd Example of the light reception amount control method implement | achieved by brightness control ECU20. It is a flowchart which shows the 4th Example of the light reception amount control method implement | achieved by brightness control ECU20. It is a flowchart which shows the 5th Example of the light reception amount control method implement | achieved by brightness control ECU20.

Explanation of symbols

DESCRIPTION OF SYMBOLS 10 Vehicle-mounted image recognition apparatus 12 Camera 14 Image processing apparatus 20 Brightness control ECU
30 Headlight ECU
32 Headlight 40 Near red lighting ECU ECU
42 Near-infrared light irradiation means 50 Lane keep ECU
52 EPS / ECU

Claims (4)

Visible light irradiation means for irradiating visible light to the first region around the vehicle;
Near-infrared light irradiation means for irradiating near-infrared light to a second region including a partial region of the first region or a region in the vicinity of the first region;
An imaging means for imaging an area including at least a part of the first area;
An in-vehicle image recognition device for recognizing an object around a vehicle based on an image captured by the imaging unit,
Further comprising brightness control means for controlling the amount of light received by the imaging means;
The brightness control unit is configured to receive the light when the near-infrared light irradiation unit emits near-infrared light and when the near-infrared light irradiation unit does not emit near-infrared light. An in-vehicle image recognition apparatus characterized by changing the control mode of the amount. The brightness control unit controls the amount of received light based on an average value of luminance values of pixels within a predetermined range in an image captured by the imaging unit,
The size or position of the predetermined range in the image is when the near infrared light irradiating means irradiates near infrared light and when the near infrared light irradiating means does not irradiate near infrared light. The in-vehicle image recognition apparatus according to claim 1, which changes depending on the time. The brightness control unit controls the amount of received light so that an average value of luminance values of pixels within a predetermined range in an image captured by the imaging unit becomes a predetermined target value;
The predetermined target value is changed between when the near-infrared light irradiating means irradiates near-infrared light and when the near-infrared light irradiating means does not irradiate near-infrared light. The in-vehicle image recognition device according to claim 1.   When the near infrared light irradiating means irradiates near infrared light, the brightness control means alternately performs near infrared brightness control and visible light brightness control in time. The in-vehicle image recognition apparatus according to claim 1, which is executed.

Images