JP2021087119A - In-vehicle imaging device and occupant monitoring device - Google Patents

Abstract

To stably image an object by an imaging unit even when the brightness of the object changes.SOLUTION: An in-vehicle imaging device 1 includes an imaging unit 7 that captures the inside of a vehicle 50, a brightness detection unit 4 that detects the brightness of the face of a driver D of the vehicle 50 in an image captured by the imaging unit 7, an exposure adjustment unit 6 that adjusts the brightness of the image captured by the imaging unit 7 by adjusting an exposure time of the imaging unit 7 to change the amount of light captured by the imaging unit 7, and a response suppression unit 5 that suppresses the responsiveness when the brightness of the image is adjusted by the exposure adjustment unit 6 to the change in the brightness of the face of the driver D detected by the brightness detection unit 4. The response suppression unit 5 changes the responsiveness on the basis of the presence or absence of the on/off switching operation of a vehicle exterior light 11 that is linked to the change in the brightness of the face of the driver D.SELECTED DRAWING: Figure 1

Description

The present invention relates to an in-vehicle imaging device that images the inside of a vehicle and an occupant monitoring device that images an occupant in the vehicle and monitors the occupant based on the image.

For example, as disclosed in Patent Document 1 and Patent Document 2, there are an in-vehicle image pickup device and an occupant monitoring device that take an image of the inside of a vehicle or an occupant in the vehicle by an image pickup unit composed of a camera. The occupant monitoring device monitors the occupant by detecting the occupant's face, line-of-sight direction, etc., based on the image captured by the in-vehicle image pickup device. The detection result and the monitoring result are used for controlling the in-vehicle device mounted on the vehicle in order to drive the vehicle safely.

In a predetermined range inside the vehicle imaged by the image pickup unit, ambient light such as sunlight passing through the window of the vehicle and light emitted by the interior light is emitted at various brightness and timing according to changes in the surrounding environment surrounding the vehicle. Incident. Under the influence of this disturbance light, the brightness of the object imaged by the imaging unit constantly changes, causing abnormalities such as overexposure and underexposure in the image captured by the imaging unit, and the imaging becomes stable. There are some things that cannot be done.

Therefore, in Patent Document 1 and Patent Document 2, the amount of light taken into the imaging unit is changed by controlling the exposure of the imaging unit according to the change in the ambient environment and the brightness of the object. The brightness of the image captured by is adjusted.

Further, in Patent Document 1, based on the map data acquired from the navigation device, the entry and exit of the vehicle into the tunnel is predicted, and the sudden change in the brightness of the surrounding environment is detected from the prediction result. Alternatively, a sudden change in the brightness of the image is detected based on the brightness of the image captured by the imaging unit. Then, the exposure time (exposure value) of the imaging unit is adjusted according to these sudden changes in brightness.

Further, in Patent Document 2, the external light state outside the vehicle is determined based on the output and time of an external light detection camera or optical sensor provided outside the vehicle, or navigation information acquired from GPS or a navigation device, and the determination result is obtained. To determine the current time zone and weather from. Then, in the sunny daytime, the reference time of the image brightness for the exposure control of the imaging unit is shortened, and the exposure time is adjusted based on the measured value of the image brightness to follow the fluctuation of the in-vehicle brightness. In the daytime or nighttime in cloudy or rainy weather, the reference time of the image brightness is lengthened, the exposure time is adjusted based on the measured value of the image brightness, and automatic exposure is performed by the light of the headlight of an oncoming vehicle. Avoid reacting.

Japanese Unexamined Patent Publication No. 2018-148423 Japanese Unexamined Patent Publication No. 2016-49260

If the exposure of the imaging unit is controlled by excessively reacting to a change in the brightness of the object imaged by the imaging unit, the control may become unstable and the device may easily malfunction. As a countermeasure, it is conceivable to suppress the responsiveness of exposure control to changes in the ambient environment and the brightness of the object. However, in a situation where the brightness of the surrounding environment and the object changes suddenly, such as when a vehicle enters or exits a tunnel, it is not possible to follow the sudden change in brightness due to the suppression of responsiveness, and the imaging unit There is a possibility that an abnormality such as overexposure may occur in the image captured by the above. Further, according to the image in which the abnormality occurs in this way, the function of monitoring the occupant of the vehicle may be temporarily interrupted.

An object of the present invention is to provide an in-vehicle imaging device and an occupant monitoring device capable of stably imaging an object by an imaging unit even if the brightness of the object changes.

The in-vehicle image pickup device according to the present invention has an image pickup unit that images the inside of a vehicle, a brightness detection unit that detects the brightness of an object imaged by the image pickup unit, and an image pickup by changing the amount of light captured by the image pickup unit. When the brightness adjustment unit adjusts the brightness of the image captured by the unit and the brightness adjustment unit adjusts the brightness of the image in response to the change in the brightness of the object detected by the brightness detection unit. It is provided with a response suppressing unit that suppresses the responsiveness of the above. The response suppression unit changes the responsiveness based on the presence or absence of a predetermined operation of the in-vehicle device mounted on the vehicle, which is linked to the change in the brightness of the object.

Further, the occupant monitoring device according to the present invention includes the in-vehicle image pickup device and an occupant monitoring unit that monitors the occupants based on an image of an occupant in the vehicle by an image pickup unit provided in the in-vehicle image pickup device.

According to the in-vehicle image pickup device and the occupant monitoring device of the present invention, the responsiveness when adjusting the brightness of an image by the brightness adjusting unit with respect to the change in the brightness of the object detected by the brightness detecting unit is determined. It is suppressed by the response suppression unit. Therefore, the brightness of the image captured by the imaging unit is appropriately adjusted by changing the amount of light taken into the imaging unit by the brightness adjusting unit without excessively reacting to the change in the brightness of the object. After adjustment, the image pickup unit can stably image the object. Further, the responsiveness is not always suppressed by a certain amount, but the responsiveness changes based on the presence or absence of a predetermined operation of the in-vehicle device linked to the change in the brightness of the object. Therefore, even if the brightness of the object suddenly changes due to the influence of ambient light and the sudden change in brightness cannot be detected from the detection result of the brightness detection unit, it is based on the presence or absence of a predetermined operation of the in-vehicle device. Therefore, the responsiveness can be changed in response to the sudden change in brightness. As a result, the brightness of the captured image can be appropriately adjusted by the brightness adjusting unit with the responsiveness corresponding to the sudden change in the brightness of the object, and the object can be stably imaged by the imaging unit. Then, based on the captured image, the occupant monitoring unit can continue to stably monitor the occupants of the vehicle.

In the present invention, the response suppression unit may promote the responsiveness when there is a predetermined operation of the in-vehicle device as compared with the case where the predetermined operation is not performed.

Further, in the present invention, the brightness detection unit may detect the brightness of the object based on the image captured by the imaging unit.

Further, in the present invention, the brightness adjusting unit may adjust the brightness of the image by changing the exposure time of the imaging unit.

Further, in the present invention, an illumination unit that illuminates a range imaged by the imaging unit may be further provided, and the brightness adjustment unit may adjust the brightness of the image by changing the brightness of the illumination unit.

Further, in the present invention, the response suppression unit refers to the time for which the detection result of the brightness detection unit is referred to for the brightness adjustment by the brightness adjustment unit, or the brightness adjustment unit refers to the image after referring to the detection result. The responsiveness may be changed by changing the time until the brightness adjustment is started.

Further, in the present invention, the in-vehicle device includes an outside light that illuminates the surroundings of the vehicle, and the response suppression unit does not have the switching operation when the outside light is switched to the on or off state. It may be more responsive than it is.

According to the present invention, it is possible to provide an in-vehicle imaging device and an occupant monitoring device that can stably image an object by the imaging unit even if the brightness of the object changes.

It is a block diagram of the occupant monitoring apparatus of 1st Embodiment of this invention. It is a flowchart which showed the imaging operation of the vehicle interior imaging apparatus of 1st Embodiment. It is a block diagram of the occupant monitoring apparatus of the 2nd Embodiment of this invention. It is a flowchart which showed the imaging operation of the vehicle interior imaging apparatus of 2nd Embodiment.

Hereinafter, embodiments of the present invention will be described with reference to the drawings. In each figure, the same parts or corresponding parts are designated by the same reference numerals.

FIG. 1 is a configuration diagram of the occupant monitoring device 100 of the first embodiment. The occupant monitoring device 100 is mounted on a vehicle 50 composed of a four-wheeled vehicle.

The occupant monitoring device 100 includes an in-vehicle image pickup device 1 and an occupant monitoring unit 3. The in-vehicle image pickup device 1 includes a face detection section 2, a brightness detection section 4, a response suppression section 5, an exposure adjustment section 6, and an image pickup section 7. The blocks 2 to 6 are built in a microcomputer (not shown) that controls the overall operation of the occupant monitoring device 100, and the functions of each block (details will be described later) are realized by software.

The imaging unit 7 is composed of, for example, an infrared camera, and images the face of the driver (occupant) D seated in the driver's seat of the vehicle 50.

The face detection unit 2 detects the face of the driver D and the feature points of the face from the image (moving image) captured by the image pickup unit 7. The occupant monitoring unit 3 monitors the driver D by authenticating the face of the driver D and detecting the line-of-sight direction and the awakening state of the driver D based on the detection result of the face detection unit 2. .. The detection result of the face detection unit 2 and the monitoring result of the occupant monitoring unit 3 are transmitted to a vehicle control device (not shown) and used for controlling an in-vehicle device mounted on the vehicle 50 to safely drive the vehicle 50. To.

The brightness detection unit 4 detects the brightness of the driver D's face in the image captured by the image pickup unit 7. In this example, the brightness detection unit 4 detects the brightness of the driver D's face in the image. As another example, the brightness or shading of the driver D's face in the image may be detected.

The exposure adjustment unit 6 adjusts the exposure time of the image pickup unit 7. When the exposure adjustment unit 6 adjusts the exposure time, the amount of light captured by the image pickup unit 7 changes, and the brightness of the image captured by the image pickup unit 7 is also adjusted.

Specifically, when the exposure adjustment unit 6 lengthens the exposure time, the amount of light captured by the image pickup unit 7 increases, and the image captured by the image pickup unit 7 becomes brighter as a whole. Further, when the exposure adjusting unit 6 shortens the exposure time, the amount of light captured by the imaging unit 7 is reduced, and the image captured by the imaging unit 7 becomes dark as a whole. The exposure adjustment unit 6 is an example of the "brightness adjustment unit" of the present invention.

A light sensor 12 mounted on the vehicle 50 is electrically connected to the occupant monitoring device 100. The light sensor 12 detects the lighting operation and the extinguishing operation of the vehicle exterior light 11 mounted on the vehicle 50. The vehicle exterior light 11 includes a headlight, a small lamp, a fog lamp, a tail lamp, and the like. The vehicle exterior light 11 illuminates the surroundings of the vehicle 50 by emitting light.

Further, the vehicle exterior light 11 automatically switches between a lighting state and an extinguishing state based on an output from an optical sensor (not shown) that detects the brightness of the surrounding environment of the vehicle 50. Further, when the driver D operates the operation unit (not shown) according to the change in the brightness of the surrounding environment, the vehicle exterior light 11 is manually switched between the on and off states. The vehicle exterior light 11 is an example of the "vehicle-mounted device" of the present invention.

By changing the brightness of the surrounding environment of the vehicle 50, ambient light (sunlight, street light, light of another vehicle, light of the interior light of the vehicle 50, etc.) can be emitted outside the vehicle 50 at various brightnesses and timings. The brightness of the face of the driver D in the car changes as the light enters the car. Further, the vehicle exterior light 11 is automatically or manually switched to a lighting state or an extinguishing state according to a change in the brightness of the surrounding environment.

That is, the operation of switching the outside light 11 to the on or off state is linked to the change in the brightness of the surrounding environment of the vehicle 50 and the brightness of the driver D's face. Therefore, the occupant monitoring device 100 changes (suddenly changes) the brightness of the surrounding environment and the brightness of the driver D's face by detecting that the outside light 11 has been switched to the on or off state. ) Can be detected. Based on the output of the light sensor 12, the response suppression unit 5 detects whether or not the outside light 11 is switched to a lighting state or an extinguishing state.

The response suppression unit 5 adjusts the brightness of the captured image by adjusting the exposure time in the exposure adjustment unit 6 in response to the change in the brightness of the driver D's face detected by the brightness detection unit 4. Suppress responsiveness. Further, the response suppression unit 5 changes the responsiveness based on the presence or absence of the switching operation of the vehicle exterior light 11 to the lighting state or the extinguishing state. In the present embodiment, suppressing the responsiveness means lengthening the response time to a change in brightness (slowing the response speed), and promoting responsiveness means shortening the response time (reducing the response speed). To do (fast). The same applies to the second embodiment described later.

Specifically, for example, when adjusting the brightness of an image by the exposure adjusting unit 6, the response suppressing unit 5 suppresses the responsiveness by lengthening the time for referring to the detection result of the brightness detecting unit 4. , By shortening the reference time, responsiveness is promoted. Alternatively, the time from when the response suppressing unit 5 refers to the detection result of the brightness detecting unit 4 until the exposure adjusting unit 6 starts adjusting the exposure of the imaging unit 7 (adjusting the brightness of the image) is delayed. Therefore, the responsiveness is suppressed and the start time is accelerated to promote the responsiveness.

FIG. 2 is a flowchart showing an imaging operation of the in-vehicle imaging device 1 of the first embodiment. The imaging operation is started when the vehicle 50 is started (ignition is turned on) and is terminated when the vehicle 50 is stopped (ignition is turned off). The same applies to the imaging operation of FIG. 4 described later.

In FIG. 2, first, the exposure adjustment unit 6 sets the exposure time of the image pickup unit 7 to an initial value (step S1), and the image pickup unit 7 executes an image pickup in the vehicle (step S2). At this time, the face detection unit 2 detects the driver D's face based on the image captured by the image pickup unit 7, and the occupant monitoring unit 3 detects the driver D based on the detection result of the face detection unit 2. Monitor.

Next, the brightness detection unit 4 detects the brightness of the driver D's face in the image captured by the image pickup unit 7 (step S3). Then, the brightness detection unit 4 compares the detected value of the detected brightness with the predetermined brightness allowable range.

As a result of the comparison, if the detected value of the brightness is within the permissible range, the brightness of the face of the driver D is appropriate (step S4: YES), and the exposure adjusting unit 6 changes the exposure time of the imaging unit 7. The imaging unit 7 continues imaging (step S2).

On the other hand, if the detected value of brightness is out of the permissible range, the brightness of the driver D's face is not appropriate (step S4: NO), and in this case, the response suppression unit 5 is based on the output of the light sensor 12. The presence or absence of automatic or manual switching operation of the vehicle exterior light 11 is detected.

Within a predetermined time after determining that the brightness of the driver D's face is not appropriate in step S4, it is detected that there is no switching operation from the off state to the on state of the outside light 11 (step S5: NO). It is also assumed that it is detected that there is no switching operation from the lighting state of the vehicle exterior light 11 to the extinguishing state (step S6: NO). In this case, the exposure adjustment unit 6 determines the exposure time of the imaging unit 7 based on the brightness detection value of the brightness detection unit 4 with the first response (long response time) suppressed by the response suppression unit 5. The brightness of the image captured by the imaging unit 7 is adjusted (step S7).

In step S7, the exposure is adjusted so as to eliminate the difference between the brightness detection value by the brightness detection unit 4 and the permissible range (that is, the brightness of the driver D's face becomes an appropriate brightness). Part 6 adjusts the exposure time. (The same applies to step S8 described later.)

The reference time of the detection result of the brightness detection unit 4 or the start time of adjustment by the exposure adjustment unit 6 for realizing the first responsiveness is stored in advance in a memory (not shown). The reference time of the detection result of the brightness detection unit 4 or the start time of adjustment by the exposure adjustment unit 6 for realizing the second responsiveness described later is also stored in the memory in advance.

For example, when the brightness of the surrounding environment of the vehicle 50 suddenly changes (bright → dark) due to the vehicle 50 entering the tunnel, the outside light 11 automatically or manually switches from the off state to the on state in conjunction with this. .. In this case, the response suppression unit 5 detects that the outside light 11 has been switched from the off state to the on state based on the output of the light sensor 12 (step S5: YES).

Further, when the brightness of the surrounding environment of the vehicle 50 suddenly changes (dark → bright), for example, when the vehicle 50 exits the tunnel, the outside light 11 automatically or manually changes from the lit state to the extinguished state in conjunction with this. Switch. In this case, the response suppression unit 5 detects that the outside light 11 has been switched from the lit state to the extinguished state based on the output of the light sensor 12 (step S6: YES).

When the switching operation of turning on / off the vehicle exterior light 11 in any of steps S5 and S6 is detected, the exposure adjustment unit 6 has a second responsiveness (short response time) that is promoted rather than the first responsiveness, and the brightness. Based on the brightness detection value of the detection unit 4, the exposure time of the image pickup unit 7 is adjusted to adjust the brightness of the image captured by the image pickup unit 7 (step S8).

After step S7 or step S8, the process returns to step S2, the imaging by the imaging unit 7 is continued, and the subsequent processes are repeated.

According to the first embodiment described above, the exposure adjusting unit 6 adjusts the exposure time of the imaging unit 7 in response to the change in the brightness of the driver D's face detected by the brightness detecting unit 4, and the imaging unit 7 adjusts the exposure time. The responsiveness when adjusting the brightness of the image captured by 7 is suppressed by the response suppressing unit 5. Therefore, the exposure adjusting unit 6 appropriately adjusts the brightness of the image according to the change in the brightness of the driver D's face without excessively reacting, and the imaging unit 7 stabilizes the driver D's face. Can be imaged.

Further, in the first embodiment, the response suppression unit 5 does not always suppress the responsiveness by a certain amount, but the operation of switching the on / off of the vehicle exterior light 11 linked to the change in the brightness of the driver D's face. The responsiveness changes depending on the presence or absence. Therefore, even if the brightness of the driver D's face suddenly changes due to the influence of the ambient light and the sudden change in the brightness cannot be detected from the detection result of the brightness detection unit 4, the outside light 11 is turned on. -The responsiveness can be changed to respond to a sudden change in the brightness based on the presence or absence of the extinguishing switching operation. As a result, the exposure adjustment unit 6 appropriately adjusts the brightness of the image with the responsiveness corresponding to the sudden change in the brightness of the driver D's face, and the image pickup unit 7 stably images the driver D's face. be able to. Then, based on the captured image, the occupant monitoring unit 3 can stably monitor the driver D.

Further, in the first embodiment, when there is a switching operation of turning on / off the vehicle exterior light 11, the responsiveness is promoted as compared with the case where there is no such switching operation. Therefore, when the outside light 11 is switched on / off, that is, when the brightness of the driver D's face suddenly changes, the exposure is adjusted by following the sudden change in the brightness with a short response time. The brightness of the image can be appropriately adjusted by the unit 6, and the face of the driver D can be stably imaged by the image pickup unit 7.

Further, in the first embodiment, the brightness of the driver D's face in the image captured by the imaging unit 7 is detected by the brightness detecting unit 4. Therefore, the exposure adjustment unit 6 adjusts the exposure time according to the change in the brightness of the driver D's face in the image, the amount of light captured by the image pickup unit 7 is changed, and the image pickup unit 7 takes an image. By adjusting the brightness of the image, the face of the driver D can be stably imaged. Further, even if an abnormality such as overexposure or underexposure occurs temporarily in the image captured by the imaging unit 7 due to a sudden change in the brightness of the face of the driver D, the face of the driver D in the image may be temporarily abnormal. Since it is possible to accurately change the responsiveness based on the presence or absence of switching operation of turning on / off the outside light 11 instead of the brightness, it is possible to stably image the face of the driver D thereafter. ..

Further, in the first embodiment, the brightness of the image captured by the imaging unit 7 is adjusted by adjusting the exposure time by the exposure adjusting unit 6. Therefore, while the brightness of the driver D's face does not change suddenly, it is possible to adjust the captured image to be brighter as a whole by lengthening the exposure time to make the dark part in the image clear. it can. In addition, when the brightness of the driver D's face suddenly changes, by shortening the exposure time, the captured image is adjusted to be dark as a whole, and abnormalities such as overexposure occur in the image. Can be prevented.

Further, in the first embodiment, the time for which the response suppression unit 5 refers to the detection result of the brightness detection unit 4 for exposure adjustment, or until the exposure adjustment unit 6 starts the adjustment after referring to the detection result. By changing the time of, the responsiveness can be suppressed or promoted.

Further, in the first embodiment, the first responsiveness (long) suppressed by the response suppressing unit 5 while the brightness of the surrounding environment of the vehicle 50 does not change suddenly and the outside light 11 does not switch to the lit state or the extinguished state. In response time), the exposure adjustment unit 6 adjusts the brightness of the image according to the change in the brightness of the driver D's face, and the image pickup unit 7 can stably image the driver D's face. .. Further, when the brightness of the surrounding environment of the vehicle 50 suddenly changes, the outside light 11 automatically or manually switches to the lighting state or the extinguishing state, so that the second responsiveness (short response time) is promoted rather than the first responsiveness. ), The exposure adjustment unit 6 adjusts the brightness of the image according to the change in the brightness of the driver D's face, and the image pickup unit 7 can stably image the driver D's face. Further, the responsiveness can be automatically changed according to the automatic switching operation of the exterior light 11.

FIG. 3 is a configuration diagram of the occupant monitoring device 100 of the second embodiment. In the second embodiment, in addition to the configuration shown in FIG. 1, the in-vehicle image pickup apparatus 1 includes a lighting unit 9 and a lighting adjusting unit 8.

The illumination unit 9 illuminates a predetermined range in the vehicle imaged by the image pickup unit 7 by emitting light. For example, when the image pickup unit 7 is composed of an infrared camera, the illumination unit 9 is composed of a plurality of infrared LEDs. The illumination adjustment unit 8 is provided in the microcomputer (not shown) together with the other blocks 2 to 6, and adjusts the brightness of the illumination unit 9. The function of the lighting adjustment unit 8 is also realized by software.

In the second embodiment, not only the exposure time of the image pickup unit 7 is adjusted by the exposure adjustment unit 6, but also the brightness of the illumination unit 9 is adjusted by the illumination adjustment unit 8, so that the amount of light captured by the image pickup unit 7 is determined. Change to adjust the brightness of the captured image.

Specifically, when the illumination adjustment unit 8 brightens the brightness of the illumination unit 9, the amount of light captured by the image pickup unit 7 increases, and the captured image becomes brighter as a whole. Further, when the illumination adjusting unit 8 darkens the brightness of the illumination unit 9, the amount of light captured by the imaging unit 7 is reduced, and the captured image becomes dark as a whole. The illumination adjustment unit 8 is an example of the “brightness adjustment unit” of the present invention together with the exposure adjustment unit 6.

Further, in the second embodiment, the response suppression unit 5 adjusts the exposure time or the lighting in the exposure adjustment unit 6 in response to the change in the brightness of the driver D's face detected by the brightness detection unit 4. By adjusting the brightness in the unit 8, the responsiveness when the brightness of the captured image is adjusted by the imaging unit 7 is suppressed. Further, the response suppressing unit 5 changes the responsiveness based on the presence or absence of the switching operation of the vehicle exterior light 11 to the lighting state or the extinguishing state.

Specifically, for example, when adjusting the brightness of an image by the exposure adjusting unit 6 or the lighting adjusting unit 8, the response suppressing unit 5 increases the time for referring to the detection result of the brightness detecting unit 4. By suppressing the responsiveness and shortening the reference time, the responsiveness is promoted. Alternatively, the response is responsive by delaying the time from when the response suppression unit 5 refers to the detection result of the brightness detection unit 4 until the exposure adjustment unit 6 or the illumination adjustment unit 8 starts adjusting the brightness of the image. By suppressing the above and accelerating the start time, responsiveness is promoted.

FIG. 4 is a flowchart showing the imaging operation of the in-vehicle imaging device 1 of the second embodiment. In FIG. 4, the same reference numerals are given to the same processes as those in the first embodiment shown in FIG.

In FIG. 4, first, the exposure adjustment unit 6 sets the exposure time of the image pickup unit 7 to the initial value, and the illumination adjustment unit 8 sets the brightness of the illumination unit 9 to the initial value (step S1') to perform imaging. The inside of the vehicle is imaged by the unit 7 (step S2). Then, the brightness of the driver D's face in the image captured by the imaging unit 7 is detected by the brightness detecting unit 4 (step S3), and if the brightness is appropriate (step S4: YES), Imaging is continued by the imaging unit 7 without changing the exposure time of the imaging unit 7 and the brightness of the illumination unit 9 (step S2).

On the other hand, when the brightness of the driver D's face is not appropriate (step S4: NO), there is no switching operation from the off state to the on state of the outside light 11 (step S5: NO), and the outside light 11 is turned on. If there is no switching operation from the state to the extinguished state (step S6: NO), the exposure adjusting unit 6 determines the brightness by the brightness detecting unit 4 with the first responsiveness (long response time) suppressed by the response suppressing unit 5. The exposure time of the image pickup unit 7 is adjusted based on the detected value, or the illumination adjustment unit 8 adjusts the brightness of the illumination unit 9 to adjust the brightness of the image captured by the image pickup unit 7. (Step S7').

In step S7', the exposure is made so as to eliminate the difference between the brightness detection value by the brightness detection unit 4 and the permissible range (that is, the brightness of the driver D's face becomes an appropriate brightness). The exposure time is adjusted by the adjustment unit 6 and the brightness is adjusted by the illumination adjustment unit 8. However, both the exposure time and the brightness may be adjusted, or one of them may be adjusted. .. (The same applies to step S8'described later.)

The reference time of the detection result of the brightness detection unit 4 or the start time of adjustment by the exposure adjustment unit 6 and the illumination adjustment unit 8 for realizing the first responsiveness is stored in advance in a memory (not shown). There is. The reference time of the detection result of the brightness detection unit 4 or the start time of adjustment by the exposure adjustment unit 6 and the illumination adjustment unit 8 for realizing the second responsiveness described later is also stored in the memory in advance.

Further, when the brightness of the driver D's face is not appropriate (step S4: NO), when the outside light 11 is switched from the off state to the on state (step S5: YES), or outside the vehicle. When there is a switching operation from the lighting state of the light 11 to the extinguishing state (step S6: YES), the exposure adjusting unit 6 has a second responsiveness (short response time) that is promoted rather than the first responsiveness. , The exposure time of the image pickup unit 7 is adjusted based on the brightness detection value by the brightness detection unit 4, or the illumination adjustment unit 8 adjusts the brightness of the illumination unit 9 and is imaged by the image pickup unit 7. The brightness of the image is adjusted (step S8').

According to the second embodiment described above, the exposure adjusting unit 6 adjusts the exposure time of the imaging unit 7 or adjusts the lighting in response to the change in the brightness of the driver D's face detected by the brightness detecting unit 4. The response suppression unit 5 suppresses the responsiveness when the brightness of the illumination unit 9 is adjusted by the unit 8 and the brightness of the image captured by the image pickup unit 7 is adjusted. Therefore, the brightness of the image is appropriately adjusted by the exposure adjusting unit 6 or the lighting adjusting unit 8 without excessively reacting according to the change in the brightness of the driver D's face, and the driver 7 adjusts the brightness of the image appropriately. The face of D can be stably imaged.

Further, in the second embodiment, as in the first embodiment, the responsiveness changes based on the presence or absence of the on / off switching operation of the vehicle exterior light 11. Therefore, even if the brightness of the driver D's face suddenly changes due to the influence of ambient light, the responsiveness is changed so as to correspond to the sudden change in the brightness, and the exposure adjustment unit 6 or the illumination adjustment unit 8 displays the image. The brightness of the driver D can be adjusted appropriately, and the image pickup unit 7 can stably image the driver D's face. Then, based on the captured image, the occupant monitoring unit 3 can stably monitor the driver D.

Further, in the second embodiment, as in the first embodiment, when the surrounding environment or the brightness of the driver D's face suddenly changes and the outside light 11 is turned on / off, the operation is concerned. Responsiveness is promoted compared to when there is no switching operation. Therefore, following a sudden change in the brightness of the driver D's face in a short response time, the exposure adjustment unit 6 or the illumination adjustment unit 8 appropriately adjusts the brightness of the image, and the image pickup unit 7 causes the driver D to adjust the brightness of the image appropriately. The face can be stably imaged.

Further, in the second embodiment, the brightness of the driver D's face in the image captured by the imaging unit 7 is detected by the brightness detecting unit 4, and the exposure adjusting unit 6 responds to the change in the brightness. The exposure time of the imaging unit 7 is adjusted by, or the brightness of the illumination unit 9 is adjusted by the illumination adjustment unit 8. Therefore, the change width of the amount of light captured by the image pickup unit 7 and the adjustment range of the brightness of the image captured by the image pickup unit 7 are further widened, and the face of the driver D is stably imaged by the image pickup unit 7. Can be done.

Further, in the second embodiment, not only the exposure adjustment unit 6 adjusts the exposure time of the image pickup unit 7, but also the illumination adjustment unit 8 adjusts the brightness of the illumination unit 9, thereby adjusting the brightness of the captured image. adjust. Therefore, while the brightness of the driver D's face does not change suddenly, the exposure time of the imaging unit 7 is lengthened or the brightness of the illumination unit 9 is increased to make the captured image as a whole. It can be adjusted to be brighter to sharpen dark areas in the image. Further, when the brightness of the driver D's face suddenly changes, the exposure time of the image pickup unit 7 is shortened or the brightness of the illumination unit 9 is darkened so that the captured image becomes dark as a whole. It can be adjusted to prevent abnormalities such as overexposure in the image.

Further, in the second embodiment, the response suppression unit 5 refers to the detection result of the brightness detection unit 4 for exposure adjustment or brightness adjustment, or the exposure adjustment unit 6 or illumination after referring to the detection result. By changing the time until the adjusting unit 8 starts the adjustment, the responsiveness can be suppressed or promoted.

Further, in the second embodiment, the first responsiveness (long) suppressed by the response suppressing unit 5 while the brightness of the surrounding environment of the vehicle 50 does not change suddenly and the outside light 11 does not switch to the lit state or the extinguished state. In response time), the brightness of the image is adjusted by the exposure adjustment unit 6 or the illumination adjustment unit 8 according to the change in the brightness of the driver D's face, and the image pickup unit 7 stabilizes the driver D's face. It can be imaged. Further, when the brightness of the surrounding environment of the vehicle 50 suddenly changes, the vehicle exterior light 11 automatically or manually switches to the on state or the off state. Therefore, when the outside light 11 is switched on / off, the brightness of the driver D's face changes with the second responsiveness (short response time) promoted rather than the first responsiveness. The brightness of the image can be adjusted by the exposure adjustment unit 6 or the illumination adjustment unit 8, and the driver D's face can be stably imaged by the image pickup unit 7.

In addition to the above-described embodiments, the present invention can employ various embodiments as described below.

In the above embodiment, an example in which the responsiveness is changed based on the presence or absence of the switching operation of turning on / off the vehicle exterior light 11 is shown, but the present invention is not limited to this. In addition to this, for example, when an in-vehicle device such as a power window, sunroof, sun visor, wiper, steering wheel, accelerator, brake, or power seat performs a predetermined operation automatically or manually, the state of incident light incident on the inside of the vehicle Since the brightness of the imaged object such as the driver changes due to the change, the responsiveness may be changed based on the presence or absence of a predetermined operation of those in-vehicle devices.

In addition, since the brightness of the object to be imaged in the vehicle changes even when the interior light of the vehicle is switched to the on or off state, the responsiveness is determined based on the presence or absence of the switching operation of turning on / off the interior light. It may be changed. Further, the responsiveness may be changed based on the result of detecting the presence or absence of a predetermined operation of two or more in-vehicle devices among the above-mentioned in-vehicle devices. In this case, for example, the responsiveness may be changed in two or more steps for each operation of the in-vehicle device.

In the above embodiment, as an example of the response time to the change in brightness, the response time until the brightness of the image is adjusted by the exposure adjustment unit 6 or the illumination adjustment unit 8 is given as an example, but the present invention is limited to this. It's not a thing. For example, when the exposure adjustment unit 6 adjusts the exposure time or the illumination adjustment unit 8 adjusts the brightness of the illumination unit 9, the exposure time and the accuracy of the brightness (adjustment accuracy) may be set as the responsiveness. ..

In the above embodiment, an example is shown in which the brightness detection unit 4 for detecting the brightness of the driver D's face in the image captured by the image pickup unit 7 is provided, but the present invention is not limited to this. Absent. In addition to this, for example, an optical sensor, an illuminance sensor, or the like may be provided in the vehicle as a brightness detection unit to measure the brightness of the driver D's face. Further, the brightness of an object other than the driver D's face may be detected by the brightness detection unit 4.

In the above embodiment, an example in which the present invention is applied to the in-vehicle imaging device 1 that images the face of the driver D by the imaging unit 7, but other occupants in the vehicle and an object in the vehicle are captured by the imaging unit. The present invention can also be applied to an in-vehicle imaging device for imaging.

1 In-vehicle image pickup device 3 Crew monitoring unit 4 Brightness detection unit 5 Response suppression unit 6 Exposure adjustment unit (brightness adjustment unit)
7 Imaging unit 8 Lighting adjustment unit (brightness adjustment unit)
9 Lighting unit 11 Outside light (vehicle-mounted device)
50 Vehicle 100 Crew monitoring device

Claims (8)

An imaging unit that captures the inside of the vehicle and
A brightness detection unit that detects the brightness of an object imaged by the imaging unit, and a brightness detection unit.
In an in-vehicle imaging device including a brightness adjusting unit that adjusts the brightness of an image captured by the imaging unit by changing the amount of light captured by the imaging unit.
Further provided with a response suppression unit that suppresses the responsiveness when adjusting the brightness of the image by the brightness adjustment unit in response to a change in the brightness of the object detected by the brightness detection unit.
The response suppressing unit is an in-vehicle imaging device that changes the responsiveness based on the presence or absence of a predetermined operation of an in-vehicle device mounted on the vehicle, which is linked to a change in the brightness of the object. .. In the in-vehicle imaging device according to claim 1,
The response suppressing unit is an in-vehicle imaging device, characterized in that when a predetermined operation of the in-vehicle device is performed, the response is promoted as compared with the case where the predetermined operation is not performed. In the in-vehicle imaging device according to claim 1 or 2.
The in-vehicle image pickup device is characterized in that the brightness detection unit detects the brightness of the object based on the image captured by the image pickup unit. In the in-vehicle imaging device according to any one of claims 1 to 3.
The in-vehicle image pickup device is characterized in that the brightness adjustment unit adjusts the brightness of the image by changing the exposure time of the image pickup unit. In the in-vehicle imaging device according to any one of claims 1 to 4.
A lighting unit that illuminates the range imaged by the imaging unit is further provided.
The in-vehicle image pickup device is characterized in that the brightness adjusting unit adjusts the brightness of the image by changing the brightness of the lighting unit. In the in-vehicle imaging device according to any one of claims 1 to 5.
The response suppression unit refers to the time for which the detection result of the brightness detection unit is referred to for brightness adjustment by the brightness adjustment unit, or after the detection result is referred to, the brightness adjustment unit refers to the brightness of the image. An in-vehicle image pickup device, characterized in that the responsiveness is changed by changing the time until the adjustment of the lightness is started. In the in-vehicle imaging device according to any one of claims 1 to 6.
The in-vehicle device includes an exterior light that illuminates the surroundings of the vehicle.
The response suppressing unit is an in-vehicle imaging device, characterized in that when there is a switching operation of the outside light to a lighting state or an extinguishing state, the responsiveness is promoted as compared with the case where the switching operation is not performed. The in-vehicle imaging device according to any one of claims 1 to 7.
An occupant monitoring device including an occupant monitoring unit that monitors the occupant based on an image of an occupant in the vehicle by the imaging unit provided in the in-vehicle image pickup device.

Images