JP5288189B2 - Information processing apparatus and method, and program - Google Patents

Description

  The present invention relates to an information processing device and method, and a program, and more particularly, to an information processing device and method, and a program that further improve the face recognition rate.

  2. Description of the Related Art Conventionally, there is a surveillance camera system that monitors the state of a driver driving a vehicle with a camera (see, for example, Patent Document 1). Such a monitoring camera system captures an image including the inside of a vehicle with a monitoring camera, and recognizes a face from the captured image.

Japanese Patent Laid-Open No. 10-275212

  However, when a conventional surveillance camera system is used in a vehicle equipped with a sunroof, the face recognition rate may be reduced.

  The present invention has been made in view of such a situation, and further improves the face recognition rate.

  An information processing device according to one aspect of the present invention includes a first state in which external light is blocked, or external light as a state of a light blocking body having a function of blocking external light that is directly irradiated toward the inside of a vehicle. Detecting means for detecting a second state that does not block the light, control means for controlling the interior lighting of the vehicle by the illumination section, and imaging of the interior of the vehicle by the imaging section, and the imaging section under the control of the control means And a recognition means for recognizing a human face existing inside the vehicle based on the first image obtained as a result of being imaged by the control means, and when the first state is detected by the detection means, the control means If the second state is detected by the detection means, the illumination unit has more light than the first light amount. Illuminate with the second amount of light and cause the imaging unit to be less than the first exposure amount Performing second control for the imaging in the second exposure are.

  In the information processing apparatus according to one aspect of the present invention, as the state of the light blocking body having a function of blocking the external light directly irradiated toward the inside of the vehicle, the first state where the external light is blocked, or the external state The second state where the light is not blocked is detected, and the illumination inside the vehicle by the illumination unit and the imaging inside the vehicle by the imaging unit are controlled, and the result obtained by the imaging unit under the control is obtained. Based on the first image, a human face existing inside the vehicle is recognized. When the first state is detected, the first control is performed to cause the illumination unit to illuminate with the first light amount, and the imaging unit to capture an image with the first exposure amount. When the second state is detected, A second control is performed in which the illumination unit is illuminated with a second light amount greater than the first light amount, and the imaging unit is imaged with a second exposure amount that is less than the first exposure amount.

  Thereby, the face recognition rate can be further improved.

  The detection means is composed of, for example, a sunroof switch. The recognition means and the control means are constituted by, for example, a CPU (Central Processing Unit).

  When the first state is detected by the detection unit, the control unit further performs a third control for causing the imaging unit to capture an image with the first exposure amount in addition to the first control. When the second state is detected by the detection unit, in addition to the second control, the illumination unit is turned off, and a fourth control for causing the imaging unit to capture an image with the second exposure amount is performed. The image obtained by the image pickup unit under the first control or the second control is the image obtained by the image pickup unit under the third control or the fourth control. By using the obtained image as a second image, a calculation unit that calculates a difference image between the first image and the second image, and by performing predetermined processing on the difference image calculated by the calculation unit, Recognizing face image processing means.

  Thereby, the face recognition rate can be further improved.

  The detection means is composed of, for example, a sunroof switch. The recognition means and the control means are constituted by a CPU, for example.

  The recognition means further includes a shadow detection means for detecting the presence or absence of a shadow in the difference image calculated by the calculation means, and the face image processing means, when the shadow detection means detects no shadow, as a predetermined process, When the first process is performed on the difference image and the shadow detection unit detects that there is a shadow, a second process different from the first process can be performed on the difference image as a predetermined process. .

  Thereby, the face recognition rate can be further improved.

  The light blocking body blocks near-infrared light, and the detection means can detect the first state or the second state based on the intensity of near-infrared light inside the vehicle.

  Thereby, the face can be recognized more easily.

  The apparatus further includes light detection means for detecting the intensity of light outside the vehicle, and the control means performs the first control or the second control when the light intensity detected by the light detection means exceeds a threshold value. If the intensity of the light detected by the light detection means is equal to or less than the threshold value, the first control can be performed regardless of the state of the light shield detected by the detection means.

  Thereby, the face can be recognized more easily.

  The light detection means is composed of, for example, an illuminance sensor.

  An information processing method and program according to one aspect of the present invention are a method and program corresponding to the information processing apparatus according to one aspect of the present invention described above.

  According to the present invention, the face recognition rate can be further improved.

It is a figure which shows an example of the surveillance camera system to which this invention is applied. It is a block diagram which shows the structural example of the surveillance camera system of FIG. It is a figure for demonstrating the process example of the vehicle-mounted apparatus of FIG. It is a flowchart explaining the process example of the vehicle-mounted apparatus of FIG. It is a flowchart explaining the process example of the vehicle-mounted apparatus of FIG. It is a figure which shows an example of the surveillance camera system to which this invention is applied. It is a block diagram which shows the structural example of the surveillance camera system of FIG. It is a figure which shows the structural example of the hardware of the computer to which this invention is applied.

Hereinafter, two embodiments (hereinafter, referred to as first and second embodiments, respectively) will be described as embodiments of an information processing system to which the present invention is applied, with reference to the drawings. Therefore, description will be given in the following order.
1. First Embodiment (Example of detecting sunroof opening / closing state with sunroof sensor)
2. Second Embodiment (Example of Detecting Open / Closed State of Sunroof with Near Infrared Light Sensor)

<1. First Embodiment>

[Configuration example of surveillance camera system]

  FIG. 1 is a diagram showing an example of a surveillance camera system as a first embodiment of an information processing system to which the present invention is applied.

  In the present specification, the term “system” means an overall apparatus composed of a plurality of apparatuses and means. That is, it is possible to grasp the monitoring camera system as one device.

  The surveillance camera system 21 in the example of FIG. 1 is provided inside the vehicle 11.

  In the example of FIG. 1, the left direction in the figure is the front of the vehicle 11. Therefore, in the following description, the left direction in the figure is referred to as the front, the upper direction in the figure is referred to as the upper side, and the front and back directions in the figure are referred to as the side.

  FIG. 1 is a diagram illustrating an example of a compartment SS of the vehicle 11.

  In the example of FIG. 1, the passenger compartment SS is formed of a front window FG, a roof RF, and the like. The front window FG covers the upper front part of the passenger compartment SS. The roof RF forms the upper part of the passenger compartment SS. The roof RF is provided with a sunroof SR. A side window SW (not shown) forms the side upper part of the passenger compartment SS.

  A driver's seat US is provided slightly in front of the passenger compartment SS. The driver's seat US is located below the sunroof SR, for example. In front of the driver's seat US, a handle HD and a dashboard DB are provided. The dashboard DB is located below the front window FG.

  For example, the driver TS is facing the front, sitting in the driver's seat US, and driving the vehicle 11. For example, the driver TS's head is located near the headrest HR of the driver's seat US.

  The surveillance camera system 21 is provided inside the vehicle 11 having the vehicle interior SS described above. The surveillance camera system 21 is provided with a surveillance camera 31, an LED (Light Emitting Diode) illumination device 32, and an illuminance sensor 33. The surveillance camera system 21 is also provided with a sunroof switch 34, an in-vehicle device 35, a display 36, and a speaker 37. A monitoring camera 31, a lighting device 32, an illuminance sensor 33, a sunroof switch 34, a display 36, and a speaker 37 are connected to the in-vehicle device 35.

  The monitoring camera 31 and the illumination device 32 are disposed, for example, near the handle HD. The illuminance sensor 33 is disposed on the dashboard DB, for example. The sunroof switch 34 is disposed in the vicinity of the sunroof SR. The in-vehicle device 35 can be arranged at an arbitrary position inside the vehicle 11. The display 36 and the speaker 37 are disposed on the dashboard DB, for example.

  The monitoring camera system 21 illuminates a predetermined space inside the vehicle 11 (for example, the vicinity of the headrest HR) with strong light by the illumination device 32, and the monitoring camera 31 causes a predetermined space (for example, the headrest HR) inside the vehicle 11 to be illuminated. Near). The surveillance camera system 21 recognizes a face based on the captured image. For example, the monitoring camera system 21 determines the state of the driver TS based on the recognition result, and presents the determination result to the driver TS using the display 36 and the speaker 37. In this way, the monitoring camera system 21 monitors the state of the driver TS.

  In the first embodiment and the second embodiment to be described later, near-infrared light is used as illumination light by the illumination device 32. However, the illumination light by the illumination device 32 is not limited to near-infrared light, and light in an arbitrary band can be adopted.

  With further reference to FIG. 1, a detailed example of the sunroof SR will be described.

  The sunroof SR is provided with a glass part GB and a shade part SB. Both the glass part GB and the shade part SB can block sunlight outside the vehicle 11 (hereinafter referred to as “external light”) that irradiates directly toward the inside of the vehicle 11.

  Here, the open / closed state of the sunroof SR is defined as follows. That is, the closed state of the sunroof SR is defined as a state in which at least one of the glass part GB and the shade part SB blocks outside light. This state is, for example, a state in which at least one of the glass part GB and the shade part SB covers the entire opening KK of the roof RF. The open state of the sunroof SR is defined as a state where both the glass part GB and the shade part SB do not block the external light directly radiating toward the inside of the vehicle 11. This state is a state where, for example, both the glass part GB and the shade part SB do not cover at least a part of the opening KK of the roof RF.

  The method for switching the open / close state of the sunroof SR is not particularly limited. However, in the first embodiment and the second embodiment to be described later, for example, a method is adopted in which the in-vehicle device 35 switches the open / close state of the sunroof SR by driving the glass part GB and the shade part SB with motors, respectively. . In this method, for example, the in-vehicle device 35 controls the motor based on an operation of the driver TS to a switch (not shown) or the like.

[Description of conventional imaging illumination control method]

  Here, in order to facilitate understanding of the present invention and to clarify the background, an example of a control method (hereinafter referred to as a conventional control method) of an illumination unit and an imaging unit in a conventional surveillance camera system will be described. Note that a surveillance camera system (conventional surveillance camera system) to which a conventional control method is applied basically has the same configuration as the surveillance camera system 21. Therefore, description will be made below using components of the monitoring camera system 21. However, it should be noted that the illuminance sensor 33 and the sunroof switch 34 are not provided in the surveillance camera system 21 to which the conventional control method is applied.

  When the surveillance camera system 21 to which a conventional control method is applied is used in a vehicle equipped with a sunroof, the face recognition rate may be lowered.

  For example, the vehicle 11 in the example of FIG. 1 will be described. When the sunroof SR is opened (when the sunroof SR is in the open state), the driver TS is irradiated with external light, and the driver TS has a strong shadow on the face. Occurs. However, in the conventional control method, since it is not assumed that a strong shadow is generated on the face of the driver TS, a shadow (light spot) also appears in the image captured by the monitoring camera 31. The inventor has found that the recognition rate of the face is reduced due to this.

  In order to reduce the influence of the shadow, the present inventors have invented the following control method for the illumination unit and the imaging unit. That is, when the closed state of the light shield (for example, the sunroof SR) is detected, the illumination unit (for example, the illumination device 32) is illuminated with the normal light amount, and the imaging unit (for example, the monitoring camera 31) has the normal exposure amount. Let's take an image. And when the open state of the light-shielding body was detected, the lighting part was made to illuminate with light quantity larger than normal light quantity, and the method of making an imaging part image with exposure amount smaller than normal exposure amount was invented. Such a control method is hereinafter referred to as an open / close state control method. Hereinafter, the surveillance camera system 21 to which the open / close state control method of the present invention is applied will be described.

[Configuration Example of Surveillance Camera System of the Present Invention]

  Next, a configuration example of the surveillance camera system 21 to which the open / close state control method of the present invention is applied will be described.

  FIG. 2 is a block diagram illustrating a configuration example of the monitoring camera system 21.

  The surveillance camera 31 of the surveillance camera system 21 is composed of, for example, a digital camera. The monitoring camera 31 captures a predetermined space inside the vehicle 11 with a predetermined exposure time based on the control of the in-vehicle device 35 described later. In the present embodiment, for example, the monitoring camera 31 has an imaging device (imager) that has sensitivity to near infrared light and can selectively receive near infrared light.

  In the first embodiment and the second embodiment to be described later, it is assumed that the exposure amount of the monitoring camera 31 is controlled by the exposure time of the monitoring camera 31. However, the exposure amount control method of the monitoring camera 31 is not particularly limited. That is, for example, the exposure amount of the monitoring camera 31 may be controlled by the diaphragm of the monitoring camera 31, or may be controlled by both the exposure time and the diaphragm.

  The illuminating device 32 illuminates a predetermined space inside the vehicle 11 with a predetermined amount of light based on control of the in-vehicle device 35 described later. The illumination device 32 includes, for example, an LED that emits near-infrared light, and illuminates with near-infrared light by causing the LED to emit light. Note that the light emitter of the illumination device 32 is not particularly limited to an LED, and may be any light emitter that emits light in a desired band.

  In the first embodiment and the second embodiment to be described later, the monitoring camera 31 and the lighting device 32 are separate devices. For example, the monitoring camera 31 and the lighting device 32 are integrated into one device. It is also possible to do.

  The illuminance sensor 33 detects the intensity of external light by detecting the intensity of light radiated to the inside of the vehicle 11 via the front window FG, and notifies the in-vehicle device 35 of the intensity.

  The sunroof switch 34 detects the open / closed state of the sunroof SR. For example, when detecting the closed state of the sunroof SR, the sunroof switch 34 notifies the in-vehicle device 35 of this.

  In the first embodiment and the second embodiment to be described later, the sunroof switch 34 notifies that the sunroof SR is in the closed state. However, for example, the sunroof SR is in the open state. You may make it notify.

  The in-vehicle device 35 controls each part of the monitoring camera system 21. For example, the in-vehicle device 35 controls the monitoring camera 31 and the illumination device 32 based on notification from the illuminance sensor 33 and the sunroof switch 34. Moreover, the vehicle-mounted apparatus 35 acquires the image imaged with the monitoring camera 31, and recognizes a face based on the image. For example, the in-vehicle device 35 determines the state of the driver TS based on the recognition result, and presents the determination result to the driver TS using the display 36 or the speaker 37.

  Next, a configuration example of the in-vehicle device 35 will be described.

  The in-vehicle device 35 includes a camera illumination control unit 41, a drive pulse output unit 42, an image processing unit 43, a state determination unit 44, and an output control unit 45.

  Based on the notification from the illuminance sensor 33 and the sunroof switch 34, the camera illumination control unit 41 outputs the drive pulse width output from the drive pulse output unit 42 (hereinafter referred to as drive pulse width) and the illumination light of the illumination device 32. (Hereinafter referred to as illumination light quantity). Further, the camera illumination control unit 41 instructs the drive pulse output unit 42 to start capturing the reference image and the illumination image. The reference image refers to an image obtained by causing the monitoring camera 31 to capture an image when illuminated by the illumination device 32. The illumination image refers to an image obtained by causing the monitoring camera 31 to capture an image when the illumination device 32 turns off the illumination image.

  When starting to capture a reference image, the drive pulse output unit 42 outputs a drive pulse having a set width only to the monitoring camera 31. When starting to capture an illumination image, the drive pulse output unit 42 outputs a drive pulse having a set width to the monitoring camera 31 and the illumination device 32.

  The image processing unit 43 acquires a reference image and an illumination image captured by the control of the camera illumination control unit 41 from the monitoring camera 31, recognizes a face by performing predetermined image processing, and determines a state of the recognition result. Supplied to the unit 44. If the face cannot be recognized, the image processing unit 43 notifies the output control unit 45 of that.

  The state determination unit 44 determines the state of the driver TS based on the recognition result, and supplies the determination result to the output control unit 45.

  For example, the output control unit 45 creates an audio signal that reads out the determination result and supplies the audio signal to the speaker 37 to output the audio. When notified that the face could not be recognized, the output control unit 45 supplies, for example, text indicating that fact to the display 36 for display.

  Next, a configuration example of the image processing unit 43 will be described.

  The image processing unit 43 includes an image acquisition unit 61, a difference image calculation unit 62, a shadow detection unit 63, and a face image processing unit 64.

  The image acquisition unit 61 acquires the reference image and the illumination image from the monitoring camera 31 and supplies them to the difference image calculation unit 62.

  The difference image calculation unit 62 calculates a difference image between the illumination image and the reference image and supplies the difference image to the shadow detection unit 63 and the face image processing unit 64.

  Here, the advantage which takes the difference of an illumination image and a reference | standard image is demonstrated.

[Example of reference image, illumination image, and difference image]

  FIG. 3 is a diagram illustrating an example of the reference image, the illumination image, and the difference image.

  FIG. 3A is a diagram illustrating an example of a reference image. FIG. 3B is a diagram illustrating an example of an illumination image. FIG. 3C is a diagram illustrating an example of a difference image calculated using the reference image of the example of FIG. 3A and the illumination image of the example of FIG. 3B.

  In the reference image 81, the illumination image 82, and the difference image 83 in the example of FIGS. 3A to 3C, the outline of the subject is indicated by a black line. In these images, a region with a small pixel value is shown as a gray region, and a region with a large pixel value is shown as a white region.

  The reference image 81 in the example of FIG. 3A includes a driver. However, in the reference image 81, the pixel value of the driver's region (hereinafter referred to as the driver region) TR81 is small. Some pixel values of the driver's background area (hereinafter referred to as background area) are large.

  In this example, it is assumed that the state of the sunroof SR is a closed state. In this case, the driver TS is hardly irradiated with infrared light via the sunroof SR. For this reason, in the reference image 81, the pixel value of the driver area TR81 becomes small. Further, the intensity of near infrared light through the side window SW or the like is not weak. For this reason, a part of the background of the driver TS is irradiated with near-infrared light via the side window SW or the like. As a result, in the reference image 81, some pixel values of the background area are increased.

  Therefore, in the reference image 81, the pattern of the driver area TR81 is unclear, and a part of the background pattern remains. For this reason, if the reference image 81 is used as it is, the face recognition rate is lowered.

  The driver 82 is included in the illumination image 82 in the example of FIG. 3B. In the illumination image 82, the pixel value of the face area KR82 in the driver area TR82 is large. The pixel value of the background area of the illumination image 82 substantially matches the pixel value of the background area of the reference image 81.

  In this example, as described above, the driver TS is hardly irradiated with infrared light through the sunroof SR. The face of the driver TS is illuminated with strong near infrared light by the illumination device 32. For this reason, in the illumination image 82, the pixel value of the face area KR82 increases. Further, the background of the driver TS is hardly illuminated by the lighting device 32. Therefore, the pixel value of the background area of the illumination image 82 substantially matches the pixel value of the background area of the reference image 81.

  Therefore, an unnecessary background pattern remains in the illumination image 82. Therefore, when the illumination image 82 is used as it is, the face recognition rate is lowered.

  In the difference image 83 in the example of FIG. 3C calculated using the reference image 81 and the illumination image 82, only the pixel value of the face area KR83 in the driver area TR83 is large.

  In the difference image 83, the pixel value of a region where the difference between the reference image 81 and the illumination image 82 is large is large. For this reason, in the difference image 83, the pixel value of the face region KR83 is increased. Further, in the difference image 83, the pixel value of the region where the difference between the reference image 81 and the illumination image 82 is small is small. For this reason, in the difference image 83, the pixel value of a background area becomes small. As a result, only the pixel value of the face area KR83 is increased.

  Therefore, in the difference image 83, the pattern of the face region TR83 is clear and unnecessary background patterns are removed. For this reason, if the difference image 83 is used, the recognition rate of a face will become high.

  Thus, the recognition rate of the face can be improved by using the difference image.

  In the illumination image 82 of the example of FIG. 3B, the pixel values of the dotted line portions in the face region KR82, that is, the eyebrow edge, the eyelid edge, the black eye edge, the nose outline, and the mouth outline part, It is larger than other parts of KR82. Correspondingly, in the difference image 83 in the example of FIG. 3C, the pixel values of the dotted line portion in the face area KR83, that is, the eyebrow, eyelid edge, black eye edge, nose outline, mouth outline part. Is larger than other parts of the face region KR83.

  The shadow detection unit 63 detects the presence or absence of a shadow in the difference image.

  For example, when the sunroof SR is in the open state, the near-infrared light through the sunroof SR is not weak. For this reason, the driver TS is irradiated with near-infrared light via the sunroof SR, and a certain amount of strong shadow is generated on the face of the driver TS. However, as described above, the illumination device 32 illuminates with a light amount greater than the normal light amount, so that the influence of the shadow is reduced in the difference image. However, the shadow may remain in the difference image without being completely removed. The shadow detection unit 63 detects the presence or absence of such a shadow.

  The face image processing unit 64 performs face image processing for recognizing a face by performing predetermined image processing on the difference image or the illumination image, and supplies the recognition result to the state determination unit 44. The face image processing unit 64 performs face image processing that is not easily affected by the shadow, instead of the face image processing, on the difference image detected as having shadow by the shadow detecting unit 63. This is because when the normal face image processing is performed on the difference image in which the shadow remains in the difference image, the face recognition rate may be lowered. Details of face image processing that is not easily affected by shadows will be described later.

[In-vehicle device processing example]

  Next, an example of processing (hereinafter referred to as monitoring control processing) of the in-vehicle device 35 will be described.

  FIGS. 4 and 5 are flowcharts for explaining an example of the monitoring control process of the in-vehicle device 35.

  In step S <b> 11, the camera illumination control unit 41 determines whether the intensity of external light is equal to or greater than a certain level.

  For example, when the illuminance sensor 33 notifies that the intensity of external light is not greater than a certain level, it is determined as NO in Step S11, and the process proceeds to Step S25. The processing after step S25 will be described later.

  On the other hand, when the illuminance sensor 33 notifies that the intensity of external light is greater than or equal to a certain level, it is determined as YES in Step S11, and the process proceeds to Step S12. In step S12, the camera illumination control unit 41 determines whether or not the sunroof SR is in an open state.

  For example, when the sunroof switch 34 has not been notified that the sunroof SR is in the closed state, it is determined as YES in Step S12, and the process proceeds to Step S18. The processing after step S18 will be described later.

  On the other hand, when the sunroof switch 34 notifies that the closed state of the sunroof SR is detected, it is determined as NO in Step S12, and the process proceeds to Step S13. In step S13, the camera illumination control unit 41 sets the drive pulse width of the drive pulse output unit 42 to the normal width.

  In step S14, the camera illumination control unit 41 sets the illumination light amount of the illumination device 32 to the normal light amount. More specifically, the camera illumination control unit 41 sets, for example, the current value that flows to the LED when the illumination device 32 is illuminated to a normal value. Alternatively, the lighting device 32 includes, for example, a plurality of LEDs, and the camera lighting control unit 41 sets the number of LEDs that emit light when the lighting device 32 is illuminated to a normal number.

  In step S15, the camera illumination control unit 41 instructs the drive pulse output unit 42 to start capturing the reference image and the illumination image.

  The drive pulse output unit 42 executes first and second processing described below in that order based on an instruction from the camera illumination control unit 41.

  First, the drive pulse output unit 42 supplies a drive pulse having a set width (normal width) to the monitoring camera 31 as a first process. The monitoring camera 31 captures an image with an exposure time (normal exposure time) corresponding to the drive pulse width in synchronization with the drive pulse. Since the drive pulse is not supplied to the illumination device 32, the illumination device 32 is not illuminated. Accordingly, a reference image is captured.

  Next, the drive pulse output part 42 supplies the drive pulse of the set width | variety (normal width | variety) to the monitoring camera 31 and the illuminating device 32 as 2nd process. The monitoring camera 31 captures an image with an exposure time (normal exposure time) corresponding to the drive pulse width in synchronization with the drive pulse. The illumination device 32 illuminates with an illumination time (normal illumination time) corresponding to the drive pulse width in synchronization with the drive pulse. For this reason, an illumination image is captured.

  In this way, the reference image and the illumination image are captured. Note that the order of the first process and the second process is not particularly limited. For example, the order of the second process and the first process may be used.

  In the first embodiment and the second embodiment to be described later, the illumination time of the illumination device 32 is varied along with the exposure time of the monitoring camera 31, but, for example, other than the maximum exposure time of the monitoring camera 31, for example. It may be fixed at a predetermined time.

  In step S <b> 16, the image acquisition unit 61 of the image processing unit 43 acquires a captured image (reference image and illumination image) from the monitoring camera 31 and supplies the acquired image to the difference image calculation unit 62.

  In step S <b> 17, the difference image calculation unit 62 calculates a difference image using the illumination image and the reference image and supplies the difference image to the face image processing unit 64. Thereafter, the process proceeds to step S29. The processing after step S29 will be described later.

  By the way, as mentioned above, if it determines with it being YES in step S12, a process will progress to step S18. In step S18, the camera illumination control unit 41 sets the drive pulse width of the drive pulse output unit 42 to a width shorter than the normal width.

  In step S19, the camera illumination control unit 41 sets the illumination light amount of the illumination device 32 to a light amount larger than the normal light amount. More specifically, the camera illumination control unit 41 sets, for example, a current value that flows through the LED during illumination of the illumination device 32 to a value greater than a normal value. Alternatively, for example, the illumination device 32 includes a plurality of LEDs, and the camera illumination control unit 41 sets the number of LEDs that emit light when the illumination device 32 is illuminated to a number greater than the normal number.

  In step S20, the camera illumination control unit 41 instructs the drive pulse output unit 42 to start capturing the reference image and the illumination image.

  The drive pulse output unit 42 executes first and second processing described below in that order based on an instruction from the camera illumination control unit 41.

  First, the drive pulse output unit 42 supplies a drive pulse having a set width (a width shorter than the normal width) to the monitoring camera 31 as a first process. The monitoring camera 31 captures an image with an exposure time (exposure time shorter than normal) according to the drive pulse width in synchronization with the drive pulse. Since the drive pulse is not supplied to the illumination device 32, the illumination device 32 is not illuminated. Accordingly, a reference image is captured.

  Next, the drive pulse output part 42 supplies the monitoring pulse 31 and the illuminating device 32 with the drive pulse of the set width | variety (width shorter than normal width) as 2nd processing. The monitoring camera 31 captures an image with an exposure time (exposure time shorter than normal) according to the drive pulse width in synchronization with the drive pulse. Further, the illumination device 32 illuminates with an illumination time (an illumination time shorter than usual) according to the drive pulse width in synchronization with the drive pulse. For this reason, an illumination image is captured.

  In this way, the reference image and the illumination image are captured. Note that the order of the first process and the second process is not particularly limited. For example, the order of the second process and the first process may be used.

  In step S <b> 21, the image acquisition unit 61 of the image processing unit 43 acquires a captured image (reference image and illumination image) from the monitoring camera 31 and supplies the acquired image to the difference image calculation unit 62.

  In step S <b> 22, the difference image calculation unit 62 calculates a difference image using the illumination image and the reference image, and supplies the difference image to the shadow detection unit 63 and the face image processing unit 64.

  In step S23, the shadow detection unit 63 detects the presence or absence of a shadow in the difference image. In addition, the detection method of the presence or absence of a shadow is not specifically limited. For example, a method of detecting the presence or absence of a shadow from a luminance difference between two predetermined regions in an image (see, for example, JP-A-2007-112300) can be employed. Alternatively, for example, a method of detecting the presence or absence of a shadow based on the position of an object that creates a shadow (see, for example, JP-A-2008-191784) can be employed.

  In step S24, the shadow detection unit 63 determines whether it is detected that there is a shadow.

  When it is detected that there is a shadow, it is determined as YES in Step S24, and the process proceeds to Step S30. The processing after step S30 will be described later.

  On the other hand, when it is detected that there is no shadow, it is determined as NO in Step S24, and the process proceeds to Step S29. The processing after step S29 will be described later.

  By the way, as mentioned above, when it is determined NO in step S11, the process proceeds to step S25. The processes in steps S25 and S26 are the same as the processes in steps S13 and S14, respectively. Therefore, explanation of these processes is omitted.

  In step S27, the camera illumination control unit 41 instructs the drive pulse output unit 42 to start capturing an illumination image.

  The drive pulse output unit 42 supplies a drive pulse having a set width (normal width) to the monitoring camera 31 and the illumination device 32 based on an instruction from the camera illumination control unit 41. The monitoring camera 31 captures an image with an exposure time (normal exposure time) corresponding to the drive pulse width in synchronization with the drive pulse. The illumination device 32 illuminates with an illumination time (normal illumination time) corresponding to the drive pulse width in synchronization with the drive pulse. For this reason, an illumination image is captured. In this way, only the illumination image is captured.

  For example, if the intensity of external light is weak because the weather is cloudy or rainy, the time zone is nighttime, or the vehicle 11 is traveling in a tunnel, the near red color included in the external light The intensity of outside light is also weakening. Therefore, regardless of the open / closed state of the sunroof SR, the driver TS and the background thereof are hardly irradiated with near-infrared light via the sunroof SR. The face of the driver TS is illuminated with strong near infrared light by the illumination device 32. For this reason, in the illumination image, only the pixel value of the face area increases. Therefore, the face can be recognized using the illumination image as it is. Therefore, it is not necessary to capture the reference image, and control processing for that purpose can be omitted.

  In step S <b> 28, the image acquisition unit 61 of the image processing unit 43 acquires a captured image (illumination image) from the monitoring camera 31 and supplies it to the face image processing unit 64. Thereafter, the process proceeds to step S29.

  In step S29, the face image processing unit 64 performs face image processing for recognizing a face by performing predetermined image processing on the difference image or the illumination image. The face image processing method is not particularly limited. However, in the first embodiment and the second embodiment to be described later, the following method is adopted as the processing method of the face image processing. That is, for example, a method of extracting a face area from a captured image and obtaining a face direction, or a technique of obtaining a gaze direction by specifying an eye position from the face area (for example, Japanese Patent Laid-Open No. 2006-259900). reference). Thereafter, the process proceeds to step S31. The processing after step S31 will be described later.

  By the way, as mentioned above, if it determines with YES in step S24, a process will progress to step S30. In step S <b> 30, the face image processing unit 64 performs face image processing that is not easily affected by the shadow on the difference image. Note that the face image processing method that is not easily affected by the shadow is not particularly limited. However, in the first embodiment and the second embodiment, which will be described later, for example, the following method is adopted as a face image processing method that is not easily affected by shadows. That is, a method is employed in which the threshold value for determining the position of the facial organ is varied in accordance with the decrease in luminance caused by the shadow (see, for example, Japanese Patent Laid-Open No. 10-275212). Face image processing that is less susceptible to shading is slightly slower than normal face image processing, but can recognize a face even if there is a shadow in the difference image. For this reason, the face recognition rate can be increased.

  In step S31, the face image processing unit 64 determines whether the face has been recognized.

  For example, when the face direction or the line-of-sight direction as the face recognition result cannot be obtained, it is determined as NO in Step S31, and the process proceeds to Step S32. In step S32, the face image processing unit 64 notifies the output control unit 45 that the face could not be recognized. The output control unit 45 supplies the display 36 with text indicating that the face could not be recognized, for example, “face recognition could not be performed” and displays the text. Thereafter, the process proceeds to step S36. The processing after step S36 will be described later.

  On the other hand, when the face direction or the line-of-sight direction as the face recognition result can be obtained, it is determined as YES in Step S31, and the process proceeds to Step S33. In step S <b> 33, the face image processing unit 64 supplies the face direction and the line-of-sight direction to the state determination unit 44.

  In step S34, the state determination unit 44 determines the state of the driver TS based on the face direction and the line-of-sight direction. For example, the state determination unit 44 determines that the state of the driver TS is a look-ahead state when the face direction is greatly deviated from the front for a predetermined time or more, and otherwise, the state of the driver TS Is determined to be in a normal state. Alternatively, for example, the state determination unit 44 determines that the state of the driver TS is a doze state when the line-of-sight direction does not move for a predetermined time or more. In other cases, the state determination unit 44 sets the state of the driver TS to the normal state. judge. The state determination unit 44 supplies such a determination result to the output control unit 45.

  In step S35, the output control unit 45, for example, in the case of the determination result that the state of the driver TS is a side-by-side state or a dozing state, the texts indicating the determination result “I am doing a side-by-side” or “I do not sleep. Create an audio signal that corresponds to the audio that reads “I am”. The output control unit 45 supplies a sound signal corresponding to the sound to the speaker 37 to output the sound. However, for example, when the determination result indicates that the state of the driver TS is the normal state, the output control unit 45 can also prevent the voice from being output without generating the voice signal.

  In step S36, the camera illumination control unit 41 determines whether or not to end the process.

  For example, when the driver TS operates an operation unit (not shown) and a command to end the process is supplied to the camera illumination control unit 41, it is determined as YES in Step S36, and the monitoring control process ends. .

  On the other hand, unless a process end command is supplied to the camera illumination control unit 41, it is determined as NO in Step S36, and the process returns to Step S11.

  As described above, the monitoring camera system 21 changes the exposure amount of the monitoring camera 31, the illumination light amount of the illumination device 32, and the contents of the face image processing according to the intensity of external light and the open / close state of the sunroof SR. Thereby, when the surveillance camera system 21 is used for a vehicle equipped with a sunroof, the face recognition rate is improved. That is, the surveillance camera system 21 can further improve the face recognition rate.

  Next, a second embodiment will be described.

  In the monitoring camera system 21 described above, as a method for detecting the open / closed state of the sunroof SR, a method for detecting the open / closed state of the sunroof SR with the closed sensor of the sunroof SR is adopted. However, the method for detecting the open / closed state of the sunroof SR is not particularly limited to this method. Therefore, for example, a method of detecting the open / closed state of the sunroof SR by detecting the intensity of near infrared light inside the vehicle 11 with a near infrared light sensor can be employed. A surveillance camera system to which such a detection technique is applied will be described with reference to FIGS.

[Example of surveillance camera system]

  FIG. 6 is a diagram showing an example of a surveillance camera system as a second embodiment of the information processing system to which the present invention is applied.

  In the monitoring camera system 91 of the example of FIG. 6, the same reference numerals are given to the corresponding parts in the monitoring camera system 21 of the example of FIG. 3, and the description thereof will be omitted as appropriate.

  In the monitoring camera system 91 in the example of FIG. 6, a near infrared light sensor 101 is provided instead of the sunroof switch 34 in the monitoring camera system 21 in the example of FIG.

  The near-infrared light sensor 101 is provided, for example, on the dashboard DB.

  When the sunroof SR is closed, the intensity of near-infrared light through the sunroof SR is weakened. For this reason, the intensity of the near infrared light inside the vehicle 11 becomes weak. On the other hand, when the sunroof SR is in the open state, the intensity of near infrared light via the sunroof SR is increased. For this reason, the intensity of near infrared light inside the vehicle 11 is increased. That is, the intensity of near-infrared light inside the vehicle 11 changes according to the open / close state of the sunroof SR. Therefore, the near-infrared light sensor 101 can detect the open / closed state of the sunroof SR by detecting the intensity of near-infrared light inside the vehicle 11.

[Configuration example of surveillance camera system]

  FIG. 7 is a block diagram illustrating a configuration example of the monitoring camera system 91.

  The near infrared light sensor 101 detects the intensity of near infrared light inside the vehicle 11. If the near-infrared light intensity is below a certain level, the near-infrared light sensor 101 determines that the sunroof SR is in a closed state and notifies the in-vehicle device 35 that the sunroof SR is in a closed state. As described above, the near-infrared light sensor 101 can detect the open / closed state of the sunroof SR, like the sunroof switch 34, although the detection method is different.

  As described above, the monitoring camera system 91 in the example of FIG. 6 can monitor the state of the driver TS, similarly to the monitoring camera system 21 in the example of FIG.

  In the first and second embodiments, various settings are changed according to the intensity of external light and the open / closed state of the sunroof SR. In addition, for example, various settings are set according to the irradiation direction of external light with respect to the vehicle 11. Can also be changed. For example, external light around the day is emitted from the direction close to the vertical direction with respect to the vehicle 11. For this reason, the driver TS is irradiated with strong near-infrared light via the sunroof SR. On the other hand, evening outdoor light irradiates the vehicle 11 from a direction close to the horizontal direction. For this reason, the driver TS is hardly irradiated with near-infrared light via the sunroof SR. Therefore, various settings are changed according to the irradiation direction of external light to the vehicle 11. In addition, the irradiation direction with respect to the vehicle 11 of external light is calculated based on the present date, time, etc., for example.

  In the first and second embodiments, the sunroof is employed as the light shielding body. However, for example, a side window, a rear window, an open car roof, and the like can be employed.

  In the first and second embodiments, the surveillance camera system is provided inside the vehicle, but can also be provided inside various vehicles such as trains and ships.

[Hardware configuration example]

  The series of processes described above can be executed by hardware or can be executed by software. When a series of processing is executed by software, a program constituting the software is installed from a program recording medium. This program is installed in, for example, a computer incorporated in dedicated hardware. Alternatively, this program is installed in, for example, a general-purpose personal computer that can execute various functions by installing various programs.

  FIG. 8 is a block diagram illustrating an example of a hardware configuration of a computer that executes the above-described series of processes using a program.

  In the computer, a CPU 201, a ROM (Read Only Memory) 202, and a RAM (Random Access Memory) 203 are connected to each other via a bus 204.

  An input / output interface 205 is further connected to the bus 204. Connected to the input / output interface 205 are an input unit 206 made up of a keyboard, mouse, microphone, etc., an output unit 207 made up of a display, a speaker, etc., and a storage unit 208 made up of a hard disk, a non-volatile memory, or the like. Furthermore, the input / output interface 205 is connected to a communication unit 209 including a network interface and the like, and a drive 210 that drives a removable medium 211 such as a magnetic disk, an optical disk, a magneto-optical disk, or a semiconductor memory.

  In the computer configured as described above, the CPU 201 loads, for example, the program stored in the storage unit 208 to the RAM 203 via the input / output interface 205 and the bus 204 and executes the program. Is performed.

  The program executed by the computer (CPU 201) is provided by being recorded on the removable medium 211 that is a magnetic disk (including a flexible disk), for example. The program is provided by being recorded on a removable medium 211 that is a package medium. As the package medium, an optical disc (CD-ROM (Compact Disc-Read Only Memory), DVD (Digital Versatile Disc), etc.), a magneto-optical disc, or a semiconductor memory is used. Alternatively, the program is provided via a wired or wireless transmission medium such as a local area network, the Internet, or digital satellite broadcasting.

  The program can be installed in the storage unit 208 via the input / output interface 205 by attaching the removable medium 211 to the drive 210. The program can be received by the communication unit 209 via a wired or wireless transmission medium and installed in the storage unit 208. In addition, the program can be installed in the ROM 202 or the storage unit 208 in advance.

  The program executed by the computer may be a program that is processed in time series in the order described in this specification, or in parallel or at a necessary timing such as when a call is made. It may be a program for processing.

  Embodiments of the present invention are not limited to the above-described embodiments, and various modifications can be made without departing from the scope of the present invention.

  DESCRIPTION OF SYMBOLS 11 Vehicle, 21 Surveillance camera system, 31 Surveillance camera, 32 Illumination device, 33 Illuminance sensor, 34 Sunroof switch, 35 In-vehicle device, 36 Display, 37 Speaker, 41 Camera illumination control part, 42 Drive pulse output part, 43 Image processing part , 44 state determination unit, 45 output control unit, 61 image acquisition unit, 62 difference image calculation unit, 63 shadow detection unit, 64 face image processing unit, 91 surveillance camera system, 101 near infrared light sensor, 201 CPU, 202 ROM , 203 RAM, 204 bus, 205 input / output interface, 206 input unit, 207 output unit, 208 storage unit, 209 communication unit, 210 drive, 211 removable media

Claims (7)

As a state of the light blocking body having a function of blocking external light directly irradiated toward the inside of the vehicle, the first state where the external light is blocked or the second state where the external light is not blocked Detecting means for detecting;
Control means for controlling illumination inside the vehicle by the illumination unit and imaging inside the vehicle by the imaging unit;
Recognizing means for recognizing a human face existing inside the vehicle based on a first image obtained as a result of being imaged by the imaging unit under the control of the control means,
The control means includes
When the first state is detected by the detection unit, the first control is performed to cause the illumination unit to illuminate with a first light amount, and to cause the imaging unit to capture an image with a first exposure amount,
When the second state is detected by the detection unit, the illumination unit is illuminated with a second light amount greater than the first light amount, and the imaging unit is exposed to a second exposure less than the first exposure amount. An information processing apparatus that performs second control for capturing an image with a quantity. The control means includes
In the case where the first state is detected by the detection means, in addition to the first control, the illumination unit is further turned off, and the third control for causing the imaging unit to capture an image with the first exposure amount. And
In the case where the second state is detected by the detection means, in addition to the second control, the illumination control unit is further turned off, and a fourth control is performed to cause the imaging unit to capture an image with the second exposure amount. And
The recognition means is
An image obtained as a result of imaging by the imaging unit under the first control or the second control is used as the first image, and the imaging unit under the third control or the fourth control. Calculating means for calculating a difference image between the first image and the second image, using the image obtained as a result of imaging by the second image as a second image;
The information processing apparatus according to claim 1, further comprising: a face image processing unit that recognizes the face by performing a predetermined process on the difference image calculated by the calculation unit. The recognition means is
A shadow detection means for detecting the presence or absence of a shadow in the difference image calculated by the calculation means;
The face image processing means includes
When it is detected by the shadow detection means that there is no shadow, as the predetermined process, a first process is performed on the difference image,
The information processing apparatus according to claim 2, wherein when the shadow detection unit detects that there is a shadow, the second process different from the first process is performed on the difference image as the predetermined process. The light blocking body blocks near infrared light,
The information processing apparatus according to claim 1, wherein the detection unit detects the first state or the second state based on intensity of near-infrared light inside the vehicle. A light detecting means for detecting the intensity of light outside the vehicle;
The control means includes
When the intensity of the light detected by the light detection means exceeds a threshold value, the first control or the second control is performed,
2. The information processing according to claim 1, wherein when the intensity of the light detected by the light detection unit is equal to or less than a threshold value, the first control is performed regardless of a state of the light shielding body detected by the detection unit. apparatus. As steps executed by the information processing apparatus,
As a state of the light blocking body having a function of blocking external light directly irradiated toward the inside of the vehicle, the first state where the external light is blocked or the second state where the external light is not blocked A detection step to detect;
A control step for controlling illumination inside the vehicle by the illumination unit and imaging inside the vehicle by the imaging unit;
Recognizing a human face existing inside the vehicle based on a first image obtained as a result of being imaged by the imaging unit under the control of the control step, and
The control step includes
When the first state is detected by the processing of the detection step, the first control is performed to cause the illumination unit to illuminate with a first light amount, and to cause the imaging unit to capture an image with a first exposure amount,
When the second state is detected by the processing of the detection step, the illumination unit is illuminated with a second light amount greater than the first light amount, and the imaging unit is second less than the first exposure amount. The information processing method including the step which performs the 2nd control made to image with the exposure amount of. In the computer that controls the information processing device,
As a state of the light blocking body having a function of blocking external light directly irradiated toward the inside of the vehicle, the first state where the external light is blocked or the second state where the external light is not blocked A detection step to detect;
A control step for controlling illumination inside the vehicle by the illumination unit and imaging inside the vehicle by the imaging unit;
Recognizing a human face existing inside the vehicle based on a first image obtained as a result of being imaged by the imaging unit under the control of the control step; and
In the computer, as the control step,
When the first state is detected by the processing of the detection step, the first control is performed to cause the illumination unit to illuminate with a first light amount, and to cause the imaging unit to capture an image with a first exposure amount,
When the second state is detected by the processing of the detection step, the illumination unit is illuminated with a second light amount greater than the first light amount, and the imaging unit is second less than the first exposure amount. The program which performs the step which performs the 2nd control which images with the exposure amount of.

Images