JP2017168960A - Image processing apparatus - Google Patents

Abstract

PROBLEM TO BE SOLVED: To provide an image processing apparatus that does not have a movable part and picks up images in two different directions without dividing a region of an imaging element so as to identify an object from the picked up images.SOLUTION: An image processing apparatus 100 comprises: a first projection part T1 which projects light of a polarization component in a first direction on a first region A1; a second projection part T2 which projects light of a polarization component in a second direction orthogonal to the first direction on a second region A2; a first optical path P1 which guides the light from the first direction; a second optical path P2 which guides the light from the second direction; a luminous flux splitter LD which transmits the polarization component in the first direction and reflects the polarization component in the second direction; a third optical path P3 which guides reflected light of the light guided to the first optical path and reflected light of the light guided to the second optical path; a lens 21 which is arranged on the third optical path, and converges the reflected light and transmitted light; an imaging element 20 which receives light from the lens; and an image processing part which performs predetermined image processing on the basis of a signal of the imaging element.SELECTED DRAWING: Figure 4

Description

  The present invention relates to an image processing apparatus, and more particularly to a face image processing apparatus that processes a face image based on a plurality of face images obtained by imaging a human face.

  2. Description of the Related Art Conventionally, a technique for capturing images in a plurality of directions with a single imaging device is known. For example, Patent Literature 1 discloses a vehicle side monitoring device for the purpose of a driver to easily obtain vehicle side traffic information. The vehicle side monitoring device includes a reflection member that captures the left and right sides of the vehicle at the vehicle end, and one imaging camera that images the left and right sides of the vehicle via the reflection member, and an image from the imaging camera. An image of the side of the vehicle is displayed on the display means provided in the passenger compartment by the signal. Thereby, light from the right side is incident on the right half of the imaging camera, and light from the left side is incident on the left half, and the left and right directions of the vehicle can be imaged with one imaging camera.

  Further, Patent Document 2 discloses a vehicular visual aid device that can visually recognize the situation in the front and left and right directions of a vehicle body with a single fixed TV camera. This vehicular visual aid device is incident on the front end of the vehicle body, a first mode in which light incident from the front is transmitted in front of the camera and directed toward the camera as it is, and incident from both the left and right directions. And an optical device mechanically switchable to a second mode in which light is reflected and directed to the camera at the same time. At low speeds of the vehicle, such as when entering an intersection with poor visibility, the optical device is switched to the second mode. Therefore, the left and right images are simultaneously captured by the camera and displayed on the monitor television.

  Patent Document 3 discloses an imaging device that realizes photographing of a stereo image by hand without dividing one imaging element. This imaging apparatus includes two incident optical systems. The first incident optical system is disposed on one of the two optical paths, has a shutter, opens the shutter for a predetermined time, and exposes and images the first subject light on the image sensor. The second incident optical system is arranged on an optical path different from that of the first incident optical system, has a shutter, opens the shutter for a predetermined time, and exposes the second object light by the first incident optical system. An image is exposed and imaged on the image sensor at the timing when the charges on the imaged image sensor are transferred to the transfer path of the image sensor and the first incident optical system is shielded from light.

  Further, Patent Document 4 discloses a front monitor camera device for obtaining images on both the left and right sides in the traveling direction of the vehicle by acquiring images on both the left and right sides in the traveling direction of the vehicle by using the entire image pickup device. And the vehicle right-and-left both-sides confirmation apparatus is disclosed. In this device, when the left and right sides are photographed by the front monitor camera device, the light transmitted through the first lens is only the light of the S polarization component from the reflection mirror and the polarization beam splitter through the polarizer region A of the region dividing filter. Light is received by the image sensor. Further, only the light of the P-polarized component is received by the image sensor through the polarizing beam splitter and the polarizer region B of the region dividing filter. When the S- and P-polarized light components are received by the imaging device, the entire imaging device can be used as an imaging range by arranging a plurality of polarizer regions A and polarizer regions B alternately. Can be secured.

Japanese Utility Model Publication No. 01-109447 Japanese Patent Application Laid-Open No. 06-278531 JP-A-11-285026 JP 2010-254085 A

As described above, in order to reduce costs and reduce the size, a technique for capturing images in a plurality of directions outside the vehicle with a single imaging device is known. Sitting persons may also need to be monitored.
However, in the prior arts of Patent Documents 1 and 4, since each direction is imaged using half of the image sensor, there is a problem that the image is coarse when the object is to be identified. Moreover, in the prior arts of Patent Documents 2 and 3, movable parts such as a reflecting mirror and a shutter are required, and there are problems in cost and life.
Therefore, the present invention provides an image processing apparatus that has no movable part, images two different directions without dividing the area of the image sensor, and identifies an object from the captured image.

In order to solve the above problem, a first light projecting unit that projects light having a polarized light component in a first direction onto a first region, and a light having a polarization component in a second direction perpendicular to the first direction is directed to a second region. A second light projecting unit for projecting light, a first optical path for guiding light from the direction of the first region, a second optical path for guiding light from the direction of the second region, and a first surface facing the first optical path. The light beam splitter is disposed so that the second surface faces the second optical path, transmits the polarization component in the first direction, and reflects the polarization component in the second direction, and the light guided to the first optical path is A third optical path for guiding reflected light reflected by one surface and transmitted light transmitted from the second surface to the first surface by light guided to the second optical path, and disposed on the third optical path, and reflected light and transmitted light. An image processing apparatus comprising: a lens that collects light; an image sensor that receives light from the lens; and an image processing unit that performs predetermined image processing based on a signal from the image sensor. It is.
According to this, light having orthogonal polarization components is imaged from two different directions via a light beam splitter, which is a polarizer, so that there is no movable part, and the two different without dividing the region of the image sensor. It is possible to provide an image processing apparatus that captures a direction and identifies an object from the captured image. Further, when an image of a person wearing spectacles is taken, the effect of specular reflection of the spectacles can be reduced, which is effective for eye identification and eye condition determination.

And a controller that controls the turning off / lighting of the first light projecting unit and the second light projecting unit, imaging of the image sensor, and image processing of the image processing unit, and a storage unit that stores an image captured by the image sensor, The control unit turns off the first light projecting unit and the second light projecting unit, images the first region and the second region as a first image by the image sensor, and stores the first image in the storage unit. One of the first light projecting unit and the second light projecting unit is turned on, the other is turned off, the first area and the second area are imaged as a second image by the image sensor, and the second image is stored in the storage unit. Then, one of the first light projecting unit and the second light projecting unit is turned off, the other is turned on, and the first area and the second area are imaged as the third image by the image sensor and the third image is stored in the storage unit. When the first image, the second image, and the third image are stored in the storage unit, the image processing unit is instructed to perform image processing. In response to the image processing instruction, the image processing unit generates a difference image between the first image and the second image as the first difference image, and generates a difference image between the first image and the third image as the second difference image. It may be characterized by.
According to this, by generating the difference image, the influence of the reflected light common to the original image is offset, and the image is generated based on the reflected light of the light projected by the light projecting unit. Therefore, it is possible to obtain a clear image based on the reflected light from a necessary area among the reflected lights from two different areas.

  According to the present invention, it is possible to provide an image processing apparatus that has no movable part, images two different directions without dividing the area of the image sensor, and identifies an object from the captured image.

1 is an external view of a face image processing apparatus according to a first embodiment of the present invention. BRIEF DESCRIPTION OF THE DRAWINGS When the face image processing apparatus of 1st Example which concerns on this invention is provided in a vehicle, (A) Explanatory drawing when it sees from a side, (B) Explanatory drawing when it sees from upper direction. 1 is a perspective view showing an internal structure of an optical system of a face image processing apparatus according to a first embodiment of the present invention. The figure which shows the internal structure of the optical system of the face image processing apparatus of 1st Example which concerns on this invention (when seen from the top). 1 is a block diagram of a face image processing apparatus according to a first embodiment of the present invention. The timing chart of the imaging in the face image processing apparatus of 1st Example which concerns on this invention. The flowchart which shows the whole operation | movement in the face image processing apparatus of 1st Example which concerns on this invention. Explanatory drawing explaining taking a 1st image in the face image processing apparatus of 1st Example which concerns on this invention, when both light projection parts are light-extinguished. Explanatory drawing explaining taking a 2nd image, when the 1st light projection part is lighted and the 2nd light projection part is light-extinguished in the face image processing apparatus of the 1st Example which concerns on this invention. Explanatory drawing explaining taking a 3rd image in the face image processing apparatus of 1st Example which concerns on this invention, when the 1st light projection part is light-extinguished and the 2nd light projection part is lighted. Explanatory drawing explaining the image processing of the 1st image imaged with the face image processing apparatus of the 1st Example which concerns on this invention. Explanatory drawing explaining the image processing of the 2nd image imaged with the face image processing apparatus of the 1st Example which concerns on this invention. Explanatory drawing explaining the difference image of the 1st image and 2nd image in the face image processing apparatus of 1st Example which concerns on this invention. FIG. 3A is a case where a person wearing glasses is imaged by the face image processing apparatus according to the first embodiment of the present invention, and (A) an image when polarization is not performed, and (B) an image when polarization is performed.

Embodiments according to the present invention will be described below with reference to the drawings.
<First Example>
A face image processing apparatus 100 according to the present embodiment will be described with reference to FIGS. The face image processing device 100 detects the driving state of the driver DR and the state of the person sitting in the passenger seat by imaging the face of the driver DR sitting in the driver seat and the passenger seat behind the steering wheel WL of the vehicle C. To do. Therefore, the face image processing apparatus 100 is provided so as to irradiate light from the front of the driver DR or the like toward the upper body centered on the face and take an image.

  The face image processing apparatus 100 is arranged at the front center (for example, a position on the rearview mirror or the instrument panel) of the driver DR and the person sitting in the front passenger seat so that both can be photographed, and the driver seat (first area A1) and the assistant A first light projecting unit T1 and a second light projecting unit T2 that project light toward the seat (second region A2), and a first light path entrance O1 and a first light projecting unit O1 that allow reflected light from the respective regions to enter the interior. And a two-light path entrance O2.

  With reference to FIGS. 3 to 5, components of the face image processing apparatus 100 in the present embodiment will be described. The face image processing apparatus 100 includes a first light projecting unit T1 and a second light projecting unit T2 that project light onto the first region A1 and the second region A2. The first projector T1 includes a first infrared emitter T1L and a first polarizer T1F, and the second projector T2 includes a second infrared emitter T2L and a second polarizer T2F. The first infrared light emitter T1L and the second infrared light emitter T2L are light emitting diodes that emit infrared light. Infrared rays are used because night vision can be performed not only in the daytime but also in the dark at night. However, light having a predetermined wavelength may be used, and is not particularly limited to infrared rays.

  The first polarizer T1F of the first light projecting unit T1 is disposed in front of the first infrared light emitter T1L, polarizes the infrared light emitted by the first infrared light emitter T1L in the first direction, and the second light projecting unit T2. The second polarizer T2F is disposed in front of the second infrared light emitter T2L and polarizes the infrared light emitted by the second infrared light emitter T2L in the second direction. That is, the first light projecting unit T1 and the second light projecting unit T2 are polarized in the respective directions when projecting infrared light, and project the infrared light into the first region A1 and the second region A2. The polarizer is typically a polarizing plate having a polarization plane in a predetermined direction and transmitting only light in a predetermined direction from natural light. In the present embodiment, the first polarizer T1F performs polarization in the first direction, and the second polarizer T2F performs polarization in the second direction orthogonal to the first direction. For convenience of explanation, it is assumed that the first polarizer T1F transmits a light component in the horizontal direction and the second polarizer T2F transmits a light component in the vertical direction, but is not limited thereto. .

  The face image processing apparatus 100 includes a first light projecting unit T1 that projects the infrared light having the horizontal polarization component to the first region A1, and an infrared light having the vertical polarization component to the second region A2. In addition to the second light projecting portion T2 that projects light, the following components for receiving light behind the first optical path incident port O1 and the second optical path incident port O2 that allow light from the respective regions to enter the inside are included. Prepare. That is, the face image processing apparatus 100 includes a first optical path P1 that guides light from the direction of the first area A1, a second optical path P2 that guides light from the direction of the second area A2, a light beam splitter LD, 3 optical path P3, the lens 21, and the image pick-up element 20 are provided.

  The first optical path P1 is an optical path that guides the reflected light of the infrared light and the reflected light of the natural light projected from the direction of the first region A1 to the inside of the face image processing apparatus 100. . Similarly, the second optical path P2 guides the reflected light of the infrared light and the reflected light of the natural light projected by the second light projecting unit T2 from the direction of the second region A2 to the inside of the face image processing apparatus 100. It is an optical path. The light guided by the first optical path P1 and the second optical path P2 reaches the light beam splitter LD. In the first optical path P1 and the second optical path P2, the first visible light cut filter O1F and the second visible light that block visible light are preferably located at the positions of the first optical path entrance O1 and the second optical path entrance O2. A cut filter O2F may be provided. The infrared light projected by the first light projecting unit T1 and the second light projecting unit T2 can be received more clearly.

  The beam splitter LD is arranged such that the first surface LD1 faces the first optical path P1 and the second surface LD2 faces the second optical path P2. The light beam splitter LD is a polarizer, and transmits the polarization component in the first direction (horizontal direction) and reflects the polarization component in the second direction (vertical direction). In other words, the light beam splitter LD transmits the light of the polarization component in the horizontal direction and reflects the light of the polarization component in the vertical direction among the light guided and arrived at the first optical path P1 and the second optical path P2. A part of the light transmitted through the light beam splitter LD and the light reflected by the light beam splitter LD is configured to reach the lens 21 and the imaging device 20 via the third optical path P3.

  In the present embodiment, the third optical path P3 is the reflected light reflected by the first surface LD1 of the beam splitter LD and the light guided to the second optical path P2 is the second surface. The light transmitted from the LD 2 to the first surface LD 1 is guided. That is, the third optical path P3 is arranged on the first optical path P1 side with respect to the light shielding wall 80 arranged in the center, and the light incident from the first optical path entrance O1 is the first surface LD1 of the light beam splitter LD. It is provided in a direction in which it travels by reflection, and in a direction in which light incident from the second optical path entrance O2 is transmitted through the second surface LD2 of the light beam splitter LD to the first surface LD1.

  The reflected light incident on the third optical path P3 and reflected from the first surface LD1 from the first optical path P1 and the transmitted light transmitted from the second surface LD2 to the first surface LD1 from the second optical path P2 are transmitted through the third optical path. The lens 21 is guided by P3. The lens 21 is disposed on the third optical path P3, collects the reflected light and the transmitted light, and causes the single image sensor 20 to receive the light. The image pickup device 20 is a CCD (Charge-Coupled Device) camera that can pick up visible light and infrared rays, but is not limited to this.

  The imaging device 20 uses the reflected light of the infrared light and the reflected light of the horizontal component projected by the first light projecting unit T1 as the light component excluding the horizontal component reflected by the first surface LD1 of the light beam splitter LD. Receive light. At the same time, the imaging device 20 transmits the reflected light of the infrared light of the vertical component and the reflected light of the natural light projected by the second light projecting unit T2 from the second surface LD2 of the light beam splitter LD to the first surface LD1. Light is received as the light component of the transmitted horizontal component. According to the face image processing apparatus 100, the light having orthogonal polarization components is imaged from two different directions via the light beam splitter LD, which is a polarizer, so that there is no moving part, and the region of the image sensor can be obtained. It is possible to provide an image processing apparatus that images two different directions without being divided and identifies an object from the captured images. As will be described later, when a driver DR wearing glasses is imaged, the influence of specular reflection of glasses can be reduced, which is effective for eye identification and eye condition determination.

  The face image processing apparatus 100 further includes a storage unit 50 that stores an image captured by the image sensor 20 and image processing that performs predetermined image processing to detect a driver DR and the like based on the image stored by the storage unit 50. Unit 30, the external output unit 60 for outputting the result processed by the image processing unit 30 to the external mechanism, turning off / lighting of the first light projecting unit T1 and the second light projecting unit T2, and the imaging device 20 A control unit 40 that controls imaging and image processing of the image processing unit 30 and a power supply unit 70 that receives an input from an ignition switch are provided. The storage unit 50 is, for example, a semiconductor memory, and the external output unit 60 and the control unit 40 are a microcomputer and control logic processed by the microcomputer.

  Note that the image processing in the image processing unit 30 is performed on the image sensor 20 based on the signal, and the image processing may be performed without using the storage unit 50, or the image sensor once stored in the storage unit 50. Image processing may be performed based on the 20 signals. Note that the predetermined image processing includes generating a differential image of a plurality of images captured as described later, and further detecting the presence or absence of a driver DR or the like based on the captured images or the generated images. This includes detection of the state of the driver DR and the like. The power supply unit 70 is configured such that when the ignition switch is turned on, power is supplied to each component and operation can be started.

  The timing at which the face image processing apparatus 100 captures an image will be described with reference to FIG. The control unit 40 of the face image processing apparatus 100 projects a first light projecting unit T1 that projects light to the first region A1 (driver's seat side) and a second light projecting unit T2 that projects light to the second region A2 (passenger seat side). The first image is acquired by causing the image sensor 20 to capture the face of the driver DR and the passenger in the front passenger seat, and stored in the storage unit 50. In addition, the control unit 40 causes the first infrared light emitter T1L of the first light projecting unit T1 that projects light to the first area A1 to emit infrared light, and projects the face of the driver DR in the first area A1. The second image is acquired by causing the image sensor 20 to capture the face of the driver DR and the passenger in the front passenger seat, and stored in the storage unit 50. Then, the control unit 40 subsequently stops the light emission of the first infrared light emitter T1L of the first light projecting unit T1, and then projects the second infrared light emitter of the second light projecting unit T2 that projects the second region A2. While T2L emits infrared light and projects the face of the passenger seat in the second area A2, the image sensor 20 causes the driver DR and the passenger seat face to capture the third image. Is stored in the storage unit 50. After storing the first image, the second image, and the third image in the storage unit 50, the image processing unit 30 processes the image based on the three images.

  The operation in the face image processing apparatus 100 will be described with reference to FIG. In the flowchart, S means a step. In S100, the control unit 40 of the face image processing apparatus 100 irradiates the first infrared light emitter T1L that irradiates the first area A1 (driver's seat side) and the second infrared light emission that irradiates the second area A2 (passenger seat side). The device T2L is turned off. In step S102, the control unit 40 causes the image sensor 20 to capture an image using only natural light while both the light emitters are turned off. In S104, the control unit 40 causes the storage unit 50 to store the image captured by the image sensor 20 as the first image.

  In S106, the control unit 40 turns on the first infrared light emitter T1L and projects it onto the first area A1. In S108, the control unit 40 causes the image sensor 20 to capture an image while the first infrared emitter T1L is turned on and the second infrared emitter T2L is turned off. In S110, the control unit 40 turns off the first infrared light emitter T1L. In S112, the control unit 40 causes the storage unit 50 to store the image captured by the imaging element 20 in S108 as the second image.

  In S114, the controller 40 turns on the second infrared light emitter T2L and projects the light onto the second area A2. In S116, the control unit 40 causes the image sensor 20 to capture an image while the first infrared light emitter T1L is turned off and the second infrared light emitter T2L is turned on. In S118, the control unit 40 turns off the second infrared light emitter T2L. In S120, the control unit 40 causes the storage unit 50 to store the image captured by the image sensor 20 in S116 as a third image.

  When the three images are stored in the storage unit 50, the image processing unit 30 generates a difference image (first difference image) between the first image and the second image in S122. Further, in S124, the image processing unit 30 generates a difference image (second difference image) between the first image and the third image. The difference image is obtained because the image portion obtained by the reflected light by the natural light existing even when irradiating the light common to the first image to the third image, that is, infrared rays can be offset, and a fine image can be obtained. Is preferable. Therefore, the generation of the difference image in this step is selective.

  In S126, the image processing unit 30 uses the second difference image to detect and determine whether there is a passenger seat situation, for example, whether there is a person in the passenger seat. Since the second difference image is a difference image between the first image with only natural light and the third image captured while the second infrared light emitter T2L on the passenger seat side is lit, the situation of the passenger seat is clear. Therefore, the image processing unit 30 detects the situation of the passenger seat using the second difference image. The external output unit 60 outputs the situation of the passenger seat determined by the image processing unit 30 in S128. Note that the determination using the second difference image may be performed only once after the ignition switch is turned on. When the ignition at the start of running is turned on, there is a case where it is only necessary to monitor whether a passenger is on the passenger seat. For example, in a vehicle equipped with a seat belt warning system, even if the passenger seat belt is not worn, the warning can be stopped if no person is sitting. In addition, for example, if an adult or a child can be identified, it can be used for adjusting the explosive force when the airbag is deployed, and can be used for adjusting the air direction of the air conditioner.

  In S130, the image processing unit 30 uses the first difference image to detect and determine the situation of the driver's seat, for example, the situation of the driver DR. The first difference image is a difference image between the first image with only natural light and the second image captured while the first infrared light emitter T1L on the driver's seat is turned on, so the driver in the driver's seat Since the DR situation appears clearly, the image processing unit 30 detects the situation of the driver DR using the first difference image.

  In S132, the control unit 40 checks whether or not the driver DR is in the same state for a predetermined time by comparing with the past first difference image. For example, if the eye is kept closed for a long time, it is determined that the patient is dozing. Alternatively, it is determined that the patient is dozing even when the face continues to face downward. When the driver DR is not in the same situation for a predetermined time, the control unit 40 waits for the suspension time T to elapse in S134 and performs S100 to S130 again. When the driver DR is in the same situation for a predetermined time, the control unit 40 outputs the situation of the driver DR determined by the image processing unit 30 to the external output unit 60 in S136.

  As described above, the control unit 40 turns off the first light projecting unit T1 and the second light projecting unit T2, and sets the first region A1 (driver's seat side) and the second region A2 (passenger seat side) to the image sensor 20. Thus, the first image is captured and the first image is stored in the storage unit 50. In addition, the control unit 40 turns on one of the first light projecting unit T1 and the second light projecting unit T2, turns off the other, and makes the first region A1 and the second region A2 as a second image by the imaging device 20. The second image is stored in the storage unit 50 while taking an image. In addition, the control unit 40 turns off one of the first light projecting unit T1 and the second light projecting unit T2, turns on the other, and sets the first region A1 and the second region A2 by the image sensor 20. The third image is stored in the storage unit 50 while being captured as three images.

  Then, when the first image, the second image, and the third image are stored in the storage unit 50, the control unit 40 instructs the image processing unit 30 to perform image processing. Upon receiving this image processing instruction, the image processing unit 30 generates a difference image between the first image and the second image as the first difference image, and generates a difference image between the first image and the third image as the second difference image. To do. By generating the difference image in this way, the influence of the reflected natural light common to the original image is offset, and the image is generated based on the reflected infrared light projected by the light projecting unit. Therefore, it is possible to obtain a clear image based on the reflected light from a necessary area among the reflected lights from two different areas.

  The imaging of the first image to the third image will be specifically described with reference to FIGS. FIG. 8 shows imaging of the first image. At the time of capturing the first image, the first infrared light emitter T1L and the second infrared light emitter T2L are turned off, and the image sensor 20 is connected to the driver DR in which natural light from the outside of the vehicle C is in the first region A1. An image is picked up based on the light irregularly reflected upon the skin of the face of the passenger seat in the second area A2. The light (dotted line) that is diffusely reflected upon hitting the driver DR in the first region A1 hits the first surface LD1 of the light beam splitter LD, and only the light component excluding the horizontal component is reflected and reaches the image sensor 20, and the horizontal component The light component is transmitted from the first surface LD1 to the second surface LD2.

  In addition, the light (dash-and-dash line) diffused and reflected by the passenger seat in the second area A2 hits the second surface LD2 of the light beam splitter LD, the light component except the horizontal component is reflected, and only the light component of the horizontal component is reflected. The light passes from the second surface LD2 to the first surface LD1 and reaches the image sensor 20. In this way, the image sensor 20 removes the light component excluding the horizontal component of the light in which natural light is irregularly reflected by the driver DR and the light component of the horizontal component in the light in which natural light is irregularly reflected by the passenger. Receive light and image.

  FIG. 9 shows the imaging of the second image. At the time of capturing the second image, the first infrared light emitter T1L is turned on, the second infrared light emitter T2L is turned off, and the image sensor 20 is a driver whose natural light from the outside of the vehicle C is in the first area A1. The light reflected irregularly on the face and the like of the face of the passenger seat in the DR and the second area A2, and the infrared light of the horizontal component projected by the first infrared light emitter T1L hits the skin etc. of the face of the driver DR and diffused. An image is taken based on the light. The light (dotted line) diffusely reflected upon the skin of the driver DR hits the first surface LD1 of the light beam splitter LD, and only the light component excluding the horizontal component is reflected and reaches the imaging device 20, and the light component of the horizontal component is The light passes from the first surface LD1 to the second surface LD2.

  In addition, the light (dash-dotted line) that is diffusely reflected upon the skin of the passenger seat hits the second surface LD2 of the light beam splitter LD, the light component excluding the horizontal component is reflected, and only the light component of the horizontal component is reflected on the second surface LD2. To the first surface LD1 to reach the image sensor 20. In this manner, the image sensor 20 has natural light and a light component excluding the horizontal component of the light that the infrared light projected by the first infrared light emitter T1L is irregularly reflected on the skin of the driver DR, and the natural light is the passenger seat. The light component of the horizontal component of the light irregularly reflected by the person's skin is received and imaged.

  When the driver DR is wearing glasses, the reflected light that is specularly reflected when the light of the horizontal component that has passed through the first polarizer T1F hits the glasses is different from the reflected light of the skin, and mainly the light of the horizontal component. Become. Then, the reflected light that is specularly reflected by the spectacles is transmitted from the first surface LD1 to the second surface LD2 and is not reflected by the first surface LD1, and thus does not reach the image sensor 20. Usually, if a portion that is specularly reflected by being projected onto spectacles without being polarized is imaged, whiteout occurs with high brightness, and it becomes particularly difficult to detect the eye state (FIG. 14A). However, as in the present embodiment, when imaging is performed through the light beam splitter LD that projects the light component of the horizontal component and reflects the light component excluding the horizontal component, the reflected light that causes overexposure is divided into the light beams. Since it passes through the detector LD and does not reach the image sensor 20, it is possible to capture a clear image without whiteout as shown in FIG. Thereby, even if the driver DR is wearing glasses, the state of the eyes can be detected.

  FIG. 10 shows the imaging of the third image. When the third image is captured, the first infrared light emitter T1L is turned off, the second infrared light emitter T2L is turned on, and the image sensor 20 is a driver in which natural light from the outside of the vehicle C is in the first area A1. DR and the light reflected diffusely on the face of the passenger in the second area A2 and the infrared light of the vertical component projected by the second infrared light emitter T2L hit the skin on the face of the passenger. Imaging is performed based on the irregularly reflected light. The light (dotted line) diffusely reflected upon the skin of the driver DR hits the first surface LD1 of the light beam splitter LD, and only the light component excluding the horizontal component is reflected and reaches the imaging device 20, and the light component of the horizontal component is The light passes from the first surface LD1 to the second surface LD2.

  In addition, the light (dash-dotted line) that is diffusely reflected upon the skin of the passenger seat hits the second surface LD2 of the light beam splitter LD, the light component excluding the horizontal component is reflected, and only the light component of the horizontal component is reflected on the second surface LD2. To the first surface LD1 to reach the image sensor 20. In this manner, the image sensor 20 is configured to receive the light component excluding the horizontal component of the light in which the natural light is diffusely reflected on the skin of the driver DR, and the infrared light projected by the natural light and the second infrared light emitter T2L. The light component of the horizontal component of the light irregularly reflected by the person's skin is received and imaged.

  Further, when the passenger seats wearing glasses, the reflected light that is specularly reflected when the light of the vertical component that has passed through the second polarizer T2F hits the glasses is different from the reflected light of the skin. It becomes. Then, the reflected light that is specularly reflected by the spectacles is reflected by the second surface LD2 and does not pass from the second surface LD2 to the first surface LD1, and therefore does not reach the image sensor 20. In the same manner as described above, when imaging is performed through the light beam splitter LD that projects the light component of the vertical component and transmits the horizontal component, reflected light that causes whiteout is reflected on the second surface LD2 of the light beam splitter LD. Since it is reflected and does not reach the image sensor 20, it is possible to capture a clear image without whiteout.

  Image processing of the first image to the third image will be described with reference to FIGS. This figure is an example in which the driver DR sits in the driver's seat and no one is in the passenger seat. That is, the seat is imaged on the passenger seat side. FIG. 11 shows image processing of the first image. FIG. 11A is an image on the driver's seat side of the first region A1 captured based on the light component except the horizontal component reflected by the image sensor 20 on the first surface LD1 of the light beam splitter LD. FIG. 11B is an image on the passenger seat side of the second region A2 that is imaged based on the horizontal light component transmitted from the second surface LD2 of the light beam splitter LD to the first surface LD1. . Since all the images are natural light, the overall image is low in luminance. Since the image sensor 20 receives the two images simultaneously, the image sensor 20 captures an image in which the two images overlap as shown in FIG.

  FIG. 12 shows image processing of the second image. FIG. 12A shows a first region in which the imaging device 20 captures an image based on a light component excluding a horizontal component reflected by the first surface LD1 of the light beam splitter LD in a state where the first light projecting unit T1 projects. It is an image on the driver's seat side of A1. FIG. 12B illustrates only the natural light, and the passenger side of the second region A2 that is imaged based on the horizontal light component transmitted from the second surface LD2 of the light beam splitter LD to the first surface LD1. It is an image. The image in FIG. 12A is an image with high brightness as a whole because the first light projecting unit T1 projects the light. Since the image sensor 20 receives the two images simultaneously, the image sensor 20 captures an image in which the two images overlap as shown in FIG. Compared to FIG. 11C, FIG. 12C has a higher overall brightness because the first light projecting unit T <b> 1 is imaging.

  FIG. 13 shows the generation of the first difference image. FIG. 13 (A) shows the first image shown in FIG. 11 (C), and FIG. 13 (B) shows the second image shown in FIG. 12 (C). The first image and the second image are common in that the reflected light of natural light is imaged, but the second image includes the reflected light of the infrared light projected by the first light projecting unit T1. In that respect, the brightness is higher than that of the first image and there is a difference. FIG. 13C shows a difference image in which the reflected light portion of the natural light common to both images is canceled in order to make this difference stand out, and is based on the infrared light projected by the first light projecting unit T1. This is a captured image.

  For example, the image on the passenger seat side in FIG. 11B and the image on the passenger seat side in FIG. 12B are the same in that the reflected light of natural light is imaged, and thus disappear in the difference image. . As a result, in the difference image of FIG. 13C, the driver's side image captured when infrared light is projected appears clearly. By doing in this way, a clear image based on the reflected light from a necessary area among the reflected lights from two different areas can be acquired. The same applies to the second difference image generated from the first image and the third image.

  In addition, this invention is not limited to the illustrated Example, The implementation by the structure of the range which does not deviate from the content described in each item of a claim is possible. That is, although the present invention has been particularly illustrated and described with respect to particular embodiments, it should be understood that the present invention has been described in terms of quantity, quantity, and amount without departing from the scope and spirit of the present invention. In other detailed configurations, various modifications can be made by those skilled in the art.

100 face image processing device T1 first light projecting unit T1L first infrared light emitter T1F first polarizer T2 second light projecting unit T2L second infrared light emitter T2F second polarizer P1 first light path P2 second light path P2 third Optical path O1 First optical path entrance port O1F First visible light cut filter O2 Second optical path entrance port O2F Second visible light cut filter LD Beam splitter LD1 First surface LD2 Second surface 20 Imaging element 21 Lens 30 Image processing unit 40 Control 50 Storage unit 60 External output unit 70 Power supply unit 80 Light shielding wall DR Driver WL Handle C Vehicle A1 First area (driver's seat)
A2 Second area (passenger seat)

Claims (2)

A first light projecting unit that projects light of a polarization component in the first direction onto the first region;
A second light projecting unit that projects light of a polarization component in a second direction orthogonal to the first direction onto a second region;
A first optical path for guiding light from the direction of the first region;
A second optical path for guiding light from the direction of the second region;
A light beam that is disposed so that the first surface faces the first optical path and the second surface faces the second optical path, transmits the polarization component in the first direction, and reflects the polarization component in the second direction. A divider,
A third light guides the reflected light reflected on the first surface by the light guided to the first optical path and the transmitted light transmitted from the second surface to the first surface by the light guided to the second optical path. The optical path,
A lens disposed on the third optical path and collecting the reflected light and the transmitted light;
An image sensor for receiving light from the lens;
An image processing unit that performs predetermined image processing based on a signal of the image sensor;
Comprising
Image processing device. A controller that controls turning off / lighting of the first light projecting unit and the second light projecting unit, imaging of the image sensor, and image processing of the image processing unit;
A storage unit for storing an image captured by the image sensor;
Further comprising
The controller is
The first light projecting unit and the second light projecting unit are turned off, the first area and the second area are imaged as a first image by the image sensor, and the first image is stored in the storage unit,
One of the first light projecting unit and the second light projecting unit is turned on, the other is turned off, and the first area and the second area are imaged as a second image by the imaging device and stored in the storage unit. Storing the second image;
The one of the first light projecting unit and the second light projecting unit is turned off, the other is turned on, and the first area and the second area are imaged as a third image by the imaging element, and Storing the third image in a storage unit;
Instructing the image processing unit to perform image processing when the first image, the second image, and the third image are stored in the storage unit;
Upon receiving the image processing instruction, the image processing unit
Generating a difference image between the first image and the second image as a first difference image;
Generating a difference image between the first image and the third image as a second difference image;
The image processing apparatus according to claim 1.