JP4585553B2 - apparatus - Google Patents

Description

  The present invention relates to a drowsiness detection device that detects drowsiness of a driver, and more particularly to a drowsiness detection technique and a driver identification technique based on the detection result.

In order to prevent an accident caused by the driver's sleepiness while driving the vehicle, it is highly necessary to monitor the driver's arousal state and determine whether or not he / she is asleep. As a technique for detecting drowsiness, a method for acquiring a driver's biological information from a steering wheel of an automobile has been devised. For example, there is a technique for measuring an electrocardiogram signal using an electrode attached to a steering wheel (for example, Patent Documents). 1). On the other hand, as a method for detecting drowsiness using information contained in the blood flow, there is a method for determining an arousal state from a pulse wave (for example, see Patent Document 2).
JP 59-25729 A JP 2002-272708 A

  However, in order to measure an electrocardiogram signal using the method disclosed in Patent Document 1 and apply it to drowsiness detection, both hands must touch the handle. Furthermore, it is necessary to take measures against noise typified by a myoelectric signal of the hand.

  On the other hand, the technique disclosed in Patent Document 2 is worn on the wrist of a subject, and other implementations have not been studied.

  In addition, when a traffic accident has occurred, or when performing health management of the driver while driving based on the driver's biometric information, the driver's identity is required. The prior art has not been studied.

  The present invention provides an apparatus capable of identifying who the driver is and detecting biological information of the driver represented by the driver's sleepiness.

  The above and other objects and novel features of the present invention will be apparent from the description of this specification and the accompanying drawings.

  In order to solve the above problems, the outline of the invention disclosed in the present application is as follows.

  Drowsiness detection that has a light source that emits light to the user's finger or hand and an imaging unit that captures reflected or transmitted light from the finger or hand, and detects the user's sleepiness by analyzing the captured image apparatus.

  Also, a personal authentication device that uses a light source that irradiates light of two or more different wavelengths and authenticates a user using data captured by each of the light of two or more wavelengths. And the personal authentication apparatus which installs a light-diffusion member between the light source and the finger | toe or hand irradiated is disclosed.

  As described above, according to the described invention, it is possible to accurately perform personal authentication and detect drowsiness of a vehicle driver.

  Of the inventions disclosed in the present application, effects obtained by typical ones will be briefly described as follows.

  (1) When the drowsiness detection method based on the vein pattern image is used and the driving is not safe, the driver and the vehicle can be warned.

  (2) Personal authentication at the time of boarding and recording of the driver's sleepiness measurement result together with the result ensure safe driving of the vehicle, and when an accident occurs, it is recorded in the storage medium The driver can be identified from the personal authentication result.

  Hereinafter, embodiments of the present invention will be described in detail with reference to the drawings.

  In the present invention, a finger vein pattern image is used to detect sleepiness and perform personal authentication. Drowsiness is detected by detecting a pulse or a temperature rise of the hand using a finger vein pattern image used for personal authentication using a finger vein pattern image. The vein pattern is not limited to the finger and may be in any place of the hand, but the finger vein pattern is easier to image than the palm portion because it travels closer to the body surface.

  The finger vein pattern image is captured based on optical measurement, and there are a method using transmitted light and a method using reflected light as a light source. Near-infrared light is mainly used as the light source. This is because the main absorber of near-infrared light in the living body is hemoglobin hemoglobin and the vein pattern image is captured by utilizing the property of being highly permeable in other biological tissues. When using transmitted light, as shown in FIG. 1, the finger (101) is placed between the light source unit (102) and the imaging unit (103). The light emitted from the light source to the finger from the back side of the hand is scattered from within the finger and then comes out of the finger. Therefore, the image is picked up by the imaging unit (103) installed on the palm side. It becomes a pattern image of the vein (104) existing directly under the skin on the side. In the case of the reflected light method, as shown in FIG. 2, the light source unit (201) and the imaging unit (103) are placed on the same side with respect to the finger (101) to capture the pattern image of the vein (104). Do. For the light source unit, a light emitting element such as an LED can be used.

  In the present embodiment, a four-wheeled vehicle is adopted as the vehicle, but the present invention includes a motorcycle, a railway vehicle, an airplane, and the like that are operated by a driver in addition to the four-wheeled vehicle. The identification of the driver by the personal authentication technique is performed, for example, when the vehicle door is opened and the driver gets into the driver's seat or when the driver sits on the seat and holds the handle (control stick). The following describes an example in which a device is installed inside a door handle or handle and imaging is performed using reflected light. In a vehicle, for example, a panel part such as a room mirror or a meter is irradiated with light toward a finger. If the light source unit is installed at a position where it can be performed, it is possible to image a vein pattern using transmitted light.

  In this case, it is effective to collect light with a mirror or use laser light. In addition, when light sources are installed separately for the right hand and left hand on the left and right of the handle, for example, one of the left and right light sources is selectively selected depending on which hand is positioned on the handle. It is possible to use it.

  When authentication is performed when the door is closed, the light source unit (303) and the imaging unit (304) are incorporated in the handle portion (302) inside the door (301) as shown in FIG. An imaging window is provided at a portion of the door handle (302) where the imaging part of the hand touches. For example, a rectangular window is provided at a position where the base part of the finger is supposed to touch from the second joint part of the four fingers excluding the thumb. It is effective that the window portion is made of a material and a size capable of blocking external ambient light as much as possible and transmitting light having the wavelength of the light source light.

  The light source unit (303) is installed at a position where light is not directly applied to the finger that is the object of imaging. This is because, if the light from the light source is directly applied to the finger, it will exceed the dynamic range of the image sensor. For example, in a 256-level grayscale image, the portion that is strongly irradiated with light is completely white. It is because it ends up. In order to prevent this, it is effective to place a material having a light diffusion function typified by ground glass on the light emitting surface of the light source.

  The light source light incident on the material having the light diffusing function is diffused and emitted with the amount of light slightly weakened. Therefore, the installation position of the light source is not limited to the position of 303, and it is possible to directly irradiate the finger with the light source light. In FIG. 4, when a near-infrared light-emitting LED is used as a light source and the finger is directly imaged with the light source light (401) and 401 does not change the intensity of the light source, the light diffusion glass is placed in front of the light source. An example in the case of installation and imaging (402) is shown. A CCD element or a CMOS element can be used for the imaging unit (304). The calculation of authentication can be incorporated into, for example, an engine control system. According to this configuration, it is not necessary to newly incorporate a control system, and the result of authentication can be used for engine start control.

  In this configuration, personal authentication is performed when the driver tries to grip the handle portion (302) inside the door when getting on. A sensor capable of sensing the approach of a living body such as an infrared sensor (not shown) is incorporated in the door (301), and the sensor detects that the human hand (305) has approached the handle. The power consumption of the unit (304) can be turned on to reduce power consumption as much as possible. Also, the handle portion (302) of the door is provided with a protruding shape (306) or a depression (307) that can accommodate an index finger, so that the place where the hand is put is limited even if it is not visually recognized. By doing so, the finger to be imaged can be positioned. With this configuration, the position of the captured image can be defined, and the collation accuracy with the registered image can be increased.

  When performing personal authentication when sitting on the driver's seat and holding the handle, an authentication device is mounted on the handle. It is also possible to perform double authentication between the door and the handle. In this case, different information is registered in the door and the handle, for example, the door is registered with an index finger and a finger different from the middle finger is registered in the handle, and the vehicle can be used only after double authentication. As a result, the security strength can be increased.

  FIG. 5 shows an example in which the authentication device is incorporated in the handle (501). The light source unit (509) and the imaging unit (510) are incorporated in the handle (501), and the authentication calculation unit (503) is incorporated in the handle central portion (504). A measurement result recording medium (505) is also incorporated. Although the recording medium is described in the horn portion in FIG. 5, it may be provided in other places.

  FIG. 5 shows an example in which the device is incorporated in a portion gripped by the right hand (305), but this may be a portion gripped by the left hand (506). It is possible to take a vein pattern image from one hand touching the handle even if it is incorporated in both hands and one hand is away from the handle.

  FIG. 5 (a) shows a diagram in which the handle is grasped with both hands. FIG. 5 (b) shows the right-hand part of (a). Further, (c) shows a cross-sectional view in a plane parallel to the wheel of the handle. As in the case where the light source unit (509) is attached to the handle portion of the door, it is desirable to install the light source unit (509) at a position where light is not directly applied to the finger as an irradiation target. However, this is not the case when a material having a light diffusion function is installed on the light emitting surface of the light source.

  Even when the device is attached to either the handle of the door or the handle, the light emitted from the light source unit is mainly absorbed by hemoglobin in venous blood that exists immediately under the skin on the palm side of the finger. Then, a vein pattern image on the ventral side of the finger can be taken. The personal authentication is performed by comparing the captured image with a finger vein pattern image registered in advance. For example, the authentication is performed using a method described in Japanese Patent Laid-Open No. 2001-184507.

  As in the case of the door (301) in FIG. 3, the handle (501) also incorporates an infrared sensor that senses the approach of a living body, and, like the handle (302), at a position suitable for capturing a vein pattern image. The authentication accuracy is improved by positioning the grip part by placing a marker (507) on the grip part immediately next to the position where the index finger is placed for the purpose of placing the hand. Is effective for.

  In order to capture the finger vein pattern image with the imaging unit (510) incorporated in the handle (501), an imaging window (508) is provided in the AA ′ portion of the handle portion in FIG. Provide. It is effective that the window is made of a material capable of blocking external ambient light as much as possible and transmitting light having the wavelength of the light source light, as in the case where the window is incorporated in the handle portion of the door.

  Further, in order to capture an image of a finger vein pattern by incorporating an element inside a handle or a handle, it is necessary to shorten the optical distance between the light source and the image sensor. In general, an imaging method using reflected light is easier to miniaturize the optical system than an imaging method using transmitted light. On the other hand, in the imaging method using reflected light, it is difficult to obtain the same contrast as the vein pattern image imaged using transmitted light. Therefore, in order to obtain a vein pattern image with higher sensitivity, an embodiment will be described in which a plurality of images are captured using two wavelengths and pattern enhancement is performed by image processing.

  As the light source, a light emitting element (for example, LED) capable of emitting two or more wavelengths is used. Of the two wavelengths used for imaging, the short wavelength can be selected from around 600 nm that can penetrate the living body, and the long wavelength can be selected from near 980 nm before the water absorption peak appears. . On the shorter wavelength side than 600 nm, since the living body absorbs light, the image becomes dark as a whole, and the vein pattern cannot be imaged with good contrast. On the longer wavelength side than 980 nm, light absorption of water, which is a component of the living body, increases, and the same thing occurs.

  Hereinafter, in this embodiment, description will be made with 680 nm and 810 nm as an example, but the wavelength to be selected is not limited to this. Incidentally, light having a wavelength of 680 nm has a characteristic that the difference in the extinction coefficient between deoxyhemoglobin and oxyhemoglobin is large, and that the difference is small at 810 nm. If the wavelength to be selected is determined each time in the program, the user does not know which wavelength is being picked up, so it is possible to make it easier to see forged fingers. It becomes.

  In order to enhance the vein pattern using two images picked up at two different wavelengths, first, in each picked-up image, the luminance value is converted into absorbance A according to (Equation 1).

Here, p _max is the maximum pixel value, and p is the pixel value. Also, the absorbance at wavelength λ is given by equation (2).

In the equation (2), ε _oxyHb (λ) and ε _deoxyHb (λ) are molecular extinction coefficients of oxyhemoglobin and deoxyhemoglobin, c _oxyHb and c _deoxyHb are the concentrations of the respective hemoglobins, d is the effective optical path length, and α ( λ) represents attenuation by living tissue other than hemoglobin. Since the pigments in the palm of the finger are mainly oxyhemoglobin and deoxyhemoglobin, assuming that α (λ) is a constant value α regardless of the measurement wavelength (in this case, 680 nm and 810 nm) Equations (3) and (Equation 4) are obtained.

  Also,

Holds. Here, it is considered that the coefficient 1 contributes to each pixel value (absorbance) as a common component, and this is ignored because it is necessary to consider the component contributing to the difference from the viewpoint of pattern enhancement.

Then, kC _deoxyHb remains, but when _CdexyHb is considered as a common component of each pixel value, it can be considered that the main component contributing to the difference between the pixel values is the value of k.

From this, k = C _oxyHb / C _dexyHb is _obtained from Equation (3) and Equation (4). Next, in order to display the result as an image, the value of k obtained for each pixel is converted into, for example, a gray scale (numerical value of 0-255). As a result, an enhanced vein pattern image is obtained.

  Here, k is the ratio of the concentration of oxyhemoglobin to the concentration of deoxyhemoglobin. That is, by plotting the ratio of oxygenated hemoglobin to non-oxygenated hemoglobin calculated using an image captured by two-wavelength measurement for each pixel, it is possible to obtain an image in which a blood vessel pattern is emphasized. . Note that the blood vessel pattern enhancement method is not limited to imaging using reflected light, but is applicable and effective in imaging using transmitted light.

Specific numerical values of k = C _oxyHb / C _dexyHb are obtained from an absorption spectrum of hemoglobin, ε _oxyHb (680) = 277.6, ε _deoxyHb (680) = 2407.9, ε _oxyHb (810) = 864, ε _deoxyHb ( Use 810) = 717.08.

  FIG. 6 shows a vein pattern image (601) when a 680 nm light source is used, a vein pattern image (602) when a 810 nm light source is used, and a pattern enhanced image (603) based on them. The pattern-enhanced image (603) is obtained by converting the calculated k value for each pixel value into a gray scale. The original image has uneven brightness, and the vein pattern itself can be emphasized even though the base of the finger is dark. For example, in the image 603, the two blood vessels are emphasized so as to draw an ellipse in the right two-third region.

  Next, a method for detecting drowsiness using a vein pattern image will be described. Although it is possible to detect drowsiness alone as an independent process, it is also possible to perform control so that this is performed only when personal authentication is successful. First, a method for detecting drowsiness by detecting pulse information from an image will be described.

  Use a single wavelength light in the near-infrared wavelength range to capture a finger vein pattern image continuously in time (for example, 30 frames / second), or use light of different wavelengths by alternately flashing two wavelengths of light. The finger vein pattern is picked up, and vein pattern emphasis by the above-described method is performed continuously over time. The contour portion of the finger is detected by image processing, and the total or average value of the pixel values inside the finger is calculated in each vein pattern image acquired in time series in this way. The time change of the sum or average value calculated here corresponds to the pulse. In order to detect drowsiness based on pulse information, for example, analysis based on chaos theory may be performed and drowsiness detection may be performed using the Lyapunov exponent as an index.

  The change of the Lyapunov exponent is measured over time by repeatedly calculating the Lyapunov exponent based on the pulse wave data measured for a certain period of time. When this index shows an increasing tendency, it is determined that sleepiness is being caused by fatigue or the like. An increase in the Lyapunov exponent is associated with a decrease in consciousness concentration due to strong fatigue. Further, since the pulse rate decreases when it becomes sleepy, the time between the peaks of the pulse wave is measured, and if this tends to increase, it can be determined that sleepiness is occurring.

  Next, a method for detecting drowsiness by detecting the temperature rise of the hand from the vein pattern image will be described. Finger vein pattern images are taken over time, and the occupied area (s) of the blood vessel portion is calculated each time. For example, the position of the blood vessel in the captured image is identified, and the number of pixels in the portion where the blood vessel exists is counted, thereby setting this as the occupied area.

  Compared with a threshold value set based on a value s calculated in a pre-registered image, when the measured value of s exceeds the threshold value, or when s shows an increasing tendency with respect to time, It is determined that the venous blood flow is increasing. An increase in hand venous blood flow is used as an index of hand temperature rise, and if the tendency continues to increase, it is determined that sleepiness is present. The threshold value can be set uniformly, but can also be set for each individual based on an image of a vein pattern that is actually drowsy. Since 2 hours after lunch is a time of sleepiness, it is also effective to provide a meal time input button so that a warning is always given 2 hours after lunch.

  By using the configuration of the present application as described above, it is possible to perform both personal authentication and drowsiness detection. The recording medium (505) of the measurement result has a structure that has strength against heat and impact that is not destroyed even in the event of a vehicle accident so that it can be taken out in the event of an accident. It is also effective to transmit data to the outside by a communication function using the above, and to store and manage data over time in an external organization. Furthermore, the measurement results should always be kept for about 5 minutes just before the start of the warning, even if not all records are left, so that it can be judged later whether the sleepiness detection function was working correctly. It is valid.

  FIG. 7 shows a series of flowcharts as described above. First, the imaging conditions are corrected by light source control or the like, the individual is authenticated (701) by the method described above as an example, and the time-lapse vein pattern image imaging (702) is started using this as a trigger. In addition to optimizing the amount of light by controlling the light source, the imaging conditions can be corrected by monitoring the environmental temperature by incorporating a temperature sensor into the handle, etc., and detecting an alarm without detecting sleepiness or personal authentication at low or high temperatures. You can list warnings such as sounding. The temperature threshold at this time can be arbitrarily set, or can be automatically set according to time and season.

  Furthermore, it is possible to make a determination at a low temperature by observing that the blood vessel is contracted to a certain ratio or more in the captured finger vein pattern image. The measurement result is recorded on the recording medium (505) as time passes, or is recorded by an external organization such as a data center by transmitting information to the outside by wireless or the like (703), and the driver is not drowsy. Whether or not (704) is determined over time. If drowsiness is not occurring, continue to capture the vein pattern image and detect drowsiness. However, if it is determined that drowsiness is occurring, use a method such as sounding from the speaker in the car. The driver and passengers are alerted by issuing a warning sound (705).

  This alerting can also be performed by linking the drowsiness detection system with the car navigation system and using a navigation screen or sound, or ventilation by opening or closing an air conditioner or a window. Further, a step of counting the number of warnings (706) and a step of determining whether the warning to the driver exceeds a certain number of times (707) are provided. This information and the result of sleepiness detection can be transmitted to a person outside the vehicle or a related organization such as a hospital, and a hazard lamp can be turned on (708) to warn the outside. The information transmission destination at this time is registered in advance.

  In the unlikely event that an accident occurs, the driver can be identified from the personal authentication result recorded on the storage medium in the vehicle or transmitted and recorded outside the vehicle.

  The above example is based on a measurement method using reflected light, but the finger vein pattern image enhancement method by two-wavelength measurement is not limited to the measurement method using reflected light. It is also possible to externally attach a portion corresponding to the light source without incorporating it into the handle, and to image the finger vein pattern using the transmitted light method. In that case, the device is configured so that the finger of the hand (305, 506) can be placed between the light source (303) and the imaging device (304). In the above embodiment, two wavelengths are used, but two or more wavelengths may be irradiated and two may be selectively selected from them. Further, even when two or more wavelengths are used, the same analysis can be performed by using a method such as obtaining the value of k for every two wavelengths using the above formula and displaying an average of those values.

  As mentioned above, the invention made by the present inventor has been specifically described based on the embodiment. However, the invention is not limited to the embodiment, and various modifications can be made without departing from the scope of the invention. Needless to say.

It is a figure which shows the apparatus structure in the transmitted light method. It is a figure which shows the apparatus structure in the reflected light method. It is a figure which shows the example of the assembly of the vein imaging device to the door of a four-wheeled vehicle, and its handle. It is a figure which shows the example of the effect of the light diffusion glass in the case of imaging with the reflected light method. It is a figure which shows the example of the assembly of the vein imaging device inside a handle. It is a figure which shows the example of the result of the imaging using 2 wavelengths, and a pattern emphasis process. It is a figure which shows the flow of sleepiness detection measurement.

Explanation of symbols

  DESCRIPTION OF SYMBOLS 101 ... Finger, 102 ... Light source part (transmitted light method), 103 ... Imaging part, 104 ... Vein, 201 ... Light source part (reflected light method), 301 ... Door, 302 ... Handle part inside door, 303 ... Light source part 304 ... Imaging unit, 305 ... Right hand, 306 ... Projection shape provided on the handle, 307 ... Recess, 401 ... Example of finger imaging with reflected light (no light diffusing glass), 402 ... Example of finger imaging with reflected light (With light diffusing glass), 501... Handle, 502... Projection shape provided on the handle, 503... Authentication calculation unit, 504... Middle portion of handle, 505 ... recording medium, 506 ... left hand, 507. 509... Light source unit, 510... Imaging unit, 601... Example of finger vein pattern image captured using 680 nm light as light source, 602... Example of finger vein pattern image captured using 810 nm light as light source, 60. 3... Pattern enhancement processing result using 601 and 602.

Claims (8)

A light source that irradiates light on a user's hand or finger, an imaging unit that images light reflected or transmitted from the finger, and an arithmetic unit that analyzes an image captured by the imaging unit,
The light source emits light of two different wavelengths,
The imaging unit captures a first image and a second image for each wavelength,
The calculation unit obtains image data in the first image and the second image, obtains an abundance ratio of oxyhemoglobin and deoxyhemoglobin for each pixel from the image data, and calculates the abundance ratio for each pixel. An apparatus for obtaining a vein pattern image by plotting. The apparatus of claim 1.
The arithmetic unit converts a luminance value for each image into an absorbance for each pixel for each of the first image and the second image based on the image data, and each pixel of the first image and the second image. An apparatus characterized in that the abundance ratio is obtained from a difference in absorbance for each. The apparatus of claim 1.
A recording unit for registering a vein pattern image of the user's hand or finger;
The said calculating part authenticates the said user using the said image data based on the said imaged image, and the said registered vein pattern image data . The apparatus of claim 1.
The said calculating part detects sleepiness by detecting a pulse from the image data of the said vein pattern , The apparatus characterized by the above-mentioned. The apparatus of claim 1.
The said calculating part detects the sleepiness of the said user by measuring the occupation area of the vein pattern in the said imaged image, The apparatus characterized by the above-mentioned. The apparatus of claim 1.
The apparatus, wherein the light source and the imaging unit are attached to a handle. The apparatus according to claim 4.
The apparatus further includes an alarm generation unit, and the alarm generation unit issues an alarm when the arithmetic unit detects the drowsiness. The apparatus of claim 1.
Having a light diffusing member,
The light from the light source passes through the light diffusing member placed between the light source and the finger and is applied to the hand or finger.