JP2010026657A - Driver state detection device - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To provide a driver state detection device for highly precisely detecting a state of a driver based on a state of eyes of the driver. <P>SOLUTION: This driver state detection device is provided with: an irradiation means for irradiating a region including the crystalline lens of eyeballs of a driver with the invisible rays of light with a striped pattern; an imaging means for picking up the image of the region including the crystalline lens of the eyeballs of the driver with the invisible rays of light; a stripped pattern detection means for detecting a stripped pattern projected on the crystalline lens of the driver from the image picked up by the imaging means; a thickness estimation means for estimation the thickness of the crystalline lens of the driver based on the change of the stripped pattern detected by the stripped pattern detection means; and a driver state decision means for deciding the state of the driver based on the change of the thickness of the crystalline lens estimated by the thickness estimation means. <P>COPYRIGHT: (C)2010,JPO&INPIT

Description

  The present invention relates to a driver state detection device.

Various devices for detecting a driver's state such as a driver's sloppy state or a dozing state have been proposed for use in driving support devices. In the apparatus described in Patent Literature 1, the pupil diameter of the driver is detected, and it is determined whether or not the driver is in a random state based on the change rate of the pupil diameter.
JP 2002-367100 A JP-A-11-213164 JP 2005-532831 A

  The diameter of the pupil changes depending on the amount of incoming light. In addition, the driver of the vehicle receives a change in light under an environment that changes during traveling. Therefore, the pupil diameter of the driver may change regardless of the driver's condition due to the influence of disturbance such as the amount of environmental light. As a result, it may not be possible to accurately determine whether the driver's pupil diameter changes or not.

  Then, this invention makes it a subject to provide the driver | operator state detection apparatus which detects a driver | operator's state with high precision based on a driver | operator's eye state.

  The driver state detection device according to the present invention includes an irradiation unit that irradiates a region including a crystalline lens of a driver's eyeball with invisible light having a striped pattern, and an imaging unit that images a region including the crystalline lens of the driver's eyeball using the invisible light. And a striped pattern detecting means for detecting a striped pattern reflected in the driver's crystalline lens from an image captured by the imaging means, and a thickness of the driver's crystalline lens based on a change in the striped pattern detected by the striped pattern detecting means. Thickness estimation means for estimation, and driver state determination means for determining the state of the driver based on a change in the thickness of the lens estimated by the thickness estimation means are provided.

  In this driver state detection device, a region of the driver's eyeball including the crystalline lens is irradiated with striped invisible light by the irradiation means. In the driver state detection device, an area including the crystalline lens irradiated with the invisible light with the striped pattern is captured by the invisible light, and an image including the crystalline lens is acquired. Furthermore, in the driver state detection device, the striped pattern reflected on the crystalline lens is detected from the image by the striped pattern detecting means. The striped pattern reflected in the crystalline lens is distorted and the pattern changes according to the curved surface (curvature) of the crystalline lens. Further, in order to change the focal length according to the visual object, the thickness of the crystalline lens changes (stretches) according to the distance to the visual object, and the curvature changes according to the thickness. Thus, the thickness of the crystalline lens depends on the distance to the visual object and is not easily affected by ambient light or the like. Therefore, in the driver state detection device, the thickness of the crystalline lens is estimated by the thickness estimating means based on the stripe pattern that changes in accordance with the state of the crystalline lens. In the driver state detection device, the driver state determination unit determines the state of the driver based on the change in the thickness of the crystalline lens. When the driver is in a normal state, the visual object is changed every moment in order to acquire information necessary for driving from various surrounding objects, and the thickness of the lens frequently changes accordingly. However, when the driver's consciousness state decreases, the frequency of changing the visual object decreases, so the change in the thickness of the lens also decreases. Thus, the driver state detection device can detect the driver's state with high accuracy by utilizing the change in the thickness of the crystalline lens.

  Examples of the striped pattern include a grid-like striped pattern, a vertical striped pattern, and a horizontal striped pattern. The driver's state includes, for example, a normal state and a sloppy state, a state in which consciousness is lowered, and a state immediately before falling asleep. The thickness of the crystalline lens used in the present invention is a concept including not only the crystalline thickness of the crystalline lens itself but also other parameters (for example, curvature) that change according to the thickness.

  In the above-described driver state detection device of the present invention, it is preferable that the irradiating means irradiates invisible light with a grid-like stripe pattern.

  In this driver state detection device, the irradiation means irradiates a region including the crystalline lens of the driver's eyeball with invisible light in a grid pattern. In the driver state detection device, the striped pattern detecting means detects the grid-like striped pattern reflected in the crystalline lens from the captured image, and the thickness estimating means drives based on the detected change in the grid-like striped pattern. The thickness of the person's lens. In the case of a lattice-like stripe pattern, the stripe pattern in the vertical direction is distorted and the stripe pattern in the horizontal direction is distorted in accordance with the change in the thickness (curvature) of the crystalline lens, so that a change in the stripe pattern in these two directions is obtained. Thus, in the driver state detection device, the thickness of the crystalline lens can be estimated with higher accuracy based on the change in the striped pattern in two directions by using the grid-like striped pattern.

  In the above-described driver state detection device of the present invention, the driver state determination unit may be configured to determine the state of the driver based on the frequency with which the thickness of the crystalline lens has changed.

  As described above, since the thickness of the crystalline lens changes according to the distance to the visual object, the thickness of the crystalline lens changes (stretches) each time the driver changes the visual object. Therefore, if the number of times the driver changes the visual object is large, the number of changes in the lens thickness increases, and if the number of times the driver changes the visual object is small, the number of changes in the lens thickness also decreases. Therefore, in the driver state detection device, the driver's state can be detected with high accuracy by determining the state of the driver based on the frequency with which the thickness of the crystalline lens is changed by the driver state determination unit.

  In the driver state detection device of the present invention, the driver state determination means may determine whether or not the driver is in a sloppy state.

  When the driver is in a state of disambiguation, the number of times the visual object is changed decreases, and the number of changes in the lens thickness also decreases. Therefore, in this driver state detection device, it is possible to accurately determine whether or not the driver is in a sloppy state by using a change in the thickness of the crystalline lens.

  The present invention can detect the state of the driver with high accuracy by utilizing the change in the thickness of the crystalline lens.

  Hereinafter, an embodiment of a driver state detection device according to the present invention will be described with reference to the drawings.

  In the present embodiment, the driver state detection device according to the present invention is applied to a casual driving determination device mounted on a vehicle. The random driving determination device according to the present invention determines whether or not it is a random state by using the frequency of expansion and contraction of the lens thickness of the driver's eyeball, and provides the determination result to a driving support device or the like.

  The crystalline lens is a convex lens that refracts incoming light and focuses it on the retina, and the ciliary muscles are connected via the Chin band. When looking at nearby objects, the ciliary muscles contract and the chin band relaxes, resulting in a thicker lens (curvature increases). Also, when looking at a distant object, the ciliary muscle relaxes and the chin band is pulled, thereby reducing the thickness of the crystalline lens (decreasing the curvature). In this way, the person adjusts the focal length by unconsciously expanding and contracting the thickness of the crystalline lens according to the distance to the object to be viewed. The thickness of the crystalline lens is generally about 3.6 mm at the normal time and about 4.0 mm at the maximum. The diameter of the crystalline lens when viewed from the front is generally about 9 to 10 mm.

  The driver of the vehicle is required to confirm visual objects (for example, other vehicles, pedestrians, bicycles, traffic lights, traffic signs, various instruments, rearview mirrors, side mirrors) for safety confirmation during normal driving (in normal conditions). ) Is constantly changing. Therefore, the driver constantly changes the lens thickness in order to adjust the focal length according to the visual object. However, when the driver is in a state of ambiguity, the visual object is not changed consciously, so the change in the thickness of the lens is also reduced.

  With reference to FIGS. 1 to 3, the casual driving determination device 1 will be described. FIG. 1 is a configuration diagram of a random driving determination device according to the present embodiment. FIG. 2 is a schematic diagram of the operation in the casual driving determination device of FIG. FIG. 3 is a graph showing an example of a temporal change in the thickness of the crystalline lens.

  The random driving determination device 1 estimates the thickness of the driver's crystalline lens, and determines whether the driver is in a normal state or a casual state based on the expansion / contraction frequency of the crystalline lens thickness. In particular, the casual driving determination device 1 irradiates the near-infrared rays of the lattice pattern on the crystalline lens, and estimates the thickness of the crystalline lens from the degree of change of the lattice pattern. For this purpose, the casual driving determination device 1 includes a near infrared irradiator 2, a polarizing film 3, a camera 4 and an ECU [Electronic Control Unit] 5.

  In the present embodiment, the near-infrared irradiator 2 and the polarizing film 3 correspond to the irradiating means described in the claims, and the camera 4 corresponds to the imaging means described in the claims. The processing corresponds to a striped pattern detection unit, a thickness estimation unit, and a driver state determination unit described in the claims.

  The near infrared irradiator 2 is an irradiator that irradiates near infrared rays. The near-infrared irradiator 2 is attached to an instrument panel, a steering column, or the like in such a direction as to irradiate around the crystalline lens of the driver's eyeball. The near-infrared irradiator 2 irradiates near infrared rays toward the driver's crystalline lens while the casual driving determination device 1 is in operation. In this way, the near infrared irradiator 2 irradiates the driver with near infrared light around the lens and the camera 4 captures the periphery of the driver's lens, thereby obtaining an image only around the driver's lens. Near-infrared rays are harmless to humans. Moreover, since near infrared rays are invisible light, they do not hinder the driving of the driver.

  In addition, since the position (especially height position) where a lens exists differs from driver | operator, the position of a driver | operator's face (especially eye) is detected, and irradiation of the near-infrared irradiator 2 according to the detected position It is good to be able to change the direction. Further, since the crystalline lens is small, the near infrared ray may be irradiated to at least a certain wide area including the crystalline lens (for example, the entire face) instead of irradiating the near infrared ray only to the periphery of the crystalline lens.

  The polarizing film 3 is a polarizing film that filters the near infrared rays irradiated from the near infrared irradiator 2 into a grid-like striped pattern. In this grid-like striped pattern, a plurality of vertical lines and horizontal lines forming the grid have the same line spacing, and the line spacing is sufficiently narrower than the diameter of the crystalline lens (for example, 1 to a fraction of the diameter). Several tenths of an interval). The polarizing film 3 is disposed at an arbitrary position through which the near infrared ray irradiated from the near infrared irradiator 2 passes, and may be provided in the near infrared irradiator 2 or provided separately from the near infrared irradiator 2. May be. In the polarizing film 3, when near-infrared rays enter, polarization filtering is performed, and lattice-patterned near-infrared rays C are emitted (see FIG. 2).

  The camera 4 is a camera for acquiring an image around the driver's crystalline lens. The camera 4 is attached to an instrument panel, a steering column, or the like in a direction around the driver's crystalline lens. The camera 4 is a near-infrared camera, takes in near-infrared rays (particularly, near-infrared rays from the near-infrared irradiator 2), and generates a near-infrared image corresponding to the brightness of the near-infrared rays. The camera 4 captures an image of the driver's crystalline lens L around every predetermined time (for example, every 1/30 seconds), and transmits the captured near-infrared image I to the ECU 5 as an image signal (see FIG. 2).

  In addition, since the position where a crystalline lens exists differs from driver | operator, it is good to detect the position of a driver | operator's face (especially eye), and to change the imaging direction of the camera 4 according to the detected position. Further, since the crystalline lens is small, the camera 4 may be used to capture the periphery of the driver's face and extract the eye region (particularly, the crystalline lens periphery) from the facial image.

  The ECU 5 includes a CPU [Central Processing Unit], a ROM [Read Only Memory], a RAM [Random Access Memory], an image processing device, and the like, and comprehensively controls the casual driving determination device 1. The ECU 5 activates the near-infrared irradiator 2 and the camera 4 when the casual driving determination device 1 is activated, and receives image signals from the camera 4 at regular intervals. Then, the ECU 5 determines whether or not the driver is in a relaxed state based on the captured images at regular intervals, and outputs the determination result to the driving support device.

  Specifically, every time a captured image is input, the ECU 5 corrects the lens distortion with respect to the input captured image I and corrects the tilt of the image to a trapezoid to obtain a corrected captured image I ′. (See FIG. 2). The lens distortion occurs according to the characteristics of the camera 4 (particularly, the curved surface of the camera lens). The inclination of the image is generated according to the positional relationship between the position of the driver's crystalline lens (face) and the camera 4. The corrected captured image I ′ corresponds to an image obtained by imaging the crystalline lens L from the front without lens distortion. Further, as preprocessing, noise removal, filtering, or the like may be performed on the captured image in order to easily detect the lattice pattern from the captured image.

  The ECU 5 traces the lattice pattern S in the corrected captured image I ′ and extracts the lattice pattern S (see FIG. 2). This lattice pattern S is distorted along the curved surface of the crystalline lens L in the portion reflected in the crystalline lens L, and is a lattice pattern composed of a curve from a lattice pattern composed of straight lines at the time of irradiation.

  The ECU 5 calculates the curvature of each curve and the interval between the curves (vertical line interval, horizontal line interval) from the extracted lattice pattern S. Further, the ECU 5 calculates the curvature of the crystalline lens L based on the curvature of the lattice pattern S and the line interval, and calculates the thickness of the crystalline lens L based on the curvature of the crystalline lens L. Then, the ECU 5 stores the calculated thickness of the crystalline lens L in time series.

  The ECU 5 reads time series data of the thickness of the crystalline lens L within a predetermined time in the past from the current time. The predetermined time is a time during which the number of expansion / contractions of the lens thickness according to the change of the visual object by the driver from time to time can be sufficiently obtained, and is set in advance by an experiment or the like. Then, the ECU 5 obtains the frequency of expansion / contraction (change) of the thickness of the crystalline lens L within the predetermined time from the time series data of the thickness of the crystalline lens L within the predetermined time.

  In the ECU 5, it is determined whether the expansion / contraction frequency of the thickness of the crystalline lens L within a predetermined time is greater than a threshold value. The threshold value is a frequency of expansion / contraction of the thickness of the lens L within a predetermined time when the driver is in a normal state (particularly, a frequency at the lowest level in the normal state), and whether the driver is in a normal state or a loose state. This is a threshold value for determining. The threshold value may be a value obtained from time-series data of the lens thickness when the driver collected in the past is in a normal state, or may be a value immediately before the driver is in a normal state after driving. Alternatively, it may be a general value obtained in advance from time-series data of the lens thickness when the driver collected from a large number of drivers is in a normal state. This general value is the initial value of the threshold. The initial value may be learned using data collected from the driver, and a threshold unique to the driver may be set.

  The ECU 5 determines that the driver is in a normal state when the expansion / contraction frequency of the thickness of the crystalline lens L within a predetermined time is equal to or greater than a threshold value (that is, when the driver frequently changes the visual object). On the other hand, in the ECU 5, when the expansion / contraction frequency of the thickness of the crystalline lens L within a predetermined time is less than the threshold value (that is, when the driver has not changed the visual object so much), the driver determines that the driver is in a loose state.

  FIG. 3 shows an example of a temporal change in the thickness of the driver's crystalline lens. In the time zone indicated by the symbol T, the number of times the thickness of the crystalline lens changes is clearly smaller than in the time zones before and after that. Therefore, in this time zone T, the number of times the driver has switched the visual object (focus target) is decreasing, and it can be estimated that the driver is in a state of being blurred.

  With reference to FIGS. 1-3, the operation | movement in the casual driving determination apparatus 1 is demonstrated along the flowchart of FIG. FIG. 4 is a flowchart showing a flow of operations in the casual driving determination device of FIG.

  While the casual driving determination device 1 is in operation, the near-infrared irradiator 2 irradiates near infrared rays toward the lens of the driver's eyeball (S1). In the polarizing film 3, the irradiated near-infrared light is subjected to polarization filtering to obtain a lattice-patterned near-infrared light (S1). This grid pattern of near-infrared rays reaches around the lens of the driver's eyeball.

  The camera 4 captures the vicinity of the crystalline lens of the driver's eyeball with near infrared rays at regular intervals, and transmits the image signal to the ECU 5 (S2). The ECU 5 receives an image signal and acquires a near-infrared captured image at regular intervals.

  The ECU 5 corrects the lens distortion of the captured image (S3), and corrects the tilt of the captured image to a trapezoid (S4). The ECU 5 extracts a lattice pattern reflected on the lens surface from the corrected captured image (S5). The ECU 5 estimates the lens thickness based on the extracted lattice pattern curvature and line spacing, and stores the lens thickness in time series (S6).

  The ECU 5 counts the expansion / contraction frequency of the lens thickness from the time-series data of the lens thickness within a predetermined time in the past from the current time (S7).

  The ECU 5 determines whether the expansion / contraction frequency of the lens thickness is equal to or greater than a threshold value (S8). When it is determined in S8 that the expansion / contraction frequency of the lens thickness is equal to or greater than the threshold value, the ECU 5 determines that the driver is in a normal state (normal operation state) (S9). On the other hand, when it is determined in S8 that the expansion / contraction frequency of the thickness of the crystalline lens is less than the threshold value, the ECU 5 determines that the driver is in a sloppy state (a sloppy driving state) (S10). Then, the ECU 5 outputs the determination result to the driving support device.

  According to this random driving determination device 1, it is possible to determine with high accuracy whether or not the driver is in a random state by making a determination based on a change in thickness (extension / contraction frequency) of the crystalline lens. By the way, the thickness of the lens changes depending on the distance to the visual object, so it is hardly affected by ambient light. In addition, since it is possible to accurately determine whether the driver frequently changes the visual object by using the expansion / contraction frequency of the thickness of the crystalline lens, it is possible to accurately determine whether the driver is in a rough state or a normal state.

  In the random driving determination device 1, the thickness of the crystalline lens can be estimated with higher accuracy based on the striped pattern in the two directions of the vertical direction and the horizontal direction by using the grid-like striped pattern.

  As mentioned above, although embodiment which concerns on this invention was described, this invention is implemented in various forms, without being limited to the said embodiment.

  For example, in the present embodiment, the present invention is applied to the casual driving determination device that determines the driver's casual state. However, the present invention is applied to a device that detects other driver states such as a state of reduced consciousness and arousal level (a state immediately before falling asleep). Alternatively, it may be applied as a driver state detection function in the driving support device.

  In the present embodiment, a grid-like striped pattern is applied as the striped pattern, but other striped patterns such as a vertical striped pattern or a horizontal striped pattern can be applied.

  In this embodiment, near-infrared light is applied as invisible light, but other invisible light may be used as long as it does not harm humans.

  In the present embodiment, the lens thickness expansion / contraction frequency is compared with a threshold value to determine whether or not the lens is in an indeterminate state. Other lens thickness change information may be used, and other methods may be used as the determination method.

It is a block diagram of the casual driving determination apparatus which concerns on this Embodiment. It is a schematic diagram of the operation | movement in the casual driving determination apparatus of FIG. It is a graph which shows an example of the time change of the thickness of a crystalline lens. It is a flowchart which shows the flow of operation | movement in the casual driving determination apparatus of FIG.

Explanation of symbols

  DESCRIPTION OF SYMBOLS 1 ... Random driving determination apparatus, 2 ... Near-infrared irradiator, 3 ... Polarizing film, 4 ... Camera, 5 ... ECU

Claims (4)

Irradiating means for irradiating a region including the crystalline lens of the driver's eyeball with invisible light in a striped pattern;
Imaging means for imaging an area including the crystalline lens of the driver's eyeball with invisible light;
Striped pattern detecting means for detecting a striped pattern reflected on the driver's crystalline lens from an image captured by the imaging means;
A thickness estimation means for estimating the thickness of the driver's crystalline lens based on the change in the stripe pattern detected by the stripe pattern detection means;
A driver state detection device comprising: a driver state determination unit that determines a driver state based on a change in thickness of the lens estimated by the thickness estimation unit.   The driver state detection apparatus according to claim 1, wherein the irradiating unit irradiates a grid-like striped invisible light.   3. The driver state detection device according to claim 1, wherein the driver state determination unit determines the state of the driver based on a frequency at which the thickness of the crystalline lens is changed.   The driver state detection device according to any one of claims 1 to 3, wherein the driver state determination unit determines whether or not the driver is in an indeterminate state.

Images