JP2008099884A - Condition estimating apparatus - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To provide a condition estimating apparatus which improves a precision of estimating a condition of a subject. <P>SOLUTION: The condition estimating apparatus 1 is equipped with a camera 2 photographing a face of a driver and an estimation processing unit 3 for estimating an awareness state (a degree of arousal) of the driver based on the captured image by the camera 2. The estimation processing unit 3 includes an image processing part 5 which performs image processing on the captured image by the camera 2 and detects the amount of opening of driver's eyes, an open eye/closed eye extracting part 6 which sorts detected values of the amount of opening of eyes for a prescribed time obtained by the image processing part 5 according to prescribed widths, creates a frequency distribution showing a relation between each detected value of the amount of opening of eyes and the detected frequency, and extracts the maximum value of an envelope approximate curve obtained from the frequency distribution, and an awareness state estimating part 7 for determining the awareness state of the driver from a time variation of the maximum value of the envelope approximate curve extracted by the part 6. <P>COPYRIGHT: (C)2008,JPO&INPIT

Description

  The present invention relates to a state estimation device that estimates, for example, a driver's consciousness state.

As a state estimation device for estimating a driver's consciousness state, for example, a device described in Patent Document 1 is known. The state estimation device described in Patent Document 1 detects an eye opening from image data of a camera, extracts a plurality of local minimum values from time series data of the opening, and uses these local minimum values as an eye opening candidate group. It separates into a closed eye candidate group, a closed eye threshold is set based on the open eye candidate group and the closed eye candidate group, and the driver's consciousness state is estimated.
JP 2004-41485 A

  However, in the above-described prior art, when estimating the driver's consciousness state, a closed eye threshold value for identifying whether the eye is open or closed is required, and it is assumed that the setting of the closed eye threshold value is incorrect. The accuracy will be adversely affected. In general, there are various open eye states of humans, and there are some people with large pupils, and there are some people who are difficult to distinguish from closed eyes, so it is difficult to set the closed eye threshold from the image data of the camera. In addition, when a low-cost camera is used in consideration of in-vehicle use, it is difficult to accurately detect a short blinking eye opening / closing, so the eye closing threshold cannot be set accurately. As a result, the estimation accuracy of the driver's consciousness state is lowered.

  The objective of this invention is providing the state estimation apparatus which can improve the estimation precision of a subject's state.

  The state estimation apparatus according to the present invention includes a detection unit that detects a feature amount related to a subject's state, an extraction unit that creates a frequency distribution of the feature amount detected by the detection unit, and extracts an extreme value of the frequency distribution. And an estimation means for estimating the state of the subject based on the temporal change of the extreme value of the frequency distribution extracted by the means.

  As described above, in the present invention, a feature amount related to the state of the subject is detected, a frequency distribution of the detected values is created, and an extreme value of the frequency distribution is extracted. Here, for example, when the consciousness (awakening) state of the subject changes, the frequency distribution of the feature amount related to the consciousness state of the subject changes, and the extreme value of the frequency distribution also changes accordingly. Therefore, the consciousness state of the subject can be directly estimated from the temporal change of the extreme value of the frequency distribution of the feature amount related to the consciousness state. Therefore, it is not necessary to set a threshold for determining the state of the subject by estimating the state of the subject based on the temporal change of the extreme value of the frequency distribution of the feature amount related to the state of the subject. It is possible to prevent erroneous estimation due to the setting error. This makes it possible to estimate the consciousness state of the subject person with high accuracy.

  In addition, the present invention is a state estimation device mounted on a vehicle, and includes a detection unit that detects a feature amount related to a driver's state, a frequency distribution of the feature amount detected by the detection unit, and the frequency distribution An extraction means for extracting an extreme value and an estimation means for estimating a driver's state based on a temporal change of the extreme value of the frequency distribution extracted by the extraction means are provided.

  As described above, in the present invention, the feature amount related to the driver's state is detected, the frequency distribution of the detected value is created, and the extreme value of the frequency distribution is extracted. Here, for example, when the driver's consciousness (awakening) state changes, the frequency distribution of the feature amount related to the driver's consciousness state changes, and the extreme value of the frequency distribution also changes accordingly. Therefore, the driver's consciousness state can be directly estimated from the temporal change of the extreme value of the frequency distribution of the feature amount related to the consciousness state. Therefore, it is not necessary to set a threshold for determining the driver's state by estimating the driver's state based on the temporal change of the extreme value of the frequency distribution of the feature amount related to the driver's state. It is possible to prevent erroneous estimation due to the setting error. This makes it possible to estimate the driver's consciousness state and the like with high accuracy.

  Preferably, the driver's state is the driver's arousal state, and the detection unit detects the degree of eye opening of the driver as a feature amount related to the driver's state. In this case, the extreme value of the frequency distribution of the driver's degree of eye opening is extracted, and the driver's arousal state is estimated based on the time change of this extreme value. It is not necessary to set a threshold value for identification.

  At this time, the extraction unit preferably creates a frequency distribution of the detection frequency of the eye opening degree. When the driver's consciousness decreases, the detection frequency of a state with a high degree of eye opening decreases, and the detection frequency of a state with a low degree of eye opening increases. Therefore, a driver's arousal state can be reliably estimated by creating a frequency distribution of the frequency of detection of the degree of eye opening and detecting temporal changes in the extreme values of the frequency distribution.

  Further, the extraction unit may create a frequency distribution of detection duration time of the eye opening degree. When the driver's consciousness decreases, the detection duration time in a state with a high degree of eye opening becomes shorter, and the detection duration time in a state with a low degree of eye opening becomes longer. Therefore, a driver's arousal state can be reliably estimated by creating a frequency distribution of the detection duration time of the eye opening degree and detecting a temporal change in the extreme value of the frequency distribution.

  Preferably, the vehicle further includes means for detecting the running state of the vehicle, and the extracting means creates a frequency distribution of the feature amount detected by the detecting means for each running state of the vehicle. As the traveling state (for example, speed) of the vehicle becomes constant, the driver's consciousness tends to decrease without continuing the tension of driving. Therefore, in order to estimate the driver's arousal state with higher accuracy, it is preferable to detect the time change of the extreme value of the frequency distribution for each traveling state of the vehicle.

  ADVANTAGE OF THE INVENTION According to this invention, the estimation precision of a subject's state can be improved. Thereby, for example, it becomes possible to effectively prevent a drowsy driving.

  DESCRIPTION OF EMBODIMENTS Hereinafter, a preferred embodiment of a state estimation device according to the present invention will be described in detail with reference to the drawings.

  FIG. 1 is a schematic configuration diagram showing a first embodiment of a state estimation apparatus according to the present invention. In the figure, the state estimation device 1 of the present embodiment is mounted on a vehicle such as an automobile. The state estimation device 1 includes a camera 2 that captures the driver's face, an estimation processing unit 3 that estimates a driver's consciousness state (wakefulness) based on an image captured by the camera 2, and the estimation processing unit 3. And an alarm device 4 that issues an alarm in accordance with the estimation result of. The camera 2 may be configured to image only a part including one eye of the driver (see FIG. 2), or may be configured to image the entire face including both eyes of the driver. good.

  The estimation processing unit 3 includes an image processing unit 5, an eye opening / closing eye extraction unit 6, and a consciousness state estimation unit 7.

  The image processing unit 5 performs image processing on the captured image of the camera 2 and detects the amount of eye opening (opening degree) of the driver. As shown in FIG. 2, the image processing unit 5 detects the maximum length P between the upper eyelid and the lower eyelid in the driver's eye as the number of pixels (pixels) as the driver's eye opening amount. To do. For example, the image processing unit 5 continuously detects the eye opening amount of the driver at a timing of a cycle of 100 ms. An example of the detection data obtained by the image processing unit 5 is shown in FIG. In FIG. 3, the horizontal axis represents the measurement time, and the vertical axis represents the detection raw value of the eye opening amount (the eye opening amount detection value).

  The eye opening / closing eye extraction unit 6 extracts the eye opening / closing state of the driver based on the detection data obtained by the image processing unit 5. The details of the processing procedure performed by the eye opening / closing eye extraction unit 6 are shown in FIG.

  In the figure, first, the eye opening amount detection values obtained by the image processing unit 5 are sequentially fetched (procedure 101). Then, the eye opening amount detection value for the latest predetermined time is stored in the memory (procedure 102). At this time, the latest data is sequentially updated every detection cycle, and the oldest data is deleted.

  Subsequently, the eye opening amount detection values (number of pixels) for a predetermined time stored in the memory are classified for each predetermined width (here, one pixel), and detection of each eye opening amount detection value for each of the classified predetermined widths is performed. The frequency is counted (procedure 103). Subsequently, a biaxial frequency distribution indicating the relationship between each eye-opening amount detection value for each predetermined width and the detection frequency is created, and envelope approximation is performed on the frequency distribution (procedure 104).

  FIG. 5 shows an example of the frequency distribution of the detection frequency of each eye opening amount detection value. In FIG. 5, the horizontal axis represents the detection frequency (number of times), and the vertical axis represents the eye opening amount detection value. A line Q in the figure indicates an envelope approximation curve obtained by performing envelope approximation on a bar graph-like frequency distribution.

  Subsequently, the maximum value (peak of the mountain) of the envelope approximation curve obtained in the procedure 104 is extracted (procedure 105). For example, in the envelope approximate curve Q shown in FIG. 5, the points a and b have maximum values.

  The consciousness state estimation unit 7 determines the driver's consciousness state based on the temporal change in the maximum value of the envelope approximation curve extracted by the open / closed eye extraction unit 6. The details of the processing procedure by the consciousness state estimation unit 7 are shown in FIG.

  In the figure, first, the maximum value data of the envelope approximation curve obtained every time the eye opening amount detection value is updated for a predetermined time is sequentially fetched (procedure 111). Then, it is determined from the degree of change in the maximum value of the envelope approximation curve over time whether or not the driver's consciousness is lowered, that is, whether or not the driver is sleepy (procedure 112).

  Specifically, as shown in FIG. 7 (a), the maximum value a of the distribution (open eye distribution) A having the larger eye opening amount is detected more frequently and the distribution (closed eye distribution) B having the smaller eye opening amount is detected. In a situation where the detection frequency of the maximum value b tends to be low, the driver's consciousness is determined to be in the normal state. Note that the closed eye distribution B as shown in FIG. 7A exists because the driver frequently blinks in the normal state. On the other hand, as shown in FIG. 7B, in the situation where the detection frequency of the maximum value a of the open eye distribution A is low and the detection frequency of the maximum value b of the closed eye distribution B tends to be high, the driver's consciousness Is determined to be in a state of falling asleep.

  And when it determines with a driver | operator's consciousness falling by the procedure 112, the alarm device 4 is controlled to generate an alarm from the alarm device 4 (procedure 113).

  As described above, in the present embodiment, the eye-opening amount detection values for a predetermined time are classified for each predetermined width, the frequency distribution of the detection frequency of each classified eye-opening amount detection value is created, and the envelope of the frequency distribution The maximum value of the approximate line is extracted, and the driver's consciousness state is estimated based on the time change of the maximum value of the envelope approximate line. Accordingly, since it is not necessary to set a threshold value for distinguishing between open / closed eyes and to perform binarization processing, it is possible to prevent deterioration of the estimation accuracy of the consciousness state due to a threshold setting error. Further, it is possible to accurately estimate the driver's consciousness state without using an expensive eye-opening / closing-eye sensor capable of high-speed image processing in order to detect a blink that occurs instantaneously. This makes it possible to estimate the driver's consciousness state with high accuracy and at low cost.

  In the first embodiment, a frequency distribution of the detection frequency of each eye opening amount detection value classified for each predetermined width is created, and the consciousness state is estimated using this frequency distribution. Instead of the frequency distribution of the detection frequency, a frequency distribution of the detection duration time of each eye opening amount detection value may be created. Hereinafter, the modification is demonstrated.

  FIG. 8 is a flowchart showing details of another processing procedure performed by the eye opening / closing eye extraction unit 6 of the estimation processing unit 3.

  In the figure, first, after executing the steps 101 and 102 shown in FIG. 4, the eye-opening amount detection values (number of pixels) for a predetermined time stored in the memory are classified for each predetermined width (here, one pixel), The detection duration time of each eye opening amount detection value for each classified predetermined width is measured (procedure 106). The detection duration time of the eye opening amount detection value is expressed as shown in FIG. 9, for example. Thereby, the change state of the detection duration time of the eye-opening amount detection value over time can be known.

  Subsequently, a triaxial frequency distribution indicating the relationship between each eye opening amount detection value for each classified predetermined width, the detection duration time of each eye opening amount detection value, and the frequency of the detection duration time is created. Envelope approximation is performed for the procedure (step 107). The envelope approximation curve obtained at this time is as shown in FIG. Then, the maximum value of the envelope approximation curve is extracted (procedure 108).

  Thereafter, the consciousness state estimation unit 7 determines the driver's consciousness state based on the temporal change of the maximum value of the envelope approximation curve obtained as described above.

Specifically, in the broken line R shown in FIG. 10, the detection duration time of the maximum value a ₁ of the distribution (eye opening distribution) A ₁ having the larger eye opening amount is longer and the frequency of the detection duration time is higher. The detection duration of the maximum value b ₁ of the distribution (closed eye distribution) B ₁ having the smaller eye opening amount is short, and the frequency of the detection duration is low. In this situation, it is determined that the driver's consciousness is in a normal state. Incidentally, there is a closed-eye distribution B ₁ represents, because performing frequent blinking driver eye under normal conditions.

On the other hand, the solid line S shown in FIG. 10, a short detection duration of the maximum value a ₂ of eye opening distribution A ₂ is and and lower the frequency of the detection duration, detection of eye closure distribution B ₂ maxima b ₂ The duration is long and the frequency of the detection duration is high. In this situation, it is determined that the driver's consciousness is lowered and sleepy.

  Therefore, also in the present modified embodiment, it is possible to estimate the driver's consciousness state with high accuracy and at low cost.

  FIG. 11: is a schematic block diagram which shows 2nd Embodiment of the state estimation apparatus concerning this invention. In the figure, the same or equivalent elements as those in the embodiment described above are denoted by the same reference numerals, and the description thereof is omitted.

  In the same figure, the state estimation apparatus 10 of this embodiment is provided with the vehicle speed sensor 11 which measures the traveling speed of a vehicle in addition to the camera 2, the estimation processing unit 3, and the alarm device 4 in the above embodiment. The estimation processing unit 3 includes an eye opening / closing eye extraction unit 12 and a consciousness state estimation unit 13 instead of the eye opening / closing eye extraction unit 6 and the consciousness state estimation unit 7 in the above embodiment.

  FIG. 12 is a flowchart showing details of a processing procedure performed by the eye opening / closing eye extraction unit 12. In the figure, first, the eye-opening amount detection value obtained by the image processing unit 5 and the measurement value (vehicle speed measurement value) of the vehicle speed sensor 11 are sequentially fetched (procedure 121). Then, the eye opening amount detection value for the latest predetermined time is stored in the memory together with the vehicle speed measurement value (step 122).

  Subsequently, the eye-opening amount detection values for a predetermined time stored in the memory are classified for each predetermined width, and the detection frequency of each eye-opening amount detection value for each classified predetermined width is counted (procedure 123). Subsequently, a biaxial frequency distribution indicating the relationship between each eye opening detection value for each predetermined width and the detection frequency is created for each predetermined vehicle speed range (for example, 20 km / h), and Envelope approximation is performed for each frequency distribution (procedure 124). Then, the maximum value of the envelope approximation curve in each vehicle speed range is extracted (procedure 125).

  FIG. 13 is a flowchart showing details of a processing procedure by the consciousness state estimation unit 13. In the figure, first, the maximum value data of the envelope approximation curve obtained every time the eye opening amount detection value for a predetermined time is updated for each vehicle speed range is sequentially fetched (procedure 131). Then, it is determined whether or not the driver's consciousness has decreased from the change in the maximum value of the envelope approximation curve over time for each vehicle speed range (procedure 132). The determination method of the driver's consciousness state is the same as the procedure 112 (shown in FIG. 7) shown in FIG. When it is determined that the driver's consciousness has decreased, the alarm device 4 is controlled to generate an alarm from the alarm device 4 (step 133).

  By the way, when the traveling speed of the vehicle is not constant, a certain degree of tension is maintained for the driver, so that the consciousness is hardly lowered, but for example, when the traveling speed of the vehicle is constant on an expressway This makes it difficult for the driver to maintain a sense of tension and causes a decrease in consciousness.

  In the present embodiment, a frequency distribution of the detection frequency of each eye opening amount detection value classified for each predetermined width is created for each vehicle speed range, and the maximum value of the envelope approximation curve of the frequency distribution is calculated for each vehicle speed range. Since the driver's consciousness state is estimated based on the time variation of the maximum value of the envelope approximation curve in the same vehicle speed range, the driver's consciousness state can be estimated with higher accuracy.

  In estimating the driver's consciousness state, the detection duration of each eye opening amount detection value is used instead of the frequency distribution of the detection frequency of each eye opening amount detection value, as in the modification of the first embodiment described above. The frequency distribution may be created. Hereinafter, the modification is demonstrated.

  FIG. 14 is a flowchart showing details of another processing procedure performed by the eye opening / closing eye extraction unit 12.

  In the figure, first, after executing the above steps 121 and 122, the eye opening amount detection values for a predetermined time stored in the memory are classified for each predetermined width, and each eye opening amount detection for each classified predetermined width is performed. The value detection duration is measured (procedure 126). Subsequently, a triaxial frequency distribution indicating the relationship between each eye opening amount detection value for each predetermined width, the detection duration time of each eye opening amount detection value, and the frequency of the detection duration time is divided into a predetermined vehicle speed range ( For example, every 20 km / h), and envelope approximation is performed on each frequency distribution for each vehicle speed range (procedure 127). Then, the maximum value of the envelope approximate curve in each vehicle speed range is extracted (procedure 128).

  Thereafter, the consciousness state estimation unit 13 determines the driver's consciousness state based on the time variation of the maximum value of the envelope approximation curve in the same vehicle speed range obtained as described above. This determination method is the same as that shown in FIG.

  Also in this modified embodiment, it is possible to estimate the driver's consciousness state with higher accuracy.

  Moreover, in said 2nd Embodiment and its modification, frequency distribution was created for every predetermined vehicle speed range, and the driver's consciousness state was estimated, but driving conditions other than the vehicle speed, for example, A frequency distribution may be created for each road type (for example, an expressway and a general road) or for each road curvature to estimate the driver's consciousness state.

  Although several preferred embodiments of the state estimation device according to the present invention have been described above, the present invention is not limited to the above embodiments. For example, although the above embodiment estimates the driver's consciousness state (wakefulness), the state estimation device of the present invention can also be applied to state estimation such as the driver's emotion and excitement level. . In this case, the driver's pulse rate, heart rate, or the like is detected as a feature amount related to the driver's state.

  Moreover, although the said embodiment sends the estimation result of the estimation process unit 3 to the alarm device 4, the state estimation apparatus of this invention can be used as a control input of various driving systems.

  Furthermore, although the said embodiment is mounted in a vehicle in order to estimate a driver | operator's state, the state estimation apparatus of this invention is applicable besides a vehicle-mounted thing.

It is a schematic block diagram which shows 1st Embodiment of the state estimation apparatus concerning this invention. It is a figure which shows an example of the site | part by which an image process is performed by the image process part shown in FIG. It is a graph which shows an example of the detection data obtained by the image processing part shown in FIG. It is a flowchart which shows the detail of the process sequence by the eye opening and closing eye extraction part shown in FIG. 3 is a graph showing an example of a frequency distribution of detection frequencies of each eye opening amount obtained by an eye opening / closing eye extraction unit shown in FIG. 1 and an envelope approximation curve. It is a flowchart which shows the detail of the process sequence by the consciousness state estimation part shown in FIG. It is a graph which shows the method of determining a driver | operator's consciousness state by the consciousness state estimation part shown in FIG. It is a flowchart which shows the detail of the other process sequence by the eye opening / closing eye extraction part shown in FIG. 3 is a graph illustrating an example of a detection duration time of an eye opening amount detection value obtained by an eye opening / closing eye extraction unit illustrated in FIG. 1. It is a graph which shows the other method of determining a driver | operator's consciousness state by the consciousness state estimation part shown in FIG. It is a schematic block diagram which shows 2nd Embodiment of the state estimation apparatus concerning this invention. It is a flowchart which shows the detail of the process sequence by the eye opening / closing eye extraction part shown in FIG. It is a flowchart which shows the detail of the process sequence by the consciousness state estimation part shown in FIG. 12 is a flowchart showing details of another processing procedure by the eye opening / closing eye extraction unit shown in FIG.

Explanation of symbols

DESCRIPTION OF SYMBOLS 1 ... State estimation apparatus, 2 ... Camera (detection means), 3 ... Estimation processing unit, 5 ... Image processing part (detection means), 6 ... Eye opening and closing eye extraction part (extraction means), 7 ... Consciousness state estimation part (estimation) Means), 10 ... state estimation device, 11 ... vehicle speed sensor, 12 ... open eye / closed eye extraction unit (extraction means), 13 ... conscious state estimation unit (estimation means).

Claims (6)

A detecting means for detecting a feature amount related to the state of the subject;
An extraction means for creating a frequency distribution of the feature amount detected by the detection means and extracting an extreme value of the frequency distribution;
A state estimation apparatus comprising: an estimation unit configured to estimate the state of the subject based on a temporal change of an extreme value of the frequency distribution extracted by the extraction unit. A state estimation device mounted on a vehicle,
Detecting means for detecting a characteristic amount related to the driver's condition;
An extraction means for creating a frequency distribution of the feature amount detected by the detection means and extracting an extreme value of the frequency distribution;
A state estimation device comprising: estimation means for estimating the state of the driver based on a temporal change of the extreme value of the frequency distribution extracted by the extraction means. The driver's state is the driver's arousal state,
The state estimation apparatus according to claim 2, wherein the detection unit detects the degree of eye opening of the driver as a feature amount related to the state of the driver.   The state estimation apparatus according to claim 3, wherein the extraction unit creates a frequency distribution of the detection frequency of the eye opening degree.   The state estimation apparatus according to claim 3, wherein the extraction unit creates a frequency distribution of the detection duration of the eye opening degree. Means for detecting a running state of the vehicle;
6. The state estimation apparatus according to claim 2, wherein the extraction unit creates a frequency distribution of the feature amount detected by the detection unit for each traveling state of the vehicle.

Images