JP2008186263A - Awakening degree calculation device - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To provide an awakening degree calculation device capable of improving the detection accuracy of sleepiness. <P>SOLUTION: The awakening degree calculation device sets a circadian variation in each driver and executes weight function calculation processing in which a weight function for correcting an awakening degree estimated on the basis of a photographed image shows a correspondence relation between a time since a mounted vehicle starts traveling and a correction coefficient and a calculation is made so that influence on an awakening degree due to low reliability of a reference value may be reduced until a second regulated time passes and so that influence exercised on a corrected awakening degree from the weight coefficient may be always constant after the second regulated time passes (S110). Then, an estimation index and a reference value are set (S150), and an awakening degree is calculated according to the estimation index and the reference value (S160). A corrected awakening degree obtained by multiplying the calculated awakening degree by the correction coefficient at the present time is calculated (S170), and warning processing based on the corrected awakening degree is executed (S180). <P>COPYRIGHT: (C)2008,JPO&INPIT

Description

  The present invention relates to an arousal level calculation device that calculates the awakening level of a vehicle driver.

  Conventionally, an image capturing unit that captures a photographed image including a driver's face, and a driver's eyelid movement (that is, a blinking state) is detected based on the photographed image photographed by the photographing unit, and the blinking state is detected. An arousal level calculating device comprising: an image processing processor that estimates a driver's arousal level based on the above; and a warning unit that issues a warning if the arousal level estimated by the image processing processor is equal to or less than a predetermined threshold value It is known (see, for example, Patent Document 1).

In this type of wakefulness calculation device, opening information indicating an opening degree of the driver's eyes derived within a predetermined time from the start of driving (an example of a blink state, that is, an upper eyelid and a lower eyelid) Are sequentially calculated, and a predetermined number is extracted in descending order of the distance. Then, a reference value (that is, an indication that the driver is in an awake state) is set according to the extracted opening degree information, and the opening degree information calculated sequentially is equal to or less than a predetermined ratio from the reference value. The degree of arousal is estimated according to the time indicating that it is.
JP-A-7-156682

  By the way, when driving a vehicle, at the initial stage of the driving (that is, within a few minutes after starting the engine), the driver looks at the meter to check the state of the vehicle or operates the air conditioner. Look at the control panel (such as looking at the control panel). Is likely to be viewed from below).

  And when the driver is looking down, etc., the driver's eyes captured in the captured image are compared to the case where the driver is in an awake state but is looking toward the gaze direction. The eye opening degree derived from the captured image is also small.

  For this reason, at the initial stage of driving, there is a possibility that the opening degree information extracted for setting the reference value includes a lot of information detected when the driver is looking down. As a result, there is a high possibility that the reference value indicating that the driver is awake is not set correctly. In order to improve the reliability of the reference value, it is necessary to extract the opening degree information used for setting the reference value from a lot of opening degree information detected during a longer time. To be precise, it took time to set the reference value.

  Therefore, in the conventional arousal level calculation device, drowsiness is generated in the driver (that is, the awakening level decreases) even though the driver is in the awake state during the period until the correct reference value is set. The possibility of misrecognizing that

Further, the conventional wakefulness calculation device has a problem that even if the time is long, the calculation result of the wakefulness in the initial stage of driving varies, and the drowsiness detection accuracy decreases.
Therefore, an object of the present invention is to provide a wakefulness calculation device capable of improving the drowsiness detection accuracy.

  The wakefulness calculation device of the present invention made to achieve the above object is based on the captured image in which the imaging unit captures a captured image including the face of the driver of the vehicle, and the index calculation unit is captured by the imaging unit. While repeatedly calculating at least one kind of estimated index used to estimate the driver's arousal level, the reference value setting means is sequentially calculated by the index calculating means during a predetermined time specified from the start of driving the vehicle. And a reference value of the estimated index is set based on the estimated index that satisfies a condition that is defined in advance as indicating that the driver is awake. Then, the wakefulness estimation means estimates the driver's wakefulness based on the estimated index sequentially calculated by the index calculation means and the reference value set by the reference value setting means, and the wakefulness correction means The reference value that is used while being updated sequentially by the degree estimation means by the specified time is less reliable than the reference value that is used after the specified time, thereby reducing the impact on the alertness estimated by the alertness estimation means As described above, the corrected arousal level obtained by correcting the arousal level is calculated according to a correction function set in advance for each driver. However, the specified time is a sufficient length for setting a highly reliable reference value.

  That is, the reference value of the estimated index is set based on the estimated index that is sequentially calculated by the index calculating unit during the specified time and satisfies the condition that is defined in advance as indicating that the driver is in an awake state. (In other words, an estimated index used to set a reference value by setting a reference value by extracting a condition that satisfies a condition from among a greater number of awakening indexes by increasing the time until the reference value is set. In the wakefulness calculation device of the present invention, the start of driving for which a highly reliable reference value is not set). The corrected arousal level is calculated so as to suppress the influence on the arousal level estimated during the period from to.

  Therefore, according to the device for calculating the arousal level of the present invention, the period immediately after the start of driving with low reliability of the reference value (for example, because the period for taking a captured image is short, the number of arousal indices is small and the reference value is uncertain). In the case where there is a high possibility that the driver is awake, it is possible to reduce the possibility of misrecognizing that the driver is drowsy even though the driver is awake. That is, according to the wakefulness calculation apparatus of the present invention, the drowsiness detection accuracy can be improved.

  The wakefulness level correcting means in the wakefulness level calculating device of the present invention, as described in claim 2, shows a correspondence relationship between the time from the start of driving of the vehicle and the correction amount to be corrected, and a prescribed time. The correction amount is constant after a predetermined determination time has elapsed as a longer time, and correction function calculation means for calculating a correction function that increases monotonically until the determination time elapses is provided. Is desirable.

  That is, the corrected arousal level calculated by such an arousal level calculation device is always affected by the correction function after the determination time has elapsed, and during the period from the start of driving until the determination time has elapsed, The shorter the time from the start (that is, the lower the reliability of the reference value), the greater the influence from the correction function.

  Therefore, according to such a wakefulness calculation device, in order to calculate a corrected wakefulness that is likely to indicate that the driver is in a wakeful state as the time from the start of driving is short, immediately after the start of driving, It is possible to detect drowsiness in accordance with the driver's feeling that many drivers feel awake.

  By the way, normally, when the driver is driving the vehicle at a high speed (that is, a specified vehicle speed or more), the driver's face may be fixed within a specific area, and the driver's line-of-sight direction may be constant. Is expensive.

  For this reason, as described in claim 3, the awakening level calculation device of the present invention includes a traveling state acquisition unit that acquires high-speed traveling information indicating that the vehicle is traveling at a predetermined vehicle speed or higher. The correction function calculation means in the wakefulness calculation device of the present invention is determined from the start of driving when the high speed driving information is acquired by the driving state acquisition means compared to when the high speed driving information is not acquired. You may comprise so that the correction function which shortened the time to time may be calculated.

  In other words, when driving at high speed, the time required to calculate a reliable reference value is reduced compared to when the vehicle is driven at low speed and the position of the driver's face and the direction of the line of sight change. The time from the start of operation to the determination time may be shortened.

  In particular, the correction function calculation means in the wakefulness calculation device according to the present invention, as set forth in claim 4, is preset for each driver and based on circadian rhythm information indicating the sleep cycle of the day, It is desirable to be configured to set a decision time.

  According to such a wakefulness calculation apparatus of the present invention, by setting the determination time according to circadian rhythm information (circular rhythm, i.e., driver's biological rhythm) indicating the driver's daily sleep cycle, It is possible to prevent the corrected arousal level from being influenced by the correction function for an excessively long time. For example, when the circadian rhythm information indicates that the driver is in an awake state when the circadian rhythm information indicates a low period of low arousal level (that is, a time zone in which the driver feels sleepy) (that is, Compared to the time zone that indicates that the degree of arousal is high), if the driver feels drowsy even though the driver is drowsy by shortening the decision time, the awakening degree It is possible to prevent the calculation device from making a determination.

  Furthermore, according to the arousal level calculation apparatus of the present invention, the correction function is calculated according to the circadian rhythm information set for each driver, and therefore the drowsiness detection accuracy varies due to individual differences for each driver. It can be prevented from occurring.

  In addition, the correction function calculation means in the arousal level calculation apparatus of the present invention acquires biological information including at least the heart rate of the driver as described in claim 5, and based on the biological information, the circadian rhythm is obtained. A circadian rhythm information generating means for generating information may be provided.

  According to the awakening level calculation apparatus of the present invention, the correction function is set in accordance with the circadian rhythm information of the driver at the actual driving time in order to generate the circadian rhythm information based on the acquired biological information. It is possible to calculate and correct the arousal level. As a result, according to the arousal level calculating apparatus of the present invention, the drowsiness detection accuracy can be further improved.

  According to the sixth aspect of the present invention, the circadian rhythm information generating means in the arousal level calculating device of the present invention varies in proportion to the heart rate of the driver, and indicates an arousal index that indicates the level of the driver's arousal level. It is desirable that the correspondence relationship with time be generated as circadian rhythm information.

  When the biological information includes at least one of heart rate variability, body movement, and sleep time, as described in claim 7, the circadian rhythm information generating means in the arousal level calculating device of the present invention is Circadian rhythm information is corrected circadian rhythm information that corrects the correspondence between time and wakefulness index according to the driver's sleep quality estimated based on at least one of heart rate variability, body movement, and sleep time. It may be configured to generate.

  However, the correction of the correspondence relationship referred to here includes shifting the time of the sleep cycle according to the bedtime, shifting the size of the awakening index according to the depth of sleep, and the like. And bedtime and the depth of sleep are estimated from heart rate variability, body movement, and sleep time by a known technique.

  By the way, in the wakefulness calculating apparatus of the present invention, as described in claim 8, when the corrected wakefulness calculated by the wakefulness correcting means is equal to or less than a predetermined threshold value, warning means for issuing a warning It is desirable to be provided.

  According to the wakefulness calculating apparatus of the present invention configured as described above, the driver recognizes that the wakefulness is lowered (i.e., drowsiness is generated) by issuing a warning. In addition, the driver can be made to take actions necessary to increase the arousal level.

  Furthermore, when the correction function indicates a correspondence relationship between the time from the start of driving and the correction amount corrected by the arousal level correction unit, the warning unit in the arousal level calculation device of the present invention is described in claim 9. As described above, if the correction amount at a specific time that is derived from the correction function and is preset as the time from the start of driving of the vehicle is equal to or greater than the preset first threshold, the value of the specified threshold is increased. It may be configured as follows.

  That is, in the wakefulness calculating apparatus according to claim 9, when the correction amount derived at the specific time is equal to or greater than the first threshold (that is, under a situation where the wakefulness is considered to be relatively high immediately after the start of driving). Then, after increasing the value of the prescribed threshold, which is a criterion for determining whether or not to issue a warning, if the corrected awakening level is equal to or less than the prescribed threshold, a warning is issued.

  Therefore, according to the wakefulness calculation device of the present invention configured as described above, under a situation that is not suitable for the driver's sense (for example, under a situation where the wakefulness is relatively high, such as immediately after the start of driving), A warning can be prevented from being issued due to misrecognition of sleepiness, an unnecessary warning can be issued, and a driver can be prevented from feeling uncomfortable.

  Further, when the correction function indicates a correspondence relationship between the time from the start of driving and the correction amount corrected by the wakefulness level correcting unit, the warning unit in the wakefulness level calculating device according to the present invention is described in claim 10. As described above, when the correction amount at a specific time that is derived from the correction function and set in advance as the time from the start of driving of the vehicle is equal to or larger than the preset second threshold value, it is less than the second threshold value. Compared to the case, it may be configured to allow a stronger warning to be issued.

  That is, in the wakefulness calculation device of the present invention configured as described above, a reference value that is reliable to some extent is set and calculated when the correction amount derived at the specific time is equal to or greater than the second threshold value. A stronger warning is issued as the reliability of the awakening level is high.

  However, the stronger warning mentioned here is to emit louder sound from the speaker or to blow cooler cold air to the driver, etc., to increase the arousal level of the driver who is in a low arousal state. It is a treatment.

  On the other hand, when the circadian rhythm information indicates a correspondence relationship between time and a wakefulness index indicating the level of the driver's wakefulness, the warning means in the wakefulness calculating device of the present invention is described in claim 11. As shown in the circadian rhythm information, if the wakefulness index at a specific time preset as the time from the start of driving indicates that the driver is in an awakening state, the specified threshold value is increased. You may be comprised so that it may make.

  According to the wakefulness calculation apparatus of the present invention configured as described above, the value of the prescribed threshold increases depending on the level of the wakefulness index (that is, the sleep cycle of the day) indicated in the circadian rhythm information. Even if the person does not feel sleepy, it is possible to prevent a warning from being issued due to misrecognition of sleepiness. Therefore, according to the arousal level calculation apparatus of the present invention, it is possible to prevent an unnecessary warning from being issued and the driver from feeling uncomfortable.

  Furthermore, when the circadian rhythm information indicates a correspondence relationship between time and a wakefulness index indicating the level of the driver's wakefulness, the warning means in the wakefulness calculating device according to the present invention is described in claim 12. If the wakefulness index at a specific time that is derived from circadian rhythm information and set in advance as the time from the start of driving indicates that the driver is in a low wakefulness state that makes the driver feel sleepy, It may be configured to allow a stronger warning to be issued as compared to a case where indicates that the state is awake.

  According to the wakefulness calculation device of the present invention configured as described above, when the wakefulness index at a specific time indicates that the driver is in a low wakefulness state, the driver is in the wakefulness state. It is possible to issue a stronger warning than when the symbol is represented. As a result, according to the wakefulness calculation device of the present invention, the wakefulness of the driver in the low wakefulness state can be more reliably increased.

  However, the stronger warning mentioned here is to emit louder sound from the speaker or to blow cooler cold air to the driver, etc., to increase the arousal level of the driver who is in a low arousal state. It is a treatment.

  According to the wakefulness calculation apparatus of the present invention, as described in claim 13, the acceleration acquisition unit acquires the acceleration of the vehicle, and the acceleration acquired by the acceleration acquisition unit determines the operation of the accelerator or the brake. When represented, the prohibiting means may be configured to prohibit the warning means from issuing a warning.

  In the wakefulness calculation apparatus according to the present invention, as a condition for the prohibiting means to prohibit the warning means from issuing a warning, the operation amount of the brake pedal is defined in advance as described in claim 14. It may be a case where the value is equal to or greater than the value, or a case where the operation amount of the accelerator pedal is equal to or greater than a predetermined value as defined in claim 15. However, in the former case, the brake operation amount acquisition means for acquiring the operation amount of the brake pedal provided in the vehicle, in the latter case, the accelerator operation amount acquisition means for acquiring the operation amount of the accelerator pedal provided in the vehicle, It is necessary to be provided in the arousal level calculation apparatus of the present invention.

  And according to these arousal level calculation devices of the present invention, a warning is not issued when the driver is operating the accelerator or the brake. Can prevent a person from feeling annoyed.

Embodiments of the present invention will be described below with reference to the drawings.
First, FIG. 1 is a block diagram showing a schematic configuration of a wakefulness calculating apparatus mounted on a vehicle and to which the present invention is applied. In the following, a vehicle equipped with an arousal level calculation device is referred to as an on-board vehicle.
<Awakening level calculation device>
The arousal level calculation device 1 includes a light source 11 that irradiates an irradiation area including a driver's face of a mounted vehicle, and a camera 12 that captures an image including the driver's face irradiated with the irradiation light from the light source 11. And information acquisition device group 15 for acquiring information such as biological information including the heart rate of the driver and acceleration of the mounted vehicle, and a circadian rhythm (that is, a circadian rhythm, which represents the driver's daily sleep cycle) The driver's awakening based on the database 20 storing biological information necessary for deriving the daily fluctuation), the image captured by the camera 12, and the information acquired by the information acquisition device group 15 An arousal level calculating ECU 10 that calculates the degree of sound, a speaker 13 that outputs a sound based on an audio signal from the arousal level calculating ECU 10, and a monitor 14 that outputs a video based on a video signal from the awakening level calculating ECU 10. It is provided.

  The light source 11 is configured around a light emitting diode that emits light in accordance with a light emission signal from the arousal level calculation ECU 10, and irradiates irradiation light with a pre-defined spread angle. And the light source 11 is arrange | positioned on the extension line of the steering column in a steering wheel so that irradiation light may be irradiated to the driver | operator's face of a mounted vehicle.

  The camera 12 has a light receiving unit composed of a CCD element. The light receiving unit receives light through a lens in accordance with an imaging signal from the arousal level calculation ECU 10 to capture a digital image (hereinafter, the captured digital image is captured). A photographed image). And the camera 12 is arrange | positioned on the instrument panel of a mounting vehicle so that the imaging | photography area | region containing a driver | operator's face may be image | photographed.

  Furthermore, the information acquisition device group 15 detects a heart rate detection sensor 16 that detects biological information including the heart rate of the driver, a time measuring unit 17 that measures time, a current position of the mounted vehicle, and a traveling direction of the mounted vehicle. The navigation apparatus 18 which performs, and the acceleration sensor 19 which detects the acceleration of a mounting vehicle are comprised.

  The heart rate detection sensor 16 is a well-known optical volume plethysmometer that optically detects a change in blood vessel volume (that is, a pulse or a pulse wave). A detection unit that detects a pulse (pulse wave), a heart rate of the driver and a heart rate fluctuation calculated from the pulse (pulse wave) detected by the detection unit, a heart rate calculated by the calculation unit, and A communication unit that communicates data between the wakefulness calculation ECU 10 and a storage unit that stores the heart rate variability in association with the time at which they were detected and sequentially updates them for a predetermined number of times of detection. And. However, the heart rate detection sensor 16 in this embodiment is of a wristwatch type that is always worn on the wrist of the driver.

  When the heartbeat detection sensor 16 receives a detection signal for detecting the heart rate from the arousal level calculation ECU 10, the heartbeat detection sensor 16 detects the pulse (pulse wave) of the driver at that time and detects the detected pulse (pulse wave). ) To calculate the heart rate and heart rate variability of the driver, and output the heart rate and heart rate variability to the arousal level calculating ECU 10. Hereinafter, the correspondence between the driver's heart rate and heart rate variability and time is referred to as heart rate information.

  The navigation device 18 is a well-known device composed of a position detector (not shown), a map data input device, a group of operation switches, a display, a speaker, and the like, and displays a map near the current position and displays a vehicle in the map. Various processes such as a current position display process for superimposing the current position mark and a route guidance process for performing route guidance can be executed. The navigation device 18 is configured to output current position information indicating whether or not the position where the on-board vehicle is currently traveling is an expressway to the awakening degree calculation ECU 10.

  The database 20 is a storage device for storing various data necessary for deriving the daily fluctuation set for each driver, and includes the daily heartbeat information of the driver detected by the heartbeat detection sensor 16. Based on the daily heartbeat information of the driver, the daily fluctuation (hereinafter referred to as the reference daily fluctuation) set in advance as a reference for the driver's sleep cycle is stored. The daily fluctuation indicates a correspondence relationship between a wakefulness index that changes in proportion to the heart rate of the driver and represents the level of wakefulness and time.

  Next, the arousal level calculation ECU 10 includes a program for executing various processes, a ROM 10a that stores data, a RAM 10b that temporarily stores data, a CPU 10c that executes processes according to the program stored in the ROM 10a, and the like. It has.

  It should be noted that the ROM 10a is used for extracting a face for simulating a human eye, nose, mouth, and the like as a template necessary for executing various processes on the photographed image and extracting the driver's face from the photographed image. A template, an eye extraction template for simulating a human eye and extracting a driver's eye from a photographed image, and the like are stored.

That is, the arousal level calculation ECU 10 corrects the driver's arousal level estimated based on the captured image captured by the camera 12 in accordance with the weight function derived based on the daily fluctuation, and the corrected arousal level (hereinafter referred to as the arousal level) The sleepiness warning process for issuing a warning to the driver can be executed based on the corrected awakening level.
<Drowsiness warning processing>
Next, drowsiness warning processing executed by the CPU 10c of the wakefulness calculation ECU 10 will be described.

Here, FIG. 2 is a flowchart showing a processing procedure of sleepiness warning processing.
This drowsiness warning process is started when the operation of the mounted vehicle is started (in the present embodiment, when the engine is started).

  When the drowsiness warning process is started, first, in S110, a daily function for each driver is set, and a weight function for calculating a weight function for correcting the arousal level estimated based on the photographed image is calculated. Execute the calculation process. However, the weighting function indicates a correspondence relationship between the time since the on-board vehicle has started driving and the correction coefficient (that is, the correction amount when correcting the arousal level).

  In subsequent S120, a light emission signal is output to the light source 11, and an image pickup signal is output to the camera 12, causing the camera 12 to capture a captured image, and the captured captured image is acquired.

  Furthermore, in S130, based on the photographed image acquired from the camera 12, the face area where the driver's face is photographed is extracted. Specifically, in the present embodiment, pattern matching using the face extraction template is performed using the entire area of the acquired captured image as the search area, and the area with the highest matching rate with the face extraction template is the face area. Extract as

  In subsequent S140, the peripheral area of the eye including the upper and lower eyelids of the driver is extracted from the face area extracted in S130. Specifically, in the present embodiment, pattern matching using an eye extraction template is executed using the face area as an exploration area, and the area having the highest matching rate with the eye extraction template is recognized as the driver's eyes. And the area | region of the magnitude | size prescribed | regulated previously so that the area | region recognized as the driver | operator's eyes may be included is extracted as a periphery area | region of eyes.

  In S150, at least one kind of estimated index, which is an index for estimating the driver's arousal level, is derived based on the peripheral area of the eye extracted in S140. Further, if the time acquired from the time measuring unit 17 is within the first specified time that is specified in advance as the time from the start of driving, it represents that the driver is in an awake state for each estimated index derived. The estimated index that is most likely to be set is set as the reference value.

  Specifically, in the present embodiment, as shown in FIG. 5A, as one of the estimation indexes, every time the eye peripheral area is extracted in S140, the eye peripheral area (that is, in the captured image). ) And the upper eyelid (ie, the highest point of the upper eyelid (position closest to the top of the head)) and the lower eyelid (ie, the lowest point of the lower eyelid (position closest to the road surface)) ) (That is, the opening degree of the driver's eyes, hereinafter referred to as opening degree information).

  Then, the calculated opening degree information is stored in the RAM 10b together with the elapsed time from the start of operation. The relationship between the opening degree information and the elapsed time from the start of operation (hereinafter referred to as blink detection relationship) is represented as a graph as shown in FIG.

  However, if the time from the start of operation is within the first specified time (in this embodiment, 10 minutes), the opening information calculated within the first specified time is specified in advance in the order of distance. The average of the specified number is stored in the RAM 10b as opening information when the driver is in an awake state (that is, the maximum value of the opening information), that is, as a reference value. That is, if it is within a specified time from the start of driving, a predetermined number of pieces of opening information that are sequentially calculated with a large distance are extracted, and a reference value for estimating the arousal level is calculated.

  Therefore, in the drowsiness warning process of the present embodiment, as the time from the start of driving elapses, more opening degree information indicating the awake state is calculated, and the condition is determined from among the much opening degree information. In order to obtain the reference value by extracting the opening degree information to be satisfied, a highly reliable reference value is set. That is, in the sleepiness warning process of the present embodiment, a highly reliable reference value is set after the first specified time has elapsed.

Further, in S160, the driver's arousal level is calculated based on the estimated index derived in S150.
Specifically, in the present embodiment, a ratio in which opening degree information (that is, an estimated index) is derived based on the blink detection relationship and compared with a reference value is defined in advance (for example, 20% in the present embodiment). The arousal level is calculated according to the following time (that is, the time when the eyes are closed, hereinafter referred to as blinking time). In this embodiment, when the blink time is short, it is assumed that the driver is awake, and the awakening degree is calculated so that the value is large.When the blink time is long, the driver is sleepy. As a feeling, the arousal level is calculated so that the value becomes low.

In subsequent S170, a corrected arousal level obtained by correcting the awakening level calculated in the previous S160 according to the weight function calculated in S110 is calculated.
Specifically, in the present embodiment, as shown in equation (1), the corrected arousal level is calculated by multiplying the awakening level calculated in S160 by the correction coefficient at the current time, which is derived from the weight function. To do.

Corrected arousal level = Arousal level × correction amount (1)
In S180, a warning process is performed in which a warning sound is emitted from the speaker 13 and a warning for the driver is displayed on the monitor 14 in accordance with the corrected awakening level calculated in S170.

Thereafter, the process returns to S120.
<Weight function calculation process>
Next, the weight function calculation process executed in the previous sleepiness warning process will be described.

Here, FIG. 3 is a flowchart showing the processing procedure of the weight function calculation processing.
When this weighting function calculation process is activated, first, in S210, a detection signal is transmitted to the heart rate detection sensor 16, and the heart rate of the driver at the current time (hereinafter referred to as the current heart rate) is set as the heart rate. The detection sensor 16 detects the current heart rate from the heart rate detection sensor 16. Furthermore, after acquiring the heart rate information stored in the heart rate detection sensor 16, the heart rate information is stored in the database 20.

  In subsequent S220, based on the current heart rate acquired in S210 and the reference daily fluctuation stored in the database 20, the daily fluctuation of the day on which the weight function calculation process was executed (that is, the day on which driving was performed) ( Hereafter, it will be referred to as fluctuation within the use day)

  Specifically, in the present embodiment, as shown in FIG. 6, it is used that the heart rate at each detection time acquired in S <b> 210 and represented as heart rate information is proportional to the driver's arousal level. Then, the time change of the arousal index (that is, the index indicating the driver's arousal level) obtained in this way is set as the daily fluctuation. That is, the daily fluctuation set in S220 is referred to so that the high / low cycle of the wakefulness index is the same as the high / low cycle of the wakefulness index indicated in the reference daily fluctuation stored in the database 20. , The current heart rate acquired in S210 is lower than the heart rate (hereinafter referred to as the reference heart rate) used when calculating the level of the arousal index (hereinafter referred to as the reference awakening index) indicated in the reference diurnal variation. For example, it indicates that the driver's arousal level is lower than the reference awakening index, and if the current heart rate is higher than the reference heart rate, the driver's arousal level is higher than the reference awakening index. (To be more precise, the standard daily fluctuation is corrected).

  In S230, based on the daily fluctuation obtained in S220 and the current heart rate acquired in S210, the time from the start of driving is the first specified time or more, and after that time, the magnitude of the correction coefficient The second specified time is set so that is constant.

Here, FIG. 7 and FIG. 8 are explanatory diagrams for explaining a method of calculating the weight function.
In the present embodiment, when the fluctuation within the use day as shown in FIG. 7A is obtained in S220, the awakening index (α in the figure) at the driving start time (ts in the figure) is the driver. If the driver's arousal level is low, the second specified time is set so that the time until the correction coefficient becomes constant is shorter than when the driver's arousal level is high. ing. That is, as shown in FIG. 8, when the mounted vehicle is driven during a time zone with a low arousal level (that is, during a low period), the mounted vehicle is driven during a time zone with a high arousal level. It is set as the time from the start of operation so as to be shorter than when the operation is performed.

In subsequent S240, the weighting function is determined according to the second specified time set in S230.
Specifically, in the present embodiment, as shown in FIG. 7B, until the second specified time (t1 in the figure) elapses from the start of operation, the reference set in S150 and used in S160. After the second specified time has elapsed, the weight coefficient is monotonically increased so that the influence on the arousal level due to the value being less reliable than the reference value used in S160 after the first specified time is reduced. The weighting function is set so that the weighting coefficient is constant at a maximum value (β in the figure, in the embodiment, the magnitude is 1).

Then, the process proceeds to S120 of the drowsiness warning process.
In other words, the corrected arousal level calculated using the weight function set in the weight function calculation process is the reference value during the period from the start of driving to the second specified time (hereinafter referred to as driving initial). The influence on the arousal level due to low reliability is reduced. Then, after the second specified time has elapsed and a highly reliable reference value has been set, the influence of the corrected arousal level from the weight function (more precisely, the weight coefficient) is always constant.
<Warning>
Next, a warning process executed in the drowsiness warning process will be described.

Here, FIG. 4 is a flowchart showing a processing procedure of warning processing.
When the warning process is executed in the drowsiness warning process, first, in S310, the corrected arousal level calculated in the previous S170 is greater than a predetermined threshold that is defined in advance as a level of arousal level in which humans feel drowsy through experiments or the like. It is determined whether or not it is small. And as a result of determination, when it is determined that the corrected arousal level is smaller than the prescribed threshold value, the process proceeds to S320.

  In S320, it is determined whether or not the accelerator pedal or the brake pedal is operated. Specifically, in the present embodiment, when the acceleration of the mounted vehicle detected by the acceleration sensor 19 is equal to or greater than a predetermined threshold value in the negative direction, it is determined that the brake pedal is operated, If it is greater than or equal to a predetermined threshold value in the positive direction, it is determined that the accelerator pedal is being operated.

If it is determined that the accelerator pedal or the brake pedal is not operated as a result of the determination, the process proceeds to S330.
Further, in S330, as shown in FIG. 9, it is determined whether or not the current correction coefficient is derived from the weight function calculated in the weight function calculation process and is equal to or greater than a preset threshold value. . If the correction coefficient is equal to or greater than the set threshold as a result of the determination, the process proceeds to S350 while maintaining the output of the predetermined audio signal so that the audio output from the speaker 13 has a predetermined level. .

  On the other hand, if the result of determination in S330 is that the correction coefficient is less than the set threshold, the audio signal is set so that the volume output from the speaker 13 is larger than a predetermined volume, and then S350 is set. Proceed to

  In S350, the currently set audio signal is output to the speaker 13, a warning sound is generated from the speaker 13, a video signal is transmitted to the monitor 14, and the warning information is displayed on the monitor 14.

Thereafter, the process returns to the drowsiness warning process.
As a result of the determination in S310, when it is determined that the corrected arousal level is greater than or equal to the predetermined threshold, or when it is determined that the accelerator pedal or the brake pedal is operated as a result of the determination in S320. Returns to the drowsiness warning process without issuing a warning to the driver.

In this embodiment, the camera 12 is an imaging unit, S150 is an index calculation unit, a reference value setting unit, S160 is an arousal level estimation unit, S170 is an awakening level correction unit, a weight function calculation process is a weight function calculation unit, a warning Processing corresponds to warning means.
[Effect of the embodiment]
As described above, in the arousal level calculation apparatus 1 according to the above-described embodiment, the influence of the reference value on the arousal level is suppressed at the initial stage of driving where a highly reliable reference value is not set. A correction function is calculated, and a corrected arousal level is calculated according to the correction function.

  Therefore, according to the arousal level calculation device 1 of the above embodiment, it is erroneously determined that sleepiness has occurred in the driver even though the driver is in the awake state in the period immediately after the start of driving with low reliability of the reference value. The possibility of recognition can be reduced. That is, according to the wakefulness calculation device of the present invention, it is possible to improve the drowsiness detection accuracy in the initial driving.

  Further, in the arousal level calculating device 1, in the period from the start of driving until the second specified time elapses, the shorter the time from the start of driving (that is, the lower the reliability of the reference value), the more affected by the correction function. The weight function is calculated so that becomes large.

  Therefore, according to the arousal level calculating device 1, the shorter the time from the start of driving, the higher the corrected awakening level that is likely to indicate that the driver is in the awake state. The drowsiness can be detected in accordance with the driver's feeling that the driver feels awake.

  Further, according to the arousal level calculation device 1, when the fluctuation in use day indicates that the period is low, the second prescribed time is shortened so that the driver feels drowsy although the driver feels drowsy. Can be determined not to feel sleepy.

  Furthermore, according to the arousal level calculation device 1, since the in-use fluctuation is set for each driver, it is possible to prevent the drowsiness detection accuracy from being varied due to individual differences for each driver. Furthermore, since the driver sets the daily fluctuation in use according to the heart rate when the driver starts driving (that is, the deviation of the daily fluctuation due to the individual difference of the driver is corrected), the driver It is possible to prevent variations in drowsiness detection accuracy due to the physical condition of the patient.

  In addition, according to the arousal level calculation device 1, by issuing a warning, it is possible to make the driver recognize that the awakening level has been lowered (that is, drowsiness has occurred). The driver can take actions necessary to ascend.

And according to the arousal level calculation device 1, since the warning is not issued when the driver is operating the accelerator or the brake, the warning is issued at the time of the operation, and the driver is bothered. Can prevent you from feeling.
[Other Embodiments]
As mentioned above, although embodiment of this invention was described, this invention is not limited to the said embodiment, In the range which does not deviate from the summary of this invention, it is possible to implement in various aspects.

  For example, in the correction of the arousal level in the arousal level calculation apparatus 1 of the above embodiment, the corrected arousal level is calculated by multiplying the arousal level estimated in S170 of the drowsiness warning process by a correction amount (that is, a correction coefficient). However, the corrected awakening level may be calculated by adding a correction amount to the awakening level estimated in S170.

  Further, in the weighting function calculation process in the wakefulness calculation device 1 of the above-described embodiment, if the in-use fluctuation is a time period indicating that it is in a low period (that is, the low wakefulness state), the wakefulness state is determined. The second specified time was set so as to shorten the time from the start of operation compared to the time zone shown, but instead of this, or in addition to this, the onboard vehicle is traveling on the expressway When the vehicle is running, the second specified time may be set so that the time from the start of driving is shorter than when driving on a normal road.

  That is, as shown in FIG. 10A, the awakening degree calculation ECU 10 obtains information obtained from current position information acquired from the navigation device 18, speed information related to the vehicle speed of the mounted vehicle acquired from a vehicle speed sensor (not shown), and the like. However, when the on-board vehicle is traveling continuously on a highway, the time from the start of driving is greater than when it is traveling intermittently on a general road. The second specified time may be set to be shorter.

  Further, in the weighting function calculation process in the arousal level calculation device 1 of the above-described embodiment, the fluctuation within the day of use is obtained according to the current heart rate of the driver detected by the heart rate detection sensor 16, but in addition to this, the driver Depending on the sleep quality estimated from the heartbeat variability, the intraday use variation may be corrected and used for calculating the weight function.

  That is, as shown in FIG. 10B, the wakefulness calculation ECU 10 indicates that the temporal change of the heart rate variability included in the heart rate information acquired from the heart rate detection sensor 16 indicates that the driver's sleep is shallow (sleep quality is low). If it represents a bad thing), the wakefulness index indicated by the in-use fluctuation set in S220 of the weight function calculation process may be corrected so that the wakefulness level is lowered.

  In addition, as a method of correcting daily fluctuations according to sleep quality, as shown in FIG. 11A, the driver is estimated from the temporal change of heartbeat fluctuations included in the heartbeat information acquired from the heartbeat detection sensor 16. Depending on the bedtime, the high and low periods of the wakefulness index may be changed (that is, only the time is shifted while maintaining the period). And as a method of estimating the quality of sleep, in addition to the method of estimating from the heart rate variability, a method of estimating from the body movement of the driver during sleep, the sleep time of the driver, or the like may be used. Note that these estimation methods use known techniques.

  Further, in the warning process of the above embodiment, the intensity of the warning is changed according to the current weighting factor derived from the weighting function. However, as shown in FIG. If the current wakefulness index indicates a low wakefulness state in which the driver feels drowsy (that is, equal to or greater than a set threshold value), the driver is in the wakefulness state. As compared with the case where it represents that, it may be made to output a louder sound from the speaker 13. In the warning process of the above embodiment, the intensity of the warning is increased by increasing the sound output from the speaker 13, but the tone of the output sound, the interval at which the sound is output, and the display on the monitor 14 are displayed. The intensity of the warning may be increased by changing the contents to be performed.

  Further, in the warning process in the above embodiment, a specified threshold value that is a criterion for determining whether or not to issue a warning according to the current weighting factor derived from the weighting function and the current awakening index indicated in the daily fluctuation. The value may be increased (i.e., raise the criterion for warning).

  Thus, according to the arousal level calculation device that increases the value of the prescribed threshold when the driver is in an awake state, it is possible to prevent a warning from being issued even though the driver does not feel sleepy. In addition, an unnecessary warning resulting from misrecognition of sleepiness can be issued, thereby preventing the driver from feeling uncomfortable.

  By the way, in the warning process in the arousal level calculation device 1 of the above embodiment, when the acceleration detected by the acceleration sensor indicates that the accelerator pedal or the brake pedal provided in the mounted vehicle is operated, a warning is given. Is prohibited (hereinafter referred to as warning execution), but the condition for prohibiting warning execution is not limited to this. For example, when the accelerator opening sensor provided in the mounted vehicle detects that the operation amount of the accelerator pedal is equal to or greater than the predetermined operation amount, or when the operation amount of the brake pedal is equal to or greater than the predetermined operation amount. It may be a case where a brake detection sensor provided in the mounted vehicle detects that there is.

  Further, in the warning processing of the above embodiment, the warning is executed by outputting sound from the speaker 13 and displaying the warning information on the monitor 14, but the driver is blown with cold air, the seat, or the steering wheel. The warning may be executed by vibrating.

  In the weighting function calculation process of the above embodiment, it is converted from the heart rate detected by the heartbeat detection sensor 16 after being matched with the high / low period of the awakening index of the reference daily fluctuation stored in the database 20. The daily fluctuation of use was obtained by shifting the level of the awakening index, but the daily fluctuation of use may be obtained by any method as long as the daily fluctuation of the driver on the day when the mounted vehicle is driven is obtained.

  Moreover, in the sleepiness warning process of the said embodiment, although the arousal level was estimated according to the time change of opening degree information, the method of estimating arousal level is not restricted to this. For example, it may be estimated from the number of blinks per unit time, the ratio between the opening degree information calculated sequentially and the reference value (that is, how close the eyes are compared with the awake state), etc. However, it may be estimated comprehensively by combining them.

  Further, in the wakefulness calculation device 1 of the above embodiment, the heart rate detection sensor 16 that calculates the heart rate from the detected pulse is used. However, a known heart rate detection that is attached to a human body and directly detects the heart rate. An apparatus may be used.

  In addition, although the arousal level calculation apparatus 1 of the said embodiment is provided with the light source 11 and the navigation apparatus 18, these may be abbreviate | omitted.

It is a block diagram which shows schematic structure of a wakefulness calculation apparatus. It is a flowchart which shows the process sequence of a drowsiness warning process. It is a flowchart which shows the process sequence of a weight function calculation process. It is a flowchart which shows the process sequence of a warning process. It is explanatory drawing for demonstrating the blink detection in a drowsiness warning process. It is explanatory drawing for demonstrating the calculation method of the daily fluctuation | variation in a weight function calculation process. It is explanatory drawing for demonstrating the calculation method of the weight function in a weight function calculation process. It is explanatory drawing for demonstrating the calculation method of the weight function in a weight function calculation process. It is explanatory drawing for demonstrating the setting threshold value in the warning process of a drowsiness warning process. It is explanatory drawing for demonstrating the modification in a wakefulness calculation apparatus. It is explanatory drawing for demonstrating the modification in a wakefulness calculation apparatus.

Explanation of symbols

DESCRIPTION OF SYMBOLS 1 ... Arousal degree calculation apparatus 10 ... Arousal degree calculation ECU 10a ... ROM 10b ... RAM 10c ... CPU 11 ... Light source 12 ... Camera 13 ... Speaker 14 ... Monitor 15 ... Information acquisition apparatus group 16 ... Heart rate detection sensor 17 ... Time measuring part 18 ... Navigation device 19 ... Acceleration sensor 20 ... Database

Claims (15)

Photographing means for photographing a photographed image including the face of the driver of the vehicle;
Index calculating means for repeatedly calculating at least one kind of estimated index used for estimating the driver's arousal level based on the captured image captured by the imaging means;
A plurality of conditions that are sequentially calculated by the index calculating means during a specified time from the start of driving of the vehicle and satisfy a predetermined condition as indicating that the driver is in an awake state A reference value setting means for setting a reference value of the estimated index based on the estimated index;
Wakefulness estimation means for estimating the driver's wakefulness based on the estimated index sequentially calculated by the index calculation means and the reference value set by the reference value setting means;
The wakefulness level estimated by the wakefulness level estimation unit when the reference value used while being sequentially updated by the predetermined time by the wakefulness level estimation unit is less reliable than the reference value used after the specified time An arousal level correction means for calculating a corrected arousal level obtained by correcting the arousal level according to a correction function set in advance for each driver so as to reduce the influence on the driver. apparatus. The arousal level correction means includes:
After showing the correspondence between the time from the start of driving of the vehicle and the correction amount to be corrected, and after a predetermined time determined as a time longer than the specified time, the correction amount becomes constant, The wakefulness calculation device according to claim 1, further comprising correction function calculation means for calculating the correction function that increases monotonously until the determination time. A traveling state acquisition means for acquiring high-speed traveling information indicating that the vehicle is traveling at a predetermined vehicle speed or higher;
The correction function calculating means includes
The correction function in which the determination time is shortened when the high-speed driving information is acquired by the driving state acquisition unit is compared with when the high-speed driving information is not acquired. 2. The arousal level calculation device according to 2. The correction function calculating means includes
The wakefulness calculation device according to claim 2 or 3, wherein the determination time is set based on circadian rhythm information that is preset for each driver and indicates a sleep cycle of a day. The correction function calculating means includes
5. The awakening according to claim 4, further comprising: circadian rhythm information generating means for acquiring biological information including at least the heart rate of the driver and generating the circadian rhythm information based on the biological information. Degree calculation device. The circadian rhythm information generating means includes:
6. The circadian rhythm information is defined as a correspondence relationship between a wakefulness index that varies in proportion to the driver's heart rate and represents a level of the wakefulness of the driver, and time. Wakefulness calculation device. The biological information includes at least one of heart rate variability, body movement, and sleep time,
The circadian rhythm information generating means includes:
According to the quality of sleep of the driver estimated based on at least one of the heart rate variability, body movement, and sleep time, the corrected circadian rhythm information obtained by correcting the correspondence relationship between the time and the wakefulness index is obtained. The wakefulness calculation apparatus according to claim 6, wherein the rhythm information is daily rhythm information.   8. The apparatus according to claim 1, further comprising a warning unit that issues a warning when the corrected arousal level calculated by the awakening level correction unit is equal to or less than a predetermined threshold value specified in advance. 9. Arousal calculation device. The correction function indicates a correspondence relationship between a time from the start of driving and a correction amount corrected by the arousal level correction unit,
The warning means is
Deriving from the correction function, when the correction amount at a specific time set in advance as a time from the start of driving of the vehicle is equal to or greater than a preset first threshold, increasing the value of the specified threshold The wakefulness calculating apparatus according to claim 8. The correction function indicates a correspondence relationship between a time from the start of driving and a correction amount corrected by the arousal level correction unit,
The warning means is
When the correction amount at a specific time that is derived from the correction function and set in advance as the time from the start of driving of the vehicle is equal to or greater than a second threshold that is set in advance, it is less than the second threshold. The wakefulness calculation apparatus according to claim 8 or 9, wherein a stronger warning is permitted as compared with a case. The circadian rhythm information indicates a correspondence relationship between time and an arousal index representing the level of arousal level of the driver,
When the corrected arousal level calculated by the awakening level correction unit is equal to or lower than a predetermined threshold value, the warning unit for issuing a warning,
The warning means is
When the wakefulness index at a specific time set in advance as the time from the start of driving shown in the circadian rhythm information indicates that the driver is in the wakefulness state, the prescribed threshold value is increased. The wakefulness calculation apparatus according to any one of claims 4 to 7. The circadian rhythm information indicates a correspondence relationship between time and an arousal index representing the level of arousal level of the driver,
Awakening at a specific time set in advance as the time from the start of driving, wherein the corrected awakening level calculated by the awakening level correcting means is equal to or less than a predetermined threshold value that is defined in advance and derived from the circadian rhythm information. When the indicator indicates that the driver is in a low arousal state where the driver feels sleepy, the indicator is provided with a warning means for issuing a stronger warning than when the driver indicates that the driver is in the awake state. The wakefulness calculating apparatus according to claim 4. Acceleration acquisition means for acquiring acceleration of the vehicle;
9. The apparatus according to claim 8, further comprising: a prohibiting unit that prohibits the warning unit from issuing a warning when the acceleration acquired by the acceleration acquisition unit represents an accelerator operation or a brake operation. The arousal level calculation apparatus according to claim 12. Brake operation amount acquisition means for acquiring an operation amount of a brake pedal provided in the vehicle;
And a prohibiting unit that prohibits the warning unit from issuing a warning when an operation amount of the brake pedal acquired by the brake operation amount acquiring unit is equal to or greater than a predetermined value defined in advance. The arousal level calculation apparatus according to any one of claims 8 to 12. An accelerator operation amount acquisition means for acquiring an operation amount of an accelerator pedal provided in the vehicle;
And a prohibiting unit that prohibits the warning unit from issuing a warning when the operation amount of the accelerator pedal acquired by the accelerator operation amount acquiring unit is equal to or greater than a predetermined value. The arousal level calculation apparatus according to any one of claims 8 to 12.

Images