JP2020091616A - Electronic device, and driver condition estimation system - Google Patents

Abstract

To provide a driver condition estimation system for estimating a condition of a driver during driving while taking a behavior of the driver before getting on a vehicle into consideration.SOLUTION: An on-vehicle device 100 provided with a function for estimating a condition of a driver comprises: a behavior history information acquisition unit for acquiring behavior history information of the driver outside of a vehicle; a vehicle outside driver condition estimation unit for estimating a condition of the driver outside of the vehicle on the basis of the behavior history information; a detection unit for detecting a situation of the driver inside of the vehicle; and a vehicle inside driver condition estimation unit for estimating a condition of the driver inside of the vehicle on the basis of the detected situation of the driver and an estimation result of the vehicle outside driver condition estimation unit.SELECTED DRAWING: Figure 1

Description

The present invention relates to an electronic device having a function of estimating a driver's state, and particularly to a technique of estimating a driver's fatigue state and presenting driving assistance information based on the estimation result.

A system that monitors the driver's condition and warns the driver of fatigue and drowsiness has been put into practical use. In such a system, the degree of fatigue of the driver is detected using biometric information such as driving time, face recognition (for example, eye open/closed state), heart rate and pulse. For example, the driving support device of Patent Document 1 discloses a method of estimating the fatigue level of a driver based on biometric information and correcting the fatigue level according to driving duration, driving operation, road type, drowsiness, and the like. ing.

International Publication No. 2017-017783

In the above-mentioned Patent Document 1, in determining the fatigue level of a driver who is traveling, the fatigue level is weighted by a destination, an experience value of driving, and the like, and it is determined whether or not to display an alarm. However, when the driver is tired before getting on the vehicle, there is a possibility that the fatigue level of the driver who is traveling cannot be accurately determined. For example, if you operate electronic devices excessively before boarding or watch for a long time, fatigue may accumulate in the driver, causing drowsiness or fatigue earlier than usual. Sometimes. Therefore, it is desired to consider the state of the driver before boarding.

The present invention is to solve such a conventional problem, and provides an electronic device and a driver state estimation system having a function of estimating the state of a traveling driver in consideration of the behavior of the driver outside the vehicle. The purpose is to provide.

The electronic device having the function of estimating the driver's state according to the present invention estimates the driver's state outside the vehicle based on the action history information and an acquisition unit capable of acquiring the driver's action history information outside the vehicle. A first driver state estimating means, a detecting means for detecting a driver's situation in the vehicle, a driver's situation detected by the detecting means, and an estimation result of the first driver state estimating means Second driver state estimating means for estimating the state of the driver in.

In one embodiment, the second driver state estimating means changes a threshold value or a condition for estimating the driver's state according to the estimation result of the first driver state estimating means. In one embodiment, the second driver state estimating means uses the threshold value or the condition to estimate the fatigue level of the driver.

The electronic device having the function of estimating the driver's state according to the present invention estimates the driver's state outside the vehicle based on the action history information and an acquisition unit capable of acquiring the driver's action history information outside the vehicle. First driver state estimating means, detecting means for detecting the driver's situation in the vehicle, and second driving for estimating the driver's state in the vehicle based on the driver's situation detected by the detecting means. And a presenting means for presenting driver assistance information based on the estimation result of the first driver state estimating means and the estimation result of the second driver state estimating means.

In one embodiment, the presenting unit changes a threshold value or a condition for presenting driving assistance information according to the estimation result of the first driver state estimating unit. In one embodiment, the presenting unit changes the timing of presenting the driving support information according to the threshold value or the condition. In one embodiment, the electronic device further includes position detection means for detecting a vehicle position, and the acquisition means selects the action history information based on the vehicle position detected by the position detection means. In one embodiment, when the current location of the own vehicle is at home, the obtaining means obtains action history information regarding the electronic device installed at the home, and when the current location of the own vehicle is outside the home, Acquire action history information about a mobile terminal. In one embodiment, the acquisition unit includes a communication unit that enables wireless communication with the outside, and the acquisition unit acquires the action history information via the communication unit. In one embodiment, the acquisition unit acquires the action history information when the electronic device is activated. In one embodiment, the first and second driver state estimating means estimate the degree of fatigue of the driver. In one embodiment, the first and second driver state estimating means estimate the degree of fatigue from the sleep time of the driver. In one embodiment, the action history information is operation information of an electronic device outside the vehicle by the driver, viewing information of the electronic device outside the vehicle by the driver, communication information of the electronic device by the driver, call information of the electronic device by the driver. , Or contains application information of the electronic device by the driver.

A driver state estimation system according to the present invention includes the electronic device described above, a server capable of communicating with the electronic device, and an external electronic device that can be used by a driver, and the external electronic device is , Uploading driver usage information to the server, and the electronic device downloads behavior history information related to driver usage information from the server. In one embodiment, the external electronic device includes a high-performance mobile phone (smartphone) or a smart home electric device installed in a driver's home.

According to the present invention, the behavior history information of the driver outside the vehicle is acquired, and the state of the driver inside the vehicle is estimated or the driving assistance information is presented by using this, so that the driver inside the vehicle is presented. Of the vehicle can be estimated more accurately, or more appropriate driving support information can be presented.

It is a figure which shows the structural example of the driver condition estimation system which concerns on the Example of this invention. It is a figure which shows the structural example of the vehicle-mounted apparatus which concerns on the Example of this invention. It is a figure which shows one structural example of the biometric information acquisition part of a present Example. It is a block diagram which shows the functional structure of the driver condition estimation program which concerns on the Example of this invention. 1 is a diagram showing a typical configuration of an electronic device such as a smart home electric device or a mobile terminal according to an embodiment of the present invention. It is a block diagram which shows the functional structure of the driver's action history extraction program which concerns on the Example of this invention. FIG. 7(A) is a diagram showing a functional configuration of the action history server of this embodiment, and FIG. 7(B) is a diagram showing an example of action history information. It is an operation|movement flow of the driver condition estimation system which concerns on the Example of this invention. It is an operation|movement flow of driver|operator state estimation of the vehicle-mounted apparatus which concerns on the 1st Example of this invention. It is an operation|movement flow of a driver|operator state estimation of the vehicle-mounted apparatus which concerns on the 2nd Example of this invention. It is an operation|movement flow of a driver|operator state estimation of the vehicle-mounted apparatus which concerns on the 2nd Example of this invention.

Next, an embodiment of the present invention will be described. The electronic device having the function of estimating the state of the driver of the present invention may be, in an embodiment, a personal computer, a portable terminal, or an on-vehicle device having an audio-visual navigation (AVN) function. In one embodiment, the electronic device of the present invention is applied to a driving support system and presents driving support information according to a driver's state. The driving support information includes driving guidance during driving, warnings, recommendations for resting, stopping driving, and the like. In the following embodiments, an in-vehicle device is exemplified as an electronic device having a function of estimating the driver's state.

FIG. 1 is a diagram showing a configuration example of a driver state estimation system according to an embodiment of the present invention. The driver state estimation system 10 according to the present embodiment includes a smart home electric device 20 installed at home, a mobile terminal 30 represented by a smartphone, an action history server 40 providing driver's action history information, and a driver. In-vehicle device 100 having a function of estimating the state of The driver's state is, for example, the degree of fatigue of the driver.

The smart home electric appliance 20 is, for example, a television, a personal computer, a game device, or the like, and is an electronic device used by a driver at home. The smart home appliance 20 is an IoT-enabled home appliance that has a wireless communication function and can be connected to the Internet, and has a function of uploading usage information on a driver's use of the smart home appliance 20 to the action history server 40. .. The mobile terminal 30 is a device carried and used by the driver. The mobile terminal 30 has a wireless communication function and a function of uploading usage information on the usage of the mobile terminal 30 by the driver to the action history server 40. ing.

Although only one user is illustrated in FIG. 1, the behavior history server 40 manages the behavior history information of a plurality of users, and transmits the behavior history information in response to a request from the in-vehicle device of each user. To do.

The in-vehicle apparatus 100 accesses the behavior history server 40 and downloads the behavior history information of the driver from the behavior history server 40, for example, when the vehicle rides on the driver vehicle M and is about to start driving, or at the time of activation. The behavior history information stored in the behavior history server 40 is a behavior history of the driver outside the vehicle, and the behavior history information is used to estimate the behavior or state of the driver before traveling.

FIG. 2 is a block diagram showing the internal configuration of the vehicle-mounted device 100 of this embodiment. The in-vehicle device 100 includes an input unit 110, a position detection unit 120, a navigation unit 130, a biometric information acquisition unit 140, a vehicle information acquisition unit 150, an output unit 160, a storage unit 170, a communication unit 180, and a control unit 190. It The configuration shown here is an example, and the vehicle-mounted device 100 may have an audio/video function, a television/radio reception function, or the like. Further, the in-vehicle device 100 may be equipped with a function of detecting an obstacle, a white line, or the like by using an imaging camera that captures an image of the surroundings of the vehicle, a radar, or the like, and supporting driving of the driver.

The input unit 110 receives an input from the user and provides it to the control unit 190. The position detection unit 120 detects the current position of the vehicle M by, for example, GPS positioning or self-contained navigation positioning (angular velocity sensor, acceleration sensor, etc.). The navigation unit 130 refers to the road map data, guides the road around the current position of the vehicle M, searches for a route to the destination, and guides the searched route.

The biometric information acquisition unit 140 acquires biometric information of the driver after getting on the vehicle, that is, in the vehicle. The biometric information acquisition unit 140 may include one or more biosensors, and may include, for example, a heart rate monitor 142, an electroencephalograph 144, a blood pressure monitor, a thermometer, etc., as shown in FIG. The biometric sensor can be attached to the driver as a wearable terminal, and the result detected by the biometric sensor is provided to the control unit 190. Furthermore, the biometric information acquisition unit 140 can include an imaging camera or a drive recorder 146 that images the driver's face. The image captured by the imaging camera 146 is provided to the control unit 190, and the control unit 190 analyzes the image data to detect the degree of opening/closing of the eyelid of the driver, the line-of-sight direction, the lack of extension, etc. To use.

The vehicle information acquisition unit 150 acquires vehicle information regarding the vehicle. As the vehicle information, for example, winker operation, headlight lighting or extinguishing, wiper operation, footbrake operation, accelerator operation, steering wheel operation, parking brake on/off, speed information, and the like are acquired via an in-vehicle bus or the like.

The output unit 160 includes a display unit and a voice output unit (speaker). For example, when the navigation unit 130 is operating, a road map is displayed on the display unit, and a voice guidance such as an intersection is output from the voice output unit. It Further, the in-vehicle device 100 can cause the output unit 160 to output a warning that wakes up the driver, an intersection guide according to the degree of fatigue, and the like according to the state of the driver (for example, fatigue state).

The storage unit 170 stores data, software, etc. required for the in-vehicle apparatus 100. For example, road map data used for the navigation unit 130, application programs executed by the control unit 190, and the like are stored. The communication unit 180 enables wired or wireless data communication with an external device or an external network. In one aspect, as shown in FIG. 1, the communication unit 180 accesses the behavior history server 40 via the Internet or the like, and downloads the behavior history information of the driver outside the vehicle from there.

The control unit 190 includes, for example, a microcontroller including a ROM/RAM, a microprocessor, an image processing processor, a digital signal processor (DSP), etc., and controls each unit of the vehicle-mounted apparatus 100 by software and/or hardware.

In one aspect, the control unit 190 executes a driver state estimation program for estimating the driver's state. FIG. 4 shows a functional configuration of the driver state estimation program 200. The driver state estimation program 200 includes an action history information acquisition unit 210, an outside driver state estimation unit 220, an inside driver state estimation unit 230, and a driving support information presentation unit 240.

The behavior history information acquisition unit 210 accesses the behavior history server 40 via the communication unit 180 and acquires the behavior history information of the driver outside the vehicle from there. In one example, when the ignition key of the vehicle M is turned on or when the vehicle-mounted device 100 is activated, the behavior history information acquisition unit 210 accesses the behavior history server 40 and acquires the behavior history information. In another example, the action history information acquisition unit 210 accesses the action history server 40 when there is an instruction from the user or according to the schedule determined by the driver state estimation program 200, and obtains the action history information from there. get.

The outside-vehicle driver state estimation unit 220 estimates the state of the driver outside the vehicle, in other words, the state of the driver before boarding the vehicle M, based on the action history information acquired by the action history information acquisition unit 210. The driver's state is, for example, the degree of fatigue of the driver. In one aspect, the vehicle exterior driver state estimation unit 220 digitizes the degree of fatigue level corresponding to the action history information according to an algorithm prepared in advance, and provides the digitized fatigue level to the vehicle interior driver state estimation unit 230. .. For example, when the fatigue level is large, the numerical value is large, and when the fatigue level is small, the numerical value is small.

The in-vehicle driver state estimation unit 230 considers the estimation result by the out-of-vehicle driver state estimation unit 220, and based on the biometric information acquired by the biometric information acquisition unit 140 and/or the vehicle information acquired by the vehicle information acquisition unit 150. Based on this, the state of the driver in the vehicle, that is, the state of the driver in the vehicle M is estimated.

The in-vehicle driver state estimation unit 230 estimates the state of the driver (here, the degree of fatigue) based on the biological information. For example, whether the driving time exceeds a certain value based on vehicle information or an image from the drive recorder 146, whether the number of blinks of the driver per unit time has become less than a certain value, or how much the driver's eyelids are opened. The driver's fatigue level is estimated based on whether the driver's heart rate is below a certain value, the presence or absence of a specific brain wave of the driver, and the like. In estimating the driver's fatigue level, the number of right and left turns during traveling, the number of brakes, the number of accelerators, and the like may be taken into consideration based on the vehicle information. In one aspect, the in-vehicle driver state estimation unit 230 digitizes the acquired biometric information and vehicle information according to a predetermined algorithm.

The in-vehicle driver state estimating unit 230 estimates the state of the driver based on the biometric information and the vehicle information as described above. When performing this estimation, the state of the driver estimated by the outside-vehicle driver state estimating unit 220 Is used as initial data. As one aspect, when the tiredness of the driver outside the vehicle is Tout, the in-vehicle driver state estimation unit 230 estimates the tiredness Tin of the driver inside the vehicle according to the tiredness Tout of the driver outside the vehicle. The threshold Th used for this is varied. For example, when the numerical value obtained by calculating the acquired biometric information or vehicle information is DET, the in-vehicle driver state estimation unit 230 has a high tiredness Tin of the driver in the vehicle when DET>Th. When DET<Th, it is estimated that the fatigue level Tin of the driver in the vehicle is small. If the driver's fatigue level Tout outside the vehicle is relatively large, the threshold value Th is reduced to Th_s so that the fatigue level can be estimated more easily than in the normal case. On the other hand, if the fatigue level Tout outside the vehicle is relatively small, the threshold Th is increased to Th_b (Th_s<Th_b) so that the fatigue level can be estimated somewhat more easily than in the normal case.

The threshold value Th for estimating the state of the driver in the vehicle is not limited to one, and a plurality of threshold values Th1, Th2,..., Thn may be prepared to estimate the fatigue level of the driver at a plurality of levels. Good. Also in this case, the plurality of thresholds Th1, Th2,..., Thn are changed according to the fatigue level Tout of the driver outside the vehicle. Further, in the above example, the threshold value Th is made variable depending on whether the driver's fatigue level Tout outside the vehicle is high or low. Alternatively, the threshold value Th may be changed depending on each level.

The driving support information presenting unit 240 presents driving information useful for the driver based on the estimation result of the in-vehicle driver state estimating unit 230 via the output unit 160. For example, the driving assistance information presenting unit 240 presents driving assistance information according to the tiredness Tin of the driver in the vehicle, and when the tiredness is large, the driving assistance information (presence of obstacles) is earlier than usual. , Guidance on intersections and the like by the navigation unit 130, warning of deviation from the white line, etc.), and driving assistance information (driving speed, presence or absence of a following vehicle, etc.) that is not normally presented. For example, when the fatigue level Tin is large, the driving support information is output earlier than the normal time by t1 seconds, and when the fatigue level Tin is small, the driving support information is output earlier than the normal time by t2 seconds ( t1>t2).

Next, functions of the smart home electric appliance 20 and the mobile terminal 30 shown in FIG. 1 will be described. In this embodiment, the smart home electric device 20 and the mobile terminal 30 have a function of extracting the behavior history information of the driver and uploading the behavior history information to the behavior history server 40. FIG. 5 shows a typical configuration of the electronic device 300 such as the smart home electric device 20 and the mobile terminal 30. As shown in the figure, the electronic device 300 includes a communication unit 310, an input unit 320, an output unit 330, a storage unit 340, and a control unit 350.

The communication unit 310 has a function of performing data communication with an external device or an external network by wire or wirelessly. Further, in the case of a mobile terminal such as a smartphone, the communication unit 310 enables data communication and voice call via the public wireless network.

The input unit 320 receives an instruction from the user and provides the instruction to the control unit 350. The input unit 320 is, for example, a keyboard, a touch panel, a remote controller, a mouse, a voice recognition module, or the like. The output unit 330 includes a display that outputs an image, a speaker that outputs sound, and the like. For example, if the electronic device is a television, the output unit 330 outputs the video/audio received by the television broadcast or the video/audio delivered on the Internet.

The storage unit 340 stores data, software, and the like necessary for the electronic device 300. The control unit 350 includes, for example, a microcontroller including a ROM/RAM, a microprocessor, an image processing processor, a digital signal processor (DSP), and the like, and controls each unit of the electronic device 300 by software and/or hardware. In one aspect, the control unit 350 executes an action history extraction program for extracting driver's action history information.

FIG. 6 shows a functional configuration of the action history extraction program 400. The action history extraction program 400 includes a usage state monitoring unit 410 that monitors a usage state of an electronic device, a usage information recording unit 420 that records usage information in a storage unit 340 based on a monitoring result of the usage state monitoring unit 410, and a recorded usage The usage information upload unit 430 is provided for uploading information to the action history server 40 via the communication unit 310.

The usage status monitoring unit 410 monitors the usage status of the electronic device 300 represented by the smart home electric device 20 and the mobile terminal 30. What kind of usage state is monitored is arbitrary and can be set in advance. As an example, the usage status monitoring unit 410 monitors the operation of the input unit 320 by the driver, and detects operation information such as the number of operations, operation time, and cumulative operation time. In addition, the usage state monitoring unit 410 detects the type of application software executed by the electronic device 300 (for example, game software, music playing software, browser software, etc.) and the usage time of the application software, and outputs the output unit. Detecting the viewing information of the driver such as the time when the video is output from 330 and the total time when the video is output, the data communication amount via the Internet or the like by the communication unit 310, the communication time, the total communication time, etc. Communication information, and call information such as the call time by the driver and the accumulated call time. Furthermore, if the electronic device 300 has a function of acquiring position information, the usage status monitoring unit 410 detects the position information. The position information of the electronic device 300 can be used by the electronic device 300 for identification from the smart home electric device 20 or the smartphone 30, and when the position information is the position information of the smartphone 30, driving is performed based on a change in the position information. It is possible to estimate the moving distance of the person.

The usage information recording unit 420 records the operation information, the viewing information, the communication information, the call information, the application information, the position information, etc. detected by the usage status monitoring unit 410 in the storage unit 340 as the usage information of the electronic device by the driver. To do.

The usage information upload unit 430 uploads the usage information recorded by the usage information recording unit 420 to the behavior history server 40 via the communication unit 310. The timing of uploading is arbitrary, and may be when the usage information recording unit 420 records, when the usage state monitoring unit 410 stops detecting the use by the driver, or for a certain period of time. It may be an interval or a predetermined time interval.

Next, the action history server 40 will be described. The action history server 40 functions as a relay that provides the in-vehicle apparatus 100 with the action history information extracted by the electronic device 300 (smart home appliance 20 or mobile terminal 30). The action history server 40 includes a use information receiving unit 42, an action history information creating unit 44, and an action history information transmitting unit 46, as shown in FIG.

The usage information receiving unit 42 receives the usage information uploaded from the electronic device 300, and the action history information creating unit 44 creates the action history information based on the received usage information together with the identification information of the uploaded electronic device 300. Then, this is stored in the storage unit. FIG. 7B shows an example of the created action history information. The action history information creation unit 44 stores the usage information with the creation time as the time when the usage information was uploaded. The usage information includes identification information of the uploaded electronic device, operation information of the electronic device 300 by the driver, viewing information, communication information, call information, application information, position information, and the like.

When the action history information transmission unit 46 receives a request for action history information from the action history information acquisition unit 210 illustrated in FIG. 4, in response to this, the action history information transmission unit transmits the action history information for a certain period of time before the request is made. .. It is possible to preset how long ago the action history information is transmitted. For example, the action history information 6 hours before or 12 hours before the request is transmitted.

Further, the action history information acquisition unit 210 can select only necessary information from the action history information, and in that case, the action history information acquisition unit 210 requests a selection condition for selecting information. Can be included in. For example, it is possible to request transmission of only operation information in the action history information, or request transmission of action history information of an electronic device other than home based on the position information. When the request from the action history information acquisition unit 210 includes a selection condition, the action history information transmission unit 46 transmits the action history information corresponding to the selection condition.

Next, the operation of the driver state estimation system according to the present embodiment will be described with reference to the flow of FIG. When the driver uses the electronic device 300 (S100), the use information is recorded in the electronic device 300 (S110), and the use information is uploaded to the action history server 40 at a predetermined timing (S120). The action history server 40 creates action history information based on the uploaded usage information (S130) and stores it in the server as the personal information of the driver (S140). After that, when the driver gets into the vehicle and, for example, the in-vehicle apparatus 100 is activated (S150), the in-vehicle apparatus 100 requests the action history server 40 to transmit the action history information. Upon receiving the request, the behavior history server 40 transmits the behavior history information of the driver to the in-vehicle device 100 (S160), and the in-vehicle device 100 acquires the behavior history information of the driver outside the vehicle (S170).

FIG. 9 is a flowchart for explaining the driver state estimation operation of the vehicle-mounted device according to the first embodiment of the present invention. In the in-vehicle apparatus 100, when the action history information acquisition unit 210 acquires the action history information (S200), the outside-vehicle driver state estimation unit 220 estimates the state of the driver outside the vehicle (S210). Next, the in-vehicle driver state estimation unit 230 takes in the result estimated by the out-of-vehicle driver state estimation unit 220 as initial data, and sets the threshold Th used for estimating the state of the in-vehicle driver as the out-of-vehicle driver. Adjustment is performed in consideration of the state of (S220). Then, the in-vehicle driver state estimating unit 230 estimates the state of the in-vehicle driver based on the acquired biometric information and vehicle information using the adjusted threshold Th (S240). Next, the driving assistance information presenting section 240 presents the driving assistance information based on the estimation result of the in-vehicle driver state estimating section 230 (S240). The driver state estimation program 200 repeats steps S230 and S240 when traveling is continued.

Thus, according to the present embodiment, the behavior history information outside the vehicle before the driver starts driving is estimated, the state of the driver outside the vehicle is estimated, and the estimation result is estimated as the state of the driver inside the vehicle. Since it is reflected in the above, it is possible to more accurately estimate the state of the driver in the vehicle, and as a result, it is possible to present the driver with appropriate timing or appropriate content of driving assistance information.

Next, a second embodiment of the present invention will be described. In the first embodiment, the estimation result of the state of the driver outside the vehicle is reflected in the estimation of the state of the driver inside the vehicle, but in the second example, the estimation result of the state of the driver outside the vehicle is reflected. It is reflected in the presentation of driving support information.

The estimation result by the out-of-vehicle driver state estimating unit 220 shown in FIG. 4 is provided to the driving assistance information presenting unit 240, and the driving assistance information presenting unit 240 presents the driving assistance information according to the estimation result. As one aspect, when the tiredness of the driver outside the vehicle is Tout, the driving assistance information presenting unit 240 changes the timing of presenting the driving assistance information according to the tiredness Tout of the driver outside the vehicle. For example, when the fatigue level Tout is relatively large, the driving assistance information presenting unit 240 presents the driving assistance information at a timing earlier than the normal time by the time T1, and provides the driving assistance information to the driver with a sufficient time. Inform. Further, when the fatigue level Tout is relatively small, the driving assistance information presenting unit 240 presents the driving assistance information at a timing T2 which is earlier than the normal time but later than the time T1. Further, in another aspect, the driving support information presenting unit 240 may change the threshold value Tp of whether or not to present the driving support information (for example, driving stop warning) according to the fatigue degree Tout. For example, when the fatigue level Tout is relatively large, the threshold value Tp is reduced to Tp_s so that the driving assistance information is easily presented, and when the fatigue level Tout is relatively small, the threshold value Tp is increased to Tp_b. The driving support information may be somewhat easier to be presented.

FIG. 10 is a flowchart for explaining the driver state estimation operation of the vehicle-mounted device according to the second embodiment of the present invention. In the in-vehicle apparatus 100, when the action history information acquisition unit 210 acquires the action history information (S300), the outside-vehicle driver state estimation unit 220 estimates the state of the driver outside the vehicle (S310). Next, the driving assistance information presenting unit 240 takes in the result estimated by the vehicle outside driver's state estimating unit 220 as initial data, and considers the state of the driver outside the vehicle as a condition for presenting the driving assistance information. Adjust (S320). Next, the in-vehicle driver state estimation unit 230 estimates the state of the driver in the vehicle based on the acquired biometric information and vehicle information (S330), and provides the estimation result to the driving support information presentation unit 240. The driving assistance information presenting unit 240 presents the driving assistance information based on the estimation result of the in-vehicle driver state estimating unit 230 using the adjusted condition (S340). The driver state estimation program 200 repeats steps S330 and S340 when traveling is continued.

As described above, according to the present embodiment, the behavior history information outside the vehicle before the driver starts driving is estimated, the state of the driver outside the vehicle is estimated, and the estimation result is used as a condition for presenting the driving assistance information. Since the information is reflected, it is possible to present the driver with appropriate timing or appropriate content of driving assistance information according to the state of the driver in the vehicle. The present invention may be a combination of the first embodiment and the second embodiment. In this case, the estimation result of the state of the driver outside the vehicle is the threshold value of the in-vehicle driver state estimating unit 230. It is reflected in each of Th and the presentation condition of the driving assistance information presenting section 240, whereby it becomes possible to appropriately present the driving assistance information while accurately estimating the state of the driver in the vehicle.

Next, a third embodiment of the present invention will be described. FIG. 11 is a flowchart for explaining the operation of the driver state estimation of the vehicle-mounted device according to the third embodiment of the present invention. First, when it is detected that the driver is seated in the driver's seat based on the image of the seat sensor or the drive recorder (S400), the action history information acquisition unit 210 obtains the current position information from the position detection unit 120 (S410). Then, it is determined whether or not the own vehicle is at home (S420). As the current location of the home, for example, the home information registered in the navigation unit 130 may be used, or the home position may be registered in the driver state estimation program 200 in advance.

The action history information acquisition unit 210 acquires the action history information of the smart home electric appliance 20 when the current location of the vehicle is at home (S430). In this case, the action history information acquisition unit 210 adds the identification information of the smart home electric appliance as a selection condition when requesting the action history server 40 to transmit the action history information. As a result, the behavior history server 40 searches the behavior history information created based on the usage information uploaded from the smart home electric appliance 20, and transmits the searched behavior history information to the in-vehicle apparatus 100.

Next, the out-of-vehicle driver state estimation unit 220 estimates whether or not the sleep time of the driver last night is X or less based on the action history information of the smart home electric appliance 20 (S440). For example, the sleep time is estimated to be 6 hours when the smart home appliance 20 is used until 1:00 in the middle of the night and the use is started from 7:00 in the morning. As a default value, the sleep start time is 0:00 am and the end time is 8:00 am, and if the smart home electric appliance 20 is used during that time, the use time may be subtracted from the default value. When the sleeping time is X or less, as in the first and second embodiments, the in-vehicle driver state estimating unit 230 and/or the driving support information presenting unit 240 notifies the driver of fatigue warning or driving support information. The alarm timing is adjusted (S450), and when the sleep time exceeds X, the alarm timing of the normal driving support information is used (S480).

On the other hand, when the current location of the vehicle is other than home (S420), the behavior history information acquisition unit 210 acquires behavior history information of the smartphone (mobile terminal 30) (S460). In other words, the driver is expected to be active outside the home. The action history information acquisition unit 210 adds the identification information of the mobile terminal 30 as a selection condition when requesting the action history server 40 to transmit the action history information. Thereby, the behavior history server 40 searches the behavior history information created based on the usage information uploaded from the mobile terminal 30, and transmits the retrieved behavior history information to the vehicle-mounted apparatus 100.

Next, the out-of-vehicle driver state estimation unit 220 estimates whether or not the operation time of the driver is Y or more based on the action history information of the mobile terminal 30 (S470). When the operation time is Y or more, as in the first and second embodiments, the in-vehicle driver state estimating unit 230 and/or the driving support information presenting unit 240 alerts the driver of fatigue or driving. The alarm timing of the support information is adjusted (S450), and when the operation time is less than Y, the normal alarm timing of the driving support information is used (S480).

As described above, according to the present embodiment, the action history information of the smart home appliance 20 or the mobile terminal 30 is selected based on the current position of the vehicle, so that the appropriate action history information of the driver outside the vehicle is acquired. As a result, the state of the driver in the vehicle can be estimated more accurately.

In the above-described embodiment, the action history server 40 is used to acquire the action history information, but this is an example, and in the present invention, the in-vehicle apparatus 100 directly connects the smart home electric appliance without the action history server 40 interposed. The communication history information may be acquired from each device by communicating with the mobile terminal 20 or the mobile terminal 30.

Although the preferred embodiments of the present invention have been described above in detail, the present invention is not limited to the specific embodiments, and various modifications are made within the scope of the gist of the invention described in the claims. It can be changed.

10: Driver state estimation system 20: Smart home appliance 30: Mobile terminal 40: Action history server 100: In-vehicle device 200: Driver state estimation program 300: Electronic device 400: Action history extraction program M: Vehicle

Claims (20)

An electronic device having a function of estimating a driver's condition,
An acquisition means capable of acquiring the behavior history information of the driver outside the vehicle,
First driver state estimating means for estimating the state of the driver outside the vehicle based on the action history information;
Detecting means for detecting the situation of the driver in the vehicle,
Second driver state estimating means for estimating the state of the driver in the vehicle based on the state of the driver detected by the detecting means and the estimation result of the first driver state estimating means;
An electronic device having. The electronic device according to claim 1, wherein the second driver state estimating means changes a threshold value or a condition for estimating a driver state according to an estimation result of the first driver state estimating means. .. The electronic device according to claim 2, wherein the second driver state estimating means uses the threshold value or the condition to estimate the fatigue level of the driver. An electronic device having a function of estimating a driver's condition,
An acquisition means capable of acquiring the behavior history information of the driver outside the vehicle,
First driver state estimating means for estimating the state of the driver outside the vehicle based on the action history information;
Detecting means for detecting the situation of the driver in the vehicle,
Second driver state estimating means for estimating the state of the driver in the vehicle based on the driver's state detected by the detecting means;
Presentation means for presenting driver assistance information based on the estimation result of the first driver state estimating means and the estimation result of the second driver state estimating means;
An electronic device having. The electronic device according to claim 4, wherein the presenting unit changes a threshold value or a condition for presenting the driving assistance information according to the estimation result of the first driver state estimating unit. The electronic device according to claim 5, wherein the presenting unit changes a timing of presenting the driving support information according to the threshold value or the condition. The electronic device further includes position detecting means for detecting the position of the own vehicle,
The electronic device according to claim 1, wherein the acquisition unit selects the action history information based on the vehicle position detected by the position detection unit. When the current location of the own vehicle is at home, the acquisition means obtains action history information regarding the electronic device installed at the home, and when the current location of the own vehicle is outside the home, the action history regarding the mobile terminal. The electronic device according to claim 7, which acquires information. The electronic device according to claim 1, wherein the acquisition unit includes a communication unit that enables wireless communication with the outside, and the acquisition unit acquires the action history information via the communication unit. The electronic device according to claim 9, wherein the acquisition unit acquires the action history information when the electronic device is activated. The electronic device according to any one of claims 1 to 10, wherein the first and second driver state estimating means estimate a driver's fatigue level. The electronic device according to claim 11, wherein the first and second driver state estimating means estimate the degree of fatigue from the sleep time of the driver. 13. The electronic device according to claim 1, wherein the action history information includes operation information of an electronic device outside the vehicle operated by a driver. The electronic device according to claim 1, wherein the action history information includes viewing information of an electronic device outside the vehicle by a driver. The electronic device according to claim 1, wherein the action history information includes communication information of a driver's electronic device. The electronic device according to claim 1, wherein the action history information includes telephone call information of a driver's electronic device. The electronic device according to claim 1, wherein the action history information includes application information of a driver's electronic device. An electronic device according to any one of claims 1 to 17,
A server capable of communicating with the electronic device,
Including external electronics that can be used by the driver,
The external electronic device uploads driver usage information to the server,
The electronic device is a driver state estimation system that downloads behavior history information related to driver usage information from the server. The driver state estimation system according to claim 18, wherein the external electronic device includes a high-performance mobile phone (smartphone). 19. The driver state estimation system according to claim 18, wherein the external electronic device includes a smart home electric device installed in a driver's home.

Images