WO2020188634A1 - Driver monitoring device, driver monitoring system and driver monitoring method - Google Patents

Driver monitoring device, driver monitoring system and driver monitoring method Download PDF

Info

Publication number
WO2020188634A1
WO2020188634A1 PCT/JP2019/010857 JP2019010857W WO2020188634A1 WO 2020188634 A1 WO2020188634 A1 WO 2020188634A1 JP 2019010857 W JP2019010857 W JP 2019010857W WO 2020188634 A1 WO2020188634 A1 WO 2020188634A1
Authority
WO
WIPO (PCT)
Prior art keywords
alarm
driver
state
output
monitoring device
Prior art date
Application number
PCT/JP2019/010857
Other languages
French (fr)
Japanese (ja)
Inventor
篤 松本
Original Assignee
三菱電機株式会社
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by 三菱電機株式会社 filed Critical 三菱電機株式会社
Priority to PCT/JP2019/010857 priority Critical patent/WO2020188634A1/en
Publication of WO2020188634A1 publication Critical patent/WO2020188634A1/en

Links

Images

Classifications

    • B—PERFORMING OPERATIONS; TRANSPORTING
    • B60—VEHICLES IN GENERAL
    • B60W—CONJOINT CONTROL OF VEHICLE SUB-UNITS OF DIFFERENT TYPE OR DIFFERENT FUNCTION; CONTROL SYSTEMS SPECIALLY ADAPTED FOR HYBRID VEHICLES; ROAD VEHICLE DRIVE CONTROL SYSTEMS FOR PURPOSES NOT RELATED TO THE CONTROL OF A PARTICULAR SUB-UNIT
    • B60W40/00—Estimation or calculation of non-directly measurable driving parameters for road vehicle drive control systems not related to the control of a particular sub unit, e.g. by using mathematical models
    • B60W40/08—Estimation or calculation of non-directly measurable driving parameters for road vehicle drive control systems not related to the control of a particular sub unit, e.g. by using mathematical models related to drivers or passengers
    • G—PHYSICS
    • G08—SIGNALLING
    • G08G—TRAFFIC CONTROL SYSTEMS
    • G08G1/00—Traffic control systems for road vehicles
    • G08G1/16—Anti-collision systems

Definitions

  • the present invention relates to a driver monitoring device, a driver monitoring system, and a driver monitoring method.
  • a driver monitoring device that outputs a warning according to the driver status is known.
  • the driving support device described in Patent Document 1 notifies the driver based on the difference between the state of the driver and the level of the proper state of driving.
  • the level of the proper state is stored in advance corresponding to the driver and the driving scene. For example, when the driving scene of a vehicle is a school road during school hours, the driver needs to frequently check the surrounding area. In that case, the driving support device described in Patent Document 1 learns a state in which the driver frequently looks aside to check the surroundings as a level of an appropriate state. As a result, the driving support device reduces the feeling of discomfort of the notified driver as much as possible.
  • the driver monitoring device that outputs a warning according to the driver status
  • the alarm method for the driver is not appropriate, there is a concern that the driver will not return to the proper operating status as a result of the alarm.
  • the driver monitoring device gives a sound alarm to the driver's doze, there is a concern that the driver cannot recognize the sound alarm and does not return to the awake state.
  • the present invention has been made to solve the above problems, and the alarm is effective for the driver in order to change the alarm from the next time onward based on the effect of the alarm on the driver. It is an object of the present invention to provide a driver monitoring device that outputs a judgment result as to whether or not it was.
  • the driver monitoring device controls the alarm device to output an alarm to the driver based on the state of the driver of the vehicle.
  • the driver monitoring device includes a driver state detection unit, an alarm effect determination unit, and an alarm information output unit.
  • the driver status detection unit detects the driver status.
  • the alarm effect determination unit issues an alarm to the driver based on the change in the driver's state before and after the alarm output from the alarm device based on the driver's state, or the change in the driving operation of the vehicle by the driver after the alarm. To determine whether it is effective or not.
  • the alarm information output unit outputs the determination result by the alarm effect determination unit. Based on the determination result output by the alarm information output unit, the alarm to the driver is changed to another alarm different from the already output alarm.
  • a driver monitoring device that outputs a determination result as to whether or not an alarm is effective for a driver in order to change the alarm from the next time onward based on the effect of the alarm on the driver. Can be provided.
  • FIG. 1 It is a block diagram which shows the structure of the driver monitoring apparatus in Embodiment 1.
  • FIG. 2 It is a figure which shows an example of the structure of the processing circuit which a driver monitoring apparatus has. It is a figure which shows another example of the structure of the processing circuit which a driver monitoring apparatus has.
  • FIG. It is a figure which shows the positional relationship between a camera mounted on a vehicle, and a driver. It is a figure which shows the positional relationship between a camera mounted on a vehicle, and a driver. It is a figure which shows an example of the alarm database in Embodiment 2.
  • FIG. 2 It is a block diagram which shows the structure of the driver monitoring apparatus in Embodiment 1.
  • FIG. 2 It is a figure which shows an example of the structure of the processing circuit which a driver monitoring apparatus has.
  • Embodiment 2 It is a flowchart which shows the driver monitoring method in Embodiment 2. It is a block diagram which shows the structure of the driver monitoring apparatus in the modification 1 of Embodiment 2. It is a block diagram which shows the structure of the driver monitoring apparatus and the driver monitoring system in Embodiment 3. It is a flowchart which shows the driver monitoring method in Embodiment 3. It is a flowchart which shows the learning processing method of the alarm database in Embodiment 3. It is a block diagram which shows the structure of the driver monitoring apparatus and the driver monitoring system in the modification of Embodiment 3. It is a block diagram which shows the structure of the driver monitoring apparatus and the apparatus which operates in connection with it in Embodiment 6.
  • FIG. 1 is a block diagram showing the configuration of the driver monitoring device 100 according to the first embodiment.
  • the driver monitoring device 100 controls the alarm device (not shown) to output an alarm to the driver based on the state of the driver of the vehicle.
  • the driver monitoring device 100 includes a driver state detection unit 10, an alarm effect determination unit 20, and an alarm information output unit 30.
  • the driver state detection unit 10 detects the driver state.
  • the alarm device outputs an alarm based on the state of the driver.
  • the alarm is effective for the driver based on the change in the driver's state before and after the alarm output from the alarm device or the change in the driving operation of the vehicle by the driver after the alarm. Judge whether or not.
  • the alarm effect determination unit 20 detects a change in the driver state based on the states of the two drivers detected by the driver state detection unit 10 before and after the alarm.
  • the alarm effect determination unit 20 detects a change in the driving operation of the vehicle by the driver after the alarm by acquiring it from a device (not shown) related to the driving control of the vehicle.
  • the alarm information output unit 30 outputs the determination result by the alarm effect determination unit 20. Based on the determination result, the alarm to the driver is changed to another alarm different from the alarm that has already been output.
  • FIG. 2 is a diagram showing an example of the configuration of the processing circuit 90 included in the driver monitoring device 100.
  • Each function of the driver state detection unit 10, the alarm effect determination unit 20, and the alarm information output unit 30 is realized by the processing circuit 90. That is, the processing circuit 90 includes a driver state detection unit 10, an alarm effect determination unit 20, and an alarm information output unit 30.
  • the processing circuit 90 may be, for example, a single circuit, a composite circuit, a programmed processor, a parallel programmed processor, an ASIC (Application Specific Integrated Circuit), or an FPGA (Field). -ProgrammableGateArray), or a circuit that combines these.
  • the functions of the driver state detection unit 10, the alarm effect determination unit 20, and the alarm information output unit 30 may be individually realized by a plurality of processing circuits, or may be collectively realized by one processing circuit.
  • FIG. 3 is a diagram showing another example of the configuration of the processing circuit included in the driver monitoring device 100.
  • the processing circuit includes a processor 91 and a memory 92.
  • each function of the driver state detection unit 10, the alarm effect determination unit 20, and the alarm information output unit 30 is realized.
  • each function is realized by executing software or firmware described as a program by the processor 91.
  • the driver monitoring device 100 has a memory 92 for storing the program and a processor 91 for executing the program.
  • the driver monitoring device 100 detects the driver's state, and the driver's state changes before and after the alarm output from the alarm device based on the driver's state, or the driver operates the vehicle after the alarm.
  • the function of determining whether or not the alarm is effective for the driver based on the change in the above and outputting the determination result is described. Further, the program causes the computer to execute the procedure or method of the driver state detection unit 10, the alarm effect determination unit 20, and the alarm information output unit 30.
  • the processor 91 is, for example, a CPU (Central Processing Unit), an arithmetic unit, a microprocessor, a microcomputer, a DSP (Digital Signal Processor), or the like.
  • the memory 92 is, for example, non-volatile or volatile such as RAM (RandomAccessMemory), ROM (ReadOnlyMemory), flash memory, EPROM (ErasableProgrammableReadOnlyMemory), EEPROM (ElectricallyErasableProgrammableReadOnlyMemory). It is a semiconductor memory.
  • the memory 92 may be any storage medium used in the future, such as a magnetic disk, a flexible disk, an optical disk, a compact disk, a mini disk, or a DVD.
  • driver state detection unit 10 Some of the functions of the driver state detection unit 10, the alarm effect determination unit 20, and the alarm information output unit 30 described above may be realized by dedicated hardware, and some may be realized by software or firmware. In this way, the processing circuit realizes each of the above-mentioned functions by hardware, software, firmware, or a combination thereof.
  • FIG. 4 is a flowchart showing the driver monitoring method according to the first embodiment.
  • step S1 the driver state detection unit 10 detects the state of the driver of the vehicle.
  • the alarm device outputs an alarm based on the state of the driver.
  • step S2 the alarm effect determination unit 20 detects a change in the driver's state before and after the alarm or a change in the driving operation of the vehicle by the driver after the alarm.
  • step S3 the alarm effect determination unit 20 determines whether or not the alarm is effective for the driver based on the change in the driver's state before and after the alarm or the change in the driving operation of the vehicle by the driver after the alarm. judge.
  • step S4 the alarm information output unit 30 outputs the determination result. Based on the determination result, the alarm to the driver is changed to another alarm different from the alarm that has already been output.
  • the driver monitoring device 100 in the first embodiment controls the alarm device to output an alarm to the driver based on the state of the driver of the vehicle.
  • the driver monitoring device 100 includes a driver state detection unit 10, an alarm effect determination unit 20, and an alarm information output unit 30.
  • the driver state detection unit 10 detects the driver state.
  • the alarm effect determination unit 20 sends an alarm to the driver based on the change in the driver's state before and after the alarm output from the alarm device based on the driver's state, or the change in the driving operation of the vehicle by the driver after the alarm. It is determined whether or not it is effective.
  • the alarm information output unit 30 outputs the determination result by the alarm effect determination unit 20. Based on the determination result output by the alarm information output unit 30, the alarm to the driver is changed to another alarm different from the already output alarm.
  • Such a driver monitoring device 100 can output a determination result as to whether or not the alarm was effective for the driver. By changing the alarm from the next time onward based on the determination result, the driver monitoring device 100 can guide the driver's state from the abnormal state to the normal state.
  • the alarm device is controlled to output an alarm to the driver based on the state of the driver of the vehicle.
  • the driver monitoring method detects the driver's condition and is based on the change in the driver's condition before and after the alarm output from the alarm device based on the driver's condition, or the change in the driving operation of the vehicle by the driver after the alarm. , Judges whether the alarm is effective for the driver and outputs the judgment result. Then, based on the determination result, the alarm to the driver is changed to another alarm different from the alarm that has already been output.
  • Such a driver monitoring method can output a judgment result as to whether or not the alarm was effective for the driver. By changing the alarm from the next time onward based on the judgment result, the driver's state is guided from the abnormal state to the normal state.
  • the driver monitoring device and the driver monitoring method according to the second embodiment will be described.
  • the second embodiment is a subordinate concept of the first embodiment, and the driver monitoring device according to the second embodiment includes each configuration of the driver monitoring device 100 according to the first embodiment. The same configuration and operation as in the first embodiment will not be described.
  • FIG. 5 is a block diagram showing the configuration of the driver monitoring device 101 according to the second embodiment.
  • the driver monitoring device 101 includes an image acquisition unit 40, a face feature detection unit 50, a driver state detection unit 10, an alarm parameter acquisition unit 70, an alarm effect determination unit 20, an alarm information output unit 30, and an alarm control unit 60.
  • FIG. 2 shows a camera 110 and an alarm device as devices that operate in connection with the driver monitoring device 101.
  • the alarm device one alarm device 120 and another alarm device 130 are shown.
  • the driver monitoring device 101 according to the second embodiment is mounted on a vehicle (not shown) as an example.
  • the camera 110 is provided inside the vehicle and captures an image around the driver's face.
  • the camera 110 for example, captures an image at a predetermined timing, or captures an image after the alarm device outputs an alarm.
  • 6 and 7 are diagrams showing the positional relationship between the camera 110 mounted on the vehicle 1 and the driver 3.
  • the camera 110 is provided near the center console of the vehicle 1 and is arranged so that an image around the face of the driver 3 can be taken.
  • the camera 110 is arranged so that the optical axis faces the driver 3.
  • the camera 110 may be arranged so that the optical axis faces the Z-axis direction shown in FIGS. 6 and 7 so that the camera 110 can image both the driver 3 and the passenger in the passenger seat.
  • the image acquisition unit 40 acquires an image around the face of the driver 3 taken by the camera 110.
  • the image acquisition unit 40 organizes the images in chronological order.
  • the face feature detection unit 50 processes the image around the face of the driver 3 to detect the face information of the driver 3.
  • the face information of the driver 3 detects, for example, the position of the face and the position of parts such as eyes constituting the face.
  • the driver state detection unit 10 detects the state of the driver 3 based on the face information of the driver 3. For example, the driver state detection unit 10 obtains the orientation of the driver 3's face from the position of the driver 3's face. Alternatively, for example, the driver state detection unit 10 obtains the open / closed state of the eyelids of the driver 3 from the position of the eyes. The driver state detection unit 10 detects the state of the driver 3 based on the orientation of the face of the driver 3 or the open / closed state of the eyelids of the driver 3.
  • the state of the driver 3 includes the dozing state of the driver 3, the inattentive driving state, and the dead man state.
  • the deadman state is a state in which a person is loose or unconscious.
  • the alarm parameter acquisition unit 70 acquires alarm parameters.
  • the alarm parameters include the attribute information of the driver 3, the environmental information inside the vehicle, the environmental information outside the vehicle, or the time information.
  • the attribute information of the driver 3 includes, for example, information such as the gender, age, and individuality of the driver 3.
  • the environmental information in the vehicle includes, for example, information such as the volume level in the vehicle.
  • the environmental information outside the vehicle includes, for example, information on the traveling area of the vehicle 1 or the traveling environment of the vehicle 1.
  • the traveling environment of the vehicle 1 is a condition of the road on which the vehicle 1 travels, a condition of traffic congestion, a weather, and the like.
  • the time information is information on the time when the vehicle 1 is traveling, and may be, for example, time information or time zone information such as daytime or nighttime.
  • the alarm parameter acquisition unit 70 acquires, for example, the attribute information of the driver 3 that has been input in advance.
  • the alarm parameter acquisition unit 70 acquires, for example, environmental information inside or outside the vehicle detected by a sensor (not shown) mounted on the vehicle 1.
  • the alarm parameter acquisition unit 70 acquires the time from, for example, a clock mounted on the vehicle 1.
  • the alarm control unit 60 includes an alarm database update unit 61, an alarm database storage unit 62, and an alarm determination unit 63.
  • the alarm database storage unit 62 stores the relationship between the alarm parameters and the effective alarm for the driver 3.
  • the alarm database storage unit 62 stores the alarm database as an example of the relationship.
  • the relationship between the state of the driver 3 and the alarm parameters and the effective alarm is stored as a table. The details of the alarm database will be described later.
  • the alarm determination unit 63 outputs to the driver 3 based on the state of the driver 3 detected by the driver state detection unit 10, the alarm parameters acquired by the alarm parameter acquisition unit 70, and the alarm database. Determine the alarm.
  • the alarm control unit 60 controls the alarm device 120 or 130 to output the alarm.
  • the alarm devices 120 and 130 may be devices mounted on the vehicle or may be devices brought into the vehicle by the occupants of the vehicle.
  • the alarm is, for example, a sound alarm.
  • the alarm device 120 or 130 that outputs the alarm of the sound is a speaker installed in the vehicle.
  • the alarm is, for example, a vibration alarm.
  • the alarm device 120 or 130 that outputs the vibration alarm is a vibration device embedded in the steering wheel or seat.
  • the alarm is, for example, a text message alarm.
  • the alarm device 120 or 130 that outputs the text message is a car navigation system or a head-up display.
  • the alarm may be a voice alarm or a vibration alarm output from a smartphone or smart watch brought into the vehicle by the occupant of the vehicle 1.
  • the alarm device 120 or 130 is a smartphone or smartwatch.
  • the alert includes, for example, attention by a call from a call center connected to a calling device present in the vehicle through a communication line.
  • the call from the call center is, for example, a call by an operator.
  • the alarm device 120 or 130 includes a calling device and a call center. The alarm may be configured by combining these.
  • the alarm effect determination unit 20 determines whether or not the alarm was effective for the driver 3 based on the change in the state of the driver 3 before and after the alarm. At this time, the alarm effect determination unit 20 compares the state of the driver 3 detected by the driver state detection unit 10 before the alarm with the state of the driver 3 detected by the driver state detection unit 10 after the alarm. And ask for the change.
  • the alarm effect determination unit 20 determines, for example, whether or not the state of the driver 3 has changed from “with aside” before the alarm to "without aside” after the alarm. When the state of the driver 3 before the alarm is "with aside” and the state after the alarm is “not aside", the alarm effect determination unit 20 determines that the alarm is “effective”. On the other hand, when the state of the driver 3 before the alarm is "with aside” and the state after the alarm is "with aside", the alarm effect determination unit 20 determines that the alarm is "no effect". To do.
  • the alarm effect determination unit 20 determines whether or not the state of the driver 3 has changed from “with doze” before the alarm to "without doze” after the alarm. Similarly, the alarm effect determination unit 20 determines whether or not the state of the driver 3 has changed from the "dead man state” before the alarm to the "not dead man state” after the alarm.
  • the alarm information output unit 30 outputs the determination result by the alarm effect determination unit 20 to the alarm control unit 60.
  • the determination result includes information on whether or not the state of the driver 3 has returned from the abnormal state to the normal state after the alarm. Further, the determination result may include information such as the degree of reaction of the driver 3 to the alarm and the time until the reaction. Further, the alarm information output unit 30 in the second embodiment outputs the alarm parameter and the state of the driver 3 when the alarm is output to the alarm control unit 60 in addition to the determination result.
  • the alarm database update unit 61 updates the alarm database stored in the alarm database storage unit 62 based on the determination result output from the alarm information output unit 30, the alarm parameter, and the state of the driver 3. For example, when the alarm in one driver 3 state and one alarm parameter is "effective", the alarm database update unit 61 updates the alarm database based on the information, so that the accuracy of the alarm database is improved. To do. Similarly, when the alarm is "no effect", the alarm database update unit 61 updates the alarm database based on the information, so that the accuracy of the alarm database is improved.
  • the alarm determination unit 63 makes the first alarm based on the alarm parameters when the alarm is output, the state of the driver 3, and the updated alarm database. Determine another alarm that is different from.
  • the first alarm is output from the alarm device 120.
  • another alarm is an alarm output by another alarm device 130 different from the alarm device 120 that first outputs the alarm. For example, if the first alarm is a sound alarm output from a speaker, another alarm is a vibration alarm output from a vibrating device, a text message alarm displayed on a display device, or a call from a call center. For example, an alarm.
  • the alarm control unit 60 controls the other alarm device 130 to output another alarm. By performing such control by the alarm control unit 60, the alarm output from the next time onward is changed to another alarm different from the first alarm that has already been output.
  • Each function of the driver state detection unit 10, the alarm parameter acquisition unit 70, the alarm control unit 60, the alarm effect determination unit 20, and the alarm information output unit 30 is realized by the processing circuit shown in FIG. 2 or FIG.
  • FIG. 8 is a diagram showing an example of the alarm database according to the second embodiment.
  • the alarm database is an example of the relationship between the driver 3 status, alarm parameters, and effective alarms, and here is a table in which the relationships are stored.
  • the state of the driver 3 includes the dozing state of the driver 3, the inattentive driving state, and the dead man state.
  • the alarm parameters include the attribute information of the driver 3, the environmental information inside the vehicle, the environmental information outside the vehicle, and the time information.
  • the alarm includes the alarm means and the details of the alarm that are effective for the driver.
  • the alarm means corresponds to the alarm device 120 or 130.
  • FIG. 9 is a flowchart showing the driver monitoring method according to the second embodiment.
  • step S10 the image acquisition unit 40 acquires an image around the face of the driver 3.
  • step S20 the face feature detection unit 50 detects the face information of the driver 3 based on the image around the face of the driver 3.
  • step S30 the driver state detection unit 10 detects the state of the driver 3 and determines whether or not the state of the driver 3 is an abnormal state.
  • the driver state detection unit 10 obtains the orientation of the face of the driver 3 or the open / closed state of the eyelids based on the information on the face of the driver 3.
  • the driver state detection unit 10 detects a dozing state, an inattentive driving state, or a dead man state of the driver 3 based on the face orientation of the driver 3 or the open / closed state of the eyelids.
  • step S40 is executed. If the state of the driver 3 is not an abnormal state, step S10 is executed again.
  • step S40 the alarm parameter acquisition unit 70 acquires the alarm parameter when an abnormality of the driver 3 is detected.
  • step S50 the alarm control unit 60 determines an alarm based on the state of the driver 3, the alarm parameters, and the alarm database. Then, the alarm control unit 60 controls the alarm device 120 to output an alarm. In this step, the alarm control unit 60 acquires the state of the driver 3 and the alarm parameters via the alarm information output unit 30.
  • step S60 the alarm effect determination unit 20 detects a change in the state of the driver 3 before and after the alarm. At this time, the alarm effect determination unit 20 compares the state of the driver 3 detected by the driver state detection unit 10 before the alarm with the state of the driver 3 detected by the driver state detection unit 10 after the alarm. And ask for the change.
  • step S70 the alarm effect determination unit 20 determines whether or not the alarm is effective for the driver 3 based on the change in the state of the driver 3 before and after the alarm.
  • step S80 the alarm information output unit 30 outputs the determination result, the alarm parameter, and the state of the driver 3 to the alarm control unit 60.
  • the alarm parameter is an alarm parameter when an abnormality of the driver 3 is detected, that is, an alarm parameter when an alarm is output.
  • the state of the driver 3 is the state of the driver 3 before the alarm.
  • step S90 the alarm database update unit 61 updates the alarm database recorded in the alarm database storage unit 62 based on the determination result, the alarm parameter, and the state of the driver 3.
  • step S100 the alarm determination unit 63 determines another alarm different from the first alarm based on the state of the driver 3, the alarm parameters, and the updated alarm database.
  • another alarm is output from another alarm device 130 different from the alarm device 120 that output the first alarm. That is, the alarm control unit 60 controls the other alarm device 130 to output another alarm. By performing such control by the alarm control unit 60, the alarm output from the next time onward is changed to another alarm different from the first alarm.
  • the driver monitoring device 101 further includes the alarm control unit 60.
  • the alarm control unit 60 controls to output another alarm to another alarm device 130 different from the alarm device 120 that outputs one alarm as the first alarm.
  • the alarm control unit 60 issues another alarm to another alarm device 130 different from the alarm device 120 that outputs one alarm when one alarm is not effective. Control to output.
  • the alarm control unit 60 controls the alarm, the alarm to the driver 3 is changed to another alarm different from the one alarm that has already been output.
  • another alarm device 130 outputs an alarm different from the one alarm as an alarm from the next time onward. To enable. Then, another alarm output by the other alarm device 130 can guide the state of the driver 3 from the abnormal state to the normal state.
  • the alarm information output unit 30 of the driver monitoring device 101 alarms the alarm parameter when one alarm is output as the first alarm and the state of the driver 3. Output to the control unit 60.
  • the alarm parameters include the attribute information of the driver 3, the environmental information inside the vehicle, the environmental information outside the vehicle, or the time information.
  • the alarm control unit 60 determines another alarm based on the determination result, the state of the driver 3, and the alarm parameters.
  • the state of the driver 3 includes the dozing state, the inattentive driving state, or the dead man state of the driver 3.
  • Such a driver monitoring device 101 determines the alarm from the next time onward based on the updated alarm database. Therefore, when one alarm is not effective, the driver monitoring device 101 makes an appropriate distinction based on not only the state of the driver 3 at that time but also information on the attributes of the driver 3, the environment inside and outside the vehicle, and the time. The alarm can be determined. Therefore, from the next time onward, an alarm that is effective for the driver 3, for example, an alarm that is easy to recognize is output.
  • the driver monitoring device 101 controls the alarm device 130 so that a sound alarm is output when the driver 3 is in the inattentive state.
  • the driver monitoring device 101 controls the alarm device 130 so that an alarm due to vibration is output when the driver 3 is in a dozing state. The alarm makes it possible to return the driver 3 to an appropriate operating state.
  • FIG. 10 is a block diagram showing the configuration of the driver monitoring device 101A in the first modification of the second embodiment.
  • the driver monitoring device 101A further includes a driving operation detection unit 80 in addition to the configuration of the driver monitoring device 101 shown in the second embodiment.
  • the driving operation detection unit 80 detects the driving operation of the vehicle 1 by the driver 3 after the alarm.
  • the driving operation detection unit 80 acquires, for example, information on the driving operation of the vehicle 1 from a device (not shown) related to the driving control of the vehicle 1 and detects the operation.
  • the driving operation detection unit 80 detects, for example, that the steering wheel of the vehicle 1 has been operated or the brake has been operated.
  • the alarm effect determination unit 20 determines whether or not the alarm was effective for the driver 3 based on the change in the driving operation of the vehicle 1 detected by the driving operation detection unit 80.
  • the alarm effect determination unit 20 determines, for example, whether or not the steering wheel has been operated by a predetermined operation amount or more after the alarm. Alternatively, for example, the alarm effect determination unit 20 determines whether or not the brake has been operated by a predetermined operation amount or more after the alarm. After the alarm, the alarm effect determination unit 20 determines that the driver 3 has executed an operation for avoiding danger when the driving operation of the vehicle 1 is operated by a predetermined operation amount or more, and the alarm is " It is judged as "effective".
  • Each function of the driving operation detection unit 80 and the alarm effect determination unit 20 is realized by the processing circuit shown in FIG. 2 or FIG.
  • step S60 the driving operation detection unit 80 detects a change in the driving operation of the vehicle 1 by the driver 3 after the alarm. .. Then, in step S70, the alarm effect determination unit 20 determines whether or not the alarm is effective for the driver 3 based on the change in the driving operation of the vehicle 1.
  • the alarm effect determination unit 20 in the first modification of the second embodiment determines whether or not the alarm is effective for the driver 3 in combination with the operation of the alarm effect determination unit 20 of the second embodiment. You may. In other words, the alarm effect determination unit 20 issues an alarm to the driver 3 based on both the change in the state of the driver 3 before and after the alarm and the change in the driving operation of the vehicle 1 by the driver 3 after the alarm. It may be determined whether it is effective or not.
  • the driver monitoring device 101A having such a configuration has the same effect as that of the second embodiment.
  • the alarm control unit 60 according to the second embodiment is different from another alarm device 130 different from the alarm device 120 that outputs the first alarm when the first alarm is not effective. It was to control to output the alarm of.
  • the alarm control unit 60 in the second modification of the second embodiment outputs another alarm to the alarm device 120 that outputs the first alarm when the first alarm is not effective. Take control.
  • the alarm control unit 60 makes a sound different from the first alarm in terms of volume, frequency, sound pattern, etc., with respect to the speaker that outputs the first alarm. Controls to output the alarm of.
  • the alarm control unit 60 differs from the first alarm in the vibration amount, frequency, vibration pattern, etc. with respect to the vibration device that outputs the first alarm. Controls to output another vibration alarm.
  • the alarm control unit 60 indicates the text content, the text display position, the text display color, and the text to the display device that outputs the first alarm. Control is performed so that the display pattern or the like outputs an alarm having a different display from the first alarm.
  • the text display pattern is a lighting display, a blinking display, a scroll display, or the like.
  • the alarm control unit 60 makes a call voice (bass, high temperature, male, female, etc.) or speaks to the call center that outputs the first alarm. Control is performed to output another alarm whose content is different from the first alarm.
  • a call voice basic, high temperature, male, female, etc.
  • the alarm control unit 60 has a first alternative to another alarm device 130 different from the alarm device 120 that outputs the first alarm when the first alarm is not effective. Control may be performed to output an alarm, and control may be performed to output a second alternative alarm to the alarm device 120 that outputs the first alarm.
  • Each function of the alarm control unit 60 in the second modification of the second embodiment is realized by the processing circuit shown in FIG. 2 or FIG.
  • the driver monitoring device having such a configuration has the same effect as that of the second embodiment.
  • the alarm effect determination unit 20 in the third modification of the second embodiment has an effect on the driver 3 among the plurality of alarms based on the change in the state of the driver 3 before and after each of the plurality of alarms output with a time difference. Judge one alarm that is the target. Alternatively, the alarm effect determination unit 20 is effective for the driver 3 among the plurality of alarms based on the change in the driving operation of the vehicle 1 by the driver 3 after each of the plurality of alarms output with a time difference. Judge one alarm. Each function of such an alarm effect determination unit 20 is realized by the processing circuit shown in FIG. 2 or FIG.
  • the alarm control unit 60 controls the alarm device 120 to output a first alarm having the “alarm pattern XX” and a second alarm having the “alarm pattern YY” after a predetermined time. ..
  • the alarm effect determination unit 20 detects a change in the state of the driver 3 or a change in the driving operation before and after the first alarm and the second alarm. Based on the change, the alarm effect determination unit 20 determines, for example, that the first alarm "alarm pattern XX has no effect" and the second alarm "alarm pattern YY has an effect".
  • the alarm device 120 has the following aspects of the driver 3 An alarm is output to.
  • the alarm device 120 When the driver 3 is in the first doze state, the alarm device 120 first outputs a bass "A" alarm as two alarms output with a time lag, and after a predetermined time has elapsed, the bass and the alarm device 120 are output. Outputs an alarm based on the high-pitched sound combination "A + B". Alternatively, the bass "A” alarm is output first, and the treble “B” alarm is output after a predetermined time has elapsed.
  • the alarm device 130 When the driver 3 is in the doze state for the second time, the alarm device 130 first outputs a treble "B" alarm as two alarms output with a time difference, and after a predetermined time elapses, the bass and the alarm device 130 are output. Outputs an alarm based on the high-pitched sound combination "A + B". Alternatively, the alarm of the high-pitched sound "B” is output first, and the alarm of the low-pitched sound "A" is output after a predetermined time has elapsed.
  • the alarm effect determination unit 20 is among the low-pitched sound “A” and high-pitched sound “B” alarms based on the change in the state of the driver 3 before and after each alarm output in such an embodiment or the change in the driving operation. It is possible to determine which alarm is effective. As a result, the driver monitoring device can determine an appropriate alarm to be adopted for the third and subsequent dozing states.
  • the driver monitoring device, the driver monitoring system, and the driver monitoring method according to the third embodiment will be described.
  • the third embodiment is a subordinate concept of the first embodiment, and the driver monitoring device according to the third embodiment includes each configuration of the driver monitoring device 100 according to the first embodiment. The description of the configuration and operation similar to those of the first or second embodiment will be omitted.
  • FIG. 11 is a block diagram showing the configurations of the driver monitoring device 102 and the driver monitoring system 200 according to the third embodiment.
  • the driver monitoring system 200 includes a driver monitoring device 102 and a server 300. Although only one driver monitoring device 102 is shown in FIG. 11 for simplification of description, the driver monitoring system 200 includes a plurality of driver monitoring devices. Each of the plurality of driver monitoring devices has a configuration similar to that of the driver monitoring device 102 shown in FIG. Further, each of the plurality of driver monitoring devices is provided in each of the plurality of vehicles, and monitors the driver of each vehicle. Each of the plurality of vehicles may be scattered all over the world.
  • the alarm information output unit 30 of the driver monitoring device 102 outputs to the server 300 the alarm parameters when the alarm is output and the state of the driver 3 before the alarm, in addition to the determination result by the alarm effect determination unit 20.
  • the determination result includes information on whether or not the state of the driver 3 has returned from the abnormal state to the normal state after the alarm. Further, the determination result may include information such as the degree of reaction of the driver 3 to the alarm and the time until the reaction.
  • the server 300 acquires the determination result by the alarm effect determination unit 20, the alarm parameter when the alarm is output, and the driver status before the alarm from each of the plurality of driver monitoring devices included in the driver monitoring system 200. To do.
  • the server 300 can collect the determination result, the alarm parameter, and the driver status from the driver monitoring devices all over the world. Then, the server 300 learns the relationship between the determination result, the alarm parameter, and the state of the driver 3 based on the information. The server 300 learns the relationship by, for example, rule-based or AI (Artificial Intelligence) technology.
  • AI Artificial Intelligence
  • the server 300 generates the relationship between the alarm parameter and the effective alarm by the learning.
  • the server 300 generates, for example, a new alarm database for seeking the optimum alarm for the driver 3 status and alarm parameters. Then, the server 300 distributes the optimum alarm database as a new alarm database to the alarm control units 60 of each of the plurality of driver monitoring devices.
  • the alarm control unit 60 of the driver monitoring device 102 receives the new alarm database.
  • the alarm database update unit 61 updates the conventional alarm database stored in the alarm database storage unit 62 with a new alarm database.
  • the alarm determination unit 63 determines another alarm based on the state of the driver 3 and the new alarm database.
  • Each function of the driver monitoring device 102 in the second modification of the second embodiment is realized by the processing circuit shown in FIG. 2 or FIG.
  • the server 300 also has a processing circuit similar to the processing circuit shown in FIG. 2 or FIG.
  • the function of the server 300 is realized by the processing circuit.
  • the driver monitoring device 102 may include a log unit (not shown) that temporarily stores the determination result, the alarm parameter, and the state of the driver 3. Then, the alarm information output unit 30 may transmit the determination result, the alarm parameter, and the state of the driver 3 stored in the log unit to the server 300.
  • FIG. 12 is a flowchart showing the driver monitoring method according to the third embodiment.
  • Steps S10 to S90 are the same as the driver monitoring method shown in FIG.
  • step S110 the alarm information output unit 30 outputs the determination result and the alarm parameter to the server 300.
  • the alarm output unit may also output the state of the driver 3 to the server 300.
  • the alarm information output unit 30 may output the alarm database updated in step S90 to the server 300. Even in that case, since the relationship between the determination result, the alarm parameter, and the state of the driver 3 is included in the alarm database, the processing after step S120 can be executed.
  • step S120 the learning process of the alarm database is executed.
  • FIG. 13 is a flowchart showing a learning processing method of the alarm database according to the third embodiment.
  • step S210 the server 300 receives the determination result and the alarm parameter from the plurality of driver monitoring devices.
  • the server 300 receives the state of the driver 3 in addition to the determination result and the alarm parameter.
  • step S220 the server 300 learns the relationship between the determination result and the alarm parameter and generates a new alarm database.
  • the server 300 learns the relationship between the driver 3 and the driver 3 in addition to the determination result and the alarm parameter.
  • step S230 the server 300 transmits a new alarm database to each driver monitoring device.
  • step S130 shown in FIG. 12 is executed.
  • step S130 the alarm database update unit 61 receives the new alarm database generated by the server 300, and updates the conventional alarm database with the new alarm database.
  • step S140 the alarm determination unit 63 determines another alarm different from the first alarm based on the driver 3 status, alarm parameters, and a new alarm database.
  • the alarm control unit 60 controls another alarm device 130 different from the alarm device 120 that outputs the first alarm, or the alarm device 120 that outputs the first alarm to output the other alarm.
  • another alarm different from the first alarm is output.
  • the alarm effect determination unit 20 gives an alarm to the driver 3 based on the change in the driving operation of the vehicle 1 detected by the driving operation detecting unit 80. It may be determined whether or not it was.
  • the alarm information output unit 30 of the driver monitoring device 102 is an alarm parameter when an alarm is output in addition to the determination result, the attribute information of the driver 3, and the environment in the vehicle.
  • An alarm parameter including information, environment information outside the vehicle, or time information is output to the server 300.
  • the alarm control unit 60 has a relationship generated by the server 300 acquiring and learning the determination result and the alarm parameter when the alarm is output, and is a relationship between the alarm parameter and an effective alarm for the driver 3. Get a relationship.
  • the alarm control unit 60 determines another alarm based on the relationship.
  • the driver monitoring system 200 includes a plurality of driver monitoring devices and a server 300.
  • the plurality of driver monitoring devices include the above driver monitoring device 102.
  • the server 300 obtains and learns the determination result by the alarm effect determination unit 20 and the alarm parameter when the alarm is output from each of the plurality of driver monitoring devices, thereby referring to the alarm parameter and the driver 3. Generate relationships with effective alerts. Then, the server 300 transmits the generated relationship to each alarm control unit 60 of the plurality of driver monitoring devices.
  • Such a driver monitoring device 102 determines another alarm based on an alarm database updated based on determination results from all over the world. Therefore, the driver monitoring device 102 enables the alarm device 130 to output an appropriate alarm to the driver 3. Further, such a driver monitoring system 200 makes it possible to construct an effective model, that is, a highly accurate alarm database at an earlier stage than collecting data such as determination results with only one driver monitoring device 102. To do.
  • FIG. 14 is a block diagram showing the configurations of the driver monitoring device 103 and the driver monitoring system 201 in the modified example of the third embodiment.
  • the modified example of the third embodiment is a subordinate concept of the first embodiment, and the driver monitoring device 103 in the modified example of the third embodiment includes each configuration of the driver monitoring device 100 in the first embodiment.
  • the alarm control unit 60 of the driver monitoring system 201 in the modified example of the third embodiment is provided in the server 300.
  • the driver monitoring device 103 is provided in the vehicle, for example. Other configurations are the same as those in the third embodiment.
  • Such a driver monitoring system 201 has the same effect as that of the third embodiment.
  • the driver monitoring device and the driver monitoring method according to the fourth embodiment will be described.
  • the fourth embodiment is a subordinate concept of the first embodiment, and the driver monitoring device according to the fourth embodiment includes each configuration of the driver monitoring device 100 according to the first embodiment.
  • the same configuration and operation as any of the first to third embodiments will be omitted.
  • the alarm control unit 60 of the driver monitoring device is an alarm of one of the plurality of alarms based on the determination result and the priority set for each of the plurality of predetermined alarms. Is determined as another alarm.
  • the priorities "1" to “7” are set as follows for each of the plurality of alarms.
  • the priority “1” has the highest priority, and the priority “7” has the lowest priority.
  • the alarm of priority “1” is an alarm based on the sound (pattern A) output from the vehicle-mounted speaker.
  • the alarm of priority “2” is an alarm based on the sound (pattern B) output from the vehicle-mounted speaker.
  • the alarm of priority “3” is an alarm by vibration (pattern A) output by a vibration device provided on the handle.
  • the alarm of priority "4" is an alarm by vibration (pattern B) output by a vibration device provided on the handle.
  • the alarm of priority "5" is an alarm by a text message output by the display device.
  • the alarm of priority "6” is an alarm uttered from the call center.
  • the alarm of priority "7” is an alarm due to vibration output by the device brought into the vehicle by the driver 3.
  • the alarm control unit 60 determines the alarm with the priority "2" as another alarm.
  • These priority settings can be changed by the driver 3. For example, if the driver 3 dislikes the vibration alarm, the driver 3 can lower the priority of the vibration alarm. For example, the driver 3 changes the priority of the vibration alarm of the above priorities "3" and "4" to the priorities "6" and "7", respectively.
  • Such a driver monitoring device can output an effective alarm to the driver 3 to the alarm device 120 or 130 based on the priority determined for each alarm.
  • the driver monitoring device and the driver monitoring method according to the fifth embodiment will be described.
  • the fifth embodiment is a subordinate concept of the first embodiment, and the driver monitoring device according to the fifth embodiment includes each configuration of the driver monitoring device 100 according to the first embodiment.
  • the same configuration and operation as any of the first to fourth embodiments will be omitted.
  • the alarm control unit 60 of the driver monitoring device is set according to the determination result and the availability information of each of the plurality of alarm devices 120 and 130 that output a plurality of predetermined alarms, respectively.
  • One of the plurality of alarms is determined as another alarm based on the priority.
  • the alarm control unit 60 determines another alarm based on the availability information of each of the plurality of alarm devices 120 and 130, or the priority of the alarm set according to the availability information. ..
  • the alarm control unit 60 determines whether or not the device can be used based on the connection status or charging status of the device (smartphone, watch, etc.) brought into the vehicle 1 by the occupant. Or, for example, when the alarm control unit 60 determines that the vehicle 1 is traveling on a rough road by the vibration sensor provided in the vehicle 1, it determines that the vibration device installed in the handle or the seat cannot be used. Decrease the priority of the alarm by the vibrating device. Alternatively, for example, when the alarm control unit 60 determines that the occupant of the vehicle 1 is talking by the microphone provided in the vehicle 1 and the sound alarm is not effective, the alarm control unit 60 lowers the priority of the alarm by the speaker. Alternatively, for example, the alarm control unit 60 determines the availability of an alarm from the call center in consideration of the communication status, and sets the priority.
  • Such a driver monitoring device can output an effective alarm to the driver 3 to the alarm device 120 or 130 based on the information on the availability of the alarm devices 120 and 130.
  • the driver monitoring device shown in each of the above embodiments can be applied to a system constructed by appropriately combining a navigation device, a communication terminal, a server, and the functions of applications installed in the navigation device.
  • the navigation device includes, for example, a PND (Portable Navigation Device) and the like.
  • Communication terminals include, for example, mobile terminals such as mobile phones, smartphones and tablets.
  • FIG. 15 is a block diagram showing the configuration of the driver monitoring device 100 and the device that operates in connection with the driver monitoring device 100 according to the sixth embodiment.
  • the driver monitoring device 100 and the communication device 140 are provided in the server 300.
  • the driver monitoring device 100 acquires an image around the face of the driver 3 from the camera 110 provided in the vehicle 1 via the communication device 150 and the communication device 140, and detects the state of the driver 3.
  • the driver monitoring device 100 controls the alarm device 120 or the alarm device 130 provided in the vehicle 1 to output another alarm via each communication device.
  • the driver monitoring device 100 By arranging the driver monitoring device 100 on the server 300 in this way, the configuration of the in-vehicle device can be simplified. Further, some of the functions or components of the driver monitoring device 100 may be provided in the server 300, and some of the other components may be provided in the vehicle 1 in a distributed manner.

Abstract

The purpose of the present invention is to provide a driver monitoring device which outputs a determination result about whether or not a warning was effective for the driver. This driver monitoring device controls a warning device so as to output a warning to the driver of a vehicle on the basis of the state of the driver. The driver monitoring device includes a driver state detection unit, a warning effect determination unit and a warning information output unit. The driver state detection unit detects the state of the driver. The warning effect determination unit determines whether or not the warning outputted from the warning device on the basis of the driver's state was effective for the driver on the basis of change in the driver's state before and after the warning or change in the driver's driving operation of the vehicle after the warning. The warning information output unit outputs the determination result from the warning effect determination unit. On the basis of the determination result, warnings to the driver are changed to another warning different from the warning already outputted.

Description

ドライバモニタリング装置、ドライバモニタリングシステムおよびドライバモニタリング方法Driver monitoring device, driver monitoring system and driver monitoring method
　本発明は、ドライバモニタリング装置、ドライバモニタリングシステムおよびドライバモニタリング方法に関する。 The present invention relates to a driver monitoring device, a driver monitoring system, and a driver monitoring method.
　ドライバの状態に応じて警告を出力するドライバモニタリング装置などが知られている。例えば、特許文献１に記載の運転支援装置は、運転者の状態と、運転の適正状態のレベルとの差分に基づいて、運転者に対して報知を行う。その適正状態のレベルは、運転者と運転場面とに対応して予め記憶されている。例えば、車両の運転場面が通学時間帯の通学路である場合、運転者は頻繁に周辺確認をすることが必要である。その場合、特許文献１に記載の運転支援装置は、運転者が周辺確認のために頻繁に脇見をしている状態を、適正状態のレベルとして学習する。これにより、運転支援装置は、報知された運転者が違和感を覚えることを極力低減している。 A driver monitoring device that outputs a warning according to the driver status is known. For example, the driving support device described in Patent Document 1 notifies the driver based on the difference between the state of the driver and the level of the proper state of driving. The level of the proper state is stored in advance corresponding to the driver and the driving scene. For example, when the driving scene of a vehicle is a school road during school hours, the driver needs to frequently check the surrounding area. In that case, the driving support device described in Patent Document 1 learns a state in which the driver frequently looks aside to check the surroundings as a level of an appropriate state. As a result, the driving support device reduces the feeling of discomfort of the notified driver as much as possible.
特開２０１７－２２００９７号公報JP-A-2017-22009
　ドライバの状態に応じて警告を出力するドライバモニタリング装置において、ドライバに対する警報方法が適切でない場合、その警報の結果、ドライバが適正な運転状態へ復帰しないことが懸念される。例えば、ドライバモニタリング装置が、ドライバの居眠りに対して、音による警報を行ったものの、ドライバがその音の警報を認識できず、覚醒状態に戻らないことが懸念される。 In the driver monitoring device that outputs a warning according to the driver status, if the alarm method for the driver is not appropriate, there is a concern that the driver will not return to the proper operating status as a result of the alarm. For example, although the driver monitoring device gives a sound alarm to the driver's doze, there is a concern that the driver cannot recognize the sound alarm and does not return to the awake state.
　本発明は、以上のような課題を解決するためになされたものであり、ドライバに対する警報の効果に基づいて次回以降の警報が変更されるようにするために、警報がドライバに対して効果的であったか否かについての判定結果を出力するドライバモニタリング装置の提供を目的とする。 The present invention has been made to solve the above problems, and the alarm is effective for the driver in order to change the alarm from the next time onward based on the effect of the alarm on the driver. It is an object of the present invention to provide a driver monitoring device that outputs a judgment result as to whether or not it was.
　本発明に係るドライバモニタリング装置は、車両のドライバの状態に基づいて、ドライバへの警報を出力させる制御を、警報装置に対して行う。ドライバモニタリング装置は、ドライバ状態検知部と、警報効果判定部と、警報情報出力部と、を含む。ドライバ状態検知部は、ドライバの状態を検知する。警報効果判定部は、ドライバの状態に基づいて警報装置から出力される警報の前後におけるドライバの状態の変化、もしくは警報の後のドライバによる車両の運転操作の変化に基づいて、警報がドライバに対して効果的であるか否かを判定する。警報情報出力部は、警報効果判定部による判定結果を出力する。警報情報出力部によって出力される判定結果に基づいて、ドライバへの警報が既に出力された警報とは異なる別の警報に変更される。 The driver monitoring device according to the present invention controls the alarm device to output an alarm to the driver based on the state of the driver of the vehicle. The driver monitoring device includes a driver state detection unit, an alarm effect determination unit, and an alarm information output unit. The driver status detection unit detects the driver status. The alarm effect determination unit issues an alarm to the driver based on the change in the driver's state before and after the alarm output from the alarm device based on the driver's state, or the change in the driving operation of the vehicle by the driver after the alarm. To determine whether it is effective or not. The alarm information output unit outputs the determination result by the alarm effect determination unit. Based on the determination result output by the alarm information output unit, the alarm to the driver is changed to another alarm different from the already output alarm.
　本発明によれば、ドライバに対する警報の効果に基づいて次回以降の警報が変更されるようにするために、警報がドライバに対して効果的であったか否かについての判定結果を出力するドライバモニタリング装置の提供が可能である。 According to the present invention, a driver monitoring device that outputs a determination result as to whether or not an alarm is effective for a driver in order to change the alarm from the next time onward based on the effect of the alarm on the driver. Can be provided.
　本発明の目的、特徴、局面、および利点は、以下の詳細な説明と添付図面とによって、より明白になる。 The object, features, aspects, and advantages of the present invention will be made clearer by the following detailed description and accompanying drawings.
実施の形態１におけるドライバモニタリング装置の構成を示すブロック図である。It is a block diagram which shows the structure of the driver monitoring apparatus in Embodiment 1. FIG. ドライバモニタリング装置が有する処理回路の構成の一例を示す図である。It is a figure which shows an example of the structure of the processing circuit which a driver monitoring apparatus has. ドライバモニタリング装置が有する処理回路の構成の別の一例を示す図である。It is a figure which shows another example of the structure of the processing circuit which a driver monitoring apparatus has. 実施の形態１におけるドライバモニタリング方法を示すフローチャートである。It is a flowchart which shows the driver monitoring method in Embodiment 1. 実施の形態２におけるドライバモニタリング装置の構成を示すブロック図である。It is a block diagram which shows the structure of the driver monitoring apparatus in Embodiment 2. FIG. 車両に搭載されたカメラとドライバとの位置関係を示す図である。It is a figure which shows the positional relationship between a camera mounted on a vehicle, and a driver. 車両に搭載されたカメラとドライバとの位置関係を示す図である。It is a figure which shows the positional relationship between a camera mounted on a vehicle, and a driver. 実施の形態２における警報データベースの一例を示す図である。It is a figure which shows an example of the alarm database in Embodiment 2. FIG. 実施の形態２におけるドライバモニタリング方法を示すフローチャートである。It is a flowchart which shows the driver monitoring method in Embodiment 2. 実施の形態２の変形例１におけるドライバモニタリング装置の構成を示すブロック図である。It is a block diagram which shows the structure of the driver monitoring apparatus in the modification 1 of Embodiment 2. 実施の形態３におけるドライバモニタリング装置およびドライバモニタリングシステムの構成を示すブロック図である。It is a block diagram which shows the structure of the driver monitoring apparatus and the driver monitoring system in Embodiment 3. 実施の形態３におけるドライバモニタリング方法を示すフローチャートである。It is a flowchart which shows the driver monitoring method in Embodiment 3. 実施の形態３における警報データベースの学習処理方法を示すフローチャートである。It is a flowchart which shows the learning processing method of the alarm database in Embodiment 3. 実施の形態３の変形例におけるドライバモニタリング装置およびドライバモニタリングシステムの構成を示すブロック図である。It is a block diagram which shows the structure of the driver monitoring apparatus and the driver monitoring system in the modification of Embodiment 3. 実施の形態６におけるドライバモニタリング装置およびそれに関連して動作する装置の構成を示すブロック図である。It is a block diagram which shows the structure of the driver monitoring apparatus and the apparatus which operates in connection with it in Embodiment 6.
　＜実施の形態１＞
　実施の形態１におけるドライバモニタリング装置およびドライバモニタリング方法を説明する。図１は、実施の形態１におけるドライバモニタリング装置１００の構成を示すブロック図である。 <Embodiment 1>
The driver monitoring device and the driver monitoring method according to the first embodiment will be described. FIG. 1 is a block diagram showing the configuration of the driver monitoring device 100 according to the first embodiment.
　ドライバモニタリング装置１００は、車両のドライバの状態に基づいて、ドライバへの警報を出力させる制御を、警報装置（図示せず）に対して行う。ドライバモニタリング装置１００は、ドライバ状態検知部１０、警報効果判定部２０および警報情報出力部３０を含む。 The driver monitoring device 100 controls the alarm device (not shown) to output an alarm to the driver based on the state of the driver of the vehicle. The driver monitoring device 100 includes a driver state detection unit 10, an alarm effect determination unit 20, and an alarm information output unit 30.
　ドライバ状態検知部１０は、ドライバの状態を検知する。警報装置は、そのドライバの状態に基づいて警報を出力する。 The driver state detection unit 10 detects the driver state. The alarm device outputs an alarm based on the state of the driver.
　警報効果判定部２０は、警報装置から出力される警報の前後におけるドライバの状態の変化、もしくは警報の後のドライバによる車両の運転操作の変化に基づいて、警報がドライバに対して効果的であるか否かを判定する。この際、警報効果判定部２０は、ドライバ状態検知部１０によって警報前後のそれぞれにおいて検知される２つのドライバの状態に基づいて、そのドライバの状態の変化を検出する。または、警報効果判定部２０は、警報後のドライバによる車両の運転操作の変化を、車両の運転制御に関連する装置（図示せず）から取得することで検出する。 In the alarm effect determination unit 20, the alarm is effective for the driver based on the change in the driver's state before and after the alarm output from the alarm device or the change in the driving operation of the vehicle by the driver after the alarm. Judge whether or not. At this time, the alarm effect determination unit 20 detects a change in the driver state based on the states of the two drivers detected by the driver state detection unit 10 before and after the alarm. Alternatively, the alarm effect determination unit 20 detects a change in the driving operation of the vehicle by the driver after the alarm by acquiring it from a device (not shown) related to the driving control of the vehicle.
　警報情報出力部３０は、警報効果判定部２０による判定結果を出力する。その判定結果に基づいて、ドライバへの警報が既に出力された警報とは異なる別の警報に変更される。 The alarm information output unit 30 outputs the determination result by the alarm effect determination unit 20. Based on the determination result, the alarm to the driver is changed to another alarm different from the alarm that has already been output.
　図２は、ドライバモニタリング装置１００が有する処理回路９０の構成の一例を示す図である。ドライバ状態検知部１０、警報効果判定部２０および警報情報出力部３０の各機能は、処理回路９０により実現される。すなわち、処理回路９０は、ドライバ状態検知部１０、警報効果判定部２０および警報情報出力部３０を有する。 FIG. 2 is a diagram showing an example of the configuration of the processing circuit 90 included in the driver monitoring device 100. Each function of the driver state detection unit 10, the alarm effect determination unit 20, and the alarm information output unit 30 is realized by the processing circuit 90. That is, the processing circuit 90 includes a driver state detection unit 10, an alarm effect determination unit 20, and an alarm information output unit 30.
　処理回路９０が専用のハードウェアである場合、処理回路９０は、例えば、単一回路、複合回路、プログラム化されたプロセッサ、並列プログラム化されたプロセッサ、ＡＳＩＣ（Application Specific Integrated Circuit）、ＦＰＧＡ（Field-Programmable Gate Array）、またはこれらを組み合わせた回路等である。ドライバ状態検知部１０、警報効果判定部２０および警報情報出力部３０の各機能は、複数の処理回路により個別に実現されてもよいし、１つの処理回路によりまとめて実現されてもよい。 When the processing circuit 90 is dedicated hardware, the processing circuit 90 may be, for example, a single circuit, a composite circuit, a programmed processor, a parallel programmed processor, an ASIC (Application Specific Integrated Circuit), or an FPGA (Field). -ProgrammableGateArray), or a circuit that combines these. The functions of the driver state detection unit 10, the alarm effect determination unit 20, and the alarm information output unit 30 may be individually realized by a plurality of processing circuits, or may be collectively realized by one processing circuit.
　図３は、ドライバモニタリング装置１００が有する処理回路の構成の別の一例を示す図である。処理回路は、プロセッサ９１とメモリ９２とを有する。プロセッサ９１がメモリ９２に格納されるプログラムを実行することにより、ドライバ状態検知部１０、警報効果判定部２０および警報情報出力部３０の各機能が実現される。例えば、プログラムとして記述されたソフトウェアまたはファームウェアがプロセッサ９１により実行されることにより各機能が実現される。このように、ドライバモニタリング装置１００は、プログラムを格納するメモリ９２と、そのプログラムを実行するプロセッサ９１とを有する。 FIG. 3 is a diagram showing another example of the configuration of the processing circuit included in the driver monitoring device 100. The processing circuit includes a processor 91 and a memory 92. By executing the program stored in the memory 92 by the processor 91, each function of the driver state detection unit 10, the alarm effect determination unit 20, and the alarm information output unit 30 is realized. For example, each function is realized by executing software or firmware described as a program by the processor 91. As described above, the driver monitoring device 100 has a memory 92 for storing the program and a processor 91 for executing the program.
　プログラムには、ドライバモニタリング装置１００が、ドライバの状態を検知し、ドライバの状態に基づいて警報装置から出力される警報の前後におけるドライバの状態の変化、もしくは警報の後のドライバによる車両の運転操作の変化に基づいて、警報がドライバに対して効果的であるか否かを判定し、判定結果を出力する機能が記述されている。また、プログラムは、ドライバ状態検知部１０、警報効果判定部２０および警報情報出力部３０の手順または方法をコンピュータに実行させるものである。 In the program, the driver monitoring device 100 detects the driver's state, and the driver's state changes before and after the alarm output from the alarm device based on the driver's state, or the driver operates the vehicle after the alarm. The function of determining whether or not the alarm is effective for the driver based on the change in the above and outputting the determination result is described. Further, the program causes the computer to execute the procedure or method of the driver state detection unit 10, the alarm effect determination unit 20, and the alarm information output unit 30.
　プロセッサ９１は、例えば、ＣＰＵ（Central Processing Unit）、演算装置、マイクロプロセッサ、マイクロコンピュータ、ＤＳＰ（Digital Signal Processor）等である。メモリ９２は、例えば、ＲＡＭ（Random Access Memory）、ＲＯＭ（Read Only Memory）、フラッシュメモリ、ＥＰＲＯＭ（Erasable Programmable Read Only Memory）、ＥＥＰＲＯＭ（Electrically Erasable Programmable Read Only Memory）等の、不揮発性または揮発性の半導体メモリである。または、メモリ９２は、磁気ディスク、フレキシブルディスク、光ディスク、コンパクトディスク、ミニディスク、ＤＶＤ等、今後使用されるあらゆる記憶媒体であってもよい。 The processor 91 is, for example, a CPU (Central Processing Unit), an arithmetic unit, a microprocessor, a microcomputer, a DSP (Digital Signal Processor), or the like. The memory 92 is, for example, non-volatile or volatile such as RAM (RandomAccessMemory), ROM (ReadOnlyMemory), flash memory, EPROM (ErasableProgrammableReadOnlyMemory), EEPROM (ElectricallyErasableProgrammableReadOnlyMemory). It is a semiconductor memory. Alternatively, the memory 92 may be any storage medium used in the future, such as a magnetic disk, a flexible disk, an optical disk, a compact disk, a mini disk, or a DVD.
　上述したドライバ状態検知部１０、警報効果判定部２０および警報情報出力部３０の各機能は、一部が専用のハードウェアによって実現され、他の一部がソフトウェアまたはファームウェアにより実現されてもよい。このように、処理回路は、ハードウェア、ソフトウェア、ファームウェア、またはこれらの組み合わせによって、上述の各機能を実現する。 Some of the functions of the driver state detection unit 10, the alarm effect determination unit 20, and the alarm information output unit 30 described above may be realized by dedicated hardware, and some may be realized by software or firmware. In this way, the processing circuit realizes each of the above-mentioned functions by hardware, software, firmware, or a combination thereof.
　図４は、実施の形態１におけるドライバモニタリング方法を示すフローチャートである。 FIG. 4 is a flowchart showing the driver monitoring method according to the first embodiment.
　ステップＳ１にて、ドライバ状態検知部１０は、車両のドライバの状態を検知する。警報装置は、そのドライバの状態に基づいて警報を出力する。 In step S1, the driver state detection unit 10 detects the state of the driver of the vehicle. The alarm device outputs an alarm based on the state of the driver.
　ステップＳ２にて、警報効果判定部２０は、警報前後のドライバの状態の変化もしくは警報後のドライバによる車両の運転操作の変化を検出する。 In step S2, the alarm effect determination unit 20 detects a change in the driver's state before and after the alarm or a change in the driving operation of the vehicle by the driver after the alarm.
　ステップＳ３にて、警報効果判定部２０は、警報前後のドライバの状態の変化もしくは警報後のドライバによる車両の運転操作の変化に基づいて、警報がドライバに対して効果的であるか否かを判定する。 In step S3, the alarm effect determination unit 20 determines whether or not the alarm is effective for the driver based on the change in the driver's state before and after the alarm or the change in the driving operation of the vehicle by the driver after the alarm. judge.
　ステップＳ４にて、警報情報出力部３０は、その判定結果を出力する。その判定結果に基づいて、ドライバへの警報が既に出力された警報とは異なる別の警報に変更される。 In step S4, the alarm information output unit 30 outputs the determination result. Based on the determination result, the alarm to the driver is changed to another alarm different from the alarm that has already been output.
　以上をまとめると、実施の形態１におけるドライバモニタリング装置１００は、車両のドライバの状態に基づいて、ドライバへの警報を出力させる制御を、警報装置に対して行う。ドライバモニタリング装置１００は、ドライバ状態検知部１０と、警報効果判定部２０と、警報情報出力部３０と、を含む。ドライバ状態検知部１０は、ドライバの状態を検知する。警報効果判定部２０は、ドライバの状態に基づいて警報装置から出力される警報の前後におけるドライバの状態の変化、もしくは警報の後のドライバによる車両の運転操作の変化に基づいて、警報がドライバに対して効果的であるか否かを判定する。警報情報出力部３０は、警報効果判定部２０による判定結果を出力する。警報情報出力部３０によって出力される判定結果に基づいて、ドライバへの警報が既に出力された警報とは異なる別の警報に変更される。 Summarizing the above, the driver monitoring device 100 in the first embodiment controls the alarm device to output an alarm to the driver based on the state of the driver of the vehicle. The driver monitoring device 100 includes a driver state detection unit 10, an alarm effect determination unit 20, and an alarm information output unit 30. The driver state detection unit 10 detects the driver state. The alarm effect determination unit 20 sends an alarm to the driver based on the change in the driver's state before and after the alarm output from the alarm device based on the driver's state, or the change in the driving operation of the vehicle by the driver after the alarm. It is determined whether or not it is effective. The alarm information output unit 30 outputs the determination result by the alarm effect determination unit 20. Based on the determination result output by the alarm information output unit 30, the alarm to the driver is changed to another alarm different from the already output alarm.
　このようなドライバモニタリング装置１００は、警報がドライバに対して効果的であったか否かについての判定結果を出力することを可能する。判定結果に基づいて次回以降の警報が変更されることにより、ドライバモニタリング装置１００は、ドライバの状態を異常状態から正常状態に誘導することができる。 Such a driver monitoring device 100 can output a determination result as to whether or not the alarm was effective for the driver. By changing the alarm from the next time onward based on the determination result, the driver monitoring device 100 can guide the driver's state from the abnormal state to the normal state.
　また、実施の形態１におけるドライバモニタリング方法は、車両のドライバの状態に基づいて、ドライバへの警報を出力させる制御を、警報装置に対して行う。ドライバモニタリング方法は、ドライバの状態を検知し、ドライバの状態に基づいて警報装置から出力される警報の前後におけるドライバの状態の変化、もしくは警報の後のドライバによる車両の運転操作の変化に基づいて、警報がドライバに対して効果的であるか否かを判定し、判定結果を出力する。そして、判定結果に基づいて、ドライバへの警報が既に出力された警報とは異なる別の警報に変更される。 Further, in the driver monitoring method in the first embodiment, the alarm device is controlled to output an alarm to the driver based on the state of the driver of the vehicle. The driver monitoring method detects the driver's condition and is based on the change in the driver's condition before and after the alarm output from the alarm device based on the driver's condition, or the change in the driving operation of the vehicle by the driver after the alarm. , Judges whether the alarm is effective for the driver and outputs the judgment result. Then, based on the determination result, the alarm to the driver is changed to another alarm different from the alarm that has already been output.
　このようなドライバモニタリング方法は、警報がドライバに対して効果的であったか否かについての判定結果を出力することを可能する。判定結果に基づいて次回以降の警報が変更されることにより、ドライバの状態が異常状態から正常状態に誘導される。 Such a driver monitoring method can output a judgment result as to whether or not the alarm was effective for the driver. By changing the alarm from the next time onward based on the judgment result, the driver's state is guided from the abnormal state to the normal state.
　＜実施の形態２＞
　実施の形態２におけるドライバモニタリング装置およびドライバモニタリング方法を説明する。実施の形態２は実施の形態１の下位概念であり、実施の形態２におけるドライバモニタリング装置は、実施の形態１におけるドライバモニタリング装置１００の各構成を含む。なお、実施の形態１と同様の構成および動作については説明を省略する。 <Embodiment 2>
The driver monitoring device and the driver monitoring method according to the second embodiment will be described. The second embodiment is a subordinate concept of the first embodiment, and the driver monitoring device according to the second embodiment includes each configuration of the driver monitoring device 100 according to the first embodiment. The same configuration and operation as in the first embodiment will not be described.
　図５は、実施の形態２におけるドライバモニタリング装置１０１の構成を示すブロック図である。ドライバモニタリング装置１０１は、画像取得部４０、顔特徴検出部５０、ドライバ状態検知部１０、警報パラメータ取得部７０、警報効果判定部２０、警報情報出力部３０および警報制御部６０を含む。また、図２には、ドライバモニタリング装置１０１と関連して動作する装置として、カメラ１１０および警報装置が示されている。ここでは、警報装置の一例として、一の警報装置１２０および別の警報装置１３０が示されている。実施の形態２におけるドライバモニタリング装置１０１は、一例として、車両（図示せず）に搭載されている。 FIG. 5 is a block diagram showing the configuration of the driver monitoring device 101 according to the second embodiment. The driver monitoring device 101 includes an image acquisition unit 40, a face feature detection unit 50, a driver state detection unit 10, an alarm parameter acquisition unit 70, an alarm effect determination unit 20, an alarm information output unit 30, and an alarm control unit 60. Further, FIG. 2 shows a camera 110 and an alarm device as devices that operate in connection with the driver monitoring device 101. Here, as an example of the alarm device, one alarm device 120 and another alarm device 130 are shown. The driver monitoring device 101 according to the second embodiment is mounted on a vehicle (not shown) as an example.
　カメラ１１０は、車両内に設けられており、ドライバの顔周辺の画像を撮影する。カメラ１１０は、例えば、予め定められたタイミングで画像を撮影する、または警報装置が警報を出力した後に画像を撮影する。図６および図７は、車両１に搭載されたカメラ１１０とドライバ３との位置関係を示す図である。カメラ１１０は、車両１のセンターコンソール付近に設けられ、ドライバ３の顔周辺の画像を撮影可能なように配置されている。図６および図７においては、カメラ１１０は、光軸がドライバ３の方向に向くように配置されている。ただし、カメラ１１０がドライバ３および助手席の乗員の両方を撮像できるように、カメラ１１０は、光軸が図６および図７に示されるＺ軸の方向に向くように配置されてもよい。 The camera 110 is provided inside the vehicle and captures an image around the driver's face. The camera 110, for example, captures an image at a predetermined timing, or captures an image after the alarm device outputs an alarm. 6 and 7 are diagrams showing the positional relationship between the camera 110 mounted on the vehicle 1 and the driver 3. The camera 110 is provided near the center console of the vehicle 1 and is arranged so that an image around the face of the driver 3 can be taken. In FIGS. 6 and 7, the camera 110 is arranged so that the optical axis faces the driver 3. However, the camera 110 may be arranged so that the optical axis faces the Z-axis direction shown in FIGS. 6 and 7 so that the camera 110 can image both the driver 3 and the passenger in the passenger seat.
　画像取得部４０は、カメラ１１０によって撮影されたドライバ３の顔周辺の画像を取得する。画像取得部４０は、画像を時系列的に整理する。 The image acquisition unit 40 acquires an image around the face of the driver 3 taken by the camera 110. The image acquisition unit 40 organizes the images in chronological order.
　顔特徴検出部５０は、ドライバ３の顔周辺の画像を処理して、ドライバ３の顔情報を検出する。ドライバ３の顔情報とは、例えば、顔の位置、および顔を構成する目などのパーツの位置を検出する。 The face feature detection unit 50 processes the image around the face of the driver 3 to detect the face information of the driver 3. The face information of the driver 3 detects, for example, the position of the face and the position of parts such as eyes constituting the face.
　ドライバ状態検知部１０は、ドライバ３の顔情報に基づいてドライバ３の状態を検知する。例えば、ドライバ状態検知部１０は、ドライバ３の顔の位置からドライバ３の顔の向きを求める。または例えば、ドライバ状態検知部１０は、目の位置からドライバ３の瞼の開閉状況を求める。ドライバ状態検知部１０は、ドライバ３の顔の向きまたはドライバ３の瞼の開閉状況に基づいて、ドライバ３の状態を検知する。ドライバ３の状態は、ドライバ３の居眠り状態、脇見運転状態およびデッドマン状態を含む。デッドマン状態とは、人がぐったりしている状態、または意識喪失状態である。 The driver state detection unit 10 detects the state of the driver 3 based on the face information of the driver 3. For example, the driver state detection unit 10 obtains the orientation of the driver 3's face from the position of the driver 3's face. Alternatively, for example, the driver state detection unit 10 obtains the open / closed state of the eyelids of the driver 3 from the position of the eyes. The driver state detection unit 10 detects the state of the driver 3 based on the orientation of the face of the driver 3 or the open / closed state of the eyelids of the driver 3. The state of the driver 3 includes the dozing state of the driver 3, the inattentive driving state, and the dead man state. The deadman state is a state in which a person is loose or unconscious.
　警報パラメータ取得部７０は、警報パラメータを取得する。警報パラメータは、ドライバ３の属性情報、車内の環境情報、車外の環境情報もしくは時間情報を含む。ドライバ３の属性情報は、例えば、ドライバ３の性別、年齢、個性等の情報を含む。車内の環境情報は、例えば、車内の音量の大きさなどの情報を含む。車外の環境情報は、例えば、車両１の走行地域または車両１の走行環境の情報を含む。車両１の走行環境とは、車両１が走行する道路の状態、渋滞の状態、天気などである。時間情報は、車両１が走行している時間に関する情報であり、例えば、時刻情報であってもよいし、昼もしくは夜等の時間帯の情報であってもよい。警報パラメータ取得部７０は、例えば、予め入力されていたドライバ３の属性情報を取得する。警報パラメータ取得部７０は、例えば、車両１に搭載されたセンサ（図示せず）によって検出される車内もしくは車外の環境情報を取得する。警報パラメータ取得部７０は、例えば、車両１に搭載された時計から時刻を取得する。 The alarm parameter acquisition unit 70 acquires alarm parameters. The alarm parameters include the attribute information of the driver 3, the environmental information inside the vehicle, the environmental information outside the vehicle, or the time information. The attribute information of the driver 3 includes, for example, information such as the gender, age, and individuality of the driver 3. The environmental information in the vehicle includes, for example, information such as the volume level in the vehicle. The environmental information outside the vehicle includes, for example, information on the traveling area of the vehicle 1 or the traveling environment of the vehicle 1. The traveling environment of the vehicle 1 is a condition of the road on which the vehicle 1 travels, a condition of traffic congestion, a weather, and the like. The time information is information on the time when the vehicle 1 is traveling, and may be, for example, time information or time zone information such as daytime or nighttime. The alarm parameter acquisition unit 70 acquires, for example, the attribute information of the driver 3 that has been input in advance. The alarm parameter acquisition unit 70 acquires, for example, environmental information inside or outside the vehicle detected by a sensor (not shown) mounted on the vehicle 1. The alarm parameter acquisition unit 70 acquires the time from, for example, a clock mounted on the vehicle 1.
　警報制御部６０は、警報データベース更新部６１、警報データベース記憶部６２および警報決定部６３を含む。 The alarm control unit 60 includes an alarm database update unit 61, an alarm database storage unit 62, and an alarm determination unit 63.
　警報データベース記憶部６２は、警報パラメータとドライバ３に対する効果的な警報との関係を記憶する。ここでは、警報データベース記憶部６２は、その関係の一例として、警報データベースを記憶する。警報データベースは、例えば、ドライバ３の状態と警報パラメータと効果的な警報との関係がテーブルとして格納されている。警報データベースの詳細は、後述する。 The alarm database storage unit 62 stores the relationship between the alarm parameters and the effective alarm for the driver 3. Here, the alarm database storage unit 62 stores the alarm database as an example of the relationship. In the alarm database, for example, the relationship between the state of the driver 3 and the alarm parameters and the effective alarm is stored as a table. The details of the alarm database will be described later.
　警報決定部６３は、ドライバ状態検知部１０にて検知されたドライバ３の状態と、警報パラメータ取得部７０にて取得された警報パラメータと、警報データベースとに基づいて、ドライバ３に対して出力される警報を決定する。 The alarm determination unit 63 outputs to the driver 3 based on the state of the driver 3 detected by the driver state detection unit 10, the alarm parameters acquired by the alarm parameter acquisition unit 70, and the alarm database. Determine the alarm.
　警報制御部６０は、警報装置１２０または１３０に対し、その警報を出力させる制御を行う。警報装置１２０および１３０は、車両に搭載された装置であってもよいし、車両の乗員が車内に持ち込んだ装置であってもよい。 The alarm control unit 60 controls the alarm device 120 or 130 to output the alarm. The alarm devices 120 and 130 may be devices mounted on the vehicle or may be devices brought into the vehicle by the occupants of the vehicle.
　警報は、例えば、音による警報である。その音の警報を出力する警報装置１２０または１３０は、車内に設置されたスピーカである。警報は、例えば、振動による警報である。その振動の警報を出力する警報装置１２０または１３０は、ハンドルまたは座席に埋め込まれた振動デバイスである。警報は、例えば、テキストメッセージによる警報である。そのテキストメッセージを出力する警報装置１２０または１３０は、カーナビまたはヘッドアップディスプレイである。警報は、車両１の乗員が車内に持ち込んだスマートフォンもしくはスマートウォッチから出力される音声による警報または振動による警報であってもよい。この場合、警報装置１２０または１３０は、スマートフォンもしくはスマートウォッチである。警報は、例えば、通信回線を通じて車内に存在する通話装置に接続されるコールセンターからの通話による注意を含む。コールセンターからの通話は、例えば、オペレータによる通話である。この場合、警報装置１２０または１３０は、通話装置およびコールセンターを含む。警報は、これらが組み合わされて構成されてもよい。 The alarm is, for example, a sound alarm. The alarm device 120 or 130 that outputs the alarm of the sound is a speaker installed in the vehicle. The alarm is, for example, a vibration alarm. The alarm device 120 or 130 that outputs the vibration alarm is a vibration device embedded in the steering wheel or seat. The alarm is, for example, a text message alarm. The alarm device 120 or 130 that outputs the text message is a car navigation system or a head-up display. The alarm may be a voice alarm or a vibration alarm output from a smartphone or smart watch brought into the vehicle by the occupant of the vehicle 1. In this case, the alarm device 120 or 130 is a smartphone or smartwatch. The alert includes, for example, attention by a call from a call center connected to a calling device present in the vehicle through a communication line. The call from the call center is, for example, a call by an operator. In this case, the alarm device 120 or 130 includes a calling device and a call center. The alarm may be configured by combining these.
　警報効果判定部２０は、警報の前後におけるドライバ３の状態の変化に基づいて、警報がドライバ３に対して効果的であったか否かを判定する。この際、警報効果判定部２０は、警報の前にドライバ状態検知部１０によって検知されたドライバ３の状態と、警報の後にドライバ状態検知部１０によって検知されたドライバ３の状態と、を比較してその変化を求める。 The alarm effect determination unit 20 determines whether or not the alarm was effective for the driver 3 based on the change in the state of the driver 3 before and after the alarm. At this time, the alarm effect determination unit 20 compares the state of the driver 3 detected by the driver state detection unit 10 before the alarm with the state of the driver 3 detected by the driver state detection unit 10 after the alarm. And ask for the change.
　警報効果判定部２０は、例えば、ドライバ３の状態が警報前の「脇見あり」から、警報後の「脇見なし」に変化したか否かを判定する。警報前のドライバ３の状態が「脇見あり」であり、かつ、警報後のその状態が「脇見なし」である場合、警報効果判定部２０は、その警報は「効果あり」と判定する。一方で、警報前のドライバ３の状態が「脇見あり」であり、かつ、警報後のその状態が「脇見あり」である場合、警報効果判定部２０は、その警報は「効果なし」と判定する。 The alarm effect determination unit 20 determines, for example, whether or not the state of the driver 3 has changed from "with aside" before the alarm to "without aside" after the alarm. When the state of the driver 3 before the alarm is "with aside" and the state after the alarm is "not aside", the alarm effect determination unit 20 determines that the alarm is "effective". On the other hand, when the state of the driver 3 before the alarm is "with aside" and the state after the alarm is "with aside", the alarm effect determination unit 20 determines that the alarm is "no effect". To do.
　同様に、警報効果判定部２０は、ドライバ３の状態が警報前の「居眠りあり」から、警報後の「居眠りなし」に変化したか否かを判定する。同様に、警報効果判定部２０は、ドライバ３の状態が警報前の「デッドマン状態である」から、警報後の「デッドマン状態でない」に変化したか否かを判定する。 Similarly, the alarm effect determination unit 20 determines whether or not the state of the driver 3 has changed from "with doze" before the alarm to "without doze" after the alarm. Similarly, the alarm effect determination unit 20 determines whether or not the state of the driver 3 has changed from the "dead man state" before the alarm to the "not dead man state" after the alarm.
　警報情報出力部３０は、警報効果判定部２０による判定結果を警報制御部６０に出力する。判定結果は、警報後にドライバ３の状態が、異常状態から正常状態に復帰したか否かの情報を含む。また、判定結果は、警報に対するドライバ３の反応の程度、反応までの時間等の情報を含んでもよい。また、実施の形態２における警報情報出力部３０は、判定結果に加えて、警報が出力された際の警報パラメータおよびドライバ３の状態を警報制御部６０に出力する。 The alarm information output unit 30 outputs the determination result by the alarm effect determination unit 20 to the alarm control unit 60. The determination result includes information on whether or not the state of the driver 3 has returned from the abnormal state to the normal state after the alarm. Further, the determination result may include information such as the degree of reaction of the driver 3 to the alarm and the time until the reaction. Further, the alarm information output unit 30 in the second embodiment outputs the alarm parameter and the state of the driver 3 when the alarm is output to the alarm control unit 60 in addition to the determination result.
　警報データベース更新部６１は、警報情報出力部３０から出力される判定結果、警報パラメータおよびドライバ３の状態に基づいて、警報データベース記憶部６２に記憶されている警報データベースを更新する。例えば、一のドライバ３の状態および一の警報パラメータにおける警報が「効果あり」である場合、警報データベース更新部６１が、その情報に基づいて警報データベースを更新することにより、警報データベースの精度が向上する。同様に、警報が「効果なし」である場合、警報データベース更新部６１が、その情報に基づいて警報データベースを更新することにより、警報データベースの精度が向上する。 The alarm database update unit 61 updates the alarm database stored in the alarm database storage unit 62 based on the determination result output from the alarm information output unit 30, the alarm parameter, and the state of the driver 3. For example, when the alarm in one driver 3 state and one alarm parameter is "effective", the alarm database update unit 61 updates the alarm database based on the information, so that the accuracy of the alarm database is improved. To do. Similarly, when the alarm is "no effect", the alarm database update unit 61 updates the alarm database based on the information, so that the accuracy of the alarm database is improved.
　警報決定部６３は、最初の警報が効果的でなかった場合に、その警報が出力された際の警報パラメータと、ドライバ３の状態と、更新された警報データベースとに基づいて、その最初の警報とは異なる別の警報を決定する。実施の形態２において、最初の警報は、警報装置１２０から出力され。また、別の警報は、最初に警報を出力した警報装置１２０とは異なる別の警報装置１３０が出力する警報である。例えば、最初の警報が、スピーカから出力される音の警報である場合、別の警報は、振動デバイスから出力される振動の警報、表示装置に表示されるテキストメッセージによる警報、コールセンターからの通話による警報などである。 When the first alarm is not effective, the alarm determination unit 63 makes the first alarm based on the alarm parameters when the alarm is output, the state of the driver 3, and the updated alarm database. Determine another alarm that is different from. In the second embodiment, the first alarm is output from the alarm device 120. Further, another alarm is an alarm output by another alarm device 130 different from the alarm device 120 that first outputs the alarm. For example, if the first alarm is a sound alarm output from a speaker, another alarm is a vibration alarm output from a vibrating device, a text message alarm displayed on a display device, or a call from a call center. For example, an alarm.
　警報制御部６０は、その別の警報装置１３０に対して、別の警報を出力させる制御を行う。警報制御部６０が、このような制御を行うことにより、次回以降に出力される警報が、既に出力された最初の警報とは異なる別の警報に変更される。 The alarm control unit 60 controls the other alarm device 130 to output another alarm. By performing such control by the alarm control unit 60, the alarm output from the next time onward is changed to another alarm different from the first alarm that has already been output.
　これらドライバ状態検知部１０、警報パラメータ取得部７０、警報制御部６０、警報効果判定部２０および警報情報出力部３０の各機能は、図２または図３に示された処理回路により実現される。 Each function of the driver state detection unit 10, the alarm parameter acquisition unit 70, the alarm control unit 60, the alarm effect determination unit 20, and the alarm information output unit 30 is realized by the processing circuit shown in FIG. 2 or FIG.
　図８は、実施の形態２における警報データベースの一例を示す図である。 FIG. 8 is a diagram showing an example of the alarm database according to the second embodiment.
　警報データベースは、ドライバ３の状態と警報パラメータと効果的な警報との関係の一例であり、ここではその関係格納されたテーブルである。ドライバ３の状態は、ドライバ３の居眠り状態、脇見運転状態およびデッドマン状態を含む。警報パラメータは、ドライバ３の属性情報、車内の環境情報、車外の環境情報および時間情報を含む。警報は、警報手段およびドライバに対して効果的な警報の詳細内容を含む。ここで警報手段とは、警報装置１２０または１３０に対応する。 The alarm database is an example of the relationship between the driver 3 status, alarm parameters, and effective alarms, and here is a table in which the relationships are stored. The state of the driver 3 includes the dozing state of the driver 3, the inattentive driving state, and the dead man state. The alarm parameters include the attribute information of the driver 3, the environmental information inside the vehicle, the environmental information outside the vehicle, and the time information. The alarm includes the alarm means and the details of the alarm that are effective for the driver. Here, the alarm means corresponds to the alarm device 120 or 130.
　図９は、実施の形態２におけるドライバモニタリング方法を示すフローチャートである。 FIG. 9 is a flowchart showing the driver monitoring method according to the second embodiment.
　ステップＳ１０にて、画像取得部４０は、ドライバ３の顔周辺の画像を取得する。 In step S10, the image acquisition unit 40 acquires an image around the face of the driver 3.
　ステップＳ２０にて、顔特徴検出部５０は、ドライバ３の顔周辺の画像に基づき、ドライバ３の顔の情報を検出する。 In step S20, the face feature detection unit 50 detects the face information of the driver 3 based on the image around the face of the driver 3.
　ステップＳ３０にて、ドライバ状態検知部１０は、ドライバ３の状態を検知し、ドライバ３の状態が異常状態であるか否かを判定する。ドライバ状態検知部１０は、ドライバ３の顔の情報に基づいて、ドライバ３の顔の向きまたは瞼の開閉状況を求める。ドライバ状態検知部１０は、それらドライバ３の顔の向きまたは瞼の開閉状況に基づいて、ドライバ３の居眠り状態、脇見運転状態、またはデッドマン状態を検知する。ドライバ３の状態が、居眠り状態、脇見運転状態、またはデッドマン状態である場合、異常状態と判定する。ドライバ３の状態が異常状態である場合、ステップＳ４０が実行される。ドライバ３の状態が異常状態でない場合、ステップＳ１０が再び実行される。 In step S30, the driver state detection unit 10 detects the state of the driver 3 and determines whether or not the state of the driver 3 is an abnormal state. The driver state detection unit 10 obtains the orientation of the face of the driver 3 or the open / closed state of the eyelids based on the information on the face of the driver 3. The driver state detection unit 10 detects a dozing state, an inattentive driving state, or a dead man state of the driver 3 based on the face orientation of the driver 3 or the open / closed state of the eyelids. When the state of the driver 3 is a dozing state, an inattentive driving state, or a dead man state, it is determined to be an abnormal state. If the state of the driver 3 is an abnormal state, step S40 is executed. If the state of the driver 3 is not an abnormal state, step S10 is executed again.
　ステップＳ４０にて、警報パラメータ取得部７０は、ドライバ３の異常が検出された際の警報パラメータを取得する。 In step S40, the alarm parameter acquisition unit 70 acquires the alarm parameter when an abnormality of the driver 3 is detected.
　ステップＳ５０にて、警報制御部６０は、ドライバ３の状態と警報パラメータと警報データベースとに基づいて、警報を決定する。そして、警報制御部６０は、警報装置１２０に対し、警報を出力させる制御を行う。なお、このステップにおいて、警報制御部６０は、ドライバ３の状態と警報パラメータとを警報情報出力部３０を介して取得する。 In step S50, the alarm control unit 60 determines an alarm based on the state of the driver 3, the alarm parameters, and the alarm database. Then, the alarm control unit 60 controls the alarm device 120 to output an alarm. In this step, the alarm control unit 60 acquires the state of the driver 3 and the alarm parameters via the alarm information output unit 30.
　ステップＳ６０にて、警報効果判定部２０は、警報前後のドライバ３の状態の変化を検出する。この際、警報効果判定部２０は、警報の前にドライバ状態検知部１０によって検知されたドライバ３の状態と、警報の後にドライバ状態検知部１０によって検知されたドライバ３の状態と、を比較してその変化を求める。 In step S60, the alarm effect determination unit 20 detects a change in the state of the driver 3 before and after the alarm. At this time, the alarm effect determination unit 20 compares the state of the driver 3 detected by the driver state detection unit 10 before the alarm with the state of the driver 3 detected by the driver state detection unit 10 after the alarm. And ask for the change.
　ステップＳ７０にて、警報効果判定部２０は、警報前後のドライバ３の状態の変化に基づいて、警報がドライバ３に対して効果的であるか否かを判定する。 In step S70, the alarm effect determination unit 20 determines whether or not the alarm is effective for the driver 3 based on the change in the state of the driver 3 before and after the alarm.
　ステップＳ８０にて、警報情報出力部３０は、その判定結果、警報パラメータおよびドライバ３の状態を警報制御部６０に出力する。警報パラメータは、ドライバ３の異常が検出された際の警報パラメータ、すなわち警報が出力された際の警報パラメータである。ドライバ３の状態は、警報前のドライバ３の状態である。 In step S80, the alarm information output unit 30 outputs the determination result, the alarm parameter, and the state of the driver 3 to the alarm control unit 60. The alarm parameter is an alarm parameter when an abnormality of the driver 3 is detected, that is, an alarm parameter when an alarm is output. The state of the driver 3 is the state of the driver 3 before the alarm.
　ステップＳ９０にて、警報データベース更新部６１は、判定結果、警報パラメータおよびドライバ３の状態に基づき、警報データベース記憶部６２に記録されている警報データベースを更新する。 In step S90, the alarm database update unit 61 updates the alarm database recorded in the alarm database storage unit 62 based on the determination result, the alarm parameter, and the state of the driver 3.
　ステップＳ１００にて、警報決定部６３は、ドライバ３の状態と警報パラメータと更新された警報データベースとに基づいて、最初の警報とは異なる別の警報を決定する。ここでは、別の警報は、最初の警報を出力した警報装置１２０とは異なる別の警報装置１３０から出力される。すなわち、警報制御部６０は、その別の警報装置１３０に対して、別の警報を出力させる制御を行う。警報制御部６０が、このような制御を行うことにより、次回以降に出力される警報が、最初の警報とは異なる別の警報に変更される。 In step S100, the alarm determination unit 63 determines another alarm different from the first alarm based on the state of the driver 3, the alarm parameters, and the updated alarm database. Here, another alarm is output from another alarm device 130 different from the alarm device 120 that output the first alarm. That is, the alarm control unit 60 controls the other alarm device 130 to output another alarm. By performing such control by the alarm control unit 60, the alarm output from the next time onward is changed to another alarm different from the first alarm.
　以上をまとめると、実施の形態２におけるドライバモニタリング装置１０１は、警報制御部６０をさらに含む。警報制御部６０は、判定結果に基づいて、最初の警報として一の警報を出力した警報装置１２０とは異なる別の警報装置１３０に対して、別の警報を出力させる制御を行う。ここでは、警報制御部６０は、判定結果として、一の警報が効果的でなかった場合に、一の警報を出力した警報装置１２０とは異なる別の警報装置１３０に対して、別の警報を出力させる制御を行う。警報制御部６０がその制御を行うことにより、ドライバ３への警報が既に出力された一の警報とは異なる別の警報に変更される。 Summarizing the above, the driver monitoring device 101 according to the second embodiment further includes the alarm control unit 60. Based on the determination result, the alarm control unit 60 controls to output another alarm to another alarm device 130 different from the alarm device 120 that outputs one alarm as the first alarm. Here, as a determination result, the alarm control unit 60 issues another alarm to another alarm device 130 different from the alarm device 120 that outputs one alarm when one alarm is not effective. Control to output. When the alarm control unit 60 controls the alarm, the alarm to the driver 3 is changed to another alarm different from the one alarm that has already been output.
　このようなドライバモニタリング装置１０１は、一の警報がドライバ３に対して効果的でなかった場合に、次回以降の警報として、別の警報装置１３０が一の警報とは別の警報を出力することを可能にする。そして、その別の警報装置１３０が出力する別の警報は、ドライバ３の状態を異常状態から正常状態に誘導することができる。 In such a driver monitoring device 101, when one alarm is not effective for the driver 3, another alarm device 130 outputs an alarm different from the one alarm as an alarm from the next time onward. To enable. Then, another alarm output by the other alarm device 130 can guide the state of the driver 3 from the abnormal state to the normal state.
　また、実施の形態２におけるドライバモニタリング装置１０１の警報情報出力部３０は、判定結果に加えて、最初の警報として一の警報が出力された際の警報パラメータと、ドライバ３の状態とを、警報制御部６０に出力する。警報パラメータは、ドライバ３の属性情報、車内の環境情報、車外の環境情報もしくは時間情報を含む。警報制御部６０は、判定結果と、ドライバ３の状態と、警報パラメータと、に基づいて、別の警報を決定する。ドライバ３の状態は、ドライバ３の居眠り状態、脇見運転状態、またはデッドマン状態を含む。 Further, in addition to the determination result, the alarm information output unit 30 of the driver monitoring device 101 according to the second embodiment alarms the alarm parameter when one alarm is output as the first alarm and the state of the driver 3. Output to the control unit 60. The alarm parameters include the attribute information of the driver 3, the environmental information inside the vehicle, the environmental information outside the vehicle, or the time information. The alarm control unit 60 determines another alarm based on the determination result, the state of the driver 3, and the alarm parameters. The state of the driver 3 includes the dozing state, the inattentive driving state, or the dead man state of the driver 3.
　このようなドライバモニタリング装置１０１は、更新された警報データベースに基づいて、次回以降の警報を決定する。そのため、ドライバモニタリング装置１０１は、一の警報が効果的でなかった場合に、その時のドライバ３の状態だけでなく、ドライバ３の属性、車内外の環境、時間に関する情報に基づいて、適切な別の警報を決定することができる。そのため、次回以降、ドライバ３にとって効果的な警報、例えば、認識しやすい警報が出力される。例えば、ドライバモニタリング装置１０１は、ドライバ３が脇見状態である場合は、音による警報が出力されるよう警報装置１３０を制御する。または例えば、ドライバモニタリング装置１０１は、ドライバ３が居眠り状態である場合には、振動による警報が出力されるよう警報装置１３０を制御する。その警報により、ドライバ３を適正な運転状態に復帰させることを可能にする。 Such a driver monitoring device 101 determines the alarm from the next time onward based on the updated alarm database. Therefore, when one alarm is not effective, the driver monitoring device 101 makes an appropriate distinction based on not only the state of the driver 3 at that time but also information on the attributes of the driver 3, the environment inside and outside the vehicle, and the time. The alarm can be determined. Therefore, from the next time onward, an alarm that is effective for the driver 3, for example, an alarm that is easy to recognize is output. For example, the driver monitoring device 101 controls the alarm device 130 so that a sound alarm is output when the driver 3 is in the inattentive state. Alternatively, for example, the driver monitoring device 101 controls the alarm device 130 so that an alarm due to vibration is output when the driver 3 is in a dozing state. The alarm makes it possible to return the driver 3 to an appropriate operating state.
　（実施の形態２の変形例１）
　図１０は、実施の形態２の変形例１におけるドライバモニタリング装置１０１Ａの構成を示すブロック図である。ドライバモニタリング装置１０１Ａは、実施の形態２に示されたドライバモニタリング装置１０１の構成に加え、運転操作検知部８０をさらに含む。 (Modification 1 of Embodiment 2)
FIG. 10 is a block diagram showing the configuration of the driver monitoring device 101A in the first modification of the second embodiment. The driver monitoring device 101A further includes a driving operation detection unit 80 in addition to the configuration of the driver monitoring device 101 shown in the second embodiment.
　運転操作検知部８０は、警報後のドライバ３による車両１の運転操作を検知する。運転操作検知部８０は、例えば、車両１の運転制御に関連する装置（図示せず）から車両１の運転操作の情報を取得して、その操作を検知する。運転操作検知部８０は、例えば、車両１のハンドルの操作またはブレーキが操作されたことを検知する。 The driving operation detection unit 80 detects the driving operation of the vehicle 1 by the driver 3 after the alarm. The driving operation detection unit 80 acquires, for example, information on the driving operation of the vehicle 1 from a device (not shown) related to the driving control of the vehicle 1 and detects the operation. The driving operation detection unit 80 detects, for example, that the steering wheel of the vehicle 1 has been operated or the brake has been operated.
　警報効果判定部２０は、運転操作検知部８０によって検知される車両１の運転操作の変化に基づいて、警報がドライバ３に対して効果的であったか否かを判定する。警報効果判定部２０は、例えば、警報後、ハンドルが予め定められた操作量以上、操作されたか否かを判定する。または例えば、警報効果判定部２０は、警報後、ブレーキが予め定められた操作量以上、操作されたか否かを判定する。警報効果判定部２０は、警報後に、車両１の運転操作が予め定められた操作量以上、操作された場合に、ドライバ３が危険を回避するための操作を実行したと判断し、警報は「効果あり」と判定する。 The alarm effect determination unit 20 determines whether or not the alarm was effective for the driver 3 based on the change in the driving operation of the vehicle 1 detected by the driving operation detection unit 80. The alarm effect determination unit 20 determines, for example, whether or not the steering wheel has been operated by a predetermined operation amount or more after the alarm. Alternatively, for example, the alarm effect determination unit 20 determines whether or not the brake has been operated by a predetermined operation amount or more after the alarm. After the alarm, the alarm effect determination unit 20 determines that the driver 3 has executed an operation for avoiding danger when the driving operation of the vehicle 1 is operated by a predetermined operation amount or more, and the alarm is " It is judged as "effective".
　これら運転操作検知部８０および警報効果判定部２０の各機能は、図２または図３に示された処理回路により実現される。 Each function of the driving operation detection unit 80 and the alarm effect determination unit 20 is realized by the processing circuit shown in FIG. 2 or FIG.
　また、実施の形態２の変形例１においては、図９のフローチャートに示されたステップＳ６０にて、運転操作検知部８０が、警報の後のドライバ３による車両１の運転操作の変化を検知する。そして、ステップＳ７０にて、警報効果判定部２０は、車両１の運転操作の変化に基づいて、その警報がドライバ３に対して効果的であったか否かを判定する。 Further, in the first modification of the second embodiment, in step S60 shown in the flowchart of FIG. 9, the driving operation detection unit 80 detects a change in the driving operation of the vehicle 1 by the driver 3 after the alarm. .. Then, in step S70, the alarm effect determination unit 20 determines whether or not the alarm is effective for the driver 3 based on the change in the driving operation of the vehicle 1.
　さらに、実施の形態２の変形例１における警報効果判定部２０は、実施の形態２の警報効果判定部２０の動作と組み合わせて、警報がドライバ３に対して効果的であるか否かを判定してもよい。言い換えると、警報効果判定部２０は、警報の前後におけるドライバ３の状態の変化、および、警報の後のドライバ３による車両１の運転操作の変化の両方に基づいて、警報がドライバ３に対して効果的であるか否かを判定してもよい。 Further, the alarm effect determination unit 20 in the first modification of the second embodiment determines whether or not the alarm is effective for the driver 3 in combination with the operation of the alarm effect determination unit 20 of the second embodiment. You may. In other words, the alarm effect determination unit 20 issues an alarm to the driver 3 based on both the change in the state of the driver 3 before and after the alarm and the change in the driving operation of the vehicle 1 by the driver 3 after the alarm. It may be determined whether it is effective or not.
　このような構成を有するドライバモニタリング装置１０１Ａは、実施の形態２と同様の効果を奏する。 The driver monitoring device 101A having such a configuration has the same effect as that of the second embodiment.
　（実施の形態２の変形例２）
　実施の形態２における警報制御部６０は、判定結果として、最初の警報が効果的でなかった場合に、その最初の警報を出力した警報装置１２０とは異なる別の警報装置１３０に対して、別の警報を出力させる制御を行うものであった。実施の形態２の変形例２における警報制御部６０は、判定結果として、最初の警報が効果的でなかった場合に、最初の警報を出力した警報装置１２０に対して、別の警報を出力させる制御を行う。 (Modification 2 of Embodiment 2)
As a determination result, the alarm control unit 60 according to the second embodiment is different from another alarm device 130 different from the alarm device 120 that outputs the first alarm when the first alarm is not effective. It was to control to output the alarm of. As a determination result, the alarm control unit 60 in the second modification of the second embodiment outputs another alarm to the alarm device 120 that outputs the first alarm when the first alarm is not effective. Take control.
　例えば、最初の警報が音の警報である場合、警報制御部６０は、その最初の警報を出力したスピーカに対して、音量、周波数、音のパターン等が、最初の警報とは異なる別の音の警報を出力させる制御を行う。 For example, when the first alarm is a sound alarm, the alarm control unit 60 makes a sound different from the first alarm in terms of volume, frequency, sound pattern, etc., with respect to the speaker that outputs the first alarm. Controls to output the alarm of.
　または例えば、最初の警報が振動の警報である場合、警報制御部６０は、その最初の警報を出力した振動デバイスに対して、振動量、周波数、振動のパターン等が、最初の警報とは異なる別の振動の警報を出力させる制御を行う。 Or, for example, when the first alarm is a vibration alarm, the alarm control unit 60 differs from the first alarm in the vibration amount, frequency, vibration pattern, etc. with respect to the vibration device that outputs the first alarm. Controls to output another vibration alarm.
　または例えば、最初の警報がテキストメッセージの警報である場合、警報制御部６０は、その最初の警報を出力した表示装置に対して、テキストの内容、テキストの表示位置、テキストの表示色、テキストの表示パターン等が、最初の警報とは異なる別の表示の警報を出力させる制御を行う。ここで、テキストの表示パターンとは、点灯表示、点滅表示、スクロール表示等である。 Or, for example, when the first alarm is a text message alarm, the alarm control unit 60 indicates the text content, the text display position, the text display color, and the text to the display device that outputs the first alarm. Control is performed so that the display pattern or the like outputs an alarm having a different display from the first alarm. Here, the text display pattern is a lighting display, a blinking display, a scroll display, or the like.
　または例えば、最初の警報がコールセンターからの通話による警報である場合、警報制御部６０は、その最初の警報を出力したコールセンターに対して、通話の声色（低音、高温、男性、女性等）または発話内容が、最初の警報とは異なる別の警報を出力させる制御を行う。 Or, for example, when the first alarm is an alarm by a call from the call center, the alarm control unit 60 makes a call voice (bass, high temperature, male, female, etc.) or speaks to the call center that outputs the first alarm. Control is performed to output another alarm whose content is different from the first alarm.
　さらに、警報制御部６０は、判定結果として、最初の警報が効果的でなかった場合に、最初の警報を出力した警報装置１２０とは異なる別の警報装置１３０に対して、第１の別の警報を出力させる制御を行い、かつ、最初の警報を出力した警報装置１２０に対して、第２の別の警報を出力させる制御を行ってもよい。 Further, as a determination result, the alarm control unit 60 has a first alternative to another alarm device 130 different from the alarm device 120 that outputs the first alarm when the first alarm is not effective. Control may be performed to output an alarm, and control may be performed to output a second alternative alarm to the alarm device 120 that outputs the first alarm.
　実施の形態２の変形例２における警報制御部６０の各機能は、図２または図３に示された処理回路により実現される。 Each function of the alarm control unit 60 in the second modification of the second embodiment is realized by the processing circuit shown in FIG. 2 or FIG.
　このような構成を有するドライバモニタリング装置は、実施の形態２と同様の効果を奏する。 The driver monitoring device having such a configuration has the same effect as that of the second embodiment.
　（実施の形態２の変形例３）
　実施の形態２の変形例３における警報効果判定部２０は、時間差で出力される複数の警報の各々の前後におけるドライバ３の状態の変化に基づいて、複数の警報のうちドライバ３に対して効果的である一の警報を判定する。もしくは、警報効果判定部２０は、時間差で出力される複数の警報の各々の後のドライバ３による車両１の運転操作の変化に基づいて、複数の警報のうちドライバ３に対して効果的である一の警報を判定する。このような警報効果判定部２０の各機能は、図２または図３に示された処理回路により実現される。 (Modification 3 of Embodiment 2)
The alarm effect determination unit 20 in the third modification of the second embodiment has an effect on the driver 3 among the plurality of alarms based on the change in the state of the driver 3 before and after each of the plurality of alarms output with a time difference. Judge one alarm that is the target. Alternatively, the alarm effect determination unit 20 is effective for the driver 3 among the plurality of alarms based on the change in the driving operation of the vehicle 1 by the driver 3 after each of the plurality of alarms output with a time difference. Judge one alarm. Each function of such an alarm effect determination unit 20 is realized by the processing circuit shown in FIG. 2 or FIG.
　例えば、警報制御部６０は、警報装置１２０に対し、「警報パターンＸＸ」を有する第１警報および「警報パターンＹＹ」を有する第２警報を、予め定められた時間を空けて出力させる制御を行う。警報効果判定部２０は、第１警報および第２警報の前後において、ドライバ３の状態の変化または運転操作の変化を検知する。警報効果判定部２０は、その変化に基づいて、例えば、第１警報の「警報パターンＸＸは効果なし」、第２警報の「警報パターンＹＹは効果あり」と判定する。 For example, the alarm control unit 60 controls the alarm device 120 to output a first alarm having the “alarm pattern XX” and a second alarm having the “alarm pattern YY” after a predetermined time. .. The alarm effect determination unit 20 detects a change in the state of the driver 3 or a change in the driving operation before and after the first alarm and the second alarm. Based on the change, the alarm effect determination unit 20 determines, for example, that the first alarm "alarm pattern XX has no effect" and the second alarm "alarm pattern YY has an effect".
　以下に具体例を示す。例えば、居眠り状態であるドライバ３に対し、低音「Ａ」の警報および高音「Ｂ」の警報の、いずれの警報が効果的であるかを判定するため、警報装置１２０は以下の態様でドライバ３に対し警報を出力する。 A specific example is shown below. For example, in order to determine which of the low-pitched sound "A" alarm and the high-pitched sound "B" alarm is effective for the driver 3 who is in a dozing state, the alarm device 120 has the following aspects of the driver 3 An alarm is output to.
　ドライバ３が１回目の居眠り状態にある場合、警報装置１２０は、時間差で出力される２つの警報として、最初に低音「Ａ」の警報を出力し、予め定められた時間の経過後、低音および高音の組み合わせ「Ａ＋Ｂ」による警報を出力する。または、最初に低音「Ａ」の警報を出力し、予め定められた時間の経過後、高音「Ｂ」の警報を出力する。 When the driver 3 is in the first doze state, the alarm device 120 first outputs a bass "A" alarm as two alarms output with a time lag, and after a predetermined time has elapsed, the bass and the alarm device 120 are output. Outputs an alarm based on the high-pitched sound combination "A + B". Alternatively, the bass "A" alarm is output first, and the treble "B" alarm is output after a predetermined time has elapsed.
　ドライバ３が２回目の居眠り状態にある場合、警報装置１３０は、時間差で出力される２つの警報として、最初に高音「Ｂ」の警報を出力し、予め定められた時間の経過後、低音および高音の組み合わせ「Ａ＋Ｂ」による警報を出力する。または、最初に高音「Ｂ」の警報を出力し、予め定められた時間の経過後、低音「Ａ」の警報を出力する。 When the driver 3 is in the doze state for the second time, the alarm device 130 first outputs a treble "B" alarm as two alarms output with a time difference, and after a predetermined time elapses, the bass and the alarm device 130 are output. Outputs an alarm based on the high-pitched sound combination "A + B". Alternatively, the alarm of the high-pitched sound "B" is output first, and the alarm of the low-pitched sound "A" is output after a predetermined time has elapsed.
　警報効果判定部２０は、このような態様で出力された各警報の前後におけるドライバ３の状態の変化、もしくは運転操作の変化に基づいて、低音「Ａ」および高音「Ｂ」の警報のうち、どちらの警報が効果的であるかを判定することができる。その結果、ドライバモニタリング装置は、３回目以降の居眠り状態に採用するべき適切な警報を決定することができる。 The alarm effect determination unit 20 is among the low-pitched sound “A” and high-pitched sound “B” alarms based on the change in the state of the driver 3 before and after each alarm output in such an embodiment or the change in the driving operation. It is possible to determine which alarm is effective. As a result, the driver monitoring device can determine an appropriate alarm to be adopted for the third and subsequent dozing states.
　＜実施の形態３＞
　実施の形態３におけるドライバモニタリング装置、ドライバモニタリングシステムおよびドライバモニタリング方法を説明する。実施の形態３は実施の形態１の下位概念であり、実施の形態３におけるドライバモニタリング装置は、実施の形態１におけるドライバモニタリング装置１００の各構成を含む。なお、実施の形態１または２と同様の構成および動作については説明を省略する。 <Embodiment 3>
The driver monitoring device, the driver monitoring system, and the driver monitoring method according to the third embodiment will be described. The third embodiment is a subordinate concept of the first embodiment, and the driver monitoring device according to the third embodiment includes each configuration of the driver monitoring device 100 according to the first embodiment. The description of the configuration and operation similar to those of the first or second embodiment will be omitted.
　図１１は、実施の形態３におけるドライバモニタリング装置１０２およびドライバモニタリングシステム２００の構成を示すブロック図である。 FIG. 11 is a block diagram showing the configurations of the driver monitoring device 102 and the driver monitoring system 200 according to the third embodiment.
　ドライバモニタリングシステム２００は、ドライバモニタリング装置１０２およびサーバ３００を含む。なお、図１１には、説明を簡単にするため、１つのドライバモニタリング装置１０２だけが示されているが、ドライバモニタリングシステム２００は、複数のドライバモニタリング装置を含む。その複数のドライバモニタリング装置の各々は、図１１に示されるドライバモニタリング装置１０２と同様の構成を有する。また、複数のドライバモニタリング装置の各々は、複数の車両の各々に設けられており、各車両のドライバをモニタリングする。複数の車両の各々は、世界中に点在していてもよい。 The driver monitoring system 200 includes a driver monitoring device 102 and a server 300. Although only one driver monitoring device 102 is shown in FIG. 11 for simplification of description, the driver monitoring system 200 includes a plurality of driver monitoring devices. Each of the plurality of driver monitoring devices has a configuration similar to that of the driver monitoring device 102 shown in FIG. Further, each of the plurality of driver monitoring devices is provided in each of the plurality of vehicles, and monitors the driver of each vehicle. Each of the plurality of vehicles may be scattered all over the world.
　ドライバモニタリング装置１０２の警報情報出力部３０は、警報効果判定部２０による判定結果に加えて、警報が出力された際の警報パラメータと、警報前のドライバ３の状態とをサーバ３００に出力する。判定結果は、警報後にドライバ３の状態が、異常状態から正常状態に復帰したか否かの情報を含む。また、判定結果は、警報に対するドライバ３の反応の程度、反応までの時間等の情報を含んでもよい。 The alarm information output unit 30 of the driver monitoring device 102 outputs to the server 300 the alarm parameters when the alarm is output and the state of the driver 3 before the alarm, in addition to the determination result by the alarm effect determination unit 20. The determination result includes information on whether or not the state of the driver 3 has returned from the abnormal state to the normal state after the alarm. Further, the determination result may include information such as the degree of reaction of the driver 3 to the alarm and the time until the reaction.
　サーバ３００は、ドライバモニタリングシステム２００に含まれる複数のドライバモニタリング装置の各々から、警報効果判定部２０による判定結果と、警報が出力された際の警報パラメータと、警報前のドライバの状態とを取得する。 The server 300 acquires the determination result by the alarm effect determination unit 20, the alarm parameter when the alarm is output, and the driver status before the alarm from each of the plurality of driver monitoring devices included in the driver monitoring system 200. To do.
　複数の車両の各々が世界中に点在している場合、サーバ３００は世界中のドライバモニタリング装置から、判定結果と警報パラメータとドライバの状態とを収集することができる。そして、サーバ３００は、それらの情報に基づいて、その判定結果と警報パラメータとドライバ３の状態との関係を学習する。サーバ３００は、例えば、ルールベースまたはＡＩ（Artificial Intelligence）技術によりその関係を学習する。 When each of the plurality of vehicles is scattered all over the world, the server 300 can collect the determination result, the alarm parameter, and the driver status from the driver monitoring devices all over the world. Then, the server 300 learns the relationship between the determination result, the alarm parameter, and the state of the driver 3 based on the information. The server 300 learns the relationship by, for example, rule-based or AI (Artificial Intelligence) technology.
　サーバ３００は、その学習により、警報パラメータと、効果的な警報との関係を生成する。サーバ３００は、例えば、ドライバ３の状態および警報パラメータに対する最適な警報を求めるための新たな警報データベースを生成する。そして、サーバ３００は、その最適な警報データベースを新たな警報データベースとして、複数のドライバモニタリング装置の各々の警報制御部６０に配信する。 The server 300 generates the relationship between the alarm parameter and the effective alarm by the learning. The server 300 generates, for example, a new alarm database for seeking the optimum alarm for the driver 3 status and alarm parameters. Then, the server 300 distributes the optimum alarm database as a new alarm database to the alarm control units 60 of each of the plurality of driver monitoring devices.
　ドライバモニタリング装置１０２の警報制御部６０は、その新たな警報データベースを受信する。警報データベース更新部６１は、警報データベース記憶部６２に記憶されている従来の警報データベースを、新たな警報データベースで更新する。警報決定部６３は、ドライバ３の状態、新たな警報データベースに基づいて、別の警報を決定する。 The alarm control unit 60 of the driver monitoring device 102 receives the new alarm database. The alarm database update unit 61 updates the conventional alarm database stored in the alarm database storage unit 62 with a new alarm database. The alarm determination unit 63 determines another alarm based on the state of the driver 3 and the new alarm database.
　実施の形態２の変形例２におけるドライバモニタリング装置１０２の各機能は、図２または図３に示された処理回路により実現される。また、サーバ３００も図２または図３に示された処理回路と同様の処理回路を有する。サーバ３００の機能は、その処理回路により実現される。 Each function of the driver monitoring device 102 in the second modification of the second embodiment is realized by the processing circuit shown in FIG. 2 or FIG. Further, the server 300 also has a processing circuit similar to the processing circuit shown in FIG. 2 or FIG. The function of the server 300 is realized by the processing circuit.
　なお、ドライバモニタリング装置１０２は、判定結果、警報パラメータおよびドライバ３の状態を一時的に記憶するログ部（図示せず）を含んでもよい。そして、警報情報出力部３０は、ログ部に記憶された判定結果、警報パラメータおよびドライバ３の状態をサーバ３００に送信してもよい。 The driver monitoring device 102 may include a log unit (not shown) that temporarily stores the determination result, the alarm parameter, and the state of the driver 3. Then, the alarm information output unit 30 may transmit the determination result, the alarm parameter, and the state of the driver 3 stored in the log unit to the server 300.
　図１２は、実施の形態３におけるドライバモニタリング方法を示すフローチャートである。 FIG. 12 is a flowchart showing the driver monitoring method according to the third embodiment.
　ステップＳ１０からステップＳ９０までは、図９に示されたドライバモニタリング方法と同様である。 Steps S10 to S90 are the same as the driver monitoring method shown in FIG.
　ステップＳ１１０にて、警報情報出力部３０は、判定結果および警報パラメータをサーバ３００に出力する。それらに加えて、警報出力部は、ドライバ３の状態もサーバ３００に出力してもよい。または、警報情報出力部３０は、ステップＳ９０にて更新された警報データベースをサーバ３００に出力してもよい。その場合であっても、判定結果、警報パラメータおよびドライバ３の状態の関係が警報データベースに含まれているため、ステップＳ１２０以降の処理の実行が可能である。 In step S110, the alarm information output unit 30 outputs the determination result and the alarm parameter to the server 300. In addition to these, the alarm output unit may also output the state of the driver 3 to the server 300. Alternatively, the alarm information output unit 30 may output the alarm database updated in step S90 to the server 300. Even in that case, since the relationship between the determination result, the alarm parameter, and the state of the driver 3 is included in the alarm database, the processing after step S120 can be executed.
　ステップＳ１２０にて、警報データベースの学習処理が実行される。図１３は、実施の形態３における警報データベースの学習処理方法を示すフローチャートである。 In step S120, the learning process of the alarm database is executed. FIG. 13 is a flowchart showing a learning processing method of the alarm database according to the third embodiment.
　ステップＳ２１０にて、サーバ３００は、複数のドライバモニタリング装置から、判定結果および警報パラメータを受信する。ここでは、サーバ３００は、判定結果および警報パラメータに加えて、ドライバ３の状態も受信する。 In step S210, the server 300 receives the determination result and the alarm parameter from the plurality of driver monitoring devices. Here, the server 300 receives the state of the driver 3 in addition to the determination result and the alarm parameter.
　ステップＳ２２０にて、サーバ３００は、判定結果と警報パラメータとの関係を学習し、新たな警報データベースを生成する。ここでは、サーバ３００は、判定結果および警報パラメータに加えて、ドライバ３の状態も合わせてそれらの関係を学習する。 In step S220, the server 300 learns the relationship between the determination result and the alarm parameter and generates a new alarm database. Here, the server 300 learns the relationship between the driver 3 and the driver 3 in addition to the determination result and the alarm parameter.
　ステップＳ２３０にて、サーバ３００は、新たな警報データベースを各ドライバモニタリング装置に送信する。以上で、警報データベースの学習処理が終了し、続いて、図１２に示されるステップＳ１３０が実行される。 In step S230, the server 300 transmits a new alarm database to each driver monitoring device. With the above, the learning process of the alarm database is completed, and subsequently, step S130 shown in FIG. 12 is executed.
　ステップＳ１３０にて、警報データベース更新部６１は、サーバ３００で生成された新たな警報データベースを受信し、従来の警報データベースをその新たな警報データベースに更新する。 In step S130, the alarm database update unit 61 receives the new alarm database generated by the server 300, and updates the conventional alarm database with the new alarm database.
　ステップＳ１４０にて、警報決定部６３は、ドライバ３の状態、警報パラメータおよび新たな警報データベースに基づいて、最初の警報とは異なる別の警報を決定する。警報制御部６０は、最初の警報を出力した警報装置１２０とは異なる別の警報装置１３０、または、最初の警報を出力した警報装置１２０に対して、その別の警報を出力させる制御を行う。警報制御部６０が、このような制御を行うことにより、最初の警報とは異なる別の警報が出力される。 In step S140, the alarm determination unit 63 determines another alarm different from the first alarm based on the driver 3 status, alarm parameters, and a new alarm database. The alarm control unit 60 controls another alarm device 130 different from the alarm device 120 that outputs the first alarm, or the alarm device 120 that outputs the first alarm to output the other alarm. When the alarm control unit 60 performs such control, another alarm different from the first alarm is output.
　なお、実施の形態２の変形例１と同様に、警報効果判定部２０は、運転操作検知部８０によって検知される車両１の運転操作の変化に基づいて、警報がドライバ３に対して効果的であったか否かを判定してもよい。 Similar to the first modification of the second embodiment, the alarm effect determination unit 20 gives an alarm to the driver 3 based on the change in the driving operation of the vehicle 1 detected by the driving operation detecting unit 80. It may be determined whether or not it was.
　以上をまとめると、実施の形態３におけるドライバモニタリング装置１０２の警報情報出力部３０は、判定結果に加えて、警報が出力された際の警報パラメータであって、ドライバ３の属性情報、車内の環境情報、車外の環境情報もしくは時間情報を含む警報パラメータを、サーバ３００に出力する。警報制御部６０は、サーバ３００が判定結果と警報が出力された際の警報パラメータとを取得して学習することにより生成される関係であって、警報パラメータとドライバ３に対する効果的な警報との関係を取得する。警報制御部６０は、その関係に基づいて、別の警報を決定する。 Summarizing the above, the alarm information output unit 30 of the driver monitoring device 102 according to the third embodiment is an alarm parameter when an alarm is output in addition to the determination result, the attribute information of the driver 3, and the environment in the vehicle. An alarm parameter including information, environment information outside the vehicle, or time information is output to the server 300. The alarm control unit 60 has a relationship generated by the server 300 acquiring and learning the determination result and the alarm parameter when the alarm is output, and is a relationship between the alarm parameter and an effective alarm for the driver 3. Get a relationship. The alarm control unit 60 determines another alarm based on the relationship.
　また、実施の形態３におけるドライバモニタリングシステム２００は、複数のドライバモニタリング装置と、サーバ３００と、を含む。複数のドライバモニタリング装置は、上記のドライバモニタリング装置１０２を含む。サーバ３００は、複数のドライバモニタリング装置の各々から、警報効果判定部２０による判定結果と、警報が出力された際の警報パラメータとを取得して学習することによって、警報パラメータとドライバ３に対して効果的な警報との関係を生成する。そして、サーバ３００は、複数のドライバモニタリング装置の各々の警報制御部６０にその生成した関係を送信する。 Further, the driver monitoring system 200 according to the third embodiment includes a plurality of driver monitoring devices and a server 300. The plurality of driver monitoring devices include the above driver monitoring device 102. The server 300 obtains and learns the determination result by the alarm effect determination unit 20 and the alarm parameter when the alarm is output from each of the plurality of driver monitoring devices, thereby referring to the alarm parameter and the driver 3. Generate relationships with effective alerts. Then, the server 300 transmits the generated relationship to each alarm control unit 60 of the plurality of driver monitoring devices.
　このようなドライバモニタリング装置１０２は、全世界からの判定結果等により更新された警報データベースに基づいて、別の警報を決定する。そのため、ドライバモニタリング装置１０２は、警報装置１３０がドライバ３に対して適切な警報を出力することを可能にする。また、このようなドライバモニタリングシステム２００は、１台のドライバモニタリング装置１０２のみで、判定結果等のデータを収集するよりも、早期に効果的なモデル、すなわち精度の高い警報データベースの構築を可能とする。 Such a driver monitoring device 102 determines another alarm based on an alarm database updated based on determination results from all over the world. Therefore, the driver monitoring device 102 enables the alarm device 130 to output an appropriate alarm to the driver 3. Further, such a driver monitoring system 200 makes it possible to construct an effective model, that is, a highly accurate alarm database at an earlier stage than collecting data such as determination results with only one driver monitoring device 102. To do.
　（実施の形態３の変形例）
　図１４は、実施の形態３の変形例におけるドライバモニタリング装置１０３およびドライバモニタリングシステム２０１の構成を示すブロック図である。実施の形態３の変形例は実施の形態１の下位概念であり、実施の形態３の変形例におけるドライバモニタリング装置１０３は、実施の形態１におけるドライバモニタリング装置１００の各構成を含む。実施の形態３の変形例におけるドライバモニタリングシステム２０１の警報制御部６０は、サーバ３００に設けられている。ドライバモニタリング装置１０３は、例えば、車両に設けられている。その他の構成は、実施の形態３と同様である。 (Modified Example of Embodiment 3)
FIG. 14 is a block diagram showing the configurations of the driver monitoring device 103 and the driver monitoring system 201 in the modified example of the third embodiment. The modified example of the third embodiment is a subordinate concept of the first embodiment, and the driver monitoring device 103 in the modified example of the third embodiment includes each configuration of the driver monitoring device 100 in the first embodiment. The alarm control unit 60 of the driver monitoring system 201 in the modified example of the third embodiment is provided in the server 300. The driver monitoring device 103 is provided in the vehicle, for example. Other configurations are the same as those in the third embodiment.
　このようなドライバモニタリングシステム２０１は、実施の形態３と同様の効果を奏する。 Such a driver monitoring system 201 has the same effect as that of the third embodiment.
　＜実施の形態４＞
　実施の形態４におけるドライバモニタリング装置およびドライバモニタリング方法を説明する。実施の形態４は実施の形態１の下位概念であり、実施の形態４におけるドライバモニタリング装置は、実施の形態１におけるドライバモニタリング装置１００の各構成を含む。なお、実施の形態１から３のいずれかと同様の構成および動作については説明を省略する。 <Embodiment 4>
The driver monitoring device and the driver monitoring method according to the fourth embodiment will be described. The fourth embodiment is a subordinate concept of the first embodiment, and the driver monitoring device according to the fourth embodiment includes each configuration of the driver monitoring device 100 according to the first embodiment. The same configuration and operation as any of the first to third embodiments will be omitted.
　実施の形態４におけるドライバモニタリング装置の警報制御部６０は、判定結果と、予め定められた複数の警報の各々に対して設定された優先度と、に基づいて、複数の警報のうち一の警報を別の警報として決定する。 The alarm control unit 60 of the driver monitoring device according to the fourth embodiment is an alarm of one of the plurality of alarms based on the determination result and the priority set for each of the plurality of predetermined alarms. Is determined as another alarm.
　例えば、優先度「１」から「７」が、複数の警報の各々に対して以下のように設定される。なお、優先度「１」が最も優先度が高く、優先度「７」が最も優先度が低い。優先度「１」の警報は、車載スピーカが出力する音（パターンＡ）による警報である。優先度「２」の警報は、車載スピーカが出力する音（パターンＢ）による警報である。優先度「３」の警報は、ハンドルに設けられた振動デバイスが出力する振動（パターンＡ）による警報である。優先度「４」の警報は、ハンドルに設けられた振動デバイスが出力する振動（パターンＢ）による警報である。優先度「５」の警報は、表示装置が出力するテキストメッセージによる警報である。優先度「６」の警報は、コールセンターからの発話による警報である。優先度「７」の警報は、ドライバ３が車内に持ち込んだ機器が出力する振動による警報である。 For example, the priorities "1" to "7" are set as follows for each of the plurality of alarms. The priority "1" has the highest priority, and the priority "7" has the lowest priority. The alarm of priority "1" is an alarm based on the sound (pattern A) output from the vehicle-mounted speaker. The alarm of priority "2" is an alarm based on the sound (pattern B) output from the vehicle-mounted speaker. The alarm of priority "3" is an alarm by vibration (pattern A) output by a vibration device provided on the handle. The alarm of priority "4" is an alarm by vibration (pattern B) output by a vibration device provided on the handle. The alarm of priority "5" is an alarm by a text message output by the display device. The alarm of priority "6" is an alarm uttered from the call center. The alarm of priority "7" is an alarm due to vibration output by the device brought into the vehicle by the driver 3.
　警報効果判定部２０が、優先度「１」の警報は効果なしと判定した場合、警報制御部６０は、優先度「２」の警報を、別の警報に決定する。 When the alarm effect determination unit 20 determines that the alarm with the priority "1" has no effect, the alarm control unit 60 determines the alarm with the priority "2" as another alarm.
　これらの優先度の設定は、ドライバ３により変更可能である。例えば、ドライバ３が振動による警報を嫌った場合に、ドライバ３は振動による警報の優先度を下げることができる。例えば、ドライバ３は、上記の優先度「３」および「４」の振動による警報の優先度を、それぞれ優先度「６」および「７」に変更する。 These priority settings can be changed by the driver 3. For example, if the driver 3 dislikes the vibration alarm, the driver 3 can lower the priority of the vibration alarm. For example, the driver 3 changes the priority of the vibration alarm of the above priorities "3" and "4" to the priorities "6" and "7", respectively.
　このようなドライバモニタリング装置は、警報ごとに定められた優先度に基づいて、ドライバ３に対して効果的な警報を警報装置１２０または１３０に出力させることができる。 Such a driver monitoring device can output an effective alarm to the driver 3 to the alarm device 120 or 130 based on the priority determined for each alarm.
　＜実施の形態５＞
　実施の形態５におけるドライバモニタリング装置およびドライバモニタリング方法を説明する。実施の形態５は実施の形態１の下位概念であり、実施の形態５におけるドライバモニタリング装置は、実施の形態１におけるドライバモニタリング装置１００の各構成を含む。なお、実施の形態１から４のいずれかと同様の構成および動作については説明を省略する。 <Embodiment 5>
The driver monitoring device and the driver monitoring method according to the fifth embodiment will be described. The fifth embodiment is a subordinate concept of the first embodiment, and the driver monitoring device according to the fifth embodiment includes each configuration of the driver monitoring device 100 according to the first embodiment. The same configuration and operation as any of the first to fourth embodiments will be omitted.
　実施の形態５におけるドライバモニタリング装置の警報制御部６０は、判定結果と、予め定められた複数の警報をそれぞれ出力する複数の警報装置１２０および１３０の各々の利用可否の情報に対応して設定される優先度と、に基づいて、複数の警報のうち一の警報を別の警報として決定する。言い換えると、警報制御部６０は、複数の警報装置１２０および１３０の各々の利用可否の情報、もしくはその利用可否の情報に応じて設定される警報の優先度に基づいて、別の警報を決定する。 The alarm control unit 60 of the driver monitoring device according to the fifth embodiment is set according to the determination result and the availability information of each of the plurality of alarm devices 120 and 130 that output a plurality of predetermined alarms, respectively. One of the plurality of alarms is determined as another alarm based on the priority. In other words, the alarm control unit 60 determines another alarm based on the availability information of each of the plurality of alarm devices 120 and 130, or the priority of the alarm set according to the availability information. ..
　例えば、警報制御部６０は、乗員が車両１に持ち込んだ機器（スマホ、ウォッチ等）の接続状況もしくは充電状況に基づいて、その機器の利用可否を判断する。または例えば、警報制御部６０は、車両１に設けられた振動センサによって車両１が悪路を走行中であると判断した場合に、ハンドルもしくは座席に設置された振動デバイスを利用不可と判断し、その振動デバイスによる警報の優先度を下げる。または例えば、警報制御部６０は、車両１に設けられたマイクによって車両１の乗員が会話中であり、音の警報が効果的でないと判断した場合に、スピーカによる警報の優先度を下げる。または例えば、警報制御部６０は、通信状況を考慮し、コールセンターからの警報の利用可否を判断して、優先度を設定する。 For example, the alarm control unit 60 determines whether or not the device can be used based on the connection status or charging status of the device (smartphone, watch, etc.) brought into the vehicle 1 by the occupant. Or, for example, when the alarm control unit 60 determines that the vehicle 1 is traveling on a rough road by the vibration sensor provided in the vehicle 1, it determines that the vibration device installed in the handle or the seat cannot be used. Decrease the priority of the alarm by the vibrating device. Alternatively, for example, when the alarm control unit 60 determines that the occupant of the vehicle 1 is talking by the microphone provided in the vehicle 1 and the sound alarm is not effective, the alarm control unit 60 lowers the priority of the alarm by the speaker. Alternatively, for example, the alarm control unit 60 determines the availability of an alarm from the call center in consideration of the communication status, and sets the priority.
　このようなドライバモニタリング装置は、警報装置１２０および１３０の利用可否の情報に基づいて、ドライバ３に対して効果的な警報を警報装置１２０または１３０に出力させることができる。 Such a driver monitoring device can output an effective alarm to the driver 3 to the alarm device 120 or 130 based on the information on the availability of the alarm devices 120 and 130.
　＜実施の形態６＞
　以上の各実施の形態に示されたドライバモニタリング装置は、ナビゲーション装置と、通信端末と、サーバと、これらにインストールされるアプリケーションの機能とを適宜に組み合わせて構築されるシステムにも適用することができる。ここで、ナビゲーション装置とは、例えば、ＰＮＤ（Portable Navigation Device）などを含む。通信端末とは、例えば、携帯電話、スマートフォンおよびタブレットなどの携帯端末を含む。 <Embodiment 6>
The driver monitoring device shown in each of the above embodiments can be applied to a system constructed by appropriately combining a navigation device, a communication terminal, a server, and the functions of applications installed in the navigation device. it can. Here, the navigation device includes, for example, a PND (Portable Navigation Device) and the like. Communication terminals include, for example, mobile terminals such as mobile phones, smartphones and tablets.
　図１５は、実施の形態６におけるドライバモニタリング装置１００およびそれに関連して動作する装置の構成を示すブロック図である。 FIG. 15 is a block diagram showing the configuration of the driver monitoring device 100 and the device that operates in connection with the driver monitoring device 100 according to the sixth embodiment.
　ドライバモニタリング装置１００および通信装置１４０がサーバ３００に設けられている。ドライバモニタリング装置１００は、車両１に設けられたカメラ１１０から通信装置１５０および通信装置１４０を介してドライバ３の顔周辺の画像を取得し、ドライバ３の状態を検知する。ドライバモニタリング装置１００は、車両１に設けられた警報装置１２０もしくは警報装置１３０に対し、各通信装置を介して別の警報を出力させる制御を行う。 The driver monitoring device 100 and the communication device 140 are provided in the server 300. The driver monitoring device 100 acquires an image around the face of the driver 3 from the camera 110 provided in the vehicle 1 via the communication device 150 and the communication device 140, and detects the state of the driver 3. The driver monitoring device 100 controls the alarm device 120 or the alarm device 130 provided in the vehicle 1 to output another alarm via each communication device.
　このように、ドライバモニタリング装置１００がサーバ３００に配置されることにより、車載装置の構成を簡素化することができる。また、ドライバモニタリング装置１００の機能あるいは構成要素の一部がサーバ３００に設けられ、他の一部が車両１に設けられるなど、分散して配置されてもよい。 By arranging the driver monitoring device 100 on the server 300 in this way, the configuration of the in-vehicle device can be simplified. Further, some of the functions or components of the driver monitoring device 100 may be provided in the server 300, and some of the other components may be provided in the vehicle 1 in a distributed manner.
　なお、本発明は、その発明の範囲内において、各実施の形態を自由に組み合わせたり、各実施の形態を適宜、変形、省略したりすることが可能である。 It should be noted that, within the scope of the invention, the present invention can be freely combined with each embodiment, and each embodiment can be appropriately modified or omitted.
　本発明は詳細に説明されたが、上記した説明は、全ての局面において、例示であって、本発明がそれに限定されるものではない。例示されていない無数の変形例が、この発明の範囲から外れることなく想定され得るものと解される。 Although the present invention has been described in detail, the above description is an example in all aspects, and the present invention is not limited thereto. It is understood that a myriad of variations not illustrated can be envisioned without departing from the scope of the invention.
　１　車両、３　ドライバ、１０　ドライバ状態検知部、２０　警報効果判定部、３０　警報情報出力部、４０　画像取得部、５０　顔特徴検出部、６０　警報制御部、６１　警報データベース更新部、６２　警報データベース記憶部、６３　警報決定部、７０　警報パラメータ取得部、８０　運転操作検知部、１１０　カメラ、１２０　警報装置、１３０　警報装置、１００　ドライバモニタリング装置、２００　ドライバモニタリングシステム、３００　サーバ。 1 vehicle, 3 drivers, 10 driver status detection unit, 20 alarm effect judgment unit, 30 alarm information output unit, 40 image acquisition unit, 50 face feature detection unit, 60 alarm control unit, 61 alarm database update unit, 62 alarm database storage Unit, 63 alarm determination unit, 70 alarm parameter acquisition unit, 80 operation operation detection unit, 110 camera, 120 alarm device, 130 alarm device, 100 driver monitoring device, 200 driver monitoring system, 300 server.

Claims (15)

  1. 　車両のドライバの状態に基づいて、前記ドライバへの警報を出力させる制御を、警報装置に対して行うドライバモニタリング装置であって、
    　前記ドライバの前記状態を検知するドライバ状態検知部と、
    　前記ドライバの前記状態に基づいて前記警報装置から出力される前記警報の前後における前記ドライバの前記状態の変化、もしくは前記警報の後の前記ドライバによる前記車両の運転操作の変化に基づいて、前記警報が前記ドライバに対して効果的であるか否かを判定する警報効果判定部と、
    　前記警報効果判定部による判定結果を出力する警報情報出力部と、を備え
    　前記警報情報出力部によって出力される前記判定結果に基づいて、前記ドライバへの前記警報が既に出力された前記警報とは異なる別の警報に変更される、ドライバモニタリング装置。     A driver monitoring device that controls the alarm device to output an alarm to the driver based on the state of the driver of the vehicle.
    A driver state detection unit that detects the state of the driver, and
    The alarm is based on a change in the state of the driver before and after the alarm output from the alarm device based on the state of the driver, or a change in the driving operation of the vehicle by the driver after the alarm. An alarm effect determination unit that determines whether or not is effective for the driver,
    The alarm information output unit that outputs the determination result by the alarm effect determination unit, and the alarm for which the alarm to the driver has already been output based on the determination result output by the alarm information output unit A driver monitoring device that is changed to a different alarm.
  2. 　前記判定結果に基づいて、前記警報を出力した前記警報装置とは異なる別の警報装置に対して、または、前記警報を出力した前記警報装置に対して、前記別の警報を出力させる制御を行う警報制御部をさらに備え、
    　前記警報制御部が前記制御を行うことにより、前記ドライバへの前記警報が前記別の警報に変更される、請求項１に記載のドライバモニタリング装置。     Based on the determination result, control is performed to output the other alarm to another alarm device different from the alarm device that outputs the alarm, or to the alarm device that outputs the alarm. Equipped with an alarm control unit
    The driver monitoring device according to claim 1, wherein the alarm control unit performs the control to change the alarm to the driver to the other alarm.
  3. 　前記警報効果判定部は、前記ドライバの前記状態に基づいて時間差で出力される複数の警報の各々の前後における前記ドライバの前記状態の前記変化、もしくは前記複数の警報の各々の後の前記ドライバによる前記車両の前記運転操作の前記変化に基づいて、前記複数の警報のうち前記ドライバに対して効果的である一の警報を判定する、請求項１に記載のドライバモニタリング装置。 The alarm effect determination unit is based on the change in the state of the driver before and after each of the plurality of alarms output with a time lag based on the state of the driver, or the driver after each of the plurality of alarms. The driver monitoring device according to claim 1, wherein one of the plurality of alarms that is effective for the driver is determined based on the change in the driving operation of the vehicle.
  4. 　前記警報情報出力部は、
    　前記判定結果に加えて、前記警報が出力された際の警報パラメータであって、前記ドライバの属性情報、前記車両内の環境情報、前記車両外の環境情報もしくは時間情報を含む前記警報パラメータを、サーバに出力し、
    　前記警報制御部は、
    　前記サーバが前記判定結果と前記警報が出力された際の前記警報パラメータとを取得して、学習することにより生成される関係であって、前記警報パラメータと前記ドライバに対する効果的な警報との前記関係を取得し、前記関係に基づいて、前記別の警報を決定する、請求項２に記載のドライバモニタリング装置。     The alarm information output unit
    In addition to the determination result, the alarm parameter when the alarm is output, which includes the attribute information of the driver, the environmental information inside the vehicle, the environmental information outside the vehicle, or the time information. Output to the server
    The alarm control unit
    The relationship is generated by the server acquiring and learning the determination result and the alarm parameter when the alarm is output, and the alarm parameter and the effective alarm for the driver. The driver monitoring device according to claim 2, wherein a relationship is acquired and the other alarm is determined based on the relationship.
  5. 　前記警報情報出力部は、
    　前記判定結果に加えて、前記警報が出力された際の警報パラメータであって、前記ドライバの属性情報、前記車両内の環境情報、前記車両外の環境情報もしくは時間情報を含む前記警報パラメータと、前記ドライバの前記状態とを、前記警報制御部に出力し、
    　前記警報制御部は、
    　前記判定結果と、前記警報パラメータと、前記ドライバの前記状態と、に基づいて、前記別の警報を決定する、請求項２に記載のドライバモニタリング装置。     The alarm information output unit
    In addition to the determination result, the alarm parameter when the alarm is output, the alarm parameter including the attribute information of the driver, the environmental information inside the vehicle, the environmental information outside the vehicle, or the time information. The state of the driver is output to the alarm control unit.
    The alarm control unit
    The driver monitoring device according to claim 2, wherein the other alarm is determined based on the determination result, the alarm parameter, and the state of the driver.
  6. 　前記ドライバの前記状態は、前記ドライバの居眠り状態、脇見運転状態、またはデッドマン状態を含む、請求項１に記載のドライバモニタリング装置。 The driver monitoring device according to claim 1, wherein the state of the driver includes a dozing state, an inattentive driving state, or a deadman state of the driver.
  7. 　前記警報効果判定部は、前記警報の前後における前記ドライバの前記状態の前記変化、および、前記警報の後の前記ドライバによる前記車両における前記運転操作の前記変化の組み合わせに基づいて、前記警報が前記ドライバに対して効果的であるか否かを判定する、請求項１に記載のドライバモニタリング装置。 The alarm effect determination unit determines that the alarm is based on a combination of the change in the state of the driver before and after the alarm and the change in the driving operation of the vehicle by the driver after the alarm. The driver monitoring device according to claim 1, which determines whether or not the driver is effective for the driver.
  8. 　前記警報は、前記警報装置に通信回線を通じて前記車両内に存在する通話装置に接続されるコールセンターからの通話による注意を含む、請求項１に記載のドライバモニタリング装置。 The driver monitoring device according to claim 1, wherein the alarm includes attention by a call from a call center connected to the communication device existing in the vehicle through a communication line to the alarm device.
  9. 　前記警報制御部は、前記判定結果として、前記警報が効果的でなかった場合に、前記警報を出力した前記警報装置とは異なる前記別の警報装置に対して、前記別の警報を出力させる前記制御を行う、請求項２に記載のドライバモニタリング装置。 As a result of the determination, the alarm control unit outputs the other alarm to the other alarm device different from the alarm device that outputs the alarm when the alarm is not effective. The driver monitoring device according to claim 2, which controls.
  10. 　前記警報制御部は、前記判定結果として、前記警報が効果的でなかった場合に、前記警報を出力した前記警報装置に対して、前記別の警報を出力させる前記制御を行う、請求項２に記載のドライバモニタリング装置。 According to claim 2, the alarm control unit performs the control to output the other alarm to the alarm device that outputs the alarm when the alarm is not effective as the determination result. The driver monitoring device described.
  11. 　前記警報制御部は、前記判定結果として、前記警報が効果的でなかった場合に、前記警報を出力した前記警報装置とは異なる前記別の警報装置に対して、前記別の警報として第１の別の警報を出力させる前記制御を行い、かつ、前記警報を出力した前記警報装置に対して、前記別の警報として第２の別の警報を出力させる前記制御を行う、請求項２に記載のドライバモニタリング装置。 As a result of the determination, when the alarm is not effective, the alarm control unit makes a first alarm as the other alarm with respect to the other alarm device different from the alarm device that outputs the alarm. The second aspect of claim 2, wherein the control for outputting another alarm is performed, and the control for outputting a second other alarm as the other alarm is performed for the alarm device that has output the alarm. Driver monitoring device.
  12. 　前記警報制御部は、前記判定結果と、予め定められた複数の警報の各々に対して設定される優先度と、に基づいて、前記予め定められた複数の警報のうち一の警報を前記別の警報として決定する、請求項２に記載のドライバモニタリング装置。 The alarm control unit determines one of the plurality of predetermined alarms based on the determination result and the priority set for each of the plurality of predetermined alarms. The driver monitoring device according to claim 2, which is determined as an alarm.
  13. 　前記警報制御部は、前記判定結果と、予め定められた複数の警報をそれぞれ出力する複数の警報装置の各々の利用可否の情報に対応して設定される優先度と、に基づいて、前記予め定められた複数の警報のうち一の警報を前記別の警報として決定する、請求項２に記載のドライバモニタリング装置。 The alarm control unit is based on the determination result and the priority set corresponding to the availability information of each of the plurality of alarm devices that output a plurality of predetermined alarms in advance. The driver monitoring device according to claim 2, wherein one of a plurality of defined alarms is determined as the other alarm.
  14. 　請求項４に記載のドライバモニタリング装置を含む複数のドライバモニタリング装置と、
    　請求項４に記載のサーバと、を備え、
    　前記サーバは、
    　前記複数のドライバモニタリング装置の各々から、前記警報効果判定部による前記判定結果と、前記警報が出力された際の前記警報パラメータとを取得して学習することによって、前記警報パラメータと前記ドライバに対して効果的な警報との前記関係を生成し、
    　前記複数のドライバモニタリング装置の各々の前記警報制御部に送信する、ドライバモニタリングシステム。     A plurality of driver monitoring devices including the driver monitoring device according to claim 4,
    The server according to claim 4 is provided.
    The server
    By acquiring and learning the determination result by the alarm effect determination unit and the alarm parameter when the alarm is output from each of the plurality of driver monitoring devices, the alarm parameter and the driver are obtained. To generate the above relationship with an effective alarm,
    A driver monitoring system that transmits to the alarm control unit of each of the plurality of driver monitoring devices.
  15. 　車両のドライバの状態に基づいて、前記ドライバへの警報を出力させる制御を、警報装置に対して行うドライバモニタリング方法であって、
    　前記ドライバの前記状態を検知し、
    　前記ドライバの前記状態に基づいて前記警報装置から出力される前記警報の前後における前記ドライバの前記状態の変化、もしくは前記警報の後の前記ドライバによる前記車両の運転操作の変化に基づいて、前記警報が前記ドライバに対して効果的であるか否かを判定し、
    　判定結果を出力し、
    　前記判定結果に基づいて、前記ドライバへの前記警報が既に出力された前記警報とは異なる別の警報に変更される、ドライバモニタリング方法。     A driver monitoring method in which an alarm device is controlled to output an alarm to the driver based on the state of the driver of the vehicle.
    Detecting the state of the driver,
    The alarm is based on a change in the state of the driver before and after the alarm output from the alarm device based on the state of the driver, or a change in the driving operation of the vehicle by the driver after the alarm. Determines whether is effective for the driver and
    Output the judgment result,
    A driver monitoring method in which the alarm to the driver is changed to another alarm different from the alarm that has already been output based on the determination result.
PCT/JP2019/010857 2019-03-15 2019-03-15 Driver monitoring device, driver monitoring system and driver monitoring method WO2020188634A1 (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
PCT/JP2019/010857 WO2020188634A1 (en) 2019-03-15 2019-03-15 Driver monitoring device, driver monitoring system and driver monitoring method

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
PCT/JP2019/010857 WO2020188634A1 (en) 2019-03-15 2019-03-15 Driver monitoring device, driver monitoring system and driver monitoring method

Publications (1)

Publication Number Publication Date
WO2020188634A1 true WO2020188634A1 (en) 2020-09-24

Family

ID=72520587

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/JP2019/010857 WO2020188634A1 (en) 2019-03-15 2019-03-15 Driver monitoring device, driver monitoring system and driver monitoring method

Country Status (1)

Country Link
WO (1) WO2020188634A1 (en)

Citations (7)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JP2005062911A (en) * 2003-06-16 2005-03-10 Fujitsu Ten Ltd Vehicle controller
JP2009151649A (en) * 2007-12-21 2009-07-09 Mitsubishi Fuso Truck & Bus Corp Alarm device for vehicle
JP2010244195A (en) * 2009-04-02 2010-10-28 Toyota Motor Corp Alarm device
JP2016018314A (en) * 2014-07-07 2016-02-01 エヌ・ティ・ティ・コミュニケーションズ株式会社 Driver crisis management device, driver crisis management method, and driver crisis management program
JP2018024290A (en) * 2016-08-08 2018-02-15 トヨタ自動車株式会社 Vehicle travel control apparatus
JP2018116381A (en) * 2017-01-17 2018-07-26 富士通株式会社 Alarm output device and alarm output control program
JP2018169906A (en) * 2017-03-30 2018-11-01 パナソニックＩｐマネジメント株式会社 Dozing prevention device, doze prevention method, and program

Patent Citations (7)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JP2005062911A (en) * 2003-06-16 2005-03-10 Fujitsu Ten Ltd Vehicle controller
JP2009151649A (en) * 2007-12-21 2009-07-09 Mitsubishi Fuso Truck & Bus Corp Alarm device for vehicle
JP2010244195A (en) * 2009-04-02 2010-10-28 Toyota Motor Corp Alarm device
JP2016018314A (en) * 2014-07-07 2016-02-01 エヌ・ティ・ティ・コミュニケーションズ株式会社 Driver crisis management device, driver crisis management method, and driver crisis management program
JP2018024290A (en) * 2016-08-08 2018-02-15 トヨタ自動車株式会社 Vehicle travel control apparatus
JP2018116381A (en) * 2017-01-17 2018-07-26 富士通株式会社 Alarm output device and alarm output control program
JP2018169906A (en) * 2017-03-30 2018-11-01 パナソニックＩｐマネジメント株式会社 Dozing prevention device, doze prevention method, and program

Similar Documents

Publication Publication Date Title
US10192171B2 (en) Method and system using machine learning to determine an automotive driver&#39;s emotional state
RU2702378C2 (en) Control system for warning vehicle driver, vehicle (embodiments)
CN112424848B (en) Warning device and driving tendency analysis method
US11353866B2 (en) Driving-automation-level lowering feasibility determination apparatus
KR101967944B1 (en) Providing a user interface experience based on inferred vehicle state
US10640123B2 (en) Driver monitoring system
US20200339133A1 (en) Driver distraction determination
US20230286513A1 (en) Safety mechanism for assuring driver engagement during autonomous drive
JP6613623B2 (en) On-vehicle device, operation mode control system, and operation mode control method
EP2942012A1 (en) Driver assistance system
US10666901B1 (en) System for soothing an occupant in a vehicle
US20180129202A1 (en) System and method of depth sensor activation
WO2018061474A1 (en) Vehicle operation management system
CN103578293A (en) Alert system and method for vehicle
CN113044043A (en) Autonomous vehicle control system and autonomous vehicle control method using the same
US11285966B2 (en) Method and system for controlling an autonomous vehicle response to a fault condition
JP2010175516A (en) Device and method for evaluating energy saving
WO2018105114A1 (en) Reporting control device and reporting control method
JP2018055445A (en) Vehicle operation management system
US10717443B2 (en) Occupant awareness monitoring for autonomous vehicles
WO2020188634A1 (en) Driver monitoring device, driver monitoring system and driver monitoring method
CN110194181B (en) Driving support method, vehicle, and driving support system
JP5435113B2 (en) Energy saving evaluation device, energy saving evaluation method
JP6377034B2 (en) Vehicle drowsiness prevention device and vehicle drowsiness prevention method
JP2019197342A (en) Notification control device and notification control method

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 19919672

Country of ref document: EP

Kind code of ref document: A1

NENP Non-entry into the national phase

Ref country code: DE

122 Ep: pct application non-entry in european phase

Ref document number: 19919672

Country of ref document: EP

Kind code of ref document: A1

NENP Non-entry into the national phase

Ref country code: JP