WO2018222028A1 - A system and a method to determine and control emotional state of a vehicle operator - Google Patents

A system and a method to determine and control emotional state of a vehicle operator Download PDF

Info

Publication number
WO2018222028A1
WO2018222028A1 PCT/MY2018/050036 MY2018050036W WO2018222028A1 WO 2018222028 A1 WO2018222028 A1 WO 2018222028A1 MY 2018050036 W MY2018050036 W MY 2018050036W WO 2018222028 A1 WO2018222028 A1 WO 2018222028A1
Authority
WO
WIPO (PCT)
Prior art keywords
vehicle
sensor
vehicle operator
emotional state
unit
Prior art date
Application number
PCT/MY2018/050036
Other languages
French (fr)
Inventor
Sawal Hamid BIN MD ALI
Md. Shabiul ISLAM
Siti Anom AHMAD
Shi Khai JONATHAN OOI
BINTI MIHAT (MINHAD), Khairun Nisa'
Noor Aishah Atiqah BINTI ZULKIFLI
Original Assignee
Universiti Kebangsaan Malaysia
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Universiti Kebangsaan Malaysia filed Critical Universiti Kebangsaan Malaysia
Publication of WO2018222028A1 publication Critical patent/WO2018222028A1/en

Links

Classifications

    • BPERFORMING OPERATIONS; TRANSPORTING
    • B60VEHICLES IN GENERAL
    • B60RVEHICLES, VEHICLE FITTINGS, OR VEHICLE PARTS, NOT OTHERWISE PROVIDED FOR
    • B60R16/00Electric or fluid circuits specially adapted for vehicles and not otherwise provided for; Arrangement of elements of electric or fluid circuits specially adapted for vehicles and not otherwise provided for
    • B60R16/02Electric or fluid circuits specially adapted for vehicles and not otherwise provided for; Arrangement of elements of electric or fluid circuits specially adapted for vehicles and not otherwise provided for electric constitutive elements
    • B60R16/023Electric or fluid circuits specially adapted for vehicles and not otherwise provided for; Arrangement of elements of electric or fluid circuits specially adapted for vehicles and not otherwise provided for electric constitutive elements for transmission of signals between vehicle parts or subsystems
    • BPERFORMING OPERATIONS; TRANSPORTING
    • B60VEHICLES IN GENERAL
    • B60KARRANGEMENT OR MOUNTING OF PROPULSION UNITS OR OF TRANSMISSIONS IN VEHICLES; ARRANGEMENT OR MOUNTING OF PLURAL DIVERSE PRIME-MOVERS IN VEHICLES; AUXILIARY DRIVES FOR VEHICLES; INSTRUMENTATION OR DASHBOARDS FOR VEHICLES; ARRANGEMENTS IN CONNECTION WITH COOLING, AIR INTAKE, GAS EXHAUST OR FUEL SUPPLY OF PROPULSION UNITS IN VEHICLES
    • B60K28/00Safety devices for propulsion-unit control, specially adapted for, or arranged in, vehicles, e.g. preventing fuel supply or ignition in the event of potentially dangerous conditions
    • B60K28/02Safety devices for propulsion-unit control, specially adapted for, or arranged in, vehicles, e.g. preventing fuel supply or ignition in the event of potentially dangerous conditions responsive to conditions relating to the driver
    • BPERFORMING OPERATIONS; TRANSPORTING
    • B60VEHICLES IN GENERAL
    • B60WCONJOINT CONTROL OF VEHICLE SUB-UNITS OF DIFFERENT TYPE OR DIFFERENT FUNCTION; CONTROL SYSTEMS SPECIALLY ADAPTED FOR HYBRID VEHICLES; ROAD VEHICLE DRIVE CONTROL SYSTEMS FOR PURPOSES NOT RELATED TO THE CONTROL OF A PARTICULAR SUB-UNIT
    • B60W50/00Details of control systems for road vehicle drive control not related to the control of a particular sub-unit, e.g. process diagnostic or vehicle driver interfaces
    • B60W50/08Interaction between the driver and the control system
    • B60W50/14Means for informing the driver, warning the driver or prompting a driver intervention
    • BPERFORMING OPERATIONS; TRANSPORTING
    • B60VEHICLES IN GENERAL
    • B60WCONJOINT CONTROL OF VEHICLE SUB-UNITS OF DIFFERENT TYPE OR DIFFERENT FUNCTION; CONTROL SYSTEMS SPECIALLY ADAPTED FOR HYBRID VEHICLES; ROAD VEHICLE DRIVE CONTROL SYSTEMS FOR PURPOSES NOT RELATED TO THE CONTROL OF A PARTICULAR SUB-UNIT
    • B60W2520/00Input parameters relating to overall vehicle dynamics
    • B60W2520/10Longitudinal speed
    • BPERFORMING OPERATIONS; TRANSPORTING
    • B60VEHICLES IN GENERAL
    • B60WCONJOINT CONTROL OF VEHICLE SUB-UNITS OF DIFFERENT TYPE OR DIFFERENT FUNCTION; CONTROL SYSTEMS SPECIALLY ADAPTED FOR HYBRID VEHICLES; ROAD VEHICLE DRIVE CONTROL SYSTEMS FOR PURPOSES NOT RELATED TO THE CONTROL OF A PARTICULAR SUB-UNIT
    • B60W2520/00Input parameters relating to overall vehicle dynamics
    • B60W2520/10Longitudinal speed
    • B60W2520/105Longitudinal acceleration
    • BPERFORMING OPERATIONS; TRANSPORTING
    • B60VEHICLES IN GENERAL
    • B60WCONJOINT CONTROL OF VEHICLE SUB-UNITS OF DIFFERENT TYPE OR DIFFERENT FUNCTION; CONTROL SYSTEMS SPECIALLY ADAPTED FOR HYBRID VEHICLES; ROAD VEHICLE DRIVE CONTROL SYSTEMS FOR PURPOSES NOT RELATED TO THE CONTROL OF A PARTICULAR SUB-UNIT
    • B60W2540/00Input parameters relating to occupants
    • B60W2540/22Psychological state; Stress level or workload
    • BPERFORMING OPERATIONS; TRANSPORTING
    • B60VEHICLES IN GENERAL
    • B60WCONJOINT CONTROL OF VEHICLE SUB-UNITS OF DIFFERENT TYPE OR DIFFERENT FUNCTION; CONTROL SYSTEMS SPECIALLY ADAPTED FOR HYBRID VEHICLES; ROAD VEHICLE DRIVE CONTROL SYSTEMS FOR PURPOSES NOT RELATED TO THE CONTROL OF A PARTICULAR SUB-UNIT
    • B60W2552/00Input parameters relating to infrastructure
    • BPERFORMING OPERATIONS; TRANSPORTING
    • B60VEHICLES IN GENERAL
    • B60WCONJOINT CONTROL OF VEHICLE SUB-UNITS OF DIFFERENT TYPE OR DIFFERENT FUNCTION; CONTROL SYSTEMS SPECIALLY ADAPTED FOR HYBRID VEHICLES; ROAD VEHICLE DRIVE CONTROL SYSTEMS FOR PURPOSES NOT RELATED TO THE CONTROL OF A PARTICULAR SUB-UNIT
    • B60W2756/00Output or target parameters relating to data
    • B60W2756/10Involving external transmission of data to or from the vehicle

Definitions

  • Embodiments of the present invention relate to safe driving technology and more particularly to a system and a method to determine and control emotional state of a vehicle operator.
  • US7042345B2 discloses an intelligent vehicle apparatus which includes radar and sensor based systems, that monitor a driver's psycho-physiological condition, including heart and respiratory rates, electro-dermal activities, blink rates, head nodding, lane changes, changes in the use of accelerator and brake pedals, and other abnormal behavior, as compared with the driver's baseline performance, feeding into computer-based decision support algorithms which initiate vehicle-shutdown when a driver falls asleep, is intoxicated, taken ill, or safety dictates.
  • this reference does not perform emotion profiling of the driver and further prediction of the emotional state of the driver based on his/her behaviour.
  • US9440657B1 discloses a method, system, and computer-readable medium, which causes the monitoring of a vehicle operator during the course of vehicle operation to determine whether the vehicle operator is impaired and causes a mitigating response when impairment is determined to exist.
  • the vehicle operator, the environment surrounding the vehicle, or forces acting on the vehicle may be monitored using a variety of sensors, including optical sensors, accelerometers, or biometric sensors (e.g., skin conductivity, heart rate, or voice modulation).
  • US8862317B2 discloses an emotion-based vehicle service system for safe driving includes a multi-emotion sensor node configured to sense emotion information of a driver and passenger in a vehicle and condition information of the vehicle.
  • the emotion-based vehicle service system further includes an emotion cognition processing apparatus configured to analyse sensing information provided from the multi-emotion sensor node to generate emotion cognition information, and a safe driving service apparatus configured to provide an emotion-based safe driving service depending on the emotion cognition information.
  • an emotion cognition processing apparatus configured to analyse sensing information provided from the multi-emotion sensor node to generate emotion cognition information
  • a safe driving service apparatus configured to provide an emotion-based safe driving service depending on the emotion cognition information.
  • US2014171752A1 discloses an apparatus for controlling emotion of a driver includes an emotion sensor unit configured to collect a biomedical signal from the driver, and generate biomedical information data based on the collected biomedical signal, a user memory unit configured to store driver information that includes biomedical signals for respective emotional states of the driver and a plurality of correspondence contents, and deliver the driver information and the correspondence content in response to a received request, and an emotion management unit configured to determine the emotional state of the driver from the driver information received from the user memory unit and the biomedical information data received from the emotion sensor unit, request a correspondence content corresponding to the determined emotional state of the driver from the user memory unit, and provide the driver with the content received from the user memory unit.
  • the reference fails to determine the responses in order to counter unfavourable driver state detected by the system.
  • Embodiments of the present invention provide a system and a method to determine and control emotional state of a vehicle operator.
  • the invention allows recognition of the emotional state of the vehicle operator while he operates a vehicle. It further initiates a response mechanism to control the emotional state of the vehicle operator, when detected as unfavourable or unsafe.
  • a system to determine and control an emotional state of a vehicle operator comprises a sensor unit installed in a vehicle, a control unit connected to the sensor unit, a storage device in communication with the control unit, and a response unit in communication with the control unit. Further, the sensor unit is configured to sense emotion information of the vehicle operator and condition information of the vehicle. Further, the control unit is configured to receive the physiological information and the condition information, and to determine the emotional state of the vehicle operator. The storage device is configured to store the emotion information and the condition information. The response unit is configured to control the emotional state of the vehicle operator.
  • control unit is further configured to receive the emotion information and the condition information from the sensor unit, combine the emotion information and the condition information to determine the emotional state, select an emotion profile matching the emotional state of the vehicle operator, and transmit a response mechanism corresponding to the selected emotion profile, to the response unit.
  • the emotion profile is selected upon comparison of the emotional state with one or more pre-trained emotion profiles stored in the storage device.
  • the sensor unit further includes an emotion sensing unit configured to sense the physiological information of the vehicle operator and a vehicle condition sensing unit configured to sense the condition information of the vehicle.
  • the emotion sensing unit comprises one or more of:
  • a heart rate sensor configured to measure heart rate of the vehicle operator
  • ECG Electro-Cardiogram
  • a galvanic skin response sensor configured to measure electrical conductance of skin of the vehicle operator
  • a body temperature sensor configured to measure body temperature of the vehicle operator
  • a blood-pressure sensor configured to measure blood pressure of the vehicle operator
  • a respiratory rate sensor configured to measure the respiratory rate of the vehicle operator
  • a voice sensor configured to sense voice of the vehicle operator
  • the emotion sensing unit further comprises an image sensor configured to measure conditions of a pupil and face of the vehicle operator via the image sensor.
  • the heart rate sensor and the ECG sensor are attached to a seat belt of the vehicle.
  • at least one of the galvanic skin response sensor and the body temperature sensor is attached to a steering wheel of the vehicle.
  • the vehicle condition sensing unit comprises one or more of:
  • a slope sensor configured to sense road condition
  • a coolant sensor configured to measure temperature inside the vehicle
  • a braking sensor configured to sense a state of at least one brake of the vehicle
  • a speedometer configured to measure the speed of the vehicle
  • an accelerometer configured to measure acceleration of the vehicle
  • an optical sensor configured to measure illumination inside the vehicle
  • a location sensor configured to detect location of the vehicle
  • a proximity sensor configured to measure proximity of the vehicle with another vehicle; a vision based sensor configured to detect rapid lane changing; and a time sensor configured to measure current time.
  • the storage device is further configured to store one or more response mechanisms corresponding to the one or more pre-trained emotion profiles.
  • the response mechanism comprises at least one of playing of therapy music, provision of a humor or a joke, alarming a concerned authority or personnel, and advising the vehicle operator.
  • a method for determining and controlling an emotional state of a vehicle operator comprises steps of sensing physiological information and condition information via a sensor unit, storing the physiological information and the condition information, determining the emotional state of the vehicle operator upon combining physiological information and the condition information, and controlling the emotional state of the vehicle operator.
  • the method further comprises comparing the emotional state with one or more pre-trained emotion profiles.
  • the method further comprises selecting an emotion profile matching the emotional state of the vehicle operator, upon comparison of the emotional state with the one or more pre-trained emotion profiles. In accordance with an embodiment of the present invention, the method further comprises initiating a response mechanism corresponding to the selected emotion profile, to control the emotional state of the vehicle operator.
  • the emotion information of the vehicle operator comprises at least one of:
  • condition information of the vehicle comprises at least one of:
  • the emotional state may comprise at least one of:
  • Fig. 1 illustrates a perspective view of a vehicle, in accordance with an embodiment of the present invention
  • Fig. 2 illustrates a block diagram of a system to determine and control an emotional state of a vehicle operator, in accordance with an embodiment of the present invention
  • Fig. 3 illustrates a schematic of a sensor unit of the system of Fig. 2, in accordance with an embodiment of the present invention.
  • Fig. 4 illustrates a method for determining and controlling the emotional state of the vehicle operator, in accordance with an embodiment of the present invention.
  • compositions or an element or a group of elements are preceded with the transitional phrase "comprising”, it is understood that we also contemplate the same composition, element or group of elements with transitional phrases “consisting of”, “consisting”, “selected from the group of consisting of, “including”, or “is” preceding the recitation of the composition, element or group of elements and vice versa.
  • FIG 1 illustrates a perspective view of a vehicle (100), in accordance with an embodiment of the present invention.
  • the vehicle (100) is equipped with a system (110) to determine and control an emotional state of a vehicle operator (120) (shown in Figure 2).
  • the vehicle (100) houses a driving seat (130), a steering wheel (140), an audio unit (150), an air-conditioning unit (160), a seat belt (170), a speedometer (174), and a gear lever (180).
  • the system (110) includes a sensor unit (190) that is strategically placed within the above mentioned components that are in direct or indirect contact with the vehicle operator (120), in order to sense multiplicity of vital signs associated with the vehicle operator (120).
  • the vital signs help in extracting physiological information about the vehicle operator (120).
  • the sensor unit (190) is installed inside the vehicle (100) in such a manner that biometric attributes such as face, skin, heart rate, electrical activity of a heart and voice are exposed and available to the system (110).
  • the components, the driving seat (130), the steering wheel (140), the audio unit (150), and the seat belt (170), may be equipped with one or more sensors of the sensor unit (190) to detect physiological information of the vehicle operator (120) and condition information of the vehicle (100).
  • Figure 1 shows a heart rate sensor (200) attached to the seat belt (170), a galvanic skin response sensor (210) and a body temperature sensor (220) attached to the steering wheel (140).
  • the speedometer (174) is attached to a dashboard (222) of the vehicle (100), and is configured to measure speed of the vehicle (100).
  • FIG. 2 illustrates a block diagram of the vehicle operator (120) integrated with the system (110) to determine and control the emotional state of the vehicle operator (120), in accordance with an embodiment of the present invention.
  • the system (110) includes the sensor unit (190), a storage device (230), a network (240), a control unit (250), and a response unit (260).
  • the sensor unit (190) is installed in the vehicle (100).
  • the sensor unit (190) is configured to sense the physiological information of the vehicle operator (120) and condition information of the vehicle (100).
  • the sensor unit (190) is connected to the control unit (250) via the network (240).
  • the sensor unit (190) transfers the physiological information and the condition information to the control unit (250) over the network (240).
  • the network (240) may include a local area network (LAN).
  • the local area network may employ a local wireless network such as Bluetooth, near field communication (NFC), radio frequency identification (RFID), and ZigBee.
  • the control unit (250) is connected to the sensor unit (190).
  • the control unit (250) is configured to receive the physiological information of the vehicle operator (120), and the condition information of the vehicle (100).
  • the control unit (250) is also configured to determine the emotional state of the vehicle operator (120) based on a method implemented by the disclosed system (110).
  • the control unit (250) is further configured to combine the physiological information and the condition information to determine the emotional state of the vehicle operator (120).
  • the control unit (250) is then configured to select an emotion profile that matches the emotional state of the vehicle operator (120).
  • the control unit (250) is further configured to transmit a response mechanism corresponding to the selected emotion profile, to the response unit (260).
  • control unit (250) is further configured to select the emotion profile upon comparison of the emotional state with one or more pre-trained emotion profiles stored in the storage device (230).
  • the storage device (230) is in control communication with the control unit (250).
  • the storage device (230) is configured to store the physiological information and the condition information, received by the control unit (250).
  • the storage device (230) is configured to store one or more response mechanisms corresponding to the one or more pre-trained emotion profiles. In this manner, the storage device (230) acts as a 'black-box" inside of the vehicle (100), storing all the information that is being generated, in real-time.
  • the response unit (260) is connected to the control unit (250). Hence, the response unit (260) is adapted to take inputs from the control unit (250) regarding the response mechanism.
  • Primary function of the response unit (260) is to control the emotional state of the vehicle operator (120), to relieve the vehicle operator (120) from the stress in the driving or to calm the anger of the vehicle operator (120) while driving, thereby preventing an otherwise-possible traffic accident.
  • the response unit (260) may be connected to the audio unit (150) and the air-conditioning unit (160), to execute the response mechanism.
  • the response mechanism comprises at least one of playing of therapy music, provision of a humour or a joke, alarming a concerned authority or personnel, and advising the vehicle operator (120) via the audio unit (150).
  • the therapy music or the humour or the joke corresponds to the emotional profile selected by the control unit (250), from the one or more pre-trained emotional profiles for the vehicle operator (120).
  • the selected emotional profile will have the therapy music and the humour or the joke as per preferences of the vehicle operator (120).
  • the response mechanism includes managing ambience of the vehicle (100) to soothe the vehicle operator (120), via the air-conditioning unit
  • Figure 3 illustrates a schematic of the sensor unit (190) of the system (110), in accordance with an embodiment of the present invention.
  • the sensor unit (190) is further divided into two sub-units namely: an emotion sensing unit (300) and a vehicle condition sensing unit (310).
  • the emotion sensing unit (300) is configured to sense the physiological information of the vehicle operator (120).
  • the physiological information may be related to at least one of heart rate, electrical activity of a heart, blood pressure, respiratory rate, electrical conductance of skin, facial expression and gesture, and voice of the vehicle operator (120).
  • the emotion sensing unit (300) includes at least one of the heart rate sensor (200), an Electro - Cardiogram (ECG) sensor (205), the galvanic skin response sensor (210), the body temperature sensor (220), a blood-pressure sensor (320), a respiratory rate sensor (330), a voice sensor (340), and an image sensor (350).
  • ECG Electro - Cardiogram
  • the emotion sensing unit (300) includes at least one of the heart rate sensor (200), an Electro - Cardiogram (ECG) sensor (205), the galvanic skin response sensor (210), the body temperature sensor (220), a blood-pressure sensor (320), a respiratory rate sensor (330), a voice sensor (340), and an image sensor (350).
  • the heart rate sensor (200) is configured to measure the heart rate of the vehicle operator (120).
  • the heart rate sensor (200) is attached to the seat belt (170) of the vehicle (100) as shown in Figure 2.
  • the galvanic skin response sensor (210) is configured to measure the electrical conductance of the skin of the vehicle operator (120).
  • the body temperature is configured to measure the body temperature of the vehicle operator (120).
  • at least one of the galvanic skin response sensor (210) and the body temperature sensor (220) is attached to the steering wheel (140) of the vehicle (100).
  • the blood-pressure sensor (320) is configured to measure the blood pressure of the vehicle operator (120).
  • the respiratory rate sensor (330) is configured to measure the respiratory rate of the vehicle operator (120).
  • the voice sensor (340) is configured to sense the voice of the vehicle operator (120).
  • the image sensor (350) is configured to detect conditions of a pupil and face of the vehicle operator (120), to assess the facial expression and gesture of the vehicle operator (120).
  • the vehicle condition sensing unit (310) is configured to sense the condition information of the vehicle (100).
  • the condition information may include, but is not limited to, at least one of speed, acceleration, illumination inside the vehicle (100), road condition the vehicle (100) is operating in, temperature inside the vehicle (100), and location.
  • the vehicle condition sensing unit (310) includes at least one of a slope sensor (360), a coolant sensor (370), a braking sensor (380), the speedometer (174), an accelerometer (390), an optical sensor (392), a location sensor (394), a proximity sensor (398), a vision based sensor (399) and a time sensor (396).
  • the slope sensor (360) is configured to sense road condition the vehicle (100) is operating in.
  • the coolant sensor (370) is configured to measure the temperature inside the vehicle (100).
  • the speedometer (174) is installed on the dashboard (222) as shown in Figure 1 .
  • the speedometer (174) measures the speed of the vehicle (100).
  • the braking sensor (380) is configured to sense a state of at least one brake of the vehicle (100).
  • the accelerometer (390) is configured to measure the acceleration of the vehicle (100).
  • the braking sensor (380) and the accelerometer (390) allow determination and monitoring of driving behavior information, such as sudden acceleration and sudden or hard braking.
  • the optical sensor (392) is configured to measure the illumination inside the vehicle (100).
  • the location sensor (394) is configured to detect the location of the vehicle (100).
  • the time sensor (396) is configured to measure the current time.
  • the proximity sensor (398) is configured to measure proximity of the vehicle with another vehicle.
  • the proximity sensor (398) allows for determination and monitoring of behaviors such as tailgating.
  • the vision based sensor (399) is configured to detect rapid lane changing by the vehicle operator (120).
  • the vision based sensor (399) essentially comprises an optical device, such as a light sensor or a camera or any other equivalent imaging device. In that manner, the vision based sensor (399) will be able to detect lane markings on a road and identify if the vehicle (100) is rapidly moving across the lane markings.
  • the vision based sensor (399) is located on at least one of the left and right side rear view mirrors of the vehicle (100), as shown in figure 1 .
  • Figure 4 illustrates a method (400) for determining and controlling the emotional state of a vehicle operator (120), in accordance with an embodiment of the present invention.
  • the method (400) begins at step (410).
  • the sensor unit (190) senses the physiological information of the vehicle operator (120) and the condition information of the vehicle (100).
  • the sensed physiological information of the vehicle operator (120) and the condition information of the vehicle (100) are stored in the storage device (230). Simultaneously, the physiological information of the vehicle operator (120) and the condition information of the vehicle (100) are transmitted to the control unit (250) for analysis.
  • the control unit (250) combines the physiological information of the vehicle operator (120) and the condition information of the vehicle (100) and determines the emotional state of the vehicle operator (120). This emotional state reflects reactions of the vehicle operator (120) in ambient conditions recorded the sensor unit (190).
  • the control unit (250) controls the emotional state of the vehicle operator (120).
  • the control unit (250) executes this by comparing the determined emotional state with the one or more pre-trained emotion profiles already existing in the storage device (230).
  • the pre-trained emotion profiles is at least one of anxiety, happy, neutral, agitation, aggression, nervousness, distraction, lethargy, drowsiness, and combination of the pre-trained emotion profiles.
  • Each of the multiple emotion profiles is associated with a corresponding relevant response mechanism.
  • the control unit (250) selects the emotion profile which matches the determined emotional state of the vehicle operator (120). Based on the selected emotion profile, the corresponding response mechanism is transmitted by the control unit (250) to the response unit (260). Finally, the response unit (260) initiates the response mechanism to manage the emotional state of the vehicle operator (120).
  • the present invention introduces the method to control the emotional state of the vehicle operator by determining the best possible and available response mechanism based on standard response mechanism and history of detections.
  • the disclosed system performs evaluation of the emotional state in real time. Further, all calculation and outcomes of the system and the method are recorded and stored in the storage device for future use. Maintaining the record of such occurrences along with the corresponding response mechanisms reduces the time required to come up with an appropriate response mechanism. This helps in preventing occurrence of unfavorable incidents while inattentive driving by the vehicle operator. For example, if the system determines that the emotional state of the vehicle operator is similar to the neutral emotion profile, (if the car driver is in a calm state), the response mechanism for emotion control of the car driver is not necessary.
  • control unit does not set out any control signal for response mechanism. If the system determines that the emotional state of the vehicle operator is similar to an aggressive emotion profile (e.g., if the vehicle operator is stressed, excited or angered), the control unit sends the control signal to the audio unit to play a music playlist for control of the emotion of the vehicle operator for change of the mood of the vehicle operator. The control unit may also send the control signal for playing some humorous content or the accident possibility notice message, depending on the emotional state of the vehicle operator.

Abstract

The present invention provides a system (110) to determine and control an emotional state of a vehicle operator (120). The system (110) comprises a sensor unit (190) installed in a vehicle (100), a control unit (250) connected to the sensor unit (190), a storage device (230) in communication with the control unit (250), and a response unit (260) in communication with the control unit. Further, the sensor unit (190) is configured to sense physiological information of the vehicle operator (120) and condition information of the vehicle (100). Further, the control unit (250) is configured to receive the physiological information and the condition information, and to determine the emotional state of the vehicle operator (120). The storage device (230) is configured to store the physiological information and the condition information. The response unit (260) is configured to control the emotional state of the vehicle operator (120).

Description

A SYSTEM AND A METHOD TO DETERMINE AND CONTROL EMOTIONAL
STATE OF A VEHICLE OPERATOR
TECHNICAL FIELD
Embodiments of the present invention relate to safe driving technology and more particularly to a system and a method to determine and control emotional state of a vehicle operator.
BACKGROUND ART
It has been observed that every year many vehicle accidents are caused by inappropriate vehicle operation. This may be cause by unfavorable moods such as, aggression, anxiety, agitation or the like, of a vehicle operator. Numerous incidents, such as traffic jams, poor driving by other drivers, vehicle malfunctions, or inclement weather conditions, occurring during the course of a trip may aggravate the mood of the vehicle operator. Additionally, other factors such as receipt of bad news, running behind schedule, passenger conduct, or any number of factors occurring prior to vehicle operation, may aggravate the mood of the vehicle operator. Such stressors may contribute to emotional responses of the vehicle operator that impair his/her ability to operate the vehicle safely. Thus, while performances of various safety devices installed in the vehicle are important for safety of the vehicle operator and environment around him/her, what is more important is to maintain an emotional balance of the vehicle operator in order to prevent an otherwise-possible traffic accident. US7042345B2 discloses an intelligent vehicle apparatus which includes radar and sensor based systems, that monitor a driver's psycho-physiological condition, including heart and respiratory rates, electro-dermal activities, blink rates, head nodding, lane changes, changes in the use of accelerator and brake pedals, and other abnormal behavior, as compared with the driver's baseline performance, feeding into computer-based decision support algorithms which initiate vehicle-shutdown when a driver falls asleep, is intoxicated, taken ill, or safety dictates. However, this reference does not perform emotion profiling of the driver and further prediction of the emotional state of the driver based on his/her behaviour.
US9440657B1 discloses a method, system, and computer-readable medium, which causes the monitoring of a vehicle operator during the course of vehicle operation to determine whether the vehicle operator is impaired and causes a mitigating response when impairment is determined to exist. The vehicle operator, the environment surrounding the vehicle, or forces acting on the vehicle may be monitored using a variety of sensors, including optical sensors, accelerometers, or biometric sensors (e.g., skin conductivity, heart rate, or voice modulation). US8862317B2 discloses an emotion-based vehicle service system for safe driving includes a multi-emotion sensor node configured to sense emotion information of a driver and passenger in a vehicle and condition information of the vehicle. The emotion-based vehicle service system further includes an emotion cognition processing apparatus configured to analyse sensing information provided from the multi-emotion sensor node to generate emotion cognition information, and a safe driving service apparatus configured to provide an emotion-based safe driving service depending on the emotion cognition information. However, there is lack of an efficient response system to act on the cognition information thus extracted by the emotion-based vehicle service system. Further, it also does not take into account the external factors affecting a current state of the vehicle.
US2014171752A1 discloses an apparatus for controlling emotion of a driver includes an emotion sensor unit configured to collect a biomedical signal from the driver, and generate biomedical information data based on the collected biomedical signal, a user memory unit configured to store driver information that includes biomedical signals for respective emotional states of the driver and a plurality of correspondence contents, and deliver the driver information and the correspondence content in response to a received request, and an emotion management unit configured to determine the emotional state of the driver from the driver information received from the user memory unit and the biomedical information data received from the emotion sensor unit, request a correspondence content corresponding to the determined emotional state of the driver from the user memory unit, and provide the driver with the content received from the user memory unit. The reference fails to determine the responses in order to counter unfavourable driver state detected by the system.
Accordingly, there is a need in the art for an improved vehicle operator emotion management system and method, which do not suffer from above mentioned deficiencies. SUMMARY OF THE INVENTION
Embodiments of the present invention provide a system and a method to determine and control emotional state of a vehicle operator. The invention allows recognition of the emotional state of the vehicle operator while he operates a vehicle. It further initiates a response mechanism to control the emotional state of the vehicle operator, when detected as unfavourable or unsafe.
In accordance with an embodiment of the present invention, a system to determine and control an emotional state of a vehicle operator comprises a sensor unit installed in a vehicle, a control unit connected to the sensor unit, a storage device in communication with the control unit, and a response unit in communication with the control unit. Further, the sensor unit is configured to sense emotion information of the vehicle operator and condition information of the vehicle. Further, the control unit is configured to receive the physiological information and the condition information, and to determine the emotional state of the vehicle operator. The storage device is configured to store the emotion information and the condition information. The response unit is configured to control the emotional state of the vehicle operator.
In accordance with an embodiment of the present invention, the control unit is further configured to receive the emotion information and the condition information from the sensor unit, combine the emotion information and the condition information to determine the emotional state, select an emotion profile matching the emotional state of the vehicle operator, and transmit a response mechanism corresponding to the selected emotion profile, to the response unit.
In accordance with an embodiment of the present invention, the emotion profile is selected upon comparison of the emotional state with one or more pre-trained emotion profiles stored in the storage device.
In accordance with an embodiment of the present invention, the sensor unit further includes an emotion sensing unit configured to sense the physiological information of the vehicle operator and a vehicle condition sensing unit configured to sense the condition information of the vehicle.
In accordance with an embodiment of the present invention, the emotion sensing unit comprises one or more of:
a heart rate sensor configured to measure heart rate of the vehicle operator;
an Electro-Cardiogram (ECG) sensor configured to monitor electrical activity of a heart of the vehicle operator;
a galvanic skin response sensor configured to measure electrical conductance of skin of the vehicle operator;
a body temperature sensor configured to measure body temperature of the vehicle operator;
a blood-pressure sensor configured to measure blood pressure of the vehicle operator;
a respiratory rate sensor configured to measure the respiratory rate of the vehicle operator; and a voice sensor configured to sense voice of the vehicle operator.
In accordance with an embodiment of the present invention, the emotion sensing unit further comprises an image sensor configured to measure conditions of a pupil and face of the vehicle operator via the image sensor.
In accordance with an embodiment of the present invention, the heart rate sensor and the ECG sensor are attached to a seat belt of the vehicle. In accordance with an embodiment of the present invention, at least one of the galvanic skin response sensor and the body temperature sensor is attached to a steering wheel of the vehicle.
In accordance with an embodiment of the present invention, the vehicle condition sensing unit comprises one or more of:
a slope sensor configured to sense road condition;
a coolant sensor configured to measure temperature inside the vehicle; a braking sensor configured to sense a state of at least one brake of the vehicle;
a speedometer configured to measure the speed of the vehicle;
an accelerometer configured to measure acceleration of the vehicle; an optical sensor configured to measure illumination inside the vehicle; a location sensor configured to detect location of the vehicle;
a proximity sensor configured to measure proximity of the vehicle with another vehicle; a vision based sensor configured to detect rapid lane changing; and a time sensor configured to measure current time.
In accordance with an embodiment of the present invention, the storage device is further configured to store one or more response mechanisms corresponding to the one or more pre-trained emotion profiles.
In accordance with an embodiment of the present invention, the response mechanism comprises at least one of playing of therapy music, provision of a humor or a joke, alarming a concerned authority or personnel, and advising the vehicle operator.
In accordance with an embodiment of the present invention, a method for determining and controlling an emotional state of a vehicle operator comprises steps of sensing physiological information and condition information via a sensor unit, storing the physiological information and the condition information, determining the emotional state of the vehicle operator upon combining physiological information and the condition information, and controlling the emotional state of the vehicle operator.
In accordance with an embodiment of the present invention, the method further comprises comparing the emotional state with one or more pre-trained emotion profiles.
In accordance with an embodiment of the present invention, the method further comprises selecting an emotion profile matching the emotional state of the vehicle operator, upon comparison of the emotional state with the one or more pre-trained emotion profiles. In accordance with an embodiment of the present invention, the method further comprises initiating a response mechanism corresponding to the selected emotion profile, to control the emotional state of the vehicle operator.
In accordance with an embodiment of the present invention, the emotion information of the vehicle operator comprises at least one of:
heart rate;
electrical activity of a heart;
blood pressure;
respiratory rate;
electrical conductance of skin;
facial expression and gesture; and
voice.
In accordance with an embodiment of the present invention, the condition information of the vehicle comprises at least one of:
speed;
acceleration;
illumination inside the vehicle;
road condition the vehicle is operating in;
temperature inside the vehicle; proximity with another vehicle;
rapid lane changing; and
location.
In accordance with an embodiment of the present invention, the emotional state may comprise at least one of:
anxiety;
happy;
neutral;
agitation;
aggression;
nervousness;
distraction;
lethargy;
drowsiness; and
combination of the pre-trained emotion profiles.
BRIEF DESCRIPTION OF THE ACCOMPANYING DRAWINGS
So that the manner in which the above recited features of the present invention can be understood in detail, a more particular description of the invention, briefly summarized above, may have been referred by embodiments, some of which are illustrated in the appended drawings. It is to be noted, however, that the appended drawings illustrate only typical embodiments of this invention and are therefore not to be considered limiting of its scope, for the invention may admit to other equally effective embodiments. These and other features, benefits, and advantages of the present invention will become apparent by reference to the following text figure, with like reference numbers referring to like structures across the views, wherein:
Fig. 1 illustrates a perspective view of a vehicle, in accordance with an embodiment of the present invention;
Fig. 2 illustrates a block diagram of a system to determine and control an emotional state of a vehicle operator, in accordance with an embodiment of the present invention;
Fig. 3 illustrates a schematic of a sensor unit of the system of Fig. 2, in accordance with an embodiment of the present invention; and
Fig. 4 illustrates a method for determining and controlling the emotional state of the vehicle operator, in accordance with an embodiment of the present invention.
DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS
While the present invention is described herein by way of example using embodiments and illustrative drawings, those skilled in the art will recognize that the invention is not limited to the embodiments of drawing or drawings described, and are not intended to represent the scale of the various components. Further, some components that may form a part of the invention may not be illustrated in certain figures, for ease of illustration, and such omissions do not limit the embodiments outlined in any way. It should be understood that the drawings and detailed description thereto are not intended to limit the invention to the particular form disclosed, but on the contrary, the invention is to cover all modifications, equivalents, and alternatives falling within the scope of the present invention as defined by the appended claim. As used throughout this description, the word "may" is used in a permissive sense (i.e. meaning having the potential to), rather than the mandatory sense, (i.e. meaning must). Further, the words "a" or "an" mean "at least one" and the word "plurality" means "one or more" unless otherwise mentioned. Furthermore, the terminology and phraseology used herein is solely used for descriptive purposes and should not be construed as limiting in scope. Language such as "including," "comprising," "having," "containing," or "involving," and variations thereof, is intended to be broad and encompass the subject matter listed thereafter, equivalents, and additional subject matter not recited, and is not intended to exclude other additives, components, integers or steps. Likewise, the term "comprising" is considered synonymous with the terms "including" or "containing" for applicable legal purposes. Any discussion of documents, acts, materials, devices, articles and the like is included in the specification solely for the purpose of providing a context for the present invention. It is not suggested or represented that any or all of these matters form part of the prior art base or were common general knowledge in the field relevant to the present invention.
In this disclosure, whenever a composition or an element or a group of elements is preceded with the transitional phrase "comprising", it is understood that we also contemplate the same composition, element or group of elements with transitional phrases "consisting of", "consisting", "selected from the group of consisting of, "including", or "is" preceding the recitation of the composition, element or group of elements and vice versa.
The present invention is described hereinafter by various embodiments with reference to the accompanying drawing, wherein reference numerals used in the accompanying drawing correspond to the like elements throughout the description. This invention may, however, be embodied in many different forms and should not be construed as limited to the embodiment set forth herein. Rather, the embodiment is provided so that this disclosure will be thorough and complete and will fully convey the scope of the invention to those skilled in the art. In the following detailed description, numeric values and ranges are provided for various aspects of the implementations described. These values and ranges are to be treated as examples only, and are not intended to limit the scope of the claims. In addition, a number of materials are identified as suitable for various facets of the implementations. These materials are to be treated as exemplary, and are not intended to limit the scope of the invention.
Referring to the drawings, the invention will now be described in more detail. Figure 1 illustrates a perspective view of a vehicle (100), in accordance with an embodiment of the present invention. The vehicle (100) is equipped with a system (110) to determine and control an emotional state of a vehicle operator (120) (shown in Figure 2). The vehicle (100) houses a driving seat (130), a steering wheel (140), an audio unit (150), an air-conditioning unit (160), a seat belt (170), a speedometer (174), and a gear lever (180). The system (110) includes a sensor unit (190) that is strategically placed within the above mentioned components that are in direct or indirect contact with the vehicle operator (120), in order to sense multiplicity of vital signs associated with the vehicle operator (120). The vital signs help in extracting physiological information about the vehicle operator (120). The sensor unit (190) is installed inside the vehicle (100) in such a manner that biometric attributes such as face, skin, heart rate, electrical activity of a heart and voice are exposed and available to the system (110). The components, the driving seat (130), the steering wheel (140), the audio unit (150), and the seat belt (170), may be equipped with one or more sensors of the sensor unit (190) to detect physiological information of the vehicle operator (120) and condition information of the vehicle (100). For example, Figure 1 shows a heart rate sensor (200) attached to the seat belt (170), a galvanic skin response sensor (210) and a body temperature sensor (220) attached to the steering wheel (140). The speedometer (174) is attached to a dashboard (222) of the vehicle (100), and is configured to measure speed of the vehicle (100).
Figure 2 illustrates a block diagram of the vehicle operator (120) integrated with the system (110) to determine and control the emotional state of the vehicle operator (120), in accordance with an embodiment of the present invention. The system (110) includes the sensor unit (190), a storage device (230), a network (240), a control unit (250), and a response unit (260). The sensor unit (190) is installed in the vehicle (100). The sensor unit (190) is configured to sense the physiological information of the vehicle operator (120) and condition information of the vehicle (100). The sensor unit (190) is connected to the control unit (250) via the network (240). The sensor unit (190) transfers the physiological information and the condition information to the control unit (250) over the network (240). The network (240), for example, may include a local area network (LAN). For example, the local area network may employ a local wireless network such as Bluetooth, near field communication (NFC), radio frequency identification (RFID), and ZigBee.
The control unit (250) is connected to the sensor unit (190). The control unit (250) is configured to receive the physiological information of the vehicle operator (120), and the condition information of the vehicle (100). The control unit (250) is also configured to determine the emotional state of the vehicle operator (120) based on a method implemented by the disclosed system (110). In an embodiment of the present invention, the control unit (250) is further configured to combine the physiological information and the condition information to determine the emotional state of the vehicle operator (120). The control unit (250) is then configured to select an emotion profile that matches the emotional state of the vehicle operator (120). Finally, the control unit (250) is further configured to transmit a response mechanism corresponding to the selected emotion profile, to the response unit (260). In an embodiment of the present invention, the control unit (250) is further configured to select the emotion profile upon comparison of the emotional state with one or more pre-trained emotion profiles stored in the storage device (230). The storage device (230) is in control communication with the control unit (250). The storage device (230) is configured to store the physiological information and the condition information, received by the control unit (250). In an embodiment, the storage device (230) is configured to store one or more response mechanisms corresponding to the one or more pre-trained emotion profiles. In this manner, the storage device (230) acts as a 'black-box" inside of the vehicle (100), storing all the information that is being generated, in real-time.
The response unit (260) is connected to the control unit (250). Hence, the response unit (260) is adapted to take inputs from the control unit (250) regarding the response mechanism. Primary function of the response unit (260) is to control the emotional state of the vehicle operator (120), to relieve the vehicle operator (120) from the stress in the driving or to calm the anger of the vehicle operator (120) while driving, thereby preventing an otherwise-possible traffic accident. The response unit (260) may be connected to the audio unit (150) and the air-conditioning unit (160), to execute the response mechanism.
In an embodiment of the present invention, the response mechanism comprises at least one of playing of therapy music, provision of a humour or a joke, alarming a concerned authority or personnel, and advising the vehicle operator (120) via the audio unit (150). It is contemplated here, that the therapy music or the humour or the joke corresponds to the emotional profile selected by the control unit (250), from the one or more pre-trained emotional profiles for the vehicle operator (120). The selected emotional profile will have the therapy music and the humour or the joke as per preferences of the vehicle operator (120). Further, the response mechanism includes managing ambience of the vehicle (100) to soothe the vehicle operator (120), via the air-conditioning unit Figure 3 illustrates a schematic of the sensor unit (190) of the system (110), in accordance with an embodiment of the present invention. The sensor unit (190) is further divided into two sub-units namely: an emotion sensing unit (300) and a vehicle condition sensing unit (310). The emotion sensing unit (300) is configured to sense the physiological information of the vehicle operator (120). The physiological information may be related to at least one of heart rate, electrical activity of a heart, blood pressure, respiratory rate, electrical conductance of skin, facial expression and gesture, and voice of the vehicle operator (120). The emotion sensing unit (300) includes at least one of the heart rate sensor (200), an Electro - Cardiogram (ECG) sensor (205), the galvanic skin response sensor (210), the body temperature sensor (220), a blood-pressure sensor (320), a respiratory rate sensor (330), a voice sensor (340), and an image sensor (350).
The heart rate sensor (200) is configured to measure the heart rate of the vehicle operator (120). In an embodiment, the heart rate sensor (200) is attached to the seat belt (170) of the vehicle (100) as shown in Figure 2. The galvanic skin response sensor (210) is configured to measure the electrical conductance of the skin of the vehicle operator (120). The body temperature is configured to measure the body temperature of the vehicle operator (120). In an embodiment, at least one of the galvanic skin response sensor (210) and the body temperature sensor (220) is attached to the steering wheel (140) of the vehicle (100). The blood-pressure sensor (320) is configured to measure the blood pressure of the vehicle operator (120). The respiratory rate sensor (330) is configured to measure the respiratory rate of the vehicle operator (120). The voice sensor (340) is configured to sense the voice of the vehicle operator (120). The image sensor (350) is configured to detect conditions of a pupil and face of the vehicle operator (120), to assess the facial expression and gesture of the vehicle operator (120).
In accordance with an embodiment of the present invention, the vehicle condition sensing unit (310) is configured to sense the condition information of the vehicle (100). The condition information may include, but is not limited to, at least one of speed, acceleration, illumination inside the vehicle (100), road condition the vehicle (100) is operating in, temperature inside the vehicle (100), and location. The vehicle condition sensing unit (310) includes at least one of a slope sensor (360), a coolant sensor (370), a braking sensor (380), the speedometer (174), an accelerometer (390), an optical sensor (392), a location sensor (394), a proximity sensor (398), a vision based sensor (399) and a time sensor (396). The slope sensor (360) is configured to sense road condition the vehicle (100) is operating in. The coolant sensor (370) is configured to measure the temperature inside the vehicle (100). The speedometer (174) is installed on the dashboard (222) as shown in Figure 1 . The speedometer (174) measures the speed of the vehicle (100). The braking sensor (380) is configured to sense a state of at least one brake of the vehicle (100). The accelerometer (390) is configured to measure the acceleration of the vehicle (100). The braking sensor (380) and the accelerometer (390) allow determination and monitoring of driving behavior information, such as sudden acceleration and sudden or hard braking. The optical sensor (392) is configured to measure the illumination inside the vehicle (100). The location sensor (394) is configured to detect the location of the vehicle (100). The time sensor (396) is configured to measure the current time. The proximity sensor (398) is configured to measure proximity of the vehicle with another vehicle. The proximity sensor (398) allows for determination and monitoring of behaviors such as tailgating. The vision based sensor (399) is configured to detect rapid lane changing by the vehicle operator (120). The vision based sensor (399) essentially comprises an optical device, such as a light sensor or a camera or any other equivalent imaging device. In that manner, the vision based sensor (399) will be able to detect lane markings on a road and identify if the vehicle (100) is rapidly moving across the lane markings. In accordance with various embodiments, it is further contemplated that the vision based sensor (399) is located on at least one of the left and right side rear view mirrors of the vehicle (100), as shown in figure 1 .
Figure 4 illustrates a method (400) for determining and controlling the emotional state of a vehicle operator (120), in accordance with an embodiment of the present invention. The method (400) begins at step (410). At step (410), the sensor unit (190) senses the physiological information of the vehicle operator (120) and the condition information of the vehicle (100). At step (420), the sensed physiological information of the vehicle operator (120) and the condition information of the vehicle (100), are stored in the storage device (230). Simultaneously, the physiological information of the vehicle operator (120) and the condition information of the vehicle (100) are transmitted to the control unit (250) for analysis. At step (430), the control unit (250) combines the physiological information of the vehicle operator (120) and the condition information of the vehicle (100) and determines the emotional state of the vehicle operator (120). This emotional state reflects reactions of the vehicle operator (120) in ambient conditions recorded the sensor unit (190).
At step (440), the control unit (250) controls the emotional state of the vehicle operator (120). The control unit (250) executes this by comparing the determined emotional state with the one or more pre-trained emotion profiles already existing in the storage device (230). In an embodiment, the pre-trained emotion profiles is at least one of anxiety, happy, neutral, agitation, aggression, nervousness, distraction, lethargy, drowsiness, and combination of the pre-trained emotion profiles. Each of the multiple emotion profiles is associated with a corresponding relevant response mechanism. Further, upon comparison, the control unit (250) selects the emotion profile which matches the determined emotional state of the vehicle operator (120). Based on the selected emotion profile, the corresponding response mechanism is transmitted by the control unit (250) to the response unit (260). Finally, the response unit (260) initiates the response mechanism to manage the emotional state of the vehicle operator (120).
The present invention introduces the method to control the emotional state of the vehicle operator by determining the best possible and available response mechanism based on standard response mechanism and history of detections. The disclosed system performs evaluation of the emotional state in real time. Further, all calculation and outcomes of the system and the method are recorded and stored in the storage device for future use. Maintaining the record of such occurrences along with the corresponding response mechanisms reduces the time required to come up with an appropriate response mechanism. This helps in preventing occurrence of unfavorable incidents while inattentive driving by the vehicle operator. For example, if the system determines that the emotional state of the vehicle operator is similar to the neutral emotion profile, (if the car driver is in a calm state), the response mechanism for emotion control of the car driver is not necessary. In this situation, the control unit does not set out any control signal for response mechanism. If the system determines that the emotional state of the vehicle operator is similar to an aggressive emotion profile (e.g., if the vehicle operator is stressed, excited or angered), the control unit sends the control signal to the audio unit to play a music playlist for control of the emotion of the vehicle operator for change of the mood of the vehicle operator. The control unit may also send the control signal for playing some humorous content or the accident possibility notice message, depending on the emotional state of the vehicle operator. Various modifications to these embodiments are apparent to those skilled in the art from the description and the accompanying drawings. The principles associated with the various embodiments described herein may be applied to other embodiments. Therefore, the description is not intended to be limited to the embodiments shown along with the accompanying drawings but is to be providing broadest scope of consistent with the principles and the novel and inventive features disclosed or suggested herein. Accordingly, the invention is anticipated to hold on to all other such alternatives, modifications, and variations that fall within the scope of the present invention and appended claim.

Claims

Claims:
A system (110) to determine and control an emotional state of a vehicle operator (120), said system (110) comprising:
a sensor unit (190) installed in a vehicle (100);
a control unit (250) connected to said sensor unit (190);
a storage device (230) in communication with said control unit (250); and
a response unit (260) in communication with said control unit (250);
wherein said sensor unit (190) is configured to sense physiological information of said vehicle operator (120) and condition information of said vehicle (100);
wherein said control unit (250) is configured to receive said physiological information and said condition information and to determine said emotional state of said vehicle operator (120); wherein said storage device (230) is configured to store said physiological information and said condition information; and wherein said response unit (260) is configured to control said emotional state of said vehicle operator (120).
The system (110) as claimed in claim 1 , wherein said control unit (250) is further configured to:
receive said physiological information and said condition information from said sensor unit (190); combine said physiological information and said condition information to determine said emotional state;
select an emotion profile matching said emotional state of said vehicle operator (120); and
transmit a response mechanism corresponding to said selected emotion profile, to said response unit (260).
3. The system (110) as claimed in claim 2, wherein said emotion profile is selected upon comparison of said emotional state with one or more pre-trained emotion profiles stored in said storage device (230).
4. The system (110) as claimed in claim 2, wherein said sensor unit (190) further includes an emotion sensing unit (300) configured to sense said physiological information of said vehicle operator (120) and a vehicle condition sensing unit (310) configured to sense said condition information of said vehicle (100).
5. The system (110) as claimed in claim 4, wherein said emotion sensing unit (300) comprises one or more of:
a heart rate sensor (200) configured to measure heart rate of said vehicle operator (120);
an Electro-Cardiogram (ECG) sensor (205) configured to monitor electrical activity of a heart of said vehicle operator (120);
a galvanic skin response sensor (210) configured to measure electrical conductance of skin of said vehicle operator (120); a body temperature sensor (220) configured to measure body temperature of said vehicle operator (120);
a blood-pressure sensor (320) configured to measure blood pressure of said vehicle operator (120);
a respiratory rate sensor (330) configured to measure said respiratory rate of said vehicle operator (120); and
a voice sensor (340) configured to sense voice of said vehicle operator (120). 6. The system (110) as claimed in claim 4, wherein said emotion sensing unit (300) further comprises an image sensor (350) configured to measure conditions of a pupil and face of said vehicle operator (120) via said image sensor (350). 7. The system (110) as claimed in claim 5, wherein said heart rate sensor (200) is attached to a seat belt (170) of said vehicle (100).
8. The system (110) as claimed in claim 5, wherein at least one of said galvanic skin response sensor (210) and said body temperature sensor (220) is attached to a steering wheel (140) of said vehicle
(100).
9. The system (110) as claimed in claim 4, wherein said vehicle condition sensing unit (310) comprises one or more of:
a slope sensor (360) configured to sense road condition; a coolant sensor (370) configured to measure temperature inside said vehicle (100);
a braking sensor (380) configured to sense a state of at least one brake of said vehicle (100);
a speedometer (174) configured to measure said speed of said vehicle (100);
an accelerometer (390) configured to measure acceleration of said vehicle (100);
an optical sensor (392) configured to measure illumination inside said vehicle (100);
a location sensor (394) configured to detect location of said vehicle (100);
a proximity sensor (398) configured to measure proximity of said vehicle (100) with another vehicle;
a vision based sensor (399) configured to detect rapid lane changing; and
a time sensor (396) configured to measure current time.
10. The system (110) as claimed in claim 1 , wherein said storage device (230) is further configured to store one or more response mechanisms corresponding to said one or more pre-trained emotion profiles.
1 1 . The system (110) as claimed in claim 11 , wherein said one or more response mechanism comprises at least one of:
playing of therapy music; provision of a humor or a joke;
alarming a concerned authority or personnel; and
advising said vehicle operator (120).
12. A method (400) for determining and controlling an emotional state of a vehicle operator (120), said method comprising steps of:
sensing (410) physiological information and condition information via a sensor unit (190);
storing (420) said physiological information and said condition information;
determining (430) said emotional state of said vehicle operator (120) upon combining physiological information and said condition information; and
controlling (440) said emotional state of said vehicle operator (120).
13. The method (400) as claimed in claim 13, wherein said method further comprises comparing said emotional state with one or more pre-trained emotion profiles.
14. The method (400) as claimed in claim 13, wherein said method further comprises selecting an emotion profile matching said emotional state of said vehicle operator upon comparison of said emotional state with said one or more pre-trained emotion profiles.
15. The method (400) as claimed in claim 13, wherein said method further comprises initiating a response mechanism corresponding to said selected emotion profile, to control said emotional state of said vehicle operator (120).
16. The method (400) as claimed in claim 13, wherein said physiological information of said vehicle operator (120) comprises at least one of: heart rate;
electrical activity of a heart;
blood pressure;
respiratory rate;
electrical conductance of skin;
facial expression and gesture; and
voice.
17. The method (400) as claimed in claim 13, wherein said condition information of a vehicle (100) comprises at least one of:
speed;
acceleration;
illumination inside said vehicle (100);
road condition said vehicle (100) is operating in;
temperature inside said vehicle (100);
proximity with another vehicle;
rapid lane changing; and
location.
18. The method (400) as claimed in claim 14, wherein said one or more pre-trained emotion profiles comprise at least one of:
anxiety;
happy;
neutral;
agitation;
aggression;
nervousness;
distraction;
lethargy;
drowsiness; and
combination of said pre-trained emotion profiles.
PCT/MY2018/050036 2017-06-01 2018-05-30 A system and a method to determine and control emotional state of a vehicle operator WO2018222028A1 (en)

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
MYPI2017702020 2017-06-01
MYPI2017702020A MY192254A (en) 2017-06-01 2017-06-01 A system and a method to determine and control emotional state of a vehicle operator

Publications (1)

Publication Number Publication Date
WO2018222028A1 true WO2018222028A1 (en) 2018-12-06

Family

ID=64455741

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/MY2018/050036 WO2018222028A1 (en) 2017-06-01 2018-05-30 A system and a method to determine and control emotional state of a vehicle operator

Country Status (2)

Country Link
MY (1) MY192254A (en)
WO (1) WO2018222028A1 (en)

Cited By (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN110826436A (en) * 2019-10-23 2020-02-21 上海能塔智能科技有限公司 Emotion data transmission and processing method and device, terminal device and cloud platform
DE102019206536A1 (en) * 2019-05-07 2020-11-12 Audi Ag Method for configuring a personal assistance device, storage medium, device for electronic data processing, motor vehicle

Citations (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20100134302A1 (en) * 2008-12-01 2010-06-03 Electronics And Telecommunications Research Institute System and method for controlling emotion of car driver
US20130054090A1 (en) * 2011-08-29 2013-02-28 Electronics And Telecommunications Research Institute Emotion-based vehicle service system, emotion cognition processing apparatus, safe driving service apparatus, and emotion-based safe driving service method
US20130073115A1 (en) * 2011-09-02 2013-03-21 Volvo Technology Corporation System And Method For Improving A Performance Estimation Of An Operator Of A Vehicle
KR20140032651A (en) * 2012-09-07 2014-03-17 전자부품연구원 Method for emotion feedback service and smart device using the same
US20160023666A1 (en) * 2014-07-23 2016-01-28 Hyundai Mobis Co., Ltd. Apparatus and method for detecting driver status

Patent Citations (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20100134302A1 (en) * 2008-12-01 2010-06-03 Electronics And Telecommunications Research Institute System and method for controlling emotion of car driver
US20130054090A1 (en) * 2011-08-29 2013-02-28 Electronics And Telecommunications Research Institute Emotion-based vehicle service system, emotion cognition processing apparatus, safe driving service apparatus, and emotion-based safe driving service method
US20130073115A1 (en) * 2011-09-02 2013-03-21 Volvo Technology Corporation System And Method For Improving A Performance Estimation Of An Operator Of A Vehicle
KR20140032651A (en) * 2012-09-07 2014-03-17 전자부품연구원 Method for emotion feedback service and smart device using the same
US20160023666A1 (en) * 2014-07-23 2016-01-28 Hyundai Mobis Co., Ltd. Apparatus and method for detecting driver status

Cited By (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
DE102019206536A1 (en) * 2019-05-07 2020-11-12 Audi Ag Method for configuring a personal assistance device, storage medium, device for electronic data processing, motor vehicle
CN110826436A (en) * 2019-10-23 2020-02-21 上海能塔智能科技有限公司 Emotion data transmission and processing method and device, terminal device and cloud platform

Also Published As

Publication number Publication date
MY192254A (en) 2022-08-11

Similar Documents

Publication Publication Date Title
US10357195B2 (en) Pupillometry and sensor fusion for monitoring and predicting a vehicle operator's condition
US11738757B2 (en) Information processing device, moving apparatus, method, and program
US10160457B1 (en) Vehicle occupant monitoring using infrared imaging
WO2019188398A1 (en) Information processing device, moving apparatus, method, and program
US10046618B2 (en) System and method for vehicle control integrating environmental conditions
US10192171B2 (en) Method and system using machine learning to determine an automotive driver's emotional state
JP6213282B2 (en) Driving assistance device
US9944228B2 (en) System and method for vehicle control integrating health priority alerts of vehicle occupants
JP7324716B2 (en) Information processing device, mobile device, method, and program
JP2000301963A (en) Accident deterrent system and method
US20140171752A1 (en) Apparatus and method for controlling emotion of driver
El Khatib et al. Driver inattention detection in the context of next-generation autonomous vehicles design: A survey
JPWO2020100539A1 (en) Information processing equipment, mobile devices, and methods, and programs
WO2021145131A1 (en) Information processing device, information processing system, information processing method, and information processing program
US20180225551A1 (en) Machine learning for olfactory mood alteration
US20060011399A1 (en) System and method for controlling vehicle operation based on a user's facial expressions and physical state
KR102272774B1 (en) Audio navigation device, vehicle having the same, user device, and method for controlling vehicle
WO2019208450A1 (en) Driving assistance device, driving assistance method, and program
KR102303023B1 (en) Taxi nomitoring system using smart band, gateway module and integrated control device connected with taxi service using smart band
WO2020145161A1 (en) Information processing device, mobile device, method, and program
WO2018147838A1 (en) Machine learning for olfactory mood alteration
JP2019086813A (en) Aimless driving control system
WO2020066757A1 (en) Dangerous driving prevention device
JP2020154976A (en) In-vehicle environment warning device and in-vehicle environment warning method
KR20200086785A (en) A method and system for preventing drowsiness driving and keeping vehicle safe

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 18809181

Country of ref document: EP

Kind code of ref document: A1

NENP Non-entry into the national phase

Ref country code: DE

122 Ep: pct application non-entry in european phase

Ref document number: 18809181

Country of ref document: EP

Kind code of ref document: A1