CN106803423B - Man-machine interaction voice control method and device based on user emotion state and vehicle - Google Patents
Man-machine interaction voice control method and device based on user emotion state and vehicle Download PDFInfo
- Publication number
- CN106803423B CN106803423B CN201611229157.7A CN201611229157A CN106803423B CN 106803423 B CN106803423 B CN 106803423B CN 201611229157 A CN201611229157 A CN 201611229157A CN 106803423 B CN106803423 B CN 106803423B
- Authority
- CN
- China
- Prior art keywords
- user
- state
- voice
- set user
- emotional state
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
- 230000008451 emotion Effects 0.000 title claims abstract description 81
- 230000003993 interaction Effects 0.000 title claims abstract description 52
- 238000000034 method Methods 0.000 title claims abstract description 26
- 230000002996 emotional effect Effects 0.000 claims abstract description 156
- 230000009471 action Effects 0.000 claims abstract description 56
- 230000014509 gene expression Effects 0.000 claims abstract description 51
- 238000012544 monitoring process Methods 0.000 claims abstract description 13
- 230000008859 change Effects 0.000 claims abstract description 10
- 238000004458 analytical method Methods 0.000 claims description 17
- 238000012545 processing Methods 0.000 claims description 12
- 230000000694 effects Effects 0.000 claims description 6
- 230000005281 excited state Effects 0.000 claims description 5
- LQIAZOCLNBBZQK-UHFFFAOYSA-N 1-(1,2-Diphosphanylethyl)pyrrolidin-2-one Chemical compound PCC(P)N1CCCC1=O LQIAZOCLNBBZQK-UHFFFAOYSA-N 0.000 claims description 3
- 102100034112 Alkyldihydroxyacetonephosphate synthase, peroxisomal Human genes 0.000 claims 2
- 101000799143 Homo sapiens Alkyldihydroxyacetonephosphate synthase, peroxisomal Proteins 0.000 claims 2
- 238000000848 angular dependent Auger electron spectroscopy Methods 0.000 claims 2
- 230000006399 behavior Effects 0.000 abstract description 33
- 230000008921 facial expression Effects 0.000 abstract description 6
- 230000006870 function Effects 0.000 description 7
- 238000010586 diagram Methods 0.000 description 4
- 230000001815 facial effect Effects 0.000 description 3
- 238000003825 pressing Methods 0.000 description 3
- 230000008569 process Effects 0.000 description 3
- 230000002159 abnormal effect Effects 0.000 description 2
- 238000004891 communication Methods 0.000 description 2
- 238000001514 detection method Methods 0.000 description 2
- 230000002452 interceptive effect Effects 0.000 description 2
- 238000012986 modification Methods 0.000 description 2
- 230000004048 modification Effects 0.000 description 2
- 230000004297 night vision Effects 0.000 description 2
- 206010039203 Road traffic accident Diseases 0.000 description 1
- 238000013473 artificial intelligence Methods 0.000 description 1
- 238000011161 development Methods 0.000 description 1
- 210000005069 ears Anatomy 0.000 description 1
- 238000005516 engineering process Methods 0.000 description 1
- 238000011156 evaluation Methods 0.000 description 1
- 210000004709 eyebrow Anatomy 0.000 description 1
- 210000003128 head Anatomy 0.000 description 1
- 230000007774 longterm Effects 0.000 description 1
- 230000036651 mood Effects 0.000 description 1
- 210000005036 nerve Anatomy 0.000 description 1
- 210000001331 nose Anatomy 0.000 description 1
- 238000011112 process operation Methods 0.000 description 1
- 230000035484 reaction time Effects 0.000 description 1
- 238000010079 rubber tapping Methods 0.000 description 1
- 238000004904 shortening Methods 0.000 description 1
- 238000003860 storage Methods 0.000 description 1
Images
Classifications
-
- G—PHYSICS
- G10—MUSICAL INSTRUMENTS; ACOUSTICS
- G10L—SPEECH ANALYSIS TECHNIQUES OR SPEECH SYNTHESIS; SPEECH RECOGNITION; SPEECH OR VOICE PROCESSING TECHNIQUES; SPEECH OR AUDIO CODING OR DECODING
- G10L15/00—Speech recognition
- G10L15/22—Procedures used during a speech recognition process, e.g. man-machine dialogue
-
- G—PHYSICS
- G10—MUSICAL INSTRUMENTS; ACOUSTICS
- G10L—SPEECH ANALYSIS TECHNIQUES OR SPEECH SYNTHESIS; SPEECH RECOGNITION; SPEECH OR VOICE PROCESSING TECHNIQUES; SPEECH OR AUDIO CODING OR DECODING
- G10L15/00—Speech recognition
- G10L15/08—Speech classification or search
-
- G—PHYSICS
- G10—MUSICAL INSTRUMENTS; ACOUSTICS
- G10L—SPEECH ANALYSIS TECHNIQUES OR SPEECH SYNTHESIS; SPEECH RECOGNITION; SPEECH OR VOICE PROCESSING TECHNIQUES; SPEECH OR AUDIO CODING OR DECODING
- G10L25/00—Speech or voice analysis techniques not restricted to a single one of groups G10L15/00 - G10L21/00
- G10L25/48—Speech or voice analysis techniques not restricted to a single one of groups G10L15/00 - G10L21/00 specially adapted for particular use
- G10L25/51—Speech or voice analysis techniques not restricted to a single one of groups G10L15/00 - G10L21/00 specially adapted for particular use for comparison or discrimination
- G10L25/63—Speech or voice analysis techniques not restricted to a single one of groups G10L15/00 - G10L21/00 specially adapted for particular use for comparison or discrimination for estimating an emotional state
-
- G—PHYSICS
- G10—MUSICAL INSTRUMENTS; ACOUSTICS
- G10L—SPEECH ANALYSIS TECHNIQUES OR SPEECH SYNTHESIS; SPEECH RECOGNITION; SPEECH OR VOICE PROCESSING TECHNIQUES; SPEECH OR AUDIO CODING OR DECODING
- G10L15/00—Speech recognition
- G10L15/08—Speech classification or search
- G10L2015/088—Word spotting
-
- G—PHYSICS
- G10—MUSICAL INSTRUMENTS; ACOUSTICS
- G10L—SPEECH ANALYSIS TECHNIQUES OR SPEECH SYNTHESIS; SPEECH RECOGNITION; SPEECH OR VOICE PROCESSING TECHNIQUES; SPEECH OR AUDIO CODING OR DECODING
- G10L15/00—Speech recognition
- G10L15/22—Procedures used during a speech recognition process, e.g. man-machine dialogue
- G10L2015/226—Procedures used during a speech recognition process, e.g. man-machine dialogue using non-speech characteristics
- G10L2015/227—Procedures used during a speech recognition process, e.g. man-machine dialogue using non-speech characteristics of the speaker; Human-factor methodology
Landscapes
- Engineering & Computer Science (AREA)
- Health & Medical Sciences (AREA)
- Computational Linguistics (AREA)
- Audiology, Speech & Language Pathology (AREA)
- Human Computer Interaction (AREA)
- Physics & Mathematics (AREA)
- Acoustics & Sound (AREA)
- Multimedia (AREA)
- Child & Adolescent Psychology (AREA)
- General Health & Medical Sciences (AREA)
- Hospice & Palliative Care (AREA)
- Psychiatry (AREA)
- Signal Processing (AREA)
- Traffic Control Systems (AREA)
- Navigation (AREA)
- Fittings On The Vehicle Exterior For Carrying Loads, And Devices For Holding Or Mounting Articles (AREA)
Abstract
The invention discloses a human-computer interaction voice control method, a human-computer interaction voice control device and a vehicle based on the emotional state of a user, wherein the method comprises the following steps: monitoring and setting the expression, voice or action of a user; determining the current emotional state of the set user according to the expression, voice or action of the set user; determining a voice control mode of the vehicle according to the current emotional state of the set user; and performing vehicle man-machine interaction according to the determined voice control mode. The method, the device and the vehicle disclosed by the invention can calculate the current emotion of the user according to the driving behavior, the speaking speed and tone, the facial expression and the like of the user, and the intelligent system can play appropriate music or adjust the change of the navigation voice and the like according to the current emotion state of the user to carry out man-machine interaction with the user, so that the emotion of the user is adjusted, and the driving of the user is safer.
Description
Technical Field
The invention relates to the field of artificial intelligence, in particular to the field of vehicle intelligent control or human-computer interaction, and particularly relates to a human-computer interaction voice control method and device based on a user emotion state and a vehicle.
Background
With the rapid development of society, automobiles are more and more popular in life; although the concept of vehicle autopilot has been long proposed, it has not yet become widespread; currently, the control of the driver is still in a decisive position during the driving of the vehicle. However, a person as a driver may be affected by various emotions during driving, and some emotions may seriously affect driving safety.
Therefore, it is necessary to provide a method or a vehicle capable of analyzing the emotion of a driver.
Disclosure of Invention
The invention aims to solve the technical problem of how to provide a method, an intelligent control system and a vehicle for man-machine interaction based on the emotion of a user aiming at a vehicle environment, so that the emotion of the user can be adjusted, and then the man-machine interaction of the vehicle is controlled based on the emotion state of the user, thereby ensuring the safety of the user in driving the vehicle.
The invention provides a human-computer interaction voice control method based on the emotional state of a user, which comprises the following steps: monitoring and setting the expression, voice or action of a user; determining the current emotional state of the set user according to the expression, voice or action of the set user; determining a voice control mode of the vehicle according to the current emotional state of the set user; and performing vehicle man-machine interaction according to the determined voice control mode.
Further, before determining the current emotional state of the set user according to the expression, voice or motion of the set user, the method includes: counting emotional state data of a plurality of users to form a general user emotional state database, wherein the emotional state database comprises the relation between expressions, voice or actions of the users and emotional states; analyzing the emotional states of a plurality of users in a big data mode according to a general user emotional state database, and calibrating a determination factor of the emotional states of the users according to the actions of the users, wherein the emotional states of the users comprise a cheerful state, an angry state, a sad state, a painful state and an excited state.
Further, the determining the current emotional state of the set user according to the expression, voice or motion of the set user includes: and evaluating and determining the current emotional state of the set user according to the general user emotional state database and the monitored expression, voice or action of the set user.
Further, the method also comprises the step of determining the control mode of the vehicle according to the current emotional state of the set user.
Further, determining the current emotional state of the setting user according to the expression, voice or motion of the setting user further includes: acquiring the change of the speed, tone and sound of the voice of the set user; and determining the current emotional state of the set user according to the average value of the speed, tone and sound of the voice of the set user.
Further, language information contained in the sound in the time period set by the set user is analyzed, and whether specific words and phrases exist in the language expression of the set user is judged; and determining the emotional state of the set user according to the specific words and phrases appearing in the current expression of the set user.
Further, analyzing the action of a set user in a set time period; and judging and setting the current emotional state of the user according to the daily action and the action of the current time of the set user.
Further, the driving behavior of a set user in a set time period is analyzed; and judging and setting the current emotional state of the user according to the daily driving behavior of the set user and the driving behavior at the current time.
Further, analyzing whether the speed, tone and sound of the voice of the set user in the set time period exceed the set threshold; if the speed of speech of the set user is increased to exceed the set speed of speech threshold, and/or the intonation is increased to exceed the set intonation threshold, and/or the voice is increased to exceed the set voice threshold, the set user is in a relative emotional state.
Further, the determining the voice control mode of the vehicle according to the current emotional state of the set user includes: and if the user is set to be in an angry state at present, selecting a gentle and comfortable sound database to perform voice control, and soothing the emotion of the set user.
And further, selecting the set human-computer interaction sound effect according to the emotion state of the set user to interact with the set user.
Further, after the emotional state of the set user is determined, the set music is selected to be played according to the emotional state of the set user so as to sooth the emotion of the set user and ensure the driving safety.
Further, if the driver is currently in the driving navigation state, the set navigation language can be selected to be played according to the set emotion state of the user so as to sooth the emotion of the user and ensure the driving safety.
The invention also provides a human-computer interaction voice control device based on the emotion state of the user, which comprises the following components: the monitoring module is used for monitoring and setting the expression, voice or action of the user; the emotion state analysis module is connected with the monitoring module and used for determining the current emotion state of the set user according to the expression, voice or action of the set user; the processing module is connected with the emotional state analysis module and used for determining a voice control mode of the vehicle according to the current emotional state of the set user; and the execution module is connected with the processing module and is used for carrying out vehicle man-machine interaction according to the determined voice control mode.
The system further comprises an emotional state database module, a general user emotional state database and a user information database, wherein the emotional state database module is used for counting emotional state data of a plurality of users to form the general user emotional state database, and the emotional state database comprises the relation between expressions, voice or actions of the users and emotional states; analyzing the emotional states of a plurality of users in a big data mode according to a general user emotional state database, and calibrating a determination factor of the emotional states of the users according to the actions of the users, wherein the emotional states of the users comprise a cheerful state, an angry state, a sad state, a painful state and an excited state.
Further, the emotion state analysis module is also used for evaluating and determining the emotion state of the set user according to the general user emotion state database and the monitored expression, voice or action of the set user.
Further, the processing module is also used for determining the control mode of the vehicle according to the current emotional state of the set user.
Further, the emotional state analysis module is further configured to: acquiring the change of the speed, tone and sound of the voice of the set user; and determining the current emotional state of the set user according to the average value of the speed, tone and sound of the voice of the set user.
Further, language information contained in the voice in the time period set by the set user is analyzed, and whether specific words and sentences exist in the language expression of the set user is judged; and determining the emotional state of the set user according to the specific words and phrases appearing in the current expression of the set user.
Further, analyzing the action of a set user in a set time period; and judging and setting the current emotional state of the user according to the daily action and the action of the current time of the set user.
Further, the driving behavior of a set user in a set time period is analyzed; and judging and setting the current emotional state of the user according to the daily driving behavior of the set user and the driving behavior at the current time.
Further, analyzing whether the speed, tone and sound of the voice of the set user in the set time period exceed the set threshold; if the speed of speech of the set user is increased to exceed the set speed of speech threshold, and/or the intonation is increased to exceed the set intonation threshold, and/or the voice is increased to exceed the set voice threshold, the set user is in a relative emotional state.
Further, the processing module is also used for selecting a gentle and comforting sound database for voice control and soothing the emotion of the set user if the set user is in an angry state at present.
Furthermore, the execution module is also used for selecting the set human-computer interaction sound effect according to the emotion state of the set user to interact with the set user.
Further, the execution module is further used for selecting the set music to play according to the emotional state of the set user after the emotional state of the set user is determined so as to sooth the emotion of the set user and ensure driving safety.
Further, the execution module is also used for selecting the set navigation language to play according to the set emotional state of the user to placate the emotion of the set user to ensure the driving safety if the driver is currently in the driving navigation state.
The invention provides a vehicle which comprises a man-machine interaction voice control device based on the emotion state of a user.
According to the method, the device and the vehicle provided by the invention, the current emotion of the user can be calculated according to the driving behavior, the speaking speed and tone, the facial expression and the like of the user, such as urgency comparison, excitement comparison, anger comparison, sadness comparison and the like, the intelligent system can play appropriate music or adjust the change of navigation voice and the like according to the current emotion state of the user to perform man-machine interaction with the user, so that the emotion of the user is adjusted, and the driving of the user is safer.
Drawings
Fig. 1 shows a flow chart of a man-machine interaction voice control method based on the emotional state of a user according to an embodiment of the invention.
Fig. 2 shows a block diagram of a human-computer interaction voice control device based on a user emotional state according to an embodiment of the invention.
Fig. 3 shows a block diagram of a vehicle according to an embodiment of the invention.
Detailed Description
The present invention now will be described more fully hereinafter with reference to the accompanying drawings, in which exemplary embodiments of the invention are shown.
Fig. 1 shows a flow chart of a man-machine interaction voice control method based on the emotional state of a user according to an embodiment of the invention, and referring to fig. 1, the method comprises the following steps:
step 101, monitoring and setting the expression, voice or action of a user.
In one embodiment, the expression, voice, or motion of a given user may be monitored or detected by a combination of various sensors.
For example, the expression, the action and the like of a user can be monitored in a vehicle through a fatigue driving camera built in the vehicle; the voice situation of the user can be set through detection of a microphone built in the vehicle.
And step 102, determining the current emotional state of the set user according to the expression, voice or action of the set user.
In one embodiment, emotional state data of a plurality of users can be counted to form a general user emotional state database, wherein the emotional state is determined according to expressions, voice or actions of the users; analyzing the emotional states of a plurality of users in a big data mode according to a general user emotional state database, and calibrating the determination factors of the happy, angry, sad, painful and excited emotional states according to the emotional states of the users; and evaluating the emotional state of the user according to the general user emotional state database and the monitored expression, voice or action of the set user.
In one embodiment, the emotion of the user can be broadly summarized into several emotions of happiness, anger, sadness and happiness. In actual life, the influence of the favorite state or the happy state on the driving of the user is relatively small; however, the emotional state of anger or sadness has a great influence on the driving behavior of the user, for example, the road anger group driving a car may cause a great safety hazard to the current driving user or other users on the road, and in real life, there are frequent accidents such as leaving car, parking hitter, and various traffic accidents caused by the driver's anger rush.
In one embodiment, the emotional state of the user, such as anger or sadness, whose influence on the driving state is greater than the happy or happy emotional state, can be set, the emotional state of the user, such as anger or sadness, is set by emphasis monitoring, and a corresponding control method is adopted in the voice interaction to adjust the emotion of the user.
In one embodiment, aiming at the environment in the vehicle, voice interaction can be an important means for interaction between a person and a vehicle intelligent system, and in the man-machine interaction process, the intelligent system can monitor the voice of a set user through a microphone to obtain the change of the speed, tone and sound size of the voice of the set user; and determining the current emotional state of the set user according to the average value of the speed, tone and sound of the voice of the set user.
In a specific embodiment, whether the speed, tone and sound size of the voice of the set user in the set time period exceed the set threshold value or not can be analyzed; if the speed of speech of the set user is increased to exceed the set speed of speech threshold, or the intonation is increased to exceed the set intonation threshold, and the voice is increased to exceed the set voice threshold, the set user is in a relative emotional state.
In one embodiment, whether a language expression of a set user in a set time period has a specific word or phrase is analyzed; and determining the emotional state of the set user according to the specific words and phrases appearing in the current expression of the set user. For example, it is possible to analyze whether a specific word or phrase, such as an expletive word or a happy word, is present in the language expression of the set user within the set time period.
For example, if a user's linguistic expression frequently has words of dirty, curer, or words of abuse, it may indicate that the user is likely to be in a relatively excited or angry state; for another example, if a sound such as a happy expression sentence or laugh appears in the language expression of the user, it can be said that the user may be currently in a relatively happy state.
In one embodiment, the current emotional state of the user can be determined by analyzing the changes of the speed, tone and sound size of the specific or common words and sentences. For example, in the vehicle human-computer interaction process, there may be some commonly used words and phrases such as a wake-up word, and the intelligent system may analyze changes of voices, tones and sound of the user when expressing the commonly used words according to expression modes of the wake-up words such as "hello, xiaozhi" or "Hi, xiaozhi" expressed by the user, so as to analyze the current emotional state of the user. For example, it is analyzed whether the user has emotional states of joy, anger, sadness, etc. when the user expresses these common words.
In one embodiment, the action of the set user in the set time period is analyzed, and the current emotional state of the set user is judged according to the daily action and the action of the current time of the set user.
In one embodiment, the driving behavior of a set user within a set time period is analyzed; and judging and setting the current emotional state of the user according to the daily driving behavior of the set user and the driving behavior at the current time. For example, the driving behavior of the user is analyzed, and if the user has a violent stepping on the accelerator, a violent tapping of the steering wheel, a leaving behavior, or the like, it can be determined that the user is currently angry.
And 103, determining a voice control mode of the vehicle according to the current emotional state of the set user.
And 104, performing vehicle man-machine interaction according to the determined voice control mode.
In one embodiment, if the user is set to be in an angry state at present, selecting a gentle and comforting sound database for voice control, and soothing the emotion of the set user; and selecting the set human-computer interaction sound effect according to the emotion state of the set user to interact with the set user.
In one embodiment, particularly in a scenario within a vehicle, it is an important function for a user to interact with the vehicle through speech. According to the interactive statistics of daily machines and people, a voice interactive database for user emotion state interaction is obtained, and influence factors of joy, anger, sadness and music can be scientifically analyzed and set when the database is established; and determining a man-machine interaction mode of the vehicle according to the current emotion of the user obtained by analysis so as to encourage, placate or prompt the current driver, so that the driver can calm down, and the driver is prevented from influencing driving safety due to bad emotion in driving.
For example, if the user is currently in an angry state, the voice interaction database of the angry state of the user is selected, and the driver can be soothed by the database, so that the emotion of the driver can be stabilized, and the influence of anger on the driving of the driver is prevented.
For example, if the user is currently in a sad state, the user sad state voice interaction database is selected, and the database is adopted to comfort the driver to prevent the driver from influencing the driving due to sadness.
In one embodiment, after determining the emotion of the user, the intelligent system can select the set music to play according to the emotion state of the user so as to sooth the emotion of the set user and ensure driving safety. For example, the user is currently in a cheerful emotional state, and may choose to play some happy and graceful music; if the user is in the angry state, some light music for soothing the nerves can be played.
In one embodiment, after determining the emotion of the user, if the user is currently in the driving navigation state, the intelligent system may select the set navigation language to play according to the set emotion state of the user to sooth the emotion of the set user to ensure driving safety.
For example, if the current emotional state of the user is a relative collapse sadness, for the male driver, the vehicle intelligence system may select a gentle female voice to interact with the user; for the female driver, the vehicle may select a magnetic male voice for human-machine interaction with the user.
As an example, when performing facial expression recognition, the information in the existing facial recognition database may be used in combination with the specific camera position in the vehicle to perform facial recognition. Firstly, the first step is face detection, namely determining the position in a vehicle lens to find the face position; secondly, detecting key points, namely accurately finding key points of facial contours such as eyebrows, ears and noses at the determined face position, and identifying the face so as to identify a set user; thirdly, determining the information of the set user based on the face recognition of the large-scale data; and fourthly, finding out the facial expression database of the set user, and identifying the current emotional state of the user based on the facial expression information database of the set user.
In one embodiment, the intelligent system determines the control mode of the vehicle based on the current emotional state of the set user. For example, the method can include increasing the warning value of the ADAS system, such as automatically increasing the safe following distance, automatically controlling the vehicle to keep a longer safe distance with vehicles in other lanes, shortening the preset reaction time of the user in the vehicle active safety system, and simultaneously automatically tightening a safety belt and warning the user through the vibration of a steering wheel, prompting the user to pay attention to the driving safety, so that the safety of the system can be improved.
In one embodiment, the control mode of the vehicle may be selected based on the current emotional state of the user after the emotional state of the user is determined. For example, if the user is currently in an angry state, the vehicle can automatically tighten the safety belt, and give an alarm through a vehicle voice system, so that the user is pacified and is prompted to pay attention to driving safety; if the user is still in an angry state, even an action of getting out of the vehicle or trying to bump others by the vehicle occurs, the vehicle intelligent operation system can temporarily take over the control of the vehicle and enable the vehicle to stop at the side; prevent the driver from being injured or self-injured due to anger. Even if the vehicle automatically selects to park at the side, the vehicle can be comprehensively analyzed according to the current situation to determine how to park, so that the danger caused by automatic driving is prevented.
In one embodiment, the action of the set user may include a set driving behavior, the driving behavior of the set user may be extracted and compared with relevant data in a driving behavior evaluation database, the current behavior of the user may be compared with a normal driving behavior of the user, and whether a civilization index, a technical index, an love car index, a stability index, and a power saving index in the current driving behavior of the user are seriously deviated from an average value of the set user may be specifically analyzed, and if the average value is seriously deviated, it may be stated that an emotional state of the current driving of the user may have a problem.
Specifically, the driving behavior that needs to be analyzed may include: the driving behaviors such as trampling force of a switch, an accelerator pedal and a brake, whether a behavior of hurrying a steering wheel exists, pressing a horn randomly, opening a high beam randomly, turning without lighting, changing lanes frequently during driving, forcibly merging lanes during driving, robbing lanes, running a red light, driving without using a safety belt, and the like; and comprehensively analyzing and evaluating the emotional state of the set user by comparing the current driving behaviors of the user with the daily driving behaviors of the user.
Generally speaking, if a person is in an angry state relatively, the person may slam the horn, the action is rough or abnormal relatively, and if the vehicle intelligent system monitors that the user has the behavior of slamming the horn and pressing the horn randomly, the user is judged to be in an excited, angry or bad emotion state; at this time, the intelligent system needs to adjust the voice interaction mode of the vehicle and interact with the user to placate the driver.
In one embodiment, if the user has the driving behaviors of hurrying the steering wheel, pressing a horn at random, opening a high beam at random, turning without lighting, changing lanes frequently during driving, forcibly merging lanes during driving, robbing lanes, running a red light and driving without using a safety belt, the vehicle can also automatically prompt the driver through voice so as to correct the bad habits of the driver and educate the user to form good driving habits.
By way of example, if a person drives normally, the steering wheel is relatively gentle and normal, and the action of stepping on an electric door, an accelerator or a brake is relatively moderate, but the behavior of suddenly stepping on the steering wheel, the accelerator or the brake suddenly exists within a set time during driving, which may indicate that the user is currently in an abnormal state, the current emotional state of the user may be determined according to the current action of the user, and the vehicle may be controlled to change a voice interaction mode or play music according to the set emotional state to sooth the user.
According to the man-machine interaction voice control method based on the emotion state of the user, provided by the embodiment of the invention, the current emotion of the user can be calculated according to the driving behavior (such as different from the daily driving behavior) of the user, the speech speed and the intonation of speaking, even the facial expression and the like, such as the condition of urgency comparison, excitement comparison, anger comparison, sadness comparison and the like, the intelligent system can play appropriate music or adjust the change of the navigation voice and the like according to the current emotion state of the user to carry out man-machine interaction with the user, the emotion of the user is adjusted, and the driving of the user is safer.
Fig. 2 is a block diagram illustrating a configuration of a human-computer interaction voice control device based on an emotional state of a user according to an embodiment of the present invention, and referring to fig. 2, the device 200 includes: a monitoring module 201, configured to monitor and set expressions, voices or actions of a user; the emotion state analysis module 202 is connected with the monitoring module 201 and is used for determining the current emotion state of the set user according to the expression, voice or action of the set user; the processing module 203 is connected with the emotional state analysis module 202 and is used for determining a voice control mode of the vehicle according to the current emotional state of the set user; and the execution module 204 is connected with the processing module 203 and is used for performing vehicle human-computer interaction according to the determined voice control mode.
In one embodiment, the apparatus further comprises: the emotion state database module 205 is used for counting emotion state data of a plurality of users to form a general user emotion state database, wherein the emotion state database comprises the relationship between expressions, voices or actions of the users and emotion states; analyzing emotional states of a plurality of users in a big data mode according to a general user emotional state database, and calibrating a determination factor of the emotional states of the users according to actions of the users, wherein the emotional states of the users comprise a cheerful state, an angry state, a sad state, a painful state and an excited state;
in one embodiment, the emotional state analysis module is further configured to evaluate and determine the emotional state of the set user according to the general user emotional state database and the monitored expression, voice or motion of the set user.
In one embodiment, the emotional state analysis module is further to: acquiring the change of the speed, tone and sound of the voice of the set user; and determining the current emotional state of the set user according to the average value of the speed, tone and sound of the voice of the set user.
In one embodiment, the emotional state analysis module is further to: analyzing language information contained in the voice in the set time period of the set user, and judging whether specific words and sentences exist in the language expression of the set user; and determining the emotional state of the set user according to the specific words and phrases appearing in the current expression of the set user.
In one embodiment, the emotional state analysis module is further to: analyzing the action of a set user in a set time period; and judging and setting the current emotional state of the user according to the daily action and the action of the current time of the set user.
In one embodiment, the emotional state analysis module is further to: analyzing the driving behavior of a set user in a set time period; and judging and setting the current emotional state of the user according to the daily driving behavior of the set user and the driving behavior at the current time.
In one embodiment, the emotional state analysis module is further to: analyzing whether the speed, tone and sound of the voice of a set user in a set time period exceed a set threshold value or not; if the speed of speech of the set user is increased to exceed the set speed of speech threshold, and/or the intonation is increased to exceed the set intonation threshold, and/or the voice is increased to exceed the set voice threshold, the set user is in a relative emotional state.
In one embodiment, the processing module is further configured to select a gentle, soothing sound database for voice control to soothe the mood of the set user if the set user is currently at an angry state.
In one embodiment, the execution module is further used for selecting the set human-computer interaction sound effect according to the emotional state of the set user to interact with the set user.
In one embodiment, the execution module is further configured to select the set music to be played according to the emotional state of the set user after determining the emotional state of the set user, so as to sooth the emotion of the set user and ensure driving safety.
In one embodiment, the execution module is further used for selecting the set navigation language to play according to the set emotional state of the user to sooth the emotion of the set user to ensure driving safety if the driving navigation state is currently in the driving navigation state.
The invention also provides a vehicle which comprises the human-computer interaction voice control device based on the emotional state of the user.
In one embodiment, the vehicle can be connected with a server located at the cloud end in real time, and data of the server is called in time to analyze the emotional state of the user.
Fig. 3 shows a block diagram of a vehicle according to an embodiment of the present invention, which may include, as shown in fig. 3: the intelligent driving system comprises a central control module, an instrument panel 310, a driving recorder 311, a HUD (Head Up Display) Head-Up Display 312, an intelligent vehicle-mounted infotainment system 313 and an intelligent driving module 313.
Dashboard 310 has a 12.3 inch LCD display device, which may be implemented with a TI J6 CPU; the operating system of the dashboard may be based on a QNX embedded system, and the dashboard may be used to display vehicle status including speed, rotational speed, electrical quantity, tire pressure, vehicle parking, gear, etc., a map, vehicle navigation information, vehicle music play, etc. The HUD heads-up display 312 may display GPS navigation information, navigation path information, time information, and the like.
In one embodiment, smart driving module 313 may be used to process operations related to smart driving, and smart driving module 313 may include Advanced Driver Assistance Systems (ADAS), active safety Systems, Attention Assistance Systems (AAS), Fatigue Warning Systems (FWS), Vehicle intelligent Acoustic warning Systems (AVAS), and the like. The vehicle can be intelligently driven by combining an ADAS system and the like, and the intelligent driving can be completely unmanned driving or can be advanced auxiliary driving functions such as auxiliary merging and lane deviation for driving control of a driver.
The central control device can be composed of a plurality of modules, and mainly comprises: a main board 301; a Serial Advanced Technology Attachment (SATA) module 302, connected to a storage device such as SSD303, may be used to store data information; an AM (Amplitude Modulation)/FM (Frequency Modulation) module 304, which provides the vehicle with the function of a radio; a power amplifier module 305 for processing sound; a WIFI (Wireless-Fidelity)/Bluetooth module 306 for providing WIFI/Bluetooth services to the vehicle; an LTE (long term Evolution) communication module 307 that provides a communication function with a telecommunications carrier for the vehicle; a power module 308, wherein the power module 308 provides power for the central control device; the Switch switching module 309, the Switch switching module 309 can be used as an expandable interface to connect with various sensors, for example, if a night vision function sensor or a PM2.5 function sensor is required to be added, the Switch switching module 309 can be connected to the main board of the central control device, so that the processor of the central control device can process data and transmit the data to the central control display.
In one embodiment, the vehicle further includes a look-around camera, an ADAS camera, a night vision camera, a millimeter wave radar, an ultrasonic radar, an ESR radar, or the like. After the vehicle hardware is produced, the intelligent driving related hardware is mounted, and in the later stage, the automatic driving related functions can be completed by using the hardware through OTA (over the air) upgrading.
The description of the present invention has been presented for purposes of illustration and description, and is not intended to be exhaustive or limited to the invention in the form disclosed. Many modifications and variations will be apparent to practitioners skilled in this art. The embodiment was chosen and described in order to best explain the principles of the invention and the practical application, and to enable others of ordinary skill in the art to understand the invention for various embodiments with various modifications as are suited to the particular use contemplated.
Claims (11)
1. A man-machine interaction voice control method based on user emotion states is characterized by comprising the following steps:
monitoring and setting the expression, voice or action of a user;
determining the current emotional state of the set user according to the expression, voice or action of the set user;
determining a control mode of the vehicle according to the current emotional state of the set user, wherein the control mode comprises the steps of increasing the warning value of an ADAS system, automatically tightening a safety belt and giving an alarm to the user through the vibration of a steering wheel; the warning value comprises increasing the safe following distance;
determining a voice control mode of the vehicle according to the current emotional state of the set user;
performing vehicle man-machine interaction according to the determined voice control mode;
and obtaining a voice interaction database interacting with the emotion state of the user according to daily human-computer interaction statistics.
2. The method of claim 1, wherein:
the determining the current emotional state of the set user according to the expression, voice or action of the set user comprises:
counting emotional state data of a plurality of users to form a general user emotional state database, wherein the emotional state database comprises the relation between expressions, voice or actions of the users and emotional states;
analyzing emotional states of a plurality of users in a big data mode according to a general user emotional state database, and calibrating a determination factor of the emotional states of the users according to actions of the users, wherein the emotional states of the users comprise a cheerful state, an angry state, a sad state, a painful state and an excited state;
and/or
The determining the current emotional state of the set user according to the expression, voice or action of the set user comprises:
and evaluating and determining the current emotional state of the set user according to the general user emotional state database and the monitored expression, voice or action of the set user.
3. The method of claim 1 or 2, wherein determining the current emotional state of the set user based on the expression, voice, or motion of the set user further comprises:
acquiring the change of the speed, tone and sound of the voice of the set user; determining the current emotional state of the set user according to the average value of the speed, tone and sound of the voice of the set user;
and/or
Analyzing language information contained in sound in a set time period of a set user, and judging whether specific words and phrases exist in language expression of the set user;
determining the emotional state of the set user according to the specific words and phrases appearing in the current expression of the set user;
and/or
Analyzing the action of a set user in a set time period;
judging and setting the current emotional state of the user according to the daily action and the action of the current time of the set user;
and/or
Analyzing the driving behavior of a set user in a set time period;
and judging and setting the current emotional state of the user according to the daily driving behavior of the set user and the driving behavior at the current time.
4. The method of claim 3, further comprising:
analyzing whether the speed, tone and sound of the voice of a set user in a set time period exceed a set threshold value or not;
if the speed of speech of the set user is increased to exceed the set speed of speech threshold, and/or the intonation is increased to exceed the set intonation threshold, and/or the voice is increased to exceed the set voice threshold, the set user is in a relative emotional state.
5. The method of claim 1, wherein determining the voice control mode of the vehicle according to the current emotional state of the set user comprises:
if the user is set to be in an angry state at present;
selecting a soft and consolation sound database to perform voice control, and pacifying and setting the emotion of the user;
and/or
Selecting a set human-computer interaction sound effect according to the emotion state of a set user to interact with the set user;
and/or
After determining to set the emotional state of the user;
the set music is selected to be played according to the emotion state of the set user so as to sooth the emotion of the set user and ensure the driving safety;
and/or
If the vehicle is currently in a driving navigation state;
and selecting the set navigation language to play according to the emotional state of the set user so as to soothe the emotion of the set user and ensure the driving safety.
6. A human-computer interaction voice control device based on user emotion states is characterized by comprising:
the monitoring module is used for monitoring and setting the expression, voice or action of the user;
the emotion state analysis module is connected with the monitoring module and used for determining the current emotion state of the set user according to the expression, voice or action of the set user;
the processing module is connected with the emotional state analysis module and used for determining a voice control mode of the vehicle according to the current emotional state of the set user; the processing module is also used for determining a control mode of the vehicle according to the current emotion state of the set user, wherein the control mode comprises the steps of increasing the warning value of an ADAS system, automatically tightening a safety belt and giving an alarm to the user through the vibration of a steering wheel; the warning value comprises increasing the safe following distance;
the execution module is connected with the processing module and is used for carrying out vehicle man-machine interaction according to the determined voice control mode; and obtaining a voice interaction database interacting with the emotion state of the user according to daily human-computer interaction statistics.
7. The apparatus of claim 6, wherein:
the system also comprises an emotional state database module which is used for counting emotional state data of a plurality of users to form a general user emotional state database, wherein the emotional state database comprises the relation between the expression, voice or action of the users and the emotional state; analyzing emotional states of a plurality of users in a big data mode according to a general user emotional state database, and calibrating a determination factor of the emotional states of the users according to actions of the users, wherein the emotional states of the users comprise a cheerful state, an angry state, a sad state, a painful state and an excited state;
and/or
And the emotion state analysis module is also used for evaluating and determining the emotion state of the set user according to the general user emotion state database and the monitored expression, voice or action of the set user.
8. The apparatus of claim 6 or 7, wherein the emotional state analysis module is further configured to:
acquiring the change of the speed, tone and sound of the voice of the set user; determining the current emotional state of the set user according to the average value of the speed, tone and sound of the voice of the set user;
and/or
Analyzing language information contained in the voice in the set time period of the set user, and judging whether specific words and sentences exist in the language expression of the set user; determining the emotional state of the set user according to the specific words and phrases appearing in the current expression of the set user;
and/or
Analyzing the action of a set user in a set time period;
judging and setting the current emotional state of the user according to the daily action and the action of the current time of the set user;
and/or
Analyzing the driving behavior of a set user in a set time period; and judging and setting the current emotional state of the user according to the daily driving behavior of the set user and the driving behavior at the current time.
9. The apparatus of claim 8, comprising:
analyzing whether the speed, tone and sound of the voice of a set user in a set time period exceed a set threshold value or not;
if the speed of speech of the set user is increased to exceed the set speed of speech threshold, and/or the intonation is increased to exceed the set intonation threshold, and/or the voice is increased to exceed the set voice threshold, the set user is in a relative emotional state.
10. The apparatus of claim 6, comprising:
the processing module is also used for selecting a soft and comforting sound database to perform voice control if the user is set to be in an angry state at present, and soothing the emotion of the set user;
and/or
The execution module is also used for selecting the set human-computer interaction sound effect according to the emotion state of the set user to interact with the set user;
and/or
The execution module is also used for selecting the set music to play according to the emotion state of the set user after the emotion state of the set user is determined so as to sooth the emotion of the set user and ensure the driving safety;
and/or
The execution module is also used for judging whether the vehicle is in a driving navigation state or not;
and selecting the set navigation language to play according to the emotional state of the set user so as to soothe the emotion of the set user and ensure the driving safety.
11. A vehicle comprising a human-computer interaction voice control device based on the emotional state of a user according to any of claims 6 to 10.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201611229157.7A CN106803423B (en) | 2016-12-27 | 2016-12-27 | Man-machine interaction voice control method and device based on user emotion state and vehicle |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201611229157.7A CN106803423B (en) | 2016-12-27 | 2016-12-27 | Man-machine interaction voice control method and device based on user emotion state and vehicle |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN106803423A CN106803423A (en) | 2017-06-06 |
| CN106803423B true CN106803423B (en) | 2020-09-04 |
Family
ID=58985118
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201611229157.7A Active CN106803423B (en) | 2016-12-27 | 2016-12-27 | Man-machine interaction voice control method and device based on user emotion state and vehicle |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN106803423B (en) |
Families Citing this family (46)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN107230384B (en) * | 2017-06-21 | 2020-09-25 | 深圳市盛路物联通讯技术有限公司 | Parking guidance system and method based on expected parking duration and weather information |
| US10381005B2 (en) | 2017-11-28 | 2019-08-13 | Toyota Motor Engineering & Manufacturing North America, Inc. | Systems and methods for determining user frustration when using voice control |
| CN108010512B (en) * | 2017-12-05 | 2021-04-30 | 广东小天才科技有限公司 | Sound effect acquisition method and recording terminal |
| CN108664123A (en) * | 2017-12-15 | 2018-10-16 | 蔚来汽车有限公司 | People's car mutual method, apparatus, vehicle intelligent controller and system |
| CN108052016A (en) * | 2017-12-29 | 2018-05-18 | 南京工程学院 | A kind of interactive intelligent mirror |
| CN110164427A (en) * | 2018-02-13 | 2019-08-23 | 阿里巴巴集团控股有限公司 | Voice interactive method, device, equipment and storage medium |
| CN108682419A (en) * | 2018-03-30 | 2018-10-19 | 京东方科技集团股份有限公司 | Sound control method and equipment, computer readable storage medium and equipment |
| CN108710821A (en) * | 2018-03-30 | 2018-10-26 | 斑马网络技术有限公司 | Vehicle user state recognition system and its recognition methods |
| CN108549720A (en) | 2018-04-24 | 2018-09-18 | 京东方科技集团股份有限公司 | It is a kind of that method, apparatus and equipment, storage medium are pacified based on Emotion identification |
| CN108896061A (en) * | 2018-05-11 | 2018-11-27 | 京东方科技集团股份有限公司 | A kind of man-machine interaction method and onboard navigation system based on onboard navigation system |
| CN109190459A (en) * | 2018-07-20 | 2019-01-11 | 上海博泰悦臻电子设备制造有限公司 | A kind of car owner's Emotion identification and adjusting method, storage medium and onboard system |
| CN108984229B (en) * | 2018-07-24 | 2021-11-26 | 广东小天才科技有限公司 | Application program starting control method and family education equipment |
| CN110825216A (en) * | 2018-08-10 | 2020-02-21 | 北京魔门塔科技有限公司 | Method and system for man-machine interaction of driver during driving |
| CN109243438B (en) * | 2018-08-24 | 2023-09-26 | 上海擎感智能科技有限公司 | Method, system and storage medium for regulating emotion of vehicle owner |
| CN109532653B (en) | 2018-10-11 | 2020-01-21 | 百度在线网络技术（北京）有限公司 | Method, device, computer equipment and storage medium for communicating with rear vehicle |
| CN109346079A (en) * | 2018-12-04 | 2019-02-15 | 北京羽扇智信息科技有限公司 | Voice interactive method and device based on Application on Voiceprint Recognition |
| CN109616109B (en) * | 2018-12-04 | 2020-05-19 | 北京蓦然认知科技有限公司 | Voice awakening method, device and system |
| CN109599094A (en) * | 2018-12-17 | 2019-04-09 | 海南大学 | The method of sound beauty and emotion modification |
| CN109669661A (en) * | 2018-12-20 | 2019-04-23 | 广东小天才科技有限公司 | Control method of dictation progress and electronic equipment |
| US11170240B2 (en) * | 2019-01-04 | 2021-11-09 | Cerence Operating Company | Interaction system and method |
| CN109712646A (en) * | 2019-02-20 | 2019-05-03 | 百度在线网络技术（北京）有限公司 | Voice broadcast method, device and terminal |
| CN110085225B (en) * | 2019-04-24 | 2024-01-02 | 北京百度网讯科技有限公司 | Voice interaction method and device, intelligent robot and computer readable storage medium |
| CN111976732A (en) * | 2019-05-23 | 2020-11-24 | 上海博泰悦臻网络技术服务有限公司 | Vehicle control method and system based on vehicle owner emotion and vehicle-mounted terminal |
| CN112009395A (en) * | 2019-05-28 | 2020-12-01 | 北京车和家信息技术有限公司 | Interaction control method, vehicle-mounted terminal and vehicle |
| CN110334669B (en) * | 2019-07-10 | 2021-06-08 | 深圳市华腾物联科技有限公司 | Morphological feature recognition method and equipment |
| CN110215683A (en) * | 2019-07-11 | 2019-09-10 | 龙马智芯（珠海横琴）科技有限公司 | A kind of electronic game system of role playing game |
| CN110534091A (en) * | 2019-08-16 | 2019-12-03 | 广州威尔森信息科技有限公司 | A kind of people-car interaction method identified based on microserver and intelligent sound |
| CN110641476A (en) * | 2019-08-16 | 2020-01-03 | 广汽蔚来新能源汽车科技有限公司 | Interaction method and device based on vehicle-mounted robot, controller and storage medium |
| CN112562661A (en) * | 2019-09-25 | 2021-03-26 | 上海汽车集团股份有限公司 | Vehicle-mounted man-machine interaction system and motor vehicle |
| CN110534135A (en) * | 2019-10-18 | 2019-12-03 | 四川大学华西医院 | A method of emotional characteristics are assessed with heart rate response based on language guidance |
| CN110689906A (en) * | 2019-11-05 | 2020-01-14 | 江苏网进科技股份有限公司 | Law enforcement detection method and system based on voice processing technology |
| JP7264079B2 (en) * | 2020-02-05 | 2023-04-25 | トヨタ自動車株式会社 | Emotion estimation device, method, program and vehicle |
| CN111329498A (en) * | 2020-03-09 | 2020-06-26 | 郑州大学 | Multi-modal driver emotion auxiliary adjusting method |
| CN111402925B (en) * | 2020-03-12 | 2023-10-10 | 阿波罗智联(北京)科技有限公司 | Voice adjustment method, device, electronic equipment, vehicle-mounted system and readable medium |
| CN111666444B (en) * | 2020-06-02 | 2021-04-27 | 中国科学院计算技术研究所 | Audio push method and system based on artificial intelligence, and related method and equipment |
| CN111605556B (en) * | 2020-06-05 | 2022-06-07 | 吉林大学 | Road rage prevention recognition and control system |
| CN111724880A (en) * | 2020-06-09 | 2020-09-29 | 百度在线网络技术（北京）有限公司 | User emotion adjusting method, device, equipment and readable storage medium |
| CN112035034B (en) * | 2020-08-27 | 2024-05-03 | 芜湖盟博科技有限公司 | Vehicle-mounted robot interaction method |
| CN112185422B (en) * | 2020-09-14 | 2022-11-08 | 五邑大学 | Prompt message generation method and voice robot thereof |
| CN112446306A (en) * | 2020-11-13 | 2021-03-05 | 阳光保险集团股份有限公司 | Emotion intensity identification method and device, electronic equipment and storage medium |
| CN113012717A (en) * | 2021-02-22 | 2021-06-22 | 上海埃阿智能科技有限公司 | Emotional feedback information recommendation system and method based on voice recognition |
| CN113658580B (en) * | 2021-06-24 | 2024-06-18 | 大众问问(北京)信息科技有限公司 | Voice prompt method, device, computer equipment and storage medium |
| CN113780062A (en) * | 2021-07-26 | 2021-12-10 | 岚图汽车科技有限公司 | Vehicle-mounted intelligent interaction method based on emotion recognition, storage medium and chip |
| CN116061821A (en) * | 2021-11-01 | 2023-05-05 | 华人运通(江苏)技术有限公司 | Control method, device and system of vehicle, mechanical arm and vehicle |
| CN114049677B (en) * | 2021-12-06 | 2023-08-25 | 中南大学 | Vehicle ADAS control method and system based on driver emotion index |
| DE102022133244A1 (en) | 2022-12-14 | 2024-06-20 | Bayerische Motoren Werke Aktiengesellschaft | Navigation system of a vehicle and method for operating a navigation system of a vehicle |
Citations (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| JP2005239117A (en) * | 2004-01-26 | 2005-09-08 | Nissan Motor Co Ltd | Driver feeling guiding device |
| CN102874259A (en) * | 2012-06-15 | 2013-01-16 | 浙江吉利汽车研究院有限公司杭州分公司 | Automobile driver emotion monitoring and automobile control system |
| CN105700682A (en) * | 2016-01-08 | 2016-06-22 | 北京乐驾科技有限公司 | Intelligent gender and emotion recognition detection system and method based on vision and voice |
| CN106114516A (en) * | 2016-08-31 | 2016-11-16 | 合肥工业大学 | The angry driver behavior modeling of a kind of drive automatically people's characteristic and tampering devic |
| CN206049658U (en) * | 2016-08-31 | 2017-03-29 | 合肥工业大学 | Angry driver behavior modeling and tampering devic based on drive automatically people's characteristic |
Family Cites Families (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| JP5088669B2 (en) * | 2007-03-23 | 2012-12-05 | 株式会社デンソー | Vehicle periphery monitoring device |
| CN101669090A (en) * | 2007-04-26 | 2010-03-10 | 福特全球技术公司 | Emotive advisory system and method |
-
2016
- 2016-12-27 CN CN201611229157.7A patent/CN106803423B/en active Active
Patent Citations (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| JP2005239117A (en) * | 2004-01-26 | 2005-09-08 | Nissan Motor Co Ltd | Driver feeling guiding device |
| CN102874259A (en) * | 2012-06-15 | 2013-01-16 | 浙江吉利汽车研究院有限公司杭州分公司 | Automobile driver emotion monitoring and automobile control system |
| CN105700682A (en) * | 2016-01-08 | 2016-06-22 | 北京乐驾科技有限公司 | Intelligent gender and emotion recognition detection system and method based on vision and voice |
| CN106114516A (en) * | 2016-08-31 | 2016-11-16 | 合肥工业大学 | The angry driver behavior modeling of a kind of drive automatically people's characteristic and tampering devic |
| CN206049658U (en) * | 2016-08-31 | 2017-03-29 | 合肥工业大学 | Angry driver behavior modeling and tampering devic based on drive automatically people's characteristic |
Also Published As
| Publication number | Publication date |
|---|---|
| CN106803423A (en) | 2017-06-06 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN106803423B (en) | Man-machine interaction voice control method and device based on user emotion state and vehicle | |
| US12032730B2 (en) | Methods and systems for using artificial intelligence to evaluate, correct, and monitor user attentiveness | |
| EP3675121B1 (en) | Computer-implemented interaction with a user | |
| EP3384475B1 (en) | System and method of conversational adjustment based on user's cognitive state | |
| US10192171B2 (en) | Method and system using machine learning to determine an automotive driver's emotional state | |
| CN112735440A (en) | Vehicle-mounted intelligent robot interaction method, robot and vehicle | |
| EP3647147B1 (en) | Method and apparatus for eveluating vehicle, device and computer readable storage medium | |
| CN111402925A (en) | Voice adjusting method and device, electronic equipment, vehicle-mounted system and readable medium | |
| KR20100062145A (en) | System and method for controlling sensibility of driver | |
| Kashevnik et al. | Multimodal corpus design for audio-visual speech recognition in vehicle cabin | |
| CN112215097A (en) | Method for monitoring driving state of vehicle, vehicle and computer readable storage medium | |
| EP4042322A1 (en) | Methods and systems for using artificial intelligence to evaluate, correct, and monitor user attentiveness | |
| CN113448433A (en) | Emotion responsive virtual personal assistant | |
| CN113771859A (en) | Intelligent driving intervention method, device and equipment and computer readable storage medium | |
| CN113879235A (en) | Method, system, equipment and storage medium for multi-screen control of automobile | |
| Sanghavi et al. | Exploring the influence of driver affective state and auditory display urgency on takeover performance in semi-automated vehicles: Experiment and modelling | |
| CN113879315A (en) | Driving support device and driving support method | |
| CN113538852A (en) | Fatigue driving reminding method and vehicle-mounted terminal | |
| CN116834749A (en) | Fatigue driving early warning method, device, equipment and storage medium | |
| CN113450788B (en) | Sound output control method and sound output control device | |
| JP6819529B2 (en) | Information processing equipment, information processing system, and information processing method | |
| CN115830724A (en) | Vehicle-mounted recognition interaction method and system based on multi-mode recognition | |
| CN114037981A (en) | Fatigue driving warning method, fatigue driving warning device, storage medium and electronic equipment | |
| KR102132058B1 (en) | Interactive voice communication system embedded in a car | |
| CN105575392A (en) | System and method for user interaction |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |