CN107919117A - A kind of active voice assistant based on recognition of face - Google Patents
A kind of active voice assistant based on recognition of face Download PDFInfo
- Publication number
- CN107919117A CN107919117A CN201610883581.7A CN201610883581A CN107919117A CN 107919117 A CN107919117 A CN 107919117A CN 201610883581 A CN201610883581 A CN 201610883581A CN 107919117 A CN107919117 A CN 107919117A
- Authority
- CN
- China
- Prior art keywords
- driver
- signal
- face
- recognition
- face recognition
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
- 230000009471 action Effects 0.000 claims abstract description 11
- 230000036651 mood Effects 0.000 claims description 3
- 230000008054 signal transmission Effects 0.000 claims description 3
- 230000002452 interceptive effect Effects 0.000 abstract description 2
- 238000010586 diagram Methods 0.000 description 2
- 230000004913 activation Effects 0.000 description 1
- 238000013473 artificial intelligence Methods 0.000 description 1
- 230000006399 behavior Effects 0.000 description 1
- 239000012141 concentrate Substances 0.000 description 1
- 230000003111 delayed effect Effects 0.000 description 1
- 230000002996 emotional effect Effects 0.000 description 1
- 230000001815 facial effect Effects 0.000 description 1
- 230000008921 facial expression Effects 0.000 description 1
- 230000003993 interaction Effects 0.000 description 1
- 238000000034 method Methods 0.000 description 1
- 230000008569 process Effects 0.000 description 1
Classifications
-
- G—PHYSICS
- G10—MUSICAL INSTRUMENTS; ACOUSTICS
- G10L—SPEECH ANALYSIS TECHNIQUES OR SPEECH SYNTHESIS; SPEECH RECOGNITION; SPEECH OR VOICE PROCESSING TECHNIQUES; SPEECH OR AUDIO CODING OR DECODING
- G10L15/00—Speech recognition
- G10L15/22—Procedures used during a speech recognition process, e.g. man-machine dialogue
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
- G06F3/16—Sound input; Sound output
- G06F3/165—Management of the audio stream, e.g. setting of volume, audio stream path
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V20/00—Scenes; Scene-specific elements
- G06V20/50—Context or environment of the image
- G06V20/59—Context or environment of the image inside of a vehicle, e.g. relating to seat occupancy, driver state or inner lighting conditions
- G06V20/597—Recognising the driver's state or behaviour, e.g. attention or drowsiness
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V40/00—Recognition of biometric, human-related or animal-related patterns in image or video data
- G06V40/10—Human or animal bodies, e.g. vehicle occupants or pedestrians; Body parts, e.g. hands
- G06V40/16—Human faces, e.g. facial parts, sketches or expressions
- G06V40/174—Facial expression recognition
-
- G—PHYSICS
- G08—SIGNALLING
- G08B—SIGNALLING OR CALLING SYSTEMS; ORDER TELEGRAPHS; ALARM SYSTEMS
- G08B21/00—Alarms responsive to a single specified undesired or abnormal condition and not otherwise provided for
- G08B21/02—Alarms for ensuring the safety of persons
- G08B21/06—Alarms for ensuring the safety of persons indicating a condition of sleep, e.g. anti-dozing alarms
-
- G—PHYSICS
- G10—MUSICAL INSTRUMENTS; ACOUSTICS
- G10L—SPEECH ANALYSIS TECHNIQUES OR SPEECH SYNTHESIS; SPEECH RECOGNITION; SPEECH OR VOICE PROCESSING TECHNIQUES; SPEECH OR AUDIO CODING OR DECODING
- G10L15/00—Speech recognition
- G10L15/28—Constructional details of speech recognition systems
- G10L15/30—Distributed recognition, e.g. in client-server systems, for mobile phones or network applications
Landscapes
- Engineering & Computer Science (AREA)
- Physics & Mathematics (AREA)
- Multimedia (AREA)
- Health & Medical Sciences (AREA)
- General Physics & Mathematics (AREA)
- Theoretical Computer Science (AREA)
- Human Computer Interaction (AREA)
- Audiology, Speech & Language Pathology (AREA)
- General Health & Medical Sciences (AREA)
- Computational Linguistics (AREA)
- Acoustics & Sound (AREA)
- Emergency Management (AREA)
- Business, Economics & Management (AREA)
- General Engineering & Computer Science (AREA)
- Oral & Maxillofacial Surgery (AREA)
- User Interface Of Digital Computer (AREA)
Abstract
The present invention relates to a kind of active voice assistant based on recognition of face, the active voice assistant includes：External equipment, relevant action is carried out for gathering the face recognition signal and voice signal of driver, while according to operational order；Cloud server, for analyzing face recognition signal, judges the state of driver, and analyzes voice signal, judges the instruction that driver sends；Central control module, face recognition signal and voice signal for external equipment to be collected are transmitted to cloud server, and send operational order to external equipment according to the analysis result of cloud server.Compared with prior art, the present invention has the advantages that interactive performance is strong, drive safety is high, strong using simple and compatible extensions.
Description
Technical field
The present invention relates to mobile unit field, more particularly, to a kind of active voice assistant based on recognition of face.
Background technology
With the popularization of automobile consumption, comfort level when people drive automobile product requires higher and higher.Full Vehicle System
More and more amusement functions are added, cause the hidden danger of some driving safeties.The introducing of voice assistant is delayed to a certain extent
Such case is solved, and then the automobile industry in today becomes more and more popular.Driver only needs to say related voice instruction, so that it may
To manipulate vehicle device, sight is avoided to deviate the distractive security risk such as road ahead or manual operation.
But the voice assistant on current market is all passivity.Voice assistant can keep silent for a long time, it is necessary to
Driver says that special instruction goes to wake up it.Its behavior is more like an instruction interpreter, rather than a voice assistant, because
Its degree of understanding to driver is very low, and such voice assistant, which is scarcely out of swaddling-clothes, many aspects to be hoisted.
The content of the invention
The purpose of the present invention is provide a kind of active voice assistant based on recognition of face regarding to the issue above.
The purpose of the present invention can be achieved through the following technical solutions：
A kind of active voice assistant based on recognition of face, the active voice assistant include：
External equipment, for gathering the face recognition signal and voice signal of driver, while carries out according to operational order
Relevant action；
Cloud server, for analyzing face recognition signal, judges the state of driver, and analyzes voice signal, judges
The instruction that driver sends；
Central control module, face recognition signal and voice signal for external equipment to be collected are transmitted to high in the clouds clothes
Business device, and operational order is sent to external equipment according to the analysis result of cloud server.
The external equipment includes：
Signal collecting device, for gathering the face recognition signal and voice signal of driver, and is transmitted to central control
Module；
Loudspeaker, for carrying out relevant action according to the operational order of central control module.
The signal collecting device includes：
Camera, for gathering the face recognition signal of driver and being transmitted to central control module；
Microphone, for gathering the voice signal of driver and being transmitted to central control module.
The relevant action includes user and greets, play entertainment information, play information on services, road conditions prompting and low notice
Warning.
The cloud server includes：
Face recognition engine, for analyzing face recognition signal, judges the state of driver, and will determine that result send to
Central control module；
Speech recognition engine, for analyzing voice signal, judges the instruction that driver sends, and will determine that result send to
Central control module.
The face recognition engine includes：
Physiological status identifies engine, for receiving face recognition signal, judges the physiological status of driver；
Psychological condition identifies engine, for analyzing face recognition signal, judges the psychological condition of driver.
The physiological status includes driver's age and driver's gender；The psychological condition includes driver's mood and drives
The person's of sailing notice.
The central control module includes：
Signal transmission unit, the face recognition signal and voice signal collected for receiving external device, and be transmitted to
Cloud server；
Main control unit, for the analysis result fed back according to cloud server, sends operational order to external equipment.
Compared with prior art, the invention has the advantages that：
(1) face recognition engine is equipped with cloud server, face recognition can be carried out to driver, and know according to face
Other result is actively initiated and the interaction of driver, Active Performance are high.
(2) speech recognition engine is equipped with cloud server, the phonetic order of driver can be analyzed and carried out
Feedback, is combined with face recognition, improves the interactive performance of whole voice assistant.
(3) the active instruction that user can send voice assistant is answered, compared with traditional voice assistant, it is not necessary to
Remember to hold confusing activation instruction too much, greatly simplify the complexity of the output order of driver.
(4) face recognition engine includes psychological condition identification engine, can be to driver's compared with traditional face recognition
Facial expression is caught, and then analyzes the state of attention and emotional state of driver, do not concentrate in driver attention or
Driver is reminded during fatigue driving in time, improves the security of driving.
(5) user's information interested is reported according to the age of user and gender automatically for user, artificial intelligence degree is high,
And the recreational of driving is improved, function is more abundant.
(6) face recognition engine and speech recognition engine are arranged on cloud server, compared with traditional voice assistant,
New hardware cost need not be increased, compatible extensions performance is strong.
Brief description of the drawings
Fig. 1 is the structure diagram of the present invention；
Fig. 2 is the correspondence schematic diagram of face recognition information and instruction action；
Wherein, 1 is central control module, and 2 be cloud server, and 3 be loudspeaker, and 4 be camera, and 5 be microphone.
Embodiment
The present invention is described in detail with specific embodiment below in conjunction with the accompanying drawings.The present embodiment is with technical solution of the present invention
Premised on implemented, give detailed embodiment and specific operating process, but protection scope of the present invention is not limited to
Following embodiments.
As shown in Figure 1, be the active voice assistant based on recognition of face, including：External equipment, for gathering driver's
Face recognition signal and voice signal, while relevant action is carried out according to operational order；Cloud server 2, for analyzing face
Identification signal, judges the state of driver, and analyzes voice signal, judges the instruction that driver sends；Central control module 1,
Face recognition signal and voice signal for external equipment to be collected are transmitted to cloud server 2, and according to cloud service
The analysis result of device 2 sends operational order to external equipment.
Wherein, external equipment includes：Signal collecting device, for gathering the face recognition signal harmony message of driver
Number, and it is transmitted to central control module 1；Loudspeaker 3, for carrying out relevant action according to the operational order of central control module 1.
Signal collecting device includes：Camera 4, for gathering the face recognition signal of driver and being transmitted to central control module 1；Wheat
Gram wind 5, for gathering the voice signal of driver and being transmitted to central control module 1.
Cloud server 2 includes：Face recognition engine, for analyzing face recognition signal, judges the state of driver, and
It will determine that result is sent to central control module 1；Speech recognition engine, for analyzing voice signal, judges what driver sent
Instruction, and will determine that result is sent to central control module 1.Above-mentioned face recognition engine includes：Physiological status identifies engine, uses
In receiving face recognition signal, the physiological status of driver is judged；Psychological condition identifies engine, for analyzing face recognition letter
Number, judge the psychological condition of driver.Wherein physiological status includes driver's age and driver's gender；Psychological condition includes driving
The person's of sailing mood and driver attention
Central control module 1 includes：Signal transmission unit, for receive face recognition signal that external device collects and
Voice signal, and it is transmitted to cloud server 2；Main control unit, for the analysis result fed back according to cloud server 2, sends
Operational order is to external equipment.
Above-mentioned voice assistant at work, by camera 4 (the present embodiment uses high-resolution camera) collection, know by face
Central control unit is connected to after other vision signal, then the face recognition engine being transferred on cloud server 2, recognition of face are drawn
Hold up and analysis result is fed back into central control unit after signal Analysis, central control unit connection loudspeaker 3 sends " hello " and refers to
Order, user send control instruction by microphone 5, finally connect vehicular amusement apparatus and complete control, and in particular to the work arrived
Principle is as follows：
High-resolution camera can catch the countenance information for collecting driver, for example, current drivers be male or
Women, expression are pleasant or gloomy.Central control module 1 can get video flowing from high-resolution camera module, in
Centre control module 1 draws the recognition of face of (or card is locally stored) on the video stream got to cloud server 2
Hold up and parsed；Face recognition engine analyzes the facial information of driver in the video flowing sended over, and then feedback result arrives
Central control unit；1 controlling loudspeaker 3 of central control module says " hello " instruction to driver, according to the result recognized
Information provides some of the recommendations or warning, for example recognizes the expression of joy, plays some easily music；What is recognized is four
The middle-aged male of ten years old or so can provide the service of some finance and economics, and recognize young woman when when can provide current popular
Still information service, specific corresponding to relation as shown in Fig. 2, such as after voice assistant provides the instruction of " broadcasting song ", user
Microphone 5 is fed back to, says the instruction similar to " song for playing Liu De China ", then central control module 1 receives user and refers to
Parsing carries out the operation of next step after order；Identification correspond to user's greeting；Age/gender identification correspond to a variety of ages/
The instruction action of gender service；Expression Recognition correspond to play relaxed music, the command operating such as tell funny stories.
Claims (8)
1. a kind of active voice assistant based on recognition of face, it is characterised in that the active voice assistant includes：
External equipment, correlation is carried out for gathering the face recognition signal and voice signal of driver, while according to operational order
Action；
Cloud server, for analyzing face recognition signal, judges the state of driver, and analyzes voice signal, judges to drive
The instruction that member sends；
Central control module, face recognition signal and voice signal for external equipment to be collected are transmitted to cloud service
Device, and operational order is sent to external equipment according to the analysis result of cloud server.
2. the active voice assistant according to claim 1 based on recognition of face, it is characterised in that the external equipment bag
Include：
Signal collecting device, for gathering the face recognition signal and voice signal of driver, and is transmitted to central control module；
Loudspeaker, for carrying out relevant action according to the operational order of central control module.
3. the active voice assistant according to claim 2 based on recognition of face, it is characterised in that the signal acquisition is set
It is standby to include：
Camera, for gathering the face recognition signal of driver and being transmitted to central control module；
Microphone, for gathering the voice signal of driver and being transmitted to central control module.
4. the active voice assistant according to claim 1 based on recognition of face, it is characterised in that the relevant action bag
User is included to greet, play entertainment information, play information on services, road conditions prompting and the warning of low notice.
5. the active voice assistant according to claim 1 based on recognition of face, it is characterised in that the cloud server
Including：
Face recognition engine, for analyzing face recognition signal, judges the state of driver, and will determine that result is sent to center
Control module；
Speech recognition engine, for analyzing voice signal, judges the instruction that driver sends, and will determine that result is sent to center
Control module.
6. the active voice assistant according to claim 5 based on recognition of face, it is characterised in that the recognition of face is drawn
Hold up including：
Physiological status identifies engine, for receiving face recognition signal, judges the physiological status of driver；
Psychological condition identifies engine, for analyzing face recognition signal, judges the psychological condition of driver.
7. the active voice assistant according to claim 6 based on recognition of face, it is characterised in that the physiological status bag
Include driver's age and driver's gender；The psychological condition includes driver's mood and driver attention.
8. the active voice assistant according to claim 1 based on recognition of face, it is characterised in that the center control mould
Block includes：
Signal transmission unit, the face recognition signal and voice signal collected for receiving external device, and it is transmitted to high in the clouds
Server；
Main control unit, for the analysis result fed back according to cloud server, sends operational order to external equipment.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201610883581.7A CN107919117A (en) | 2016-10-10 | 2016-10-10 | A kind of active voice assistant based on recognition of face |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201610883581.7A CN107919117A (en) | 2016-10-10 | 2016-10-10 | A kind of active voice assistant based on recognition of face |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN107919117A true CN107919117A (en) | 2018-04-17 |
Family
ID=61891843
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201610883581.7A Pending CN107919117A (en) | 2016-10-10 | 2016-10-10 | A kind of active voice assistant based on recognition of face |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN107919117A (en) |
Cited By (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN109858314A (en) * | 2018-07-31 | 2019-06-07 | 徐昌有 | A kind of vehicle intelligent face identification device |
| US20190279106A1 (en) * | 2018-03-06 | 2019-09-12 | Samsung Electronics Co., Ltd | Dynamically evolving hybrid personalized artificial intelligence system |
| WO2019201304A1 (en) * | 2018-04-20 | 2019-10-24 | 比亚迪股份有限公司 | Face recognition-based voice processing method, and device |
| CN110490592A (en) * | 2018-05-15 | 2019-11-22 | 上海博泰悦臻网络技术服务有限公司 | Interior consumption and payment method and cloud server based on recognition of face |
| CN111857638A (en) * | 2020-06-01 | 2020-10-30 | 江西江铃集团新能源汽车有限公司 | Voice interaction method and system based on face recognition and automobile |
| CN114312818A (en) * | 2022-01-29 | 2022-04-12 | 中国第一汽车股份有限公司 | Vehicle control method and device, vehicle and medium |
Citations (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| JP2006166970A (en) * | 2004-12-13 | 2006-06-29 | Electronic Navigation Research Institute | Speech sound collection system for driver |
| CN103956128A (en) * | 2014-05-09 | 2014-07-30 | 东华大学 | Intelligent active advertising platform based on somatosensory technology |
| CN105700682A (en) * | 2016-01-08 | 2016-06-22 | 北京乐驾科技有限公司 | Intelligent gender and emotion recognition detection system and method based on vision and voice |
-
2016
- 2016-10-10 CN CN201610883581.7A patent/CN107919117A/en active Pending
Patent Citations (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| JP2006166970A (en) * | 2004-12-13 | 2006-06-29 | Electronic Navigation Research Institute | Speech sound collection system for driver |
| CN103956128A (en) * | 2014-05-09 | 2014-07-30 | 东华大学 | Intelligent active advertising platform based on somatosensory technology |
| CN105700682A (en) * | 2016-01-08 | 2016-06-22 | 北京乐驾科技有限公司 | Intelligent gender and emotion recognition detection system and method based on vision and voice |
Cited By (7)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20190279106A1 (en) * | 2018-03-06 | 2019-09-12 | Samsung Electronics Co., Ltd | Dynamically evolving hybrid personalized artificial intelligence system |
| US11676062B2 (en) * | 2018-03-06 | 2023-06-13 | Samsung Electronics Co., Ltd. | Dynamically evolving hybrid personalized artificial intelligence system |
| WO2019201304A1 (en) * | 2018-04-20 | 2019-10-24 | 比亚迪股份有限公司 | Face recognition-based voice processing method, and device |
| CN110490592A (en) * | 2018-05-15 | 2019-11-22 | 上海博泰悦臻网络技术服务有限公司 | Interior consumption and payment method and cloud server based on recognition of face |
| CN109858314A (en) * | 2018-07-31 | 2019-06-07 | 徐昌有 | A kind of vehicle intelligent face identification device |
| CN111857638A (en) * | 2020-06-01 | 2020-10-30 | 江西江铃集团新能源汽车有限公司 | Voice interaction method and system based on face recognition and automobile |
| CN114312818A (en) * | 2022-01-29 | 2022-04-12 | 中国第一汽车股份有限公司 | Vehicle control method and device, vehicle and medium |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN107919117A (en) | A kind of active voice assistant based on recognition of face | |
| CN110390932A (en) | Method of speech processing and its equipment based on recognition of face | |
| CN108831460A (en) | A kind of interactive voice control system and method based on fatigue monitoring | |
| CN102298694A (en) | Man-machine interaction identification system applied to remote information service | |
| CN110008879A (en) | Vehicle-mounted personalization audio-video frequency content method for pushing and device | |
| CN109243462A (en) | A kind of voice awakening method and device | |
| CN203661267U (en) | Automobile audio system | |
| CN114327041B (en) | Multi-mode interaction method and system for intelligent cabin and intelligent cabin with multi-mode interaction method and system | |
| CN106847291A (en) | Speech recognition system and method that a kind of local and high in the clouds is combined | |
| CN107808191A (en) | The output intent and system of the multi-modal interaction of visual human | |
| CN114445888A (en) | Vehicle-mounted interaction system based on emotion perception and voice interaction | |
| CN105807925A (en) | Flexible electronic skin based lip language identification system and method | |
| CN110389744A (en) | Multimedia music processing method and system based on recognition of face | |
| CN109941231A (en) | Vehicle-mounted terminal equipment, vehicle-mounted interactive system and exchange method | |
| CN110154056A (en) | Service robot and its man-machine interaction method | |
| CN110103819A (en) | Fatigue driving awakening method and system | |
| CN116229977A (en) | System for realizing intelligent real-time interactive question and answer based on virtual digital person and processing method thereof | |
| CN111833875A (en) | Embedded voice interaction system | |
| CN106388713A (en) | Intelligent sweeping robot | |
| CN109835280B (en) | System for displaying vehicle state and driving behavior through voice recognition and vehicle | |
| CN209571226U (en) | A kind of speech recognition equipment and system | |
| CN206441536U (en) | A kind of active voice assistant based on recognition of face | |
| CN112562267A (en) | Vehicle-mounted safety robot and safe driving assistance method | |
| CN208652928U (en) | Electric heater and its wisdom interactive voice response system based on artificial intelligence interaction technique | |
| CN109545215A (en) | A kind of awakening method and Rouser of vehicle intelligent equipment |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| CB02 | Change of applicant information |
Address after: 215126 No. 123 Changyang street, Suzhou Industrial Park, Suzhou, Jiangsu. Applicant after: Annex Electronics (Suzhou) Co., Ltd. Address before: 215126 No. 123 Changyang street, Suzhou Industrial Park, Suzhou, Jiangsu. Applicant before: Delphi Electronics (Suzhou) Co., Ltd. |
|
| CB02 | Change of applicant information |