CN113160826B - Family member communication method and system based on face recognition - Google Patents
Family member communication method and system based on face recognition Download PDFInfo
- Publication number
- CN113160826B CN113160826B CN202110225394.0A CN202110225394A CN113160826B CN 113160826 B CN113160826 B CN 113160826B CN 202110225394 A CN202110225394 A CN 202110225394A CN 113160826 B CN113160826 B CN 113160826B
- Authority
- CN
- China
- Prior art keywords
- user
- recognition device
- face recognition
- voice
- communication
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
- 230000006854 communication Effects 0.000 title claims abstract description 79
- 238000004891 communication Methods 0.000 title claims abstract description 71
- 238000000034 method Methods 0.000 title claims abstract description 62
- 230000008569 process Effects 0.000 claims abstract description 29
- 230000007613 environmental effect Effects 0.000 claims abstract description 24
- 238000013135 deep learning Methods 0.000 claims abstract description 13
- 238000010276 construction Methods 0.000 claims description 3
- 230000002996 emotional effect Effects 0.000 claims description 3
- 230000003213 activating effect Effects 0.000 claims 1
- 230000006870 function Effects 0.000 description 7
- 238000010586 diagram Methods 0.000 description 6
- 230000009471 action Effects 0.000 description 3
- 230000000694 effects Effects 0.000 description 2
- 230000000977 initiatory effect Effects 0.000 description 2
- 239000000463 material Substances 0.000 description 2
- 230000003287 optical effect Effects 0.000 description 2
- 230000003111 delayed effect Effects 0.000 description 1
- 201000010099 disease Diseases 0.000 description 1
- 208000037265 diseases, disorders, signs and symptoms Diseases 0.000 description 1
- 238000005516 engineering process Methods 0.000 description 1
- 230000004438 eyesight Effects 0.000 description 1
- 230000008676 import Effects 0.000 description 1
- 238000012986 modification Methods 0.000 description 1
- 230000004048 modification Effects 0.000 description 1
- 239000013307 optical fiber Substances 0.000 description 1
- 230000001960 triggered effect Effects 0.000 description 1
Images
Classifications
-
- G—PHYSICS
- G10—MUSICAL INSTRUMENTS; ACOUSTICS
- G10L—SPEECH ANALYSIS TECHNIQUES OR SPEECH SYNTHESIS; SPEECH RECOGNITION; SPEECH OR VOICE PROCESSING TECHNIQUES; SPEECH OR AUDIO CODING OR DECODING
- G10L15/00—Speech recognition
- G10L15/22—Procedures used during a speech recognition process, e.g. man-machine dialogue
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F18/00—Pattern recognition
- G06F18/20—Analysing
- G06F18/22—Matching criteria, e.g. proximity measures
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/08—Learning methods
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V20/00—Scenes; Scene-specific elements
- G06V20/35—Categorising the entire scene, e.g. birthday party or wedding scene
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V40/00—Recognition of biometric, human-related or animal-related patterns in image or video data
- G06V40/10—Human or animal bodies, e.g. vehicle occupants or pedestrians; Body parts, e.g. hands
- G06V40/16—Human faces, e.g. facial parts, sketches or expressions
- G06V40/168—Feature extraction; Face representation
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V40/00—Recognition of biometric, human-related or animal-related patterns in image or video data
- G06V40/10—Human or animal bodies, e.g. vehicle occupants or pedestrians; Body parts, e.g. hands
- G06V40/16—Human faces, e.g. facial parts, sketches or expressions
- G06V40/174—Facial expression recognition
-
- G—PHYSICS
- G10—MUSICAL INSTRUMENTS; ACOUSTICS
- G10L—SPEECH ANALYSIS TECHNIQUES OR SPEECH SYNTHESIS; SPEECH RECOGNITION; SPEECH OR VOICE PROCESSING TECHNIQUES; SPEECH OR AUDIO CODING OR DECODING
- G10L15/00—Speech recognition
- G10L15/08—Speech classification or search
- G10L15/10—Speech classification or search using distance or distortion measures between unknown speech and reference templates
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04M—TELEPHONIC COMMUNICATION
- H04M1/00—Substation equipment, e.g. for use by subscribers
- H04M1/26—Devices for calling a subscriber
- H04M1/27—Devices whereby a plurality of signals may be stored simultaneously
- H04M1/274—Devices whereby a plurality of signals may be stored simultaneously with provision for storing more than one subscriber number at a time, e.g. using toothed disc
- H04M1/2745—Devices whereby a plurality of signals may be stored simultaneously with provision for storing more than one subscriber number at a time, e.g. using toothed disc using static electronic memories, e.g. chips
- H04M1/27467—Methods of retrieving data
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04M—TELEPHONIC COMMUNICATION
- H04M1/00—Substation equipment, e.g. for use by subscribers
- H04M1/26—Devices for calling a subscriber
- H04M1/27—Devices whereby a plurality of signals may be stored simultaneously
- H04M1/274—Devices whereby a plurality of signals may be stored simultaneously with provision for storing more than one subscriber number at a time, e.g. using toothed disc
- H04M1/2745—Devices whereby a plurality of signals may be stored simultaneously with provision for storing more than one subscriber number at a time, e.g. using toothed disc using static electronic memories, e.g. chips
- H04M1/27467—Methods of retrieving data
- H04M1/27475—Methods of retrieving data using interactive graphical means or pictorial representations
-
- G—PHYSICS
- G10—MUSICAL INSTRUMENTS; ACOUSTICS
- G10L—SPEECH ANALYSIS TECHNIQUES OR SPEECH SYNTHESIS; SPEECH RECOGNITION; SPEECH OR VOICE PROCESSING TECHNIQUES; SPEECH OR AUDIO CODING OR DECODING
- G10L15/00—Speech recognition
- G10L15/22—Procedures used during a speech recognition process, e.g. man-machine dialogue
- G10L2015/223—Execution procedure of a spoken command
Landscapes
- Engineering & Computer Science (AREA)
- Physics & Mathematics (AREA)
- Theoretical Computer Science (AREA)
- Health & Medical Sciences (AREA)
- Multimedia (AREA)
- General Physics & Mathematics (AREA)
- Human Computer Interaction (AREA)
- General Health & Medical Sciences (AREA)
- Computational Linguistics (AREA)
- Data Mining & Analysis (AREA)
- Oral & Maxillofacial Surgery (AREA)
- Life Sciences & Earth Sciences (AREA)
- General Engineering & Computer Science (AREA)
- Computer Vision & Pattern Recognition (AREA)
- Signal Processing (AREA)
- Audiology, Speech & Language Pathology (AREA)
- Artificial Intelligence (AREA)
- Acoustics & Sound (AREA)
- Evolutionary Computation (AREA)
- Bioinformatics & Cheminformatics (AREA)
- Bioinformatics & Computational Biology (AREA)
- Evolutionary Biology (AREA)
- Biomedical Technology (AREA)
- Biophysics (AREA)
- Molecular Biology (AREA)
- Computing Systems (AREA)
- Mathematical Physics (AREA)
- Software Systems (AREA)
- Collating Specific Patterns (AREA)
- Telephonic Communication Services (AREA)
Abstract
The embodiment of the application provides a family member communication method and system based on face recognition. The method comprises the following steps: a face recognition device and a voice recognition device are arranged in the communication terminal, and the voice recognition device prestores at least one user password template for triggering the face recognition process; after the voice recognition device catches at least two times of sounds with the similarity smaller than a specified threshold value with any one password template in the user password templates, starting the face recognition device; the face recognition device captures a face image through rotation, recognizes the expression characteristics of a user and forms a user state characteristic set; collecting the environment around the user in the rotating process, identifying the environmental characteristics around the user and forming an environmental characteristic set; and importing the expression feature set and the environment feature set of the user into a deep learning network, predicting family members needing communication of the user, and starting a communication process. The method and the device improve the efficiency of family member communication through the human body recognition algorithm.
Description
Technical Field
The application relates to the field of face recognition technology and communication management, in particular to a family member communication method and system based on face recognition.
Background
The family members generally have the members like the old, children or other members needing help, and the members are often inconvenient in the communication process, for example, the old have poor eyesight and inconvenient hands and feet, and are often inconvenient to operate the communication terminal quickly and accurately. However, the demand for communication among such family members is not reduced due to limitations in their operational capabilities, particularly in emergencies such as sudden illness, accidents, etc., where immediate contact with the family members is required for the first time. The current common mode is carried out by a voice dialing mode, but on one hand, the accuracy of the voice dialing is difficult to ensure due to the imperfect language functions of family members needing help, such as old people, children and the like; on the other hand, it is difficult for these family members to make a sound for voice dialing in cases such as natural disasters, criminal crimes, sudden diseases, and the like.
Therefore, it is necessary to develop a more convenient method for family member connection to solve the above problems.
Disclosure of Invention
In view of this, the present application aims to provide a method and a system for family member communication based on face recognition, so as to improve the communication efficiency of family members and solve the technical problem that some special family members cannot normally contact the family members in emergency at present.
Based on the above purpose, the present application provides a family member communication method based on face recognition, which includes:
setting a face recognition device and a voice recognition device in the communication terminal, wherein the voice recognition device prestores at least one user password template for triggering the face recognition process;
after the voice recognition device catches at least two times of sounds with the similarity smaller than a specified threshold value with any one password template in the user password templates, starting the face recognition device;
the face recognition device captures a face image through rotation, and recognizes the expression characteristics of the user to form a user state characteristic set;
collecting the environment around the user in the rotation process, identifying the environmental characteristics around the user and forming an environmental characteristic set;
and importing the expression feature set and the environment feature set of the user into a deep learning network, predicting family members needing communication of the user, and starting a communication process.
In some embodiments, the method further comprises:
if the human body recognition algorithm judges that the standard action sequence is not met, an error instruction is sent to the user, and a replacement action suggestion is provided;
and if the human body recognition algorithm judges that the times of the action sequences which do not accord with the standard exceed the times of a specified threshold aiming at the same user, sending an alarm to an administrator.
In some embodiments, the method further comprises:
and if the communication equipment of the family member cannot be connected, the communication equipment of the user selects the family member with the next rank from the deep learning result to carry out communication.
In some embodiments, the speech recognition apparatus pre-stores at least one user password template for triggering a face recognition process, including:
setting a white list user password template in the voice recognition device, and immediately starting the face recognition process after the voice recognition device catches the voice in the white list user password template;
and setting a blacklist user password template in the voice recognition device, and after capturing the voice in the blacklist user password template, carrying out neglect processing by the voice recognition device.
In some embodiments, after the voice recognition device captures at least two sounds with similarity smaller than a specified threshold with any one of the user password templates, the face recognition device is started, including:
after the voice recognition device captures a first voice for the first time, obtaining a first comparison result by calculating the similarity between the first voice and a corresponding password in the password template;
within a specified time interval, if the voice recognition device catches a second voice, a second comparison result is obtained by calculating the similarity between the second voice and a corresponding password in the password template;
and if the first comparison result and the second comparison result both fall into a preset similarity interval, starting the face recognition device.
In some embodiments, the face recognition device performs face image capture by rotation, identifies expression features of the user, and forms a user status feature set, including:
the face recognition device adjusts the image acquisition direction of the face recognition device according to the voice direction captured by the voice recognition device, and obtains the expression characteristics of the user;
the face recognition device obtains the emotional characteristics and the body state characteristics of the user through the expression characteristics and the body state of the user to form the user state characteristic set.
In some embodiments, the collecting of the environment around the user during the rotating, the identifying of the environmental features around the user, and the forming of the environmental feature set include:
the face recognition device takes the user as a circle center, adjusts the image acquisition direction of the face recognition device and obtains the environmental information around the user;
the face recognition device obtains surrounding crowd and geographic state features of the user through the surrounding environment information of the user to form the user environment feature set.
In some embodiments, each element in the user environment feature set is calculated by the following formula:
wherein j is a discrete distance series, i is an environment type series, f i () And the characteristic value is the environment characteristic value of the ith type of environment, P is the characteristic value of the surrounding crowd, and G is the characteristic value of the geographic state.
Based on the above purpose, the present application further provides a family member communication system based on face recognition, including:
the construction module is used for arranging a face recognition device and a voice recognition device in the communication terminal, wherein the voice recognition device prestores at least one user password template for triggering the face recognition process;
the triggering module is used for starting the face recognition device after the voice recognition device captures at least two times of sounds with the similarity smaller than a specified threshold value with any one password template in the user password templates;
the face recognition module is used for capturing a face image by the face recognition device through rotation, recognizing the expression characteristics of the user and forming a user state characteristic set;
the environment identification module is used for collecting the environment around the user in the rotation process, identifying the environmental characteristics around the user and forming an environmental characteristic set;
and the communication module is used for importing the expression feature set and the environment feature set of the user into a deep learning network, predicting family members needing communication of the user and starting a communication process.
In some embodiments, the system further comprises:
and the delay module is used for selecting the family member with the next rank from the deep learning result by the user's communication equipment to carry out communication if the communication equipment of the family member cannot be connected.
In some embodiments, the building module comprises:
the white list unit is used for setting a white list user password template in the voice recognition device, and immediately starting the face recognition process after the voice recognition device catches the voice in the white list user password template;
and the blacklist unit is used for setting a blacklist user password template in the voice recognition device, and performing ignoring processing after the voice recognition device catches the voice in the blacklist user password template.
In general, the advantages of the present application and the experience brought to the user are: the method and the device improve the efficiency of family member communication through the human body recognition algorithm. The technical problem that some special family members cannot normally contact the family members in emergency at present is solved.
Drawings
In the drawings, like reference numerals refer to the same or similar parts or elements throughout the several views unless otherwise specified. The figures are not necessarily to scale. It is appreciated that these drawings depict only some embodiments in accordance with the disclosure and are therefore not to be considered limiting of its scope.
Fig. 1 is a flowchart illustrating a family member communication method based on face recognition according to an embodiment of the present invention.
Fig. 2 is a flowchart illustrating a family member communication method based on face recognition according to an embodiment of the present invention.
Fig. 3 is a block diagram illustrating a family member communication system based on face recognition according to an embodiment of the present invention.
Fig. 4 is a block diagram illustrating a family member communication system based on face recognition according to an embodiment of the present invention.
Fig. 5 shows a composition diagram of a building block according to an embodiment of the present invention.
Detailed Description
The present application will be described in further detail with reference to the drawings and examples. It is to be understood that the specific embodiments described herein are merely illustrative of the relevant invention and not restrictive of the invention. It should be noted that, for convenience of description, only the portions related to the related invention are shown in the drawings.
It should be noted that, in the present application, the embodiments and features of the embodiments may be combined with each other without conflict. The present application will be described in detail below with reference to the accompanying drawings in conjunction with embodiments.
Fig. 1 shows a flow chart of a family member communication method based on face recognition according to an embodiment of the invention. As shown in fig. 1, the family member communication method based on face recognition includes:
and step S11, setting a face recognition device and a voice recognition device in the communication terminal, wherein the voice recognition device prestores at least one user password template for triggering the face recognition process.
Specifically, the communication terminal may refer to an electronic device having a communication function, such as a mobile phone, a telephone, a computer, and the like. The face recognition device and the voice recognition device can be used as a part of the communication terminal and embedded into the communication terminal, so that the whole equipment is portable and light.
In one embodiment, the voice recognition device prestores at least one user password template for triggering a face recognition process, and the method comprises the following steps:
setting a white list user password template in the voice recognition device, and immediately starting the face recognition process after the voice recognition device captures the voice in the white list user password template;
setting a blacklist user password template in the voice recognition device, and after the voice recognition device captures the voice in the blacklist user password template, performing ignoring processing.
In particular, a user may trigger a face recognition process through at least one password template. For example, when a user calls "make a call to dad" or "make a call to mom," the face recognition process can be triggered.
And step S12, after the voice recognition device captures at least two times of voice with the similarity smaller than a specified threshold value with any password template in the user password template, starting the face recognition device.
Specifically, since the user may inadvertently hit the password in the password template during a daily conversation or conversation, it is contemplated that the face recognition device is activated after hearing two or more times of speech similar to the speech in the password template.
In one embodiment, after the voice recognition device captures at least two sounds with similarity smaller than a specified threshold with any one of the user password templates, the face recognition device is started, and the method comprises the following steps:
after the voice recognition device captures a first voice for the first time, obtaining a first comparison result by calculating the similarity between the first voice and a corresponding password in the password template;
within a specified time interval, if the voice recognition device catches a second voice, a second comparison result is obtained by calculating the similarity between the second voice and a corresponding password in the password template;
and if the first comparison result and the second comparison result both fall into a preset similarity interval, starting the face recognition device.
And step S13, the face recognition device captures a face image through rotation, the expression features of the user are recognized, and a user state feature set is formed.
In one embodiment, the face recognition device performs face image capture by rotation, recognizes expressive features of the user, and forms a user status feature set, including:
the face recognition device adjusts the image acquisition direction of the face recognition device according to the voice direction captured by the voice recognition device, and obtains the expression characteristics of the user;
the face recognition device obtains the emotional characteristics and the body state characteristics of the user through the expression characteristics and the body state of the user to form the user state characteristic set.
Particularly, the collection angle of the face recognition device may affect the collection effect, and therefore, it is necessary to select the face recognition device to achieve the best collection effect.
Step S14, collecting the environment around the user in the rotation process, identifying the environmental features around the user, and forming an environmental feature set.
In one embodiment, the collecting of the environment around the user during the rotation, the identifying of the environmental features around the user, and the forming of the environmental feature set comprise:
the face recognition device takes the user as a circle center, adjusts the image acquisition direction of the face recognition device and obtains the environmental information around the user;
the face recognition device obtains surrounding crowd and geographic state features of the user through the surrounding environment information of the user to form the user environment feature set.
In particular, the face recognition device needs to collect the environmental information around the user to determine the environmental characteristics of the user. For example, if a feature such as a sharp, criminal, etc. is found around the child, the environmental characteristics of the child may be determined.
In one embodiment, each element in the user environment feature set is calculated by the following formula:
wherein j is a discrete distance series, i is an environment type series, f i () And the characteristic value is the environment characteristic value of the ith type of environment, P is the characteristic value of the surrounding crowd, and G is the characteristic value of the geographic state.
And step S15, importing the expression feature set and the environment feature set of the user into a deep learning network, predicting family members needing to be communicated by the user, and starting a communication process.
Specifically, initiating the communicative process refers to initiating communication with a communicative object. For example, if it is predicted to communicate with "dad," then "dad" can be reached.
Fig. 2 is a flowchart illustrating a family member communication method based on face recognition according to an embodiment of the present invention. As shown in fig. 2, the family member communication method based on face recognition further includes:
and step S16, if the communication equipment of the family member can not be connected, the communication equipment of the user selects the family member with the next rank from the deep learning result to carry out communication.
Specifically, it is likely that the communication object cannot be normally connected at some time, but at an emergency, the communication object needs to be immediately contacted with the corresponding communication object, so that the communication object can be immediately contacted with the next communication object, and the problem that the first contact person cannot be contacted and the solving of things is delayed is prevented.
Fig. 3 is a block diagram illustrating a family member communication system based on face recognition according to an embodiment of the present invention. As shown in fig. 3, the whole family member communication system based on face recognition may be divided into:
the building module 31 is used for setting a face recognition device and a voice recognition device in the communication terminal, wherein the voice recognition device prestores at least one user password template for triggering the face recognition process;
the triggering module 32 is configured to start the face recognition device after the voice recognition device captures at least two sounds with similarity smaller than a specified threshold value with any one of the user password templates;
a face recognition module 33, configured to capture a face image by the face recognition device through rotation, recognize an expression feature of the user, and form a user state feature set;
an environment identification module 34, configured to collect an environment around the user in the rotation process, identify an environment feature around the user, and form an environment feature set;
and the communication module 35 is configured to import the expression feature set and the environment feature set of the user into a deep learning network, predict family members of the user who need to communicate, and start a communication process.
Fig. 4 is a block diagram of a family member communication system based on face recognition according to an embodiment of the present invention. As shown in fig. 4, the whole family member communication system based on face recognition further includes:
and a delay module 36, configured to select the family member with the next rank from the deep learning result to perform the communication by the user's communication device if the communication device of the family member cannot be connected.
Fig. 5 shows a composition diagram of a building block according to an embodiment of the present invention. As shown in fig. 5, the construction module 31 of the family member communication system based on face recognition includes:
a white list unit 311, configured to set a white list user password template in the speech recognition apparatus, and immediately start the face recognition process after the speech recognition apparatus captures the speech in the white list user password template;
a blacklist unit 312, configured to set a blacklist user password template in the speech recognition apparatus, and perform an ignoring process after the speech recognition apparatus captures the speech in the blacklist user password template.
The functions of the modules in the systems in the embodiments of the present application may refer to the corresponding descriptions in the above methods, and are not described herein again.
In the description herein, references to the description of the term "one embodiment," "some embodiments," "an example," "a specific example," or "some examples," etc., mean that a particular feature, structure, material, or characteristic described in connection with the embodiment or example is included in at least one embodiment or example of the invention. Furthermore, the particular features, structures, materials, or characteristics described may be combined in any suitable manner in any one or more embodiments or examples. Furthermore, various embodiments or examples and features of different embodiments or examples described in this specification can be combined and combined by one skilled in the art without contradiction.
Any process or method descriptions in flow charts or otherwise described herein may be understood as representing modules, segments, or portions of code which include one or more executable instructions for implementing specific logical functions or steps of the process, and alternate implementations are included within the scope of the preferred embodiment of the present invention in which functions may be executed out of order from that shown or discussed, including substantially concurrently or in reverse order, depending on the functionality involved, as would be understood by those reasonably skilled in the art of the present invention.
The logic and/or steps represented in the flowcharts or otherwise described herein, such as an ordered listing of executable instructions that can be considered to implement logical functions, can be embodied in any computer-readable medium for use by or in connection with an instruction execution system, apparatus, or device, such as a computer-based system, processor-containing system, or other system that can fetch the instructions from the instruction execution system, apparatus, or device and execute the instructions. For the purposes of this description, a "computer-readable medium" can be any means that can contain, store, communicate, propagate, or transport the program for use by or in connection with the instruction execution system, apparatus, or device. More specific examples (a non-exhaustive list) of the computer-readable medium would include the following: an electrical connection (electronic device) having one or more wires, a portable computer diskette (magnetic device), a Random Access Memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber device, and a portable read-only memory (CDROM). Further, the computer-readable medium could even be paper or another suitable medium upon which the program is printed, as the program can be electronically captured, via for instance optical scanning of the paper or other medium, then compiled, interpreted or otherwise processed in a suitable manner if necessary, and then stored in a computer memory.
It should be understood that portions of the present invention may be implemented in hardware, software, firmware, or a combination thereof. In the above embodiments, the various steps or methods may be implemented in software or firmware stored in memory and executed by a suitable instruction execution system. For example, if implemented in hardware, as in another embodiment, any one or combination of the following techniques, which are known in the art, may be used: a discrete logic circuit having a logic gate circuit for implementing a logic function on a data signal, an application specific integrated circuit having an appropriate combinational logic gate circuit, a Programmable Gate Array (PGA), a Field Programmable Gate Array (FPGA), or the like.
It will be understood by those skilled in the art that all or part of the steps carried by the method for implementing the above embodiments may be implemented by hardware related to instructions of a program, which may be stored in a computer readable storage medium, and when the program is executed, the program includes one or a combination of the steps of the method embodiments.
In addition, functional units in the embodiments of the present invention may be integrated into one processing module, or each unit may exist alone physically, or two or more units are integrated into one module. The integrated module can be realized in a hardware mode, and can also be realized in a software functional module mode. The integrated module, if implemented in the form of a software functional module and sold or used as a separate product, may also be stored in a computer readable storage medium. The storage medium may be a read-only memory, a magnetic or optical disk, or the like.
While the invention has been described with reference to specific embodiments, it will be understood by those skilled in the art that various changes and modifications may be made without departing from the spirit and scope of the invention. Therefore, the protection scope of the present invention shall be subject to the protection scope of the claims.
Claims (10)
1. A family member communication method based on face recognition is characterized by comprising the following steps:
a face recognition device and a voice recognition device are arranged in the communication terminal, and the voice recognition device prestores at least one user password template for triggering the face recognition process;
after the voice recognition device catches at least two times of sounds with the similarity smaller than a specified threshold value with any one password template in the user password templates, starting the face recognition device;
the face recognition device captures a face image through rotation, and recognizes the expression characteristics of the user to form a user state characteristic set;
collecting the environment around the user in the rotation process, identifying the environmental characteristics around the user and forming an environmental characteristic set;
and importing the expression feature set and the environment feature set of the user into a deep learning network, predicting family members needing communication of the user, and starting a communication process.
2. The method of claim 1, further comprising:
and if the communication equipment of the family member cannot be connected, the communication equipment of the user selects the family member with the next rank from the deep learning result to carry out communication.
3. The method according to claim 1, wherein the voice recognition device pre-stores at least one user password template for triggering a face recognition process, comprising:
setting a white list user password template in the voice recognition device, and immediately starting the face recognition process after the voice recognition device captures the voice in the white list user password template;
and setting a blacklist user password template in the voice recognition device, and after capturing the voice in the blacklist user password template, carrying out neglect processing by the voice recognition device.
4. The method of claim 1, wherein activating the face recognition device after the voice recognition device captures at least two sounds having a similarity less than a specified threshold with any of the user password templates comprises:
after the voice recognition device captures a first voice for the first time, obtaining a first comparison result by calculating the similarity between the first voice and a corresponding password in the password template;
within a specified time interval, if the voice recognition device captures a second voice, obtaining a second comparison result by calculating the similarity between the second voice and a corresponding password in the password template;
and if the first comparison result and the second comparison result both fall into a preset similarity interval, starting the face recognition device.
5. The method of claim 1, wherein the face recognition device performs face image capture by rotation, recognizes the expression features of the user, and forms a user status feature set, comprising:
the face recognition device adjusts the image acquisition direction of the face recognition device according to the voice direction captured by the voice recognition device, and obtains the expression characteristics of the user;
the face recognition device obtains the emotional characteristics and the body state characteristics of the user through the expression characteristics and the body state of the user to form the user state characteristic set.
6. The method of claim 1, wherein collecting the environment around the user during the rotation, identifying the environmental features around the user, and forming an environmental feature set comprises:
the face recognition device takes the user as a circle center, adjusts the image acquisition direction of the face recognition device, and acquires the environmental information around the user;
the face recognition device obtains surrounding crowd and geographic state features of the user through the surrounding environment information of the user to form the user environment feature set.
7. The method of claim 6, wherein each element in the user environment feature set is calculated by the following formula:
wherein j is a discrete distance series, i is an environment type series, f i () And the characteristic value is the environment characteristic value of the i-th environment, P is the characteristic value of the surrounding crowd, and G is the characteristic value of the geographic state.
8. A family member communication system based on face recognition is characterized by comprising:
the construction module is used for arranging a face recognition device and a voice recognition device in the communication terminal, wherein the voice recognition device prestores at least one user password template for triggering the face recognition process;
the triggering module is used for starting the face recognition device after the voice recognition device catches at least two times of sounds with the similarity smaller than a specified threshold value with any one password template in the user password templates;
the face recognition module is used for capturing a face image by the face recognition device through rotation, recognizing the expression characteristics of the user and forming a user state characteristic set;
the environment identification module is used for collecting the environment around the user in the rotation process, identifying the environmental characteristics around the user and forming an environmental characteristic set;
and the communication module is used for importing the expression feature set and the environment feature set of the user into a deep learning network, predicting family members needing communication of the user and starting a communication process.
9. The system of claim 8, further comprising:
and the delay module is used for selecting the family member with the next rank from the deep learning result by the user's communication equipment to carry out communication if the communication equipment of the family member cannot be connected.
10. The system of claim 8, wherein the building block comprises:
a white list unit, configured to set a white list user password template in the speech recognition device, and immediately start the face recognition process after the speech recognition device captures the speech in the white list user password template;
and the blacklist unit is used for setting a blacklist user password template in the voice recognition device, and performing ignoring processing after the voice recognition device catches the voice in the blacklist user password template.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202110225394.0A CN113160826B (en) | 2021-03-01 | 2021-03-01 | Family member communication method and system based on face recognition |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202110225394.0A CN113160826B (en) | 2021-03-01 | 2021-03-01 | Family member communication method and system based on face recognition |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN113160826A CN113160826A (en) | 2021-07-23 |
| CN113160826B true CN113160826B (en) | 2022-09-02 |
Family
ID=76883754
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202110225394.0A Active CN113160826B (en) | 2021-03-01 | 2021-03-01 | Family member communication method and system based on face recognition |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN113160826B (en) |
Citations (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN101715102A (en) * | 2008-10-02 | 2010-05-26 | 宝利通公司 | Displaying dynamic caller identity during point-to-point and multipoint audio/video conference |
| CN106790054A (en) * | 2016-12-20 | 2017-05-31 | 四川长虹电器股份有限公司 | Interactive authentication system and method based on recognition of face and Application on Voiceprint Recognition |
| CN107895576A (en) * | 2017-11-10 | 2018-04-10 | 北京康力优蓝机器人科技有限公司 | The identification of service robot starts method and device |
| CN111634769A (en) * | 2020-05-13 | 2020-09-08 | 北京云迹科技有限公司 | Method and device for controlling elevator operation |
Family Cites Families (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| JP6693111B2 (en) * | 2015-12-14 | 2020-05-13 | カシオ計算機株式会社 | Interactive device, robot, interactive method and program |
| TW202027062A (en) * | 2018-12-28 | 2020-07-16 | 塞席爾商元鼎音訊股份有限公司 | Sound playback system and output sound adjusting method thereof |
-
2021
- 2021-03-01 CN CN202110225394.0A patent/CN113160826B/en active Active
Patent Citations (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN101715102A (en) * | 2008-10-02 | 2010-05-26 | 宝利通公司 | Displaying dynamic caller identity during point-to-point and multipoint audio/video conference |
| CN106790054A (en) * | 2016-12-20 | 2017-05-31 | 四川长虹电器股份有限公司 | Interactive authentication system and method based on recognition of face and Application on Voiceprint Recognition |
| CN107895576A (en) * | 2017-11-10 | 2018-04-10 | 北京康力优蓝机器人科技有限公司 | The identification of service robot starts method and device |
| CN111634769A (en) * | 2020-05-13 | 2020-09-08 | 北京云迹科技有限公司 | Method and device for controlling elevator operation |
Also Published As
| Publication number | Publication date |
|---|---|
| CN113160826A (en) | 2021-07-23 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN107278369B (en) | Personnel searching method, device and communication system | |
| CN109769099B (en) | Method and device for detecting abnormality of call person | |
| KR101528086B1 (en) | System and method for providing conference information | |
| US9854439B2 (en) | Device and method for authenticating a user of a voice user interface and selectively managing incoming communications | |
| CN108122555B (en) | Communication method, voice recognition device and terminal device | |
| CN111968635B (en) | Speech recognition method, device and storage medium | |
| CN109793526B (en) | Lie detection method, device, computer equipment and storage medium | |
| CN108806684B (en) | Position prompting method and device, storage medium and electronic equipment | |
| JP2006259925A (en) | Object authentication device, cellular phone, object authentication method and object authentication program | |
| CN106878535A (en) | The based reminding method and device of mobile terminal locations | |
| CN112908325B (en) | Voice interaction method and device, electronic equipment and storage medium | |
| CN111583919A (en) | Information processing method, device and storage medium | |
| CN110730330B (en) | Sound processing method and device, doorbell and computer readable storage medium | |
| CN112509586A (en) | Method and device for recognizing voice print of telephone channel | |
| CN106327792B (en) | Alarm method and alarm system | |
| CN113160826B (en) | Family member communication method and system based on face recognition | |
| CN107729737B (en) | Identity information acquisition method and wearable device | |
| CN110491384B (en) | Voice data processing method and device | |
| JP2007037088A (en) | Intercom device | |
| CN111641751A (en) | Screen unlocking method and device of terminal equipment | |
| CN114005436A (en) | Method, device and storage medium for determining voice endpoint | |
| CN113314125A (en) | Voiceprint identification method, system and memory for monitoring room interphone | |
| JP2005348240A (en) | Telephone device | |
| CN110401775B (en) | Alarm clock setting method, alarm clock setting device and mobile terminal | |
| CN110738812B (en) | Doorbell prompting method and related product |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |