CN110767208A - Auxiliary rescue communication method and device based on unvoiced instruction recognition of facial surface muscle signals - Google Patents
Auxiliary rescue communication method and device based on unvoiced instruction recognition of facial surface muscle signals Download PDFInfo
- Publication number
- CN110767208A CN110767208A CN201911128112.4A CN201911128112A CN110767208A CN 110767208 A CN110767208 A CN 110767208A CN 201911128112 A CN201911128112 A CN 201911128112A CN 110767208 A CN110767208 A CN 110767208A
- Authority
- CN
- China
- Prior art keywords
- muscle
- processing device
- sample
- signals
- value
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
- 210000003205 muscle Anatomy 0.000 title claims abstract description 56
- 238000000034 method Methods 0.000 title claims abstract description 35
- 230000001815 facial effect Effects 0.000 title claims abstract description 18
- 238000004891 communication Methods 0.000 title claims abstract description 17
- 238000012545 processing Methods 0.000 claims abstract description 43
- 238000000605 extraction Methods 0.000 claims abstract description 13
- 238000007781 pre-processing Methods 0.000 claims abstract description 7
- 238000007637 random forest analysis Methods 0.000 claims description 15
- 230000000694 effects Effects 0.000 claims description 14
- 230000008569 process Effects 0.000 claims description 9
- 230000029058 respiratory gaseous exchange Effects 0.000 claims description 9
- 238000004422 calculation algorithm Methods 0.000 claims description 8
- 230000006870 function Effects 0.000 claims description 8
- 238000004364 calculation method Methods 0.000 claims description 4
- 238000012360 testing method Methods 0.000 claims description 4
- 238000004458 analytical method Methods 0.000 claims description 3
- 238000001914 filtration Methods 0.000 claims description 3
- 238000005259 measurement Methods 0.000 claims description 3
- 230000037433 frameshift Effects 0.000 claims description 2
- 238000005457 optimization Methods 0.000 claims description 2
- 230000035945 sensitivity Effects 0.000 claims description 2
- 239000000126 substance Substances 0.000 claims description 2
- 230000007613 environmental effect Effects 0.000 abstract description 3
- 230000008602 contraction Effects 0.000 description 3
- 238000010586 diagram Methods 0.000 description 2
- 230000003183 myoelectrical effect Effects 0.000 description 2
- 210000001087 myotubule Anatomy 0.000 description 2
- 230000002776 aggregation Effects 0.000 description 1
- 238000004220 aggregation Methods 0.000 description 1
- 210000000988 bone and bone Anatomy 0.000 description 1
- 230000008859 change Effects 0.000 description 1
- 239000003245 coal Substances 0.000 description 1
- 238000005516 engineering process Methods 0.000 description 1
- 210000001097 facial muscle Anatomy 0.000 description 1
- 238000004519 manufacturing process Methods 0.000 description 1
- 238000012544 monitoring process Methods 0.000 description 1
- 230000002232 neuromuscular Effects 0.000 description 1
- 230000009467 reduction Effects 0.000 description 1
- 238000011160 research Methods 0.000 description 1
- 238000006467 substitution reaction Methods 0.000 description 1
- 230000001755 vocal effect Effects 0.000 description 1
Images
Classifications
-
- G—PHYSICS
- G10—MUSICAL INSTRUMENTS; ACOUSTICS
- G10L—SPEECH ANALYSIS TECHNIQUES OR SPEECH SYNTHESIS; SPEECH RECOGNITION; SPEECH OR VOICE PROCESSING TECHNIQUES; SPEECH OR AUDIO CODING OR DECODING
- G10L13/00—Speech synthesis; Text to speech systems
- G10L13/08—Text analysis or generation of parameters for speech synthesis out of text, e.g. grapheme to phoneme translation, prosody generation or stress or intonation determination
-
- G—PHYSICS
- G10—MUSICAL INSTRUMENTS; ACOUSTICS
- G10L—SPEECH ANALYSIS TECHNIQUES OR SPEECH SYNTHESIS; SPEECH RECOGNITION; SPEECH OR VOICE PROCESSING TECHNIQUES; SPEECH OR AUDIO CODING OR DECODING
- G10L13/00—Speech synthesis; Text to speech systems
- G10L13/02—Methods for producing synthetic speech; Speech synthesisers
-
- G—PHYSICS
- G10—MUSICAL INSTRUMENTS; ACOUSTICS
- G10L—SPEECH ANALYSIS TECHNIQUES OR SPEECH SYNTHESIS; SPEECH RECOGNITION; SPEECH OR VOICE PROCESSING TECHNIQUES; SPEECH OR AUDIO CODING OR DECODING
- G10L15/00—Speech recognition
- G10L15/005—Language recognition
-
- G—PHYSICS
- G10—MUSICAL INSTRUMENTS; ACOUSTICS
- G10L—SPEECH ANALYSIS TECHNIQUES OR SPEECH SYNTHESIS; SPEECH RECOGNITION; SPEECH OR VOICE PROCESSING TECHNIQUES; SPEECH OR AUDIO CODING OR DECODING
- G10L15/00—Speech recognition
- G10L15/08—Speech classification or search
-
- G—PHYSICS
- G10—MUSICAL INSTRUMENTS; ACOUSTICS
- G10L—SPEECH ANALYSIS TECHNIQUES OR SPEECH SYNTHESIS; SPEECH RECOGNITION; SPEECH OR VOICE PROCESSING TECHNIQUES; SPEECH OR AUDIO CODING OR DECODING
- G10L15/00—Speech recognition
- G10L15/24—Speech recognition using non-acoustical features
-
- G—PHYSICS
- G10—MUSICAL INSTRUMENTS; ACOUSTICS
- G10L—SPEECH ANALYSIS TECHNIQUES OR SPEECH SYNTHESIS; SPEECH RECOGNITION; SPEECH OR VOICE PROCESSING TECHNIQUES; SPEECH OR AUDIO CODING OR DECODING
- G10L15/00—Speech recognition
- G10L15/26—Speech to text systems
-
- G—PHYSICS
- G10—MUSICAL INSTRUMENTS; ACOUSTICS
- G10L—SPEECH ANALYSIS TECHNIQUES OR SPEECH SYNTHESIS; SPEECH RECOGNITION; SPEECH OR VOICE PROCESSING TECHNIQUES; SPEECH OR AUDIO CODING OR DECODING
- G10L15/00—Speech recognition
- G10L15/28—Constructional details of speech recognition systems
- G10L15/30—Distributed recognition, e.g. in client-server systems, for mobile phones or network applications
-
- G—PHYSICS
- G10—MUSICAL INSTRUMENTS; ACOUSTICS
- G10L—SPEECH ANALYSIS TECHNIQUES OR SPEECH SYNTHESIS; SPEECH RECOGNITION; SPEECH OR VOICE PROCESSING TECHNIQUES; SPEECH OR AUDIO CODING OR DECODING
- G10L15/00—Speech recognition
- G10L15/08—Speech classification or search
- G10L2015/088—Word spotting
Landscapes
- Engineering & Computer Science (AREA)
- Computational Linguistics (AREA)
- Health & Medical Sciences (AREA)
- Audiology, Speech & Language Pathology (AREA)
- Human Computer Interaction (AREA)
- Physics & Mathematics (AREA)
- Acoustics & Sound (AREA)
- Multimedia (AREA)
- Measurement And Recording Of Electrical Phenomena And Electrical Characteristics Of The Living Body (AREA)
Abstract
The invention discloses an auxiliary rescue communication method and device based on the non-sounding instruction identification of facial surface muscle signals, which comprises the steps of firstly, collecting muscle electric signals at corresponding positions in real time through a measuring electrode which is attached to the facial skin of a user in advance; the data processing device carries out preprocessing, feature extraction and classification identification operations on the collected muscle electric signals, and identifies corresponding instruction words; then the language processing device converts the recognized instruction words into artificial voice; and then the artificial voice is sent to an earphone of an operator through a radio frequency device, and the voice text is sent to a command center in a wireless mode. The method and the device avoid the inherent problem that the traditional voice recognition result is interfered by the environmental background noise, and are suitable for the application scene with high noise background or incapable of receiving sound.
Description
Technical Field
The invention relates to the technical field of rescue communication, in particular to an auxiliary rescue communication method and device based on the unvoiced instruction recognition of facial surface muscle signals.
Background
Speech recognition under strong background noise environment, such as disasters, wars, etc., is always one of the important problems in the speech recognition field, and is also one of the inherent problems that have not been solved perfectly. If communication is needed in an environment with strong background noise, for example, firefighters or divers need to give instructions according to the current state, communicate among teammates, and the like, the performance of voice recognition is more important.
In the safe production and rescue work of coal mines, rescue teams or reconnaissance teams need to communicate effectively under severe and extreme environments. However, the live engine sound and the working sound can weaken the sound of communication between team members, even if the existing digital voice communication technology is adopted and the microphone is carried for receiving sound, the obtained voice still cannot completely eliminate the interference of background sound, such as intermittent hissing sound emitted by breathing equipment carried by the team members, and the breathing device covering the face can distort the emitted sound, so that the speaking is unclear. In order to solve the problem of speech recognition in such extreme environments, many researches have been made in the prior art, such as noise reduction, bone conduction, etc., but the prior arts are still not suitable for complex and variable real-world environments.
Disclosure of Invention
The invention aims to provide an auxiliary rescue communication method and device based on the unvoiced instruction recognition of facial surface muscle signals, which avoid the inherent problem that the traditional voice recognition result is interfered by environmental background noise and are suitable for application scenes with high noise backgrounds or incapable of receiving sound.
The purpose of the invention is realized by the following technical scheme:
an assisted rescue communication method based on unvoiced instruction recognition of facial surface muscle signals, the method comprising:
step 1, firstly, acquiring muscle electric signals of corresponding positions in real time through a measuring electrode which is attached to the facial skin of a user in advance;
step 2, carrying out preprocessing, feature extraction and classification identification operations on the collected muscle electric signals by a data processing device to identify corresponding instruction words;
step 3, the recognized instruction words are converted into artificial voices by the language processing device;
and 4, sending the artificial voice to an earphone of an operator through a radio frequency device, and sending the voice text to a command center in a wireless mode.
The invention also provides an auxiliary rescue communication device based on the non-sounding instruction identification of the facial surface muscle signal, which comprises a collecting device, a data processing device, a language processing device and a radio frequency device, wherein:
the acquisition device consists of five channels of measuring electrodes, and the measuring electrodes are attached to the skin of the face of a user and acquire muscle electric signals at corresponding positions in real time;
the acquisition device is in wired connection with the data processing device, and the data processing device receives the muscle electric signals transmitted by the acquisition device, performs preprocessing, feature extraction and classification identification operations on the muscle electric signals and identifies corresponding instruction words;
the language processing device is electrically connected with the data processing device and is used for converting the recognized instruction words into artificial voice;
the language processing device is in wired connection with the radio frequency device, the radio frequency device is used for receiving and sending an identification result, sending the artificial voice to an earphone of an operator through the radio frequency device, and sending a voice text to a command center in a wireless mode.
According to the technical scheme provided by the invention, the method and the device avoid the inherent problem that the traditional voice recognition result is interfered by the environmental background noise, and are suitable for the application scene with high noise background or incapable of receiving sound.
Drawings
In order to more clearly illustrate the technical solutions of the embodiments of the present invention, the drawings needed to be used in the description of the embodiments are briefly introduced below, and it is obvious that the drawings in the following description are only some embodiments of the present invention, and it is obvious for those skilled in the art to obtain other drawings based on the drawings without creative efforts.
Fig. 1 is a schematic flow chart of an auxiliary rescue communication method based on unvoiced instruction recognition of facial surface muscle signals according to an embodiment of the present invention;
FIG. 2 is a schematic view of a skin-contacting position of a measuring electrode according to an embodiment of the present invention;
fig. 3 is a schematic structural diagram of an apparatus according to an embodiment of the present invention.
Detailed Description
The technical solutions in the embodiments of the present invention are clearly and completely described below with reference to the drawings in the embodiments of the present invention, and it is obvious that the described embodiments are only a part of the embodiments of the present invention, and not all embodiments. All other embodiments, which can be derived by a person skilled in the art from the embodiments of the present invention without making any creative effort, shall fall within the protection scope of the present invention.
The embodiment of the present invention will be further described in detail with reference to the accompanying drawings, and as shown in fig. 1, a schematic flow chart of an auxiliary rescue communication method based on unvoiced instruction recognition of facial surface muscle signals provided by the embodiment of the present invention is shown, where the method includes:
step 1, firstly, acquiring muscle electric signals of corresponding positions in real time through a measuring electrode which is attached to the facial skin of a user in advance;
in this step, as shown in fig. 2, a schematic view of a skin-contacting position of the measuring electrode according to an embodiment of the present invention is shown, where the measuring electrode includes five passages, and the measuring electrodes of the five passages are respectively skin-contacted on the set positions of the facial skin of the user.
Step 2, carrying out preprocessing, feature extraction and classification identification operations on the collected muscle electric signals by a data processing device to identify corresponding instruction words;
in this step, the recognized command words need to be preset according to the use environment and requirements, the command words can be single words or two-word phrases, such as forward, backward, fire extinguishing, danger, aggregation, etc., before the device is used, the signal is collected and processed to generate a command sample set, and then the command sample set is stored in the data processing device for realizing the classification recognition operation.
In a specific implementation, the specific processing procedure of the data processing apparatus is as follows:
1) firstly, filtering and denoising collected muscle electric signals;
because the collected signal is interfered by other physiological electric signals or electronic equipment, the signal needs to be preprocessed, and the embodiment of the invention adopts a high-channel filtering of 20HZ and a comb notch filter (notch filter) of 50HZ to eliminate noise.
2) Then, judging the muscle activity state based on the preprocessed muscle electric signals, and segmenting effective signals generated during muscle activity, namely the starting point and the ending point of the muscle activity state, wherein the specific process is as follows:
firstly, carrying out sequential windowing on the muscle electric signals after pretreatment, wherein the window length is 200ms, and the standard deviation of a calculation window is compared with a threshold value, and the calculation formula of the threshold value is as follows:
Th＝mean(rest)+μ*std(rest)
wherein Th is a threshold value; rest is the signal within the first 100ms of a window signal; mean is the signal expectation; std is the standard deviation of the signal; mu is a sensitivity value, and the result is optimal when the mu value is 3 through test tests;
corresponding signal data are obtained after detecting that muscles are in an active state, and the lengths of extracted signals are different due to different speaking speeds of users, wherein the length of the signals is between 200ms and 400ms in general;
the signal is further extended to 400ms by cubic interpolation as a valid signal.
3) And then carrying out feature extraction and optimization on the effective signals in the muscle activity state, wherein the specific process is as follows:
the characteristic adopted by the characteristic extraction is a time characteristic, firstly effective signals are framed, the frame length is 30ms, the frame shift is 15 ms:
for each moving window, extracting four characteristic values, which are respectively:
wherein, N represents the number of numerical values contained in a moving window; xi is the ith numerical value of the current window; the characteristic dimensionality obtained after windowing characteristic extraction is carried out on a 5-channel effective signal is 520 dimensions;
here, considering that the activity of one muscle cluster affects the coarse activity of surrounding muscle clusters, the correlation exists in nature in the characteristics, the algorithm efficiency can be improved from a small amount of effective characteristics, then the characteristic dimension is optimized from 520 dimensions to 50 dimensions by using a linear discriminant analysis method (LDA), and information of high latitude is ensured by using a small amount of characteristic dimension, so that a classification result with similar or even same effect is obtained.
4) Then, classifying and identifying the effective signals extracted by the features by adopting a consistency prediction algorithm based on random forests, and identifying corresponding instruction words, wherein the specific process comprises the following steps:
first, a random forest based definition AnA closeness value (RandomForest proximity) P (i, j) between two samples is calculated in a random forest, i, j being 1Similarity between the samples can be defined by using the closeness value, a singular measurement function A of a consistency prediction algorithm is defined and used for calculating a singular value (Strengenses Score) α of a sample, and an ith sample ziThe singular values of (a) can be calculated as follows:
αi＝An({z1，...，zi-1，zi+1，...，zn}，zi)
wherein the data in { } is unordered, zi＝(xi，yi)，xiFeatures representing an ith sample in the sample set; y isiA label representing the specimen;
the singular metric function a based on random forests is defined as follows:
for sample (x)i，yi) Comprises the following steps:
A(xi，yi)＝A(xi，yi)-/A(xi，yi)+
wherein the content of the first and second substances,
tsfinger sample (x)i，yi) (ii) an s-th greater affinity value than samples in the subsequence of samples having the same tag as the sequence; j is a function ofsFinger sample (x)i，yi) (ii) an s-th greater affinity value than samples in the subsequence of samples with different tags to which it is tagged;
then, classifying and identifying the effective signals extracted by the features based on a consistency prediction algorithm of the random forest, wherein the specific process is as follows:
for a valid signal xnFor which a hypothetical instruction tag y is setnY, with a pre-collected and processed instruction sample set Zt＝{(x1，y1)，...，(xn-1，yn-1) Generating a new sample sequence Z ═ Zt，(xn，y)}；
Applying A based on random forest definition to new sample sequence Zn，AnFeeding sampleEach sample in the sequence is assigned a singular value to form a singular value sequence
By combining singular values in a sequenceWith others
Comparing to obtain effective signal xnCurrent hypothesis of confidence level of instruction tag y
P is aboveyIs shown when xnWhen the label of (a) is y, how consistent the sample is with the rest of the sample in the sample sequence, pyThe greater the degree of consistency, the better;
then is the valid signal xnResetting a new hypothetical instruction tag, repeating the above steps until all instructions are treated as hypothetical instruction tags, where pyThe hypothetical instruction tag y with the largest value is considered as the recognition result.
The above method is to perform analysis by acquiring surface electromyogram (sEMG) through electrodes attached to facial muscles, which may also be called merry recognition. The surface electromyogram is to convert the weak potential difference generated by muscle fiber contraction into digital signal to reflect the state of the neuromuscular, and the potential difference is changed due to the change of muscle structure and function, and the electromyogram signal generated by different muscle fiber contraction is also changed correspondingly. The speech sound is generated by the complex cooperation of a series of facial and other parts of muscle clusters, and the corresponding muscle clusters used for different pronunciations or characters are different, so that the identification of instruction words can be realized through the skin electromyographic signals, and the generation of the muscle electrical signals is related to the contraction of the muscle clusters and is not related to the real pronunciations, so that the instruction words can be in the form of small sound or uninteresting reading and non-sounding, thereby solving the problem of distorted sound caused by background noise or breathing equipment.
Step 3, the recognized instruction words are converted into artificial voices by the language processing device;
and 4, sending the artificial voice to an earphone of an operator through a radio frequency device, and sending the voice text to a command center in a wireless mode.
Based on the above method, an embodiment of the present invention further provides an auxiliary rescue communication device based on the non-vocal instruction recognition of the facial surface muscle signal, as shown in fig. 3, which is a schematic structural diagram of the device according to the embodiment of the present invention, and the device mainly includes an acquisition device, a data processing device, a language processing device, and a radio frequency device, wherein:
the acquisition device consists of five channels of measuring electrodes, and the measuring electrodes are attached to the skin of the face of a user and acquire muscle electric signals at corresponding positions in real time;
the acquisition device is in wired connection with the data processing device, and the data processing device receives the muscle electric signals transmitted by the acquisition device, performs preprocessing, feature extraction and classification identification operations on the muscle electric signals and identifies corresponding instruction words;
the language processing device is electrically connected with the data processing device and is used for converting the recognized instruction words into artificial voice;
the language processing device is in wired connection with the radio frequency device, the radio frequency device is used for receiving and sending an identification result, sending the artificial voice to an earphone of an operator through the radio frequency device, and sending a voice text to a command center in a wireless mode.
The specific implementation of each component in the above-described apparatus is described in the above-described method embodiment.
In a specific implementation, the device can be integrated into a neck hanging type independent device; or the device is combined with a breathing mask, and specifically comprises:
the acquisition device is in wired connection with the breathing mask, and the breathing mask is in wired connection with the battery; the data processing device, the language processing device and the radio frequency device are arranged on the bearing equipment.
The working process of the auxiliary rescue communication device is as follows:
(1) the device is turned on, and connection (Bluetooth, wireless) is established;
(2) the acquisition device acquires real-time electromyographic signals, and the electromyographic signals are transmitted to the data processing device through a cable;
(3) the data processing device processes the electromyographic signals in real time and enters a monitoring state during initialization, if the awakening words are monitored to enter a recognition state, the radio frequency device sends a prompt tone to the earphone of a user to indicate that the instruction can be described, the prompt tone is also sent to the earphones of other teammates through the radio frequency device, and the prompt is that the instruction is issued;
(4) the user describes the instruction words, the description mode can be voiced, the worship mode can also be silent, and the acquisition device acquires corresponding myoelectric signals and transmits the corresponding myoelectric signals to the data processing device;
(5) the data processing device in the identification state carries out identification operation on the electromyographic signals and identifies corresponding instruction words;
(6) and the language processing device converts the recognized instruction words into artificial voice, the artificial voice is sent to earphones of other teammates through the radio frequency device, and the voice text is sent to the command center in a wireless mode.
It is noted that those skilled in the art will recognize that embodiments of the present invention are not described in detail herein.
The above description is only for the preferred embodiment of the present invention, but the scope of the present invention is not limited thereto, and any changes or substitutions that can be easily conceived by those skilled in the art within the technical scope of the present invention are included in the scope of the present invention. Therefore, the protection scope of the present invention shall be subject to the protection scope of the claims.
Claims (9)
1. An auxiliary rescue communication method based on unvoiced instruction recognition of facial surface muscle signals, the method comprising:
step 1, firstly, acquiring muscle electric signals of corresponding positions in real time through a measuring electrode which is attached to the facial skin of a user in advance;
step 2, carrying out preprocessing, feature extraction and classification identification operations on the collected muscle electric signals by a data processing device to identify corresponding instruction words;
step 3, the recognized instruction words are converted into artificial voices by the language processing device;
and 4, sending the artificial voice to an earphone of an operator through a radio frequency device, and sending the voice text to a command center in a wireless mode.
2. The method according to claim 1, wherein in step 1, the measuring electrodes comprise five paths, and the measuring electrodes of the five paths are respectively attached to the skin of the face of the user at the set positions.
3. The method according to claim 1, wherein in step 2, the identified instruction words are preset according to the use environment and the requirement, and the instruction words are single words or two-word phrases, specifically including forward, backward, fire-extinguishing, dangerous and collective phrases.
4. The method according to claim 1, wherein in step 2, the specific processing procedure of the data processing device is:
firstly, filtering and denoising collected muscle electric signals;
then, judging the muscle activity state based on the muscle electric signals after the pretreatment, and segmenting effective signals generated during the muscle activity, namely the starting point and the ending point of the muscle activity state;
then, extracting and optimizing the characteristics of the effective signals in the muscle activity state;
and then classifying and identifying the effective signals extracted by the features by adopting a consistency prediction algorithm based on a random forest, and identifying corresponding instruction words.
5. The method according to claim 4, wherein the muscle activity state is determined based on the preprocessed muscle electrical signals, and the process of segmenting the effective signals generated during the muscle activity is specifically as follows:
firstly, carrying out sequential windowing on the muscle electric signals after pretreatment, wherein the window length is 200ms, and the standard deviation of a calculation window is compared with a threshold value, and the calculation formula of the threshold value is as follows:
Th＝mean(rest)+μ*std(rest)
wherein Th is a threshold value; rest is the signal within the first 100ms of a window signal; mean is the signal expectation; std is the standard deviation of the signal; mu is a sensitivity value, and the result is optimal when the mu value is 3 through test tests;
obtaining corresponding effective signal data after detecting that the muscle is in an active state, wherein the length of the effective signal is between 200ms and 400 ms;
and further expanding the effective signal to 400ms by using a cubic interpolation method for outputting.
6. The method according to claim 4, wherein the process of feature extraction and optimization of the valid signals within the muscle activity state is specifically as follows:
the characteristic adopted by the characteristic extraction is a time characteristic, firstly, effective signals are framed, the frame length is 30ms, and the frame shift is 15 ms;
for each moving window, extracting four characteristic values, which are respectively:
wherein, N represents the number of numerical values contained in a moving window; xi is the ith numerical value of the current window; the characteristic dimensionality obtained after windowing characteristic extraction is carried out on a 5-channel effective signal is 520 dimensions;
then, a linear discriminant analysis method is used for optimizing the characteristic dimension from 520 dimensions to 50 dimensions, and a small amount of characteristic dimension is used for guaranteeing high-latitude information.
7. The method as claimed in claim 4, wherein the step of classifying and identifying the effective signals extracted by the features by using a consistency prediction algorithm based on a random forest is specifically as follows:
first, a random forest based definition AnCalculating a closeness value (RandomForest proximity) P (i, j) between two samples in a random forest, wherein the value i, j is 1, the.., n, and the closeness value represents the similarity between the two samples without considering a real label, and using the closeness value, a singular measurement function A of a consistency prediction algorithm can be defined, and the singular measurement function A is used for calculating a singular value (Strangenness Score) α of the sample, and the ith sample z is used for calculating a singular value (Strangenness Score) α of the sampleiThe singular values of (a) can be calculated as follows:
αi＝An({z1，...，zi-1，zi+1，...，zn}，zi)
wherein the data in { } is unordered, zi＝(xi，yi)，xiFeatures representing an ith sample in the sample set; y isiA label representing the specimen;
the singular metric function a based on random forests is defined as follows:
for sample (x)i，yi) Comprises the following steps:
A(xi，yi)＝A(xi，yi)-/A(xi，yi)+
wherein the content of the first and second substances,
tsfinger sample (x)i，yi) (ii) an s-th greater affinity value than samples in the subsequence of samples having the same tag as the sequence; j is a function ofsFinger sample (x)i，yi) (ii) an s-th greater affinity value than samples in the subsequence of samples with different tags to which it is tagged;
then, classifying and identifying the effective signals extracted by the features based on a consistency prediction algorithm of the random forest, wherein the specific process is as follows:
for a valid signal xnFor which a hypothetical instruction tag y is setnY, with a pre-collected and processed instruction sample set Zt＝{(x1，y1)，...，(xn-1，yn-1) Generating a new sample sequence Z ═ Zt，(xn，y)}；
Applying A based on random forest definition to new sample sequence Zn，AnAllocating a singular value to each sample in the sample sequence to form a singular value sequence
By combining singular values in a sequence
With others
Comparing to obtain effective signal xnCurrent hypothesis of confidence level of instruction tag y
P is aboveyIs shown when xnWhen the label of (a) is y, how consistent the sample is with the rest of the sample in the sample sequence, pyThe greater the degree of consistency, the better;
then is the valid signal xnResetSetting a new hypothetical instruction tag, and repeating the above steps until all instructions are assumed as hypothetical instruction tags, where pyThe hypothetical instruction tag y with the largest value is considered as the recognition result.
8. An auxiliary rescue communication device based on unvoiced instruction recognition of facial surface muscle signals, the device comprising a collection device, a data processing device, a language processing device and a radio frequency device, wherein:
the acquisition device consists of five channels of measuring electrodes, and the measuring electrodes are attached to the skin of the face of a user and acquire muscle electric signals at corresponding positions in real time;
the acquisition device is in wired connection with the data processing device, and the data processing device receives the muscle electric signals transmitted by the acquisition device, performs preprocessing, feature extraction and classification identification operations on the muscle electric signals and identifies corresponding instruction words;
the language processing device is electrically connected with the data processing device and is used for converting the recognized instruction words into artificial voice;
the language processing device is in wired connection with the radio frequency device, the radio frequency device is used for receiving and sending an identification result, sending the artificial voice to an earphone of an operator through the radio frequency device, and sending a voice text to a command center in a wireless mode.
9. The device of claim 8, wherein the device is integrally configured as a hanging neck stand-alone device;
or the device is combined with a breathing mask, and specifically comprises:
the acquisition device is in wired connection with the breathing mask, and the breathing mask is in wired connection with the battery; the data processing device, the language processing device and the radio frequency device are arranged on the bearing equipment.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201911128112.4A CN110767208A (en) | 2019-11-18 | 2019-11-18 | Auxiliary rescue communication method and device based on unvoiced instruction recognition of facial surface muscle signals |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201911128112.4A CN110767208A (en) | 2019-11-18 | 2019-11-18 | Auxiliary rescue communication method and device based on unvoiced instruction recognition of facial surface muscle signals |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN110767208A true CN110767208A (en) | 2020-02-07 |
Family
ID=69338198
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201911128112.4A Pending CN110767208A (en) | 2019-11-18 | 2019-11-18 | Auxiliary rescue communication method and device based on unvoiced instruction recognition of facial surface muscle signals |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN110767208A (en) |
Cited By (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN114822508A (en) * | 2022-04-25 | 2022-07-29 | 中国人民解放军军事科学院国防科技创新研究院 | Silence communication method and system for eliminating different speaking characteristics |
Citations (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN102169690A (en) * | 2011-04-08 | 2011-08-31 | 哈尔滨理工大学 | Voice signal recognition system and method based on surface myoelectric signal |
| CN102999154A (en) * | 2011-09-09 | 2013-03-27 | 中国科学院声学研究所 | Electromyography (EMG)-based auxiliary sound producing method and device |
| CN104517608A (en) * | 2013-09-30 | 2015-04-15 | 韦伯斯特生物官能(以色列)有限公司 | Controlling a system using voiceless alaryngeal speech |
| CN109460144A (en) * | 2018-09-18 | 2019-03-12 | 逻腾（杭州）科技有限公司 | A kind of brain-computer interface control system and method based on sounding neuropotential |
| CN113288183A (en) * | 2021-05-20 | 2021-08-24 | 中国科学技术大学 | Silent voice recognition method based on facial neck surface myoelectricity |
-
2019
- 2019-11-18 CN CN201911128112.4A patent/CN110767208A/en active Pending
Patent Citations (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN102169690A (en) * | 2011-04-08 | 2011-08-31 | 哈尔滨理工大学 | Voice signal recognition system and method based on surface myoelectric signal |
| CN102999154A (en) * | 2011-09-09 | 2013-03-27 | 中国科学院声学研究所 | Electromyography (EMG)-based auxiliary sound producing method and device |
| CN104517608A (en) * | 2013-09-30 | 2015-04-15 | 韦伯斯特生物官能(以色列)有限公司 | Controlling a system using voiceless alaryngeal speech |
| CN109460144A (en) * | 2018-09-18 | 2019-03-12 | 逻腾（杭州）科技有限公司 | A kind of brain-computer interface control system and method based on sounding neuropotential |
| CN113288183A (en) * | 2021-05-20 | 2021-08-24 | 中国科学技术大学 | Silent voice recognition method based on facial neck surface myoelectricity |
Non-Patent Citations (6)
| Title |
|---|
| BRADLEY J. BETTS，等: "Small-vocabulary speech recognition using surface electromyography", 《INTERACTING WITH COMPUTERS》 * |
| PHINYOMARK A,等: "Application of Linear Discriminant Analysis in Dimensionality Reduction for Hand Motion Classification", 《MEASUREMENT SCIENCE REVIEW》 * |
| 乔波,等: "基于安卓的无声语音识别App的设计与开发", 《电脑知识与技术》 * |
| 王华珍: "带置信度分类器的研究与应用", 《中国博士学位论文全文数据库 信息科技辑》 * |
| 王鑫,等: "带置信度的基于sEMG的无声语音识别研究", 《电脑知识与技术》 * |
| 金丹彤: "基于表面肌电信号的无声语音识别算法研究", 《中国优秀硕士学位论文全文数据库 信息科技辑》 * |
Cited By (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN114822508A (en) * | 2022-04-25 | 2022-07-29 | 中国人民解放军军事科学院国防科技创新研究院 | Silence communication method and system for eliminating different speaking characteristics |
| CN114822508B (en) * | 2022-04-25 | 2024-05-10 | 中国人民解放军军事科学院国防科技创新研究院 | Communication method and system for eliminating silence of different speaking characteristics |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| US11495053B2 (en) | Systems, methods, devices and apparatuses for detecting facial expression | |
| US20190025919A1 (en) | System, method and apparatus for detecting facial expression in an augmented reality system | |
| Li et al. | A sign-component-based framework for Chinese sign language recognition using accelerometer and sEMG data | |
| CN110415728B (en) | Method and device for recognizing emotion voice | |
| US7574357B1 (en) | Applications of sub-audible speech recognition based upon electromyographic signals | |
| US10621973B1 (en) | Sub-vocal speech recognition apparatus and method | |
| JP2003255993A (en) | System, method, and program for speech recognition, and system, method, and program for speech synthesis | |
| Patil et al. | The physiological microphone (PMIC): A competitive alternative for speaker assessment in stress detection and speaker verification | |
| CN110059575A (en) | A kind of augmentative communication system based on the identification of surface myoelectric lip reading | |
| CN108427916A (en) | A kind of monitoring system and monitoring method of mood of attending a banquet for customer service | |
| US10573335B2 (en) | Methods, systems and apparatuses for inner voice recovery from neural activation relating to sub-vocalization | |
| CN105976820A (en) | Voice emotion analysis system | |
| CN103294199A (en) | Silent information identifying system based on facial muscle sound signals | |
| EP4131256A1 (en) | Voice recognition system and method using accelerometers for sensing bone conduction | |
| CN113780150A (en) | Fatigue detection method and system based on multi-dimensional body state perception | |
| Usman et al. | Heart rate detection and classification from speech spectral features using machine learning | |
| Setiawan et al. | A framework for real time emotion recognition based on human ans using pervasive device | |
| CN115089179A (en) | Psychological emotion insights analysis method and system | |
| CN110767208A (en) | Auxiliary rescue communication method and device based on unvoiced instruction recognition of facial surface muscle signals | |
| Freitas et al. | Multimodal corpora for silent speech interaction | |
| Subbarao et al. | Emotion recognition using BiLSTM classifier | |
| Li et al. | Interpreting sign components from accelerometer and sEMG data for automatic sign language recognition | |
| Koct et al. | Speech Activity Detection from EEG using a feed-forward neural network | |
| JP4381404B2 (en) | Speech synthesis system, speech synthesis method, speech synthesis program | |
| CN110693508A (en) | Multi-channel cooperative psychophysiological active sensing method and service robot |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| RJ01 | Rejection of invention patent application after publication |
Application publication date: 20200207 |
|
| RJ01 | Rejection of invention patent application after publication |