CN113053417A - Method, system, equipment and storage medium for recognizing emotion of voice with noise - Google Patents

Method, system, equipment and storage medium for recognizing emotion of voice with noise Download PDF

Info

Publication number
CN113053417A
CN113053417A CN202110332451.5A CN202110332451A CN113053417A CN 113053417 A CN113053417 A CN 113053417A CN 202110332451 A CN202110332451 A CN 202110332451A CN 113053417 A CN113053417 A CN 113053417A
Authority
CN
China
Prior art keywords
residual
speech
iteration
voice
endpoint detection
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Granted
Application number
CN202110332451.5A
Other languages
Chinese (zh)
Other versions
CN113053417B (en
Inventor
姜晓庆
陈贞翔
杨倩
郑永强
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Shandong Sizheng Information Technology Co Ltd
University of Jinan
Original Assignee
Shandong Sizheng Information Technology Co Ltd
University of Jinan
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Shandong Sizheng Information Technology Co Ltd, University of Jinan filed Critical Shandong Sizheng Information Technology Co Ltd
Priority to CN202110332451.5A priority Critical patent/CN113053417B/en
Publication of CN113053417A publication Critical patent/CN113053417A/en
Application granted granted Critical
Publication of CN113053417B publication Critical patent/CN113053417B/en
Active legal-status Critical Current
Anticipated expiration legal-status Critical

Links

Images

Classifications

    • G—PHYSICS
    • G10—MUSICAL INSTRUMENTS; ACOUSTICS
    • G10L—SPEECH ANALYSIS TECHNIQUES OR SPEECH SYNTHESIS; SPEECH RECOGNITION; SPEECH OR VOICE PROCESSING TECHNIQUES; SPEECH OR AUDIO CODING OR DECODING
    • G10L25/00—Speech or voice analysis techniques not restricted to a single one of groups G10L15/00 - G10L21/00
    • G10L25/48—Speech or voice analysis techniques not restricted to a single one of groups G10L15/00 - G10L21/00 specially adapted for particular use
    • G10L25/51—Speech or voice analysis techniques not restricted to a single one of groups G10L15/00 - G10L21/00 specially adapted for particular use for comparison or discrimination
    • G10L25/63—Speech or voice analysis techniques not restricted to a single one of groups G10L15/00 - G10L21/00 specially adapted for particular use for comparison or discrimination for estimating an emotional state
    • G—PHYSICS
    • G10—MUSICAL INSTRUMENTS; ACOUSTICS
    • G10L—SPEECH ANALYSIS TECHNIQUES OR SPEECH SYNTHESIS; SPEECH RECOGNITION; SPEECH OR VOICE PROCESSING TECHNIQUES; SPEECH OR AUDIO CODING OR DECODING
    • G10L25/00—Speech or voice analysis techniques not restricted to a single one of groups G10L15/00 - G10L21/00
    • G10L25/27—Speech or voice analysis techniques not restricted to a single one of groups G10L15/00 - G10L21/00 characterised by the analysis technique
    • G10L25/30—Speech or voice analysis techniques not restricted to a single one of groups G10L15/00 - G10L21/00 characterised by the analysis technique using neural networks
    • G—PHYSICS
    • G10—MUSICAL INSTRUMENTS; ACOUSTICS
    • G10L—SPEECH ANALYSIS TECHNIQUES OR SPEECH SYNTHESIS; SPEECH RECOGNITION; SPEECH OR VOICE PROCESSING TECHNIQUES; SPEECH OR AUDIO CODING OR DECODING
    • G10L25/00—Speech or voice analysis techniques not restricted to a single one of groups G10L15/00 - G10L21/00
    • G10L25/78—Detection of presence or absence of voice signals
    • G10L25/87—Detection of discrete points within a voice signal

Landscapes

  • Engineering & Computer Science (AREA)
  • Health & Medical Sciences (AREA)
  • Multimedia (AREA)
  • Human Computer Interaction (AREA)
  • Acoustics & Sound (AREA)
  • Physics & Mathematics (AREA)
  • Computational Linguistics (AREA)
  • Signal Processing (AREA)
  • Audiology, Speech & Language Pathology (AREA)
  • Hospice & Palliative Care (AREA)
  • General Health & Medical Sciences (AREA)
  • Psychiatry (AREA)
  • Child & Adolescent Psychology (AREA)
  • Artificial Intelligence (AREA)
  • Evolutionary Computation (AREA)
  • Circuit For Audible Band Transducer (AREA)

Abstract

本发明公开了带噪语音情感识别方法、系统、设备及存储介质，获取待识别的带噪语音信号；对待识别的带噪语音信号进行端点检测处理；根据端点得到若干个有声音的语音片段；对有声音的语音片段进行特征提取，得到语音特征；将语音特征输入到训练后的语音情感识别模型中，输出情感类别。端点检测方法可以在样本重建过程中，计算正交匹配追踪算法算法迭代过程中预测残差与上一次迭代的信号估计值之间的条件熵，根据迭代前后的残差条件熵差值，在样本重建完成的同时直接给出重建样本的端点检测结果，充分利用样本重建过程中产生的数据，节省系统后续的分析与处理时间，且由于该端点检测方法是建立在压缩感知重建算法之上的，具有抗噪性能。

Figure 202110332451

The invention discloses a noisy speech emotion recognition method, system, equipment and storage medium, which are used to obtain a noisy speech signal to be recognized; perform endpoint detection and processing on the noisy speech signal to be recognized; and obtain a number of voice segments according to the endpoint; Feature extraction is performed on speech fragments with sound to obtain speech features; the speech features are input into the trained speech emotion recognition model to output emotion categories. The endpoint detection method can calculate the conditional entropy between the predicted residual in the iteration process of the orthogonal matching pursuit algorithm and the signal estimated value of the previous iteration, according to the residual conditional entropy difference before and after the iteration, in the sample When the reconstruction is completed, the endpoint detection result of the reconstructed sample is directly given, and the data generated in the sample reconstruction process is fully utilized, which saves the subsequent analysis and processing time of the system, and since the endpoint detection method is based on the compressed sensing reconstruction algorithm, With anti-noise performance.

Figure 202110332451

Description

Method, system, equipment and storage medium for recognizing emotion of voice with noise
Technical Field
The present application relates to the field of speech emotion recognition technology, and in particular, to a method, a system, a device, and a storage medium for recognizing speech emotion with noise.
Background
The statements in this section merely provide background information related to the present disclosure and may not constitute prior art.
The voice endpoint detection method has wide and important application in the field of voice signal processing, and has important research significance in the aspects of reducing the data volume of processing, learning the effective characteristics of voice, recognizing voice and recognizing the accuracy of voice emotion and the like.
The presence of ubiquitous noise tends to degrade the accuracy of speech endpoint detection, and current research has shown that Compressed Sensing (CS) also has superior performance in denoising speech signals. According to the CS theory, the observation value obtained after the voice signal is transformed by a proper sparse basis and an observation matrix contains all useful information in the voice signal, the observation value can be used for reconstructing the voice signal at a receiving end by adopting a certain reconstruction algorithm after being transmitted, and noise cannot be reconstructed because the sparsity cannot be realized, so that the compressed sensing can greatly reduce the voice transmission data volume and simultaneously realize the denoising processing in the reconstruction process. Meanwhile, the unvoiced segment in the voice signal has the noise-like characteristic and can be inhibited in the reconstruction process, so that the unvoiced and voiced division of the reconstructed sample is more accurate, and the extraction accuracy of the subsequent voice characteristic parameters is improved. The existing research also shows that the reconstructed voice sample under the compressed sensing theory can be effectively applied to the emotion recognition of the voice with noise.
In the conventional research, attention is focused on signal reconstruction, and research and application of neglecting parameters and data characteristics generated in the reconstruction process cause waste of data resources. For example, if the reconstructed sample needs to be subjected to the voice sample endpoint detection, the reconstructed sample needs to be analyzed by using a certain endpoint detection algorithm after being obtained, and the reconstruction cannot be realized and an endpoint detection result is given at the same time, so that the existing endpoint detection processing method undoubtedly increases the time delay of system processing. In addition, the existing endpoint detection algorithms are all based on the voice signals to process, and have high data dimensionality and low operation efficiency.
Disclosure of Invention
In order to solve the defects of the prior art, the application provides a method and a system for recognizing the speech emotion with noise;
in a first aspect, the application provides a method for recognizing speech emotion with noise;
the method for recognizing the emotion of the voice with the noise comprises the following steps:
acquiring a voice signal with noise to be identified;
carrying out end point detection processing on a voice signal with noise to be recognized; obtaining a plurality of voice segments with voice according to the end points;
carrying out feature extraction on the voice fragments with voice to obtain voice features;
and inputting the voice features into the trained voice emotion recognition model, and outputting emotion types.
In a second aspect, the present application provides a noisy speech emotion recognition system;
a noisy speech emotion recognition system comprising:
an acquisition module configured to: acquiring a voice signal with noise to be identified;
an endpoint detection module configured to: carrying out end point detection processing on a voice signal with noise to be recognized; obtaining a plurality of voice segments with voice according to the end points;
a feature extraction module configured to: carrying out feature extraction on the voice fragments with voice to obtain voice features;
an output module configured to: and inputting the voice features into the trained voice emotion recognition model, and outputting emotion types.
In a third aspect, the present application further provides an electronic device, including: one or more processors, one or more memories, and one or more computer programs; wherein a processor is connected to the memory, the one or more computer programs are stored in the memory, and when the electronic device is running, the processor executes the one or more computer programs stored in the memory, so as to make the electronic device execute the method according to the first aspect.
In a fourth aspect, the present application also provides a computer-readable storage medium for storing computer instructions which, when executed by a processor, perform the method of the first aspect.
Compared with the prior art, the beneficial effects of this application are:
(1) the invention provides a voice endpoint detection method based on residual error conditional entropy difference generated in an iteration process, and the method is effectively applied to voice recognition with noise and emotion. The endpoint detection method can calculate the conditional entropy between the predicted residual error and the last iteration signal estimation value in the iteration process of an Orthogonal Matching Pursuit (OMP) algorithm in the sample reconstruction process, directly give the endpoint detection result of the reconstructed sample while completing the sample reconstruction according to the difference value of the conditional entropy of the residual error before and after the iteration, fully utilize the data generated in the sample reconstruction process, save the subsequent analysis and processing time of the system, and have the anti-noise performance because the endpoint detection method is established on the compressed sensing reconstruction algorithm.
(2) Processing an emotional voice component in an emotional video by adopting a compressed sensing theory, completing sparse transformation of the emotional voice by using discrete cosine transformation, and providing a prediction residual conditional entropy parameter of the emotional voice compressed sensing reconstruction by using a Gaussian random matrix as an observation matrix and an Orthogonal Matching Pursuit (OMP) algorithm as a reconstruction algorithm;
(3) providing a residual conditional entropy difference analysis idea before and after OMP reconstruction iteration;
(4) according to the residual error condition entropy difference value and a threshold value, giving an endpoint detection result while finishing sample reconstruction;
(5) and realizing voice emotion recognition of the voice test sample with the noise emotion based on the end point detection result.
(6) The voice signal endpoint detection method adopting the residual error conditional entropy difference is based on a compressed sensing theory, endpoint detection is completed during sample reconstruction, and because noise has no sparsity and cannot be reconstructed, the voice endpoint detection result obtained by the method has anti-noise performance;
(7) when the voice is reconstructed, the voice signal endpoint detection method adopting the residual conditional entropy difference value obtains the judgment result whether the voice frame is a voiced segment according to the calculated residual conditional entropy difference value, does not need to process the reconstructed voice sample, has small time delay and can realize quick judgment;
(8) the voice signal endpoint detection method adopting the residual conditional entropy difference value deeply and effectively excavates the data characteristics in the reconstruction process through the calculation of the information theory parameters, fully utilizes the data in the sample reconstruction process and saves the calculation resources;
(9) the voice signal endpoint detection method adopting the residual error conditional entropy difference can be effectively applied to noisy voice emotion recognition.
Advantages of additional aspects of the invention will be set forth in part in the description which follows, and in part will be obvious from the description, or may be learned by practice of the invention.
Drawings
The accompanying drawings, which are incorporated in and constitute a part of this application, illustrate embodiments of the application and, together with the description, serve to explain the application and are not intended to limit the application.
FIG. 1 is a flow chart of a method of the first embodiment;
FIG. 2(a) is a schematic time domain waveform of the first embodiment;
FIG. 2(b) is a schematic time-domain waveform of a noisy speech according to the first embodiment;
FIG. 2(c) is the residual conditional entropy difference and the threshold for endpoint detection of the first embodiment;
fig. 3 is a flowchart of endpoint detection according to the first embodiment.
Detailed Description
It should be noted that the following detailed description is exemplary and is intended to provide further explanation of the disclosure. Unless defined otherwise, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this application belongs.
It is noted that the terminology used herein is for the purpose of describing particular embodiments only and is not intended to be limiting of example embodiments according to the present application. As used herein, the singular forms "a", "an", and "the" are intended to include the plural forms as well, unless the context clearly indicates otherwise, and it should be understood that the terms "comprises" and "comprising", and any variation thereof, are intended to cover a non-exclusive inclusion, such that a process, method, system, article, or apparatus that comprises a list of steps or elements is not necessarily limited to those steps or elements expressly listed, but may include other steps or elements not expressly listed or inherent to such process, method, article, or apparatus.
The embodiments and features of the embodiments of the present invention may be combined with each other without conflict.
Example one
The embodiment provides a method for recognizing noisy speech emotion;
as shown in fig. 1, the method for recognizing emotion of noisy speech includes:
s100: acquiring a voice signal with noise to be identified;
s200: carrying out end point detection processing on a voice signal with noise to be recognized; obtaining a plurality of voice segments with voice according to the end points;
s300: carrying out feature extraction on the voice fragments with voice to obtain voice features;
s400: and inputting the voice features into the trained voice emotion recognition model, and outputting emotion types.
As one or more embodiments, the S200: carrying out end point detection processing on a voice signal with noise to be recognized; obtaining a plurality of voice segments with voice according to the end points; the method specifically comprises the following steps:
s201: carrying out sparse conversion processing on a voice signal with noise to be recognized;
s202: randomly generating a Gaussian random matrix for the voice signals after the sparse conversion processing; taking the Gaussian random matrix as an observation matrix of the voice signal;
s203: and based on the observation matrix, carrying out sample reconstruction by adopting an Orthogonal Matching Pursuit (OMP) algorithm to obtain an endpoint detection result.
Further, the step S201: carrying out sparse conversion processing on a voice signal with noise to be recognized; the method specifically comprises the following steps:
and performing sparse conversion processing on the voice signal with the noise to be recognized by adopting discrete cosine transform.
Further, the step S202: randomly generating a Gaussian random matrix for the voice signals after the sparse conversion processing; wherein, the Gaussian random matrix follows normal distribution with the mean value of 0, the variance of 1 and the standard deviation of 1.
As one or more embodiments, as shown in fig. 3, the S203: based on the observation matrix, adopting an orthogonal matching pursuit algorithm (OMP) to reconstruct a sample to obtain an endpoint detection result; the method specifically comprises the following steps:
s2031: obtaining a voice observation value of each frame according to the observation matrix;
s2032: when the sensor runs for the first time, setting the residual error as a voice observation value, and calculating a correlation coefficient of the residual error and the sensing matrix;
when the sensor is not operated for the first time, calculating a residual error between the last iteration estimation value and the voice observation value and a correlation coefficient between the residual error and the sensing matrix;
s2033: searching atoms with the maximum correlation coefficient in the sensing matrix, and updating a support set reconstructed by the signals by using the atoms with the maximum correlation coefficient;
s2034: based on the support set, approximating the observed value by using a least square method to obtain an estimated value of the signal;
s2035: updating the residual error, and calculating the conditional entropy of the residual error;
s2036: judging whether the sparsity condition is reached, if so, returning to S2032; if not, calculating a residual conditional entropy difference value between the first iteration and the last iteration, and obtaining a reconstructed sample according to an estimated value of a signal at the moment;
s2037: judging whether the difference value of the residual conditional entropy of the first iteration and the last iteration is higher than a set threshold value, and if so, considering the current frame speech as a voiced segment; if the current frame voice is lower than the set threshold, the current frame voice is considered to be a silent section, and an endpoint detection result of the current frame voice is obtained;
s2038: based on the endpoint detection results, voiced speech segments in the reconstructed samples are obtained.
Further, the S2031: obtaining a voice observation value of each frame according to the observation matrix; the method specifically comprises the following steps:
if a frame of voice signal is x, completing sparse conversion through discrete cosine transform, wherein the signal is a discrete cosine coefficient alpha, namely x ═ Ψ alpha, and Ψ is a sparse matrix formed by DCT bases; then the observation is y ═ θ α, where Θ ═ Φ Ψ, and Φ is the observation matrix.
Further, the S2032: calculating a residual error between the last iteration estimation value and the voice observation value and a correlation coefficient between the residual error and the sensing matrix; the method specifically comprises the following steps:
reconstructed residual r obtained from the t-th iterationtThe calculation formula of (2) is as follows:
Figure BDA0002996707110000071
wherein A istIs a support set formed by atoms of a sensing matrix in the t iteration process of the OMP algorithm,
Figure BDA0002996707110000072
and y is an observed value which is an estimated value calculated by a least square method in the t-th iteration process.
Further, the correlation coefficient of the residual error and the sensing matrix is calculated by using the inner product of the residual error and the column vector of the sensing matrix.
It should be understood that the sensing matrix is obtained by multiplying a sparsity matrix of sparse transformation and an observation matrix, and can ensure that signals can be sampled and compressed simultaneously.
Further, the S2033: searching atoms with the maximum correlation coefficient in the sensing matrix, and updating a support set reconstructed by the signals by using the atoms with the maximum correlation coefficient; the support set is a set of columns found from the sensing matrix according to the correlation coefficient.
Further, the S2035: updating the residual error, and calculating the conditional entropy of the residual error; the method specifically comprises the following steps:
storing the residual error obtained by each iteration and updating the residual error;
based on the updated residual, a residual conditional entropy is calculated.
Further, calculating a residual conditional entropy based on the updated residual; residual conditional entropy σeThe calculation formula of (2) is as follows:
Figure BDA0002996707110000081
At-1is a support set formed by atoms of a sensing matrix in the t-1 iteration process of the OMP algorithm,
Figure BDA0002996707110000082
is an estimated value calculated by a least square method in the process of t-1 times of iteration.
Further, the S2036: judging whether the sparsity condition is reached, if so, returning to S2032; if not, calculating a residual conditional entropy difference value between the first iteration and the last iteration; the method specifically comprises the following steps:
and subtracting the residual conditional entropy obtained by the first iteration from the residual conditional entropy obtained by the last iteration to obtain a difference value.
Further, the sparsity condition refers to that whether iteration is terminated or not is judged by judging the number of iterations and the sparsity K after each iteration is completed in the sample reconstruction process. If the iteration number is less than K, continuing the iteration, otherwise, terminating the iteration.
Further, S300: extracting the characteristics of each voice segment with voice to obtain voice characteristics; the specific voice features include: prosodic features (e.g., fundamental frequency, short-term energy, time-dependent features such as sample duration, voiced segment duration, speech rate, etc.), psychoacoustic features (e.g., first, second, third formants, etc.), spectral features (e.g., MFCC parameters), and statistical parameters (maximum, minimum, mean) of the above features, etc.
Further, the step S400: inputting the voice features into the trained voice emotion recognition model, and outputting emotion types; the training step of the trained speech emotion recognition model comprises the following steps:
constructing a neural network model; the neural network model is a convolutional neural network;
constructing a training set, wherein the training set comprises voice features of known emotion classes;
and inputting the training set into a neural network model for training, and stopping training when the loss function reaches the minimum value or reaches the set iteration times to obtain the trained speech emotion recognition model.
The compressed sensing is applied to voice signal processing, and if discrete cosine transform is selected to complete sparse transform of voice signals, a Gaussian random matrix is adopted as an observation matrix, and an Orthogonal Matching Pursuit (OMP) algorithm is adopted as a sample reconstruction algorithm.
The invention provides a voice signal endpoint detection method adopting a residual conditional entropy difference value, which is based on a prediction residual generated in an OMP iterative execution process. The OMP algorithm is a common algorithm in speech signal reconstruction, and updates a support set of signal reconstruction by calculating a residual error between an estimated value and an observed value of each iteration and a correlation between the residual error and a sensing matrix until a sparsity condition is reached, and then completes the signal reconstruction. The calculation of the residual is an important ring in the OMP algorithm, and from the information theory perspective, the acquisition of the voice information in the iterative process means the reduction of the residual entropy. The invention adopts the conditional entropy sigma between the residual error of the introduced t iteration and the signal estimation value of the last iterationeTo judge the extraction degree of the voice component in the reconstructed residual.
In the OMP algorithm, the reconstructed residual r obtained in the t-th iterationtThe calculation formula of (2) is as follows:
Figure BDA0002996707110000091
wherein A istIs a support set formed by atoms of a sensing matrix in the t iteration process of the OMP algorithm,
Figure BDA0002996707110000092
is the estimated value calculated by the least square method in the t-th iteration process.
σeThe calculation formula of (2) is as follows:
Figure BDA0002996707110000101
At-1is a support set formed by atoms of a sensing matrix in the t-1 iteration process of the OMP algorithm,
Figure BDA0002996707110000102
is an estimated value calculated by a least square method in the process of t-1 times of iteration.
And when the iteration is finished, solving a residual conditional entropy difference value between the last iteration and the first iteration, and obtaining an endpoint detection result through threshold judgment.
Fig. 2(a) shows a speech time domain waveform in a process of reconstructing a certain speech sample by using an OMP algorithm, fig. 2(b) shows a time domain waveform of a noisy speech, and fig. 2(c) shows a residual conditional entropy difference value and a threshold value between a last iteration and a first iteration.
As can be seen from the figure, the sample has a strong noise level, the signal-to-noise ratio of the noisy sample is 0dB, and the voice signal is covered by noise, but according to the algorithm, the residual conditional entropy difference is more stable in a noise environment and has better robustness, and the starting point and the ending point of the noisy voice can be detected by setting a smaller threshold.
It can be seen that the difference in residual conditional entropy in the iterative process corresponds well to the active components, σ, in the speech sampleeThe variation trend of the method is corresponding to the position of a voiced segment (including unvoiced sound and voiced sound) in an original waveform, and the starting and ending point judgment of the reconstructed voice sample can be completed by adopting an empirical threshold conditionEnd point detection of noisy speech may then be achieved using a lower threshold (e.g., 0.01) as in fig. 2 (c). And the end points of the reconstructed samples can be obtained while the samples are reconstructed by the algorithm, and other end point detection algorithms do not need to be implemented on the reconstructed samples.
The overall flow chart of the noisy speech emotion recognition of the speech signal endpoint detection method using the residual conditional entropy difference is shown in fig. 1. As can be seen from fig. 1, when the noisy emotion speech is reconstructed, an endpoint detection result of the reconstructed sample can be obtained, subsequent feature extraction and feature learning can be performed according to the endpoint detection result, and an effective emotion recognition model can be trained by using a feature parameter set of the emotion speech, thereby realizing the noisy speech emotion recognition.
Example two
The embodiment provides a system for recognizing noisy speech emotion;
a noisy speech emotion recognition system comprising:
an acquisition module configured to: acquiring a voice signal with noise to be identified;
an endpoint detection module configured to: carrying out end point detection processing on a voice signal with noise to be recognized; obtaining a plurality of voice segments with voice according to the end points;
a feature extraction module configured to: carrying out feature extraction on the voice fragments with voice to obtain voice features;
an output module configured to: and inputting the voice features into the trained voice emotion recognition model, and outputting emotion types.
It should be noted here that the above-mentioned obtaining module, the endpoint detecting module, the feature extracting module and the output module correspond to steps S100 to S400 in the first embodiment, and the above-mentioned modules are the same as the examples and application scenarios realized by the corresponding steps, but are not limited to the contents disclosed in the first embodiment. It should be noted that the modules described above as part of a system may be implemented in a computer system such as a set of computer-executable instructions.
In the foregoing embodiments, the descriptions of the embodiments have different emphasis, and for parts that are not described in detail in a certain embodiment, reference may be made to related descriptions of other embodiments.
The proposed system can be implemented in other ways. For example, the above-described system embodiments are merely illustrative, and for example, the division of the above-described modules is merely a logical division, and in actual implementation, there may be other divisions, for example, multiple modules may be combined or integrated into another system, or some features may be omitted, or not executed.
EXAMPLE III
The present embodiment also provides an electronic device, including: one or more processors, one or more memories, and one or more computer programs; wherein, a processor is connected with the memory, the one or more computer programs are stored in the memory, and when the electronic device runs, the processor executes the one or more computer programs stored in the memory, so as to make the electronic device execute the method according to the first embodiment.
It should be understood that in this embodiment, the processor may be a central processing unit CPU, and the processor may also be other general purpose processors, digital signal processors DSP, application specific integrated circuits ASIC, off-the-shelf programmable gate arrays FPGA or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, and so on. A general purpose processor may be a microprocessor or the processor may be any conventional processor or the like.
The memory may include both read-only memory and random access memory, and may provide instructions and data to the processor, and a portion of the memory may also include non-volatile random access memory. For example, the memory may also store device type information.
In implementation, the steps of the above method may be performed by integrated logic circuits of hardware in a processor or instructions in the form of software.
The method in the first embodiment may be directly implemented by a hardware processor, or may be implemented by a combination of hardware and software modules in the processor. The software modules may be located in ram, flash, rom, prom, or eprom, registers, among other storage media as is well known in the art. The storage medium is located in a memory, and a processor reads information in the memory and completes the steps of the method in combination with hardware of the processor. To avoid repetition, it is not described in detail here.
Those of ordinary skill in the art will appreciate that the various illustrative elements and algorithm steps described in connection with the embodiments disclosed herein may be implemented as electronic hardware or combinations of computer software and electronic hardware. Whether such functionality is implemented as hardware or software depends upon the particular application and design constraints imposed on the implementation. Skilled artisans may implement the described functionality in varying ways for each particular application, but such implementation decisions should not be interpreted as causing a departure from the scope of the present application.
Example four
The present embodiments also provide a computer-readable storage medium for storing computer instructions, which when executed by a processor, perform the method of the first embodiment.
The above description is only a preferred embodiment of the present application and is not intended to limit the present application, and various modifications and changes may be made by those skilled in the art. Any modification, equivalent replacement, improvement and the like made within the spirit and principle of the present application shall be included in the protection scope of the present application.

Claims (10)

1.带噪语音情感识别方法，其特征是，包括：1. A noisy speech emotion recognition method, characterized in that it comprises: 获取待识别的带噪语音信号；Obtain the noisy speech signal to be recognized; 对待识别的带噪语音信号进行端点检测处理；根据端点得到若干个有声音的语音片段；Perform endpoint detection processing on the noisy speech signal to be recognized; obtain several voice segments according to the endpoint; 对有声音的语音片段进行特征提取，得到语音特征；Perform feature extraction on voice fragments with voice to obtain voice features; 将语音特征输入到训练后的语音情感识别模型中，输出情感类别。Input speech features into the trained speech emotion recognition model and output emotion categories. 2.如权利要求1所述的带噪语音情感识别方法，其特征是，对待识别的带噪语音信号进行端点检测处理；根据端点得到若干个有声音的语音片段；具体包括：2. noisy speech emotion recognition method as claimed in claim 1, is characterized in that, carries out endpoint detection processing to the noisy speech signal to be identified; Obtain several voice fragments with sound according to endpoint; Specifically comprise: 对待识别的带噪语音信号进行稀疏转换处理；Perform sparse conversion processing on the noisy speech signal to be recognized; 对稀疏转换处理后的语音信号，随机生成高斯随机矩阵；将高斯随机矩阵作为语音信号的观测矩阵；For the sparsely transformed speech signal, randomly generate a Gaussian random matrix; use the Gaussian random matrix as the observation matrix of the speech signal; 基于观测矩阵，采用正交匹配追踪算法OMP进行样本重建，得到端点检测结果。Based on the observation matrix, the orthogonal matching pursuit algorithm OMP is used to reconstruct the sample, and the endpoint detection result is obtained. 3.如权利要求2所述的带噪语音情感识别方法，其特征是，对待识别的带噪语音信号进行稀疏转换处理；具体包括：3. The noisy speech emotion recognition method as claimed in claim 2, is characterized in that, carries out sparse conversion processing to the noisy speech signal to be recognized; Specifically comprises: 采用离散余弦变换，对待识别的带噪语音信号进行稀疏转换处理。Using discrete cosine transform, the noisy speech signal to be recognized is sparsely transformed. 4.如权利要求2所述的带噪语音情感识别方法，其特征是，基于观测矩阵，采用正交匹配追踪算法OMP进行样本重建，得到端点检测结果；具体包括：4. the noisy speech emotion recognition method as claimed in claim 2, is characterized in that, based on observation matrix, adopts orthogonal matching pursuit algorithm OMP to carry out sample reconstruction, obtains endpoint detection result; Concrete comprises: (1)：根据观测矩阵，得到每一帧的语音观测值；(1): According to the observation matrix, the speech observation value of each frame is obtained; (2)：首次运行时，残差设定为语音观测值，计算残差与传感矩阵的相关系数；(2): When running for the first time, the residual is set as the speech observation value, and the correlation coefficient between the residual and the sensing matrix is calculated; 非首次运行时，计算上一次迭代估计值与语音观测值的残差，和残差与传感矩阵的相关系数；When not running for the first time, calculate the residual between the estimated value of the last iteration and the speech observation, and the correlation coefficient between the residual and the sensing matrix; (3)：在传感矩阵中，寻找相关系数最大原子，利用相关系数最大原子更新信号重建的支撑集；(3): In the sensing matrix, find the atom with the largest correlation coefficient, and use the atom with the largest correlation coefficient to update the support set of the signal reconstruction; (4)：基于支撑集，使用最小二乘法对观测值进行逼近，得到的信号的估计值；(4): Based on the support set, use the least squares method to approximate the observed value, and obtain the estimated value of the signal; (5)：更新残差，计算残差条件熵；(5): Update the residual and calculate the residual conditional entropy; (6)：判断是否达到稀疏度条件，如果是，就返回(2)；如果否，就计算第一次迭代与最后一次迭代的残差条件熵差值，且此时的信号的估计值认为是重建样本；(6): Determine whether the sparsity condition is reached, if so, return to (2); if not, calculate the residual conditional entropy difference between the first iteration and the last iteration, and the estimated value of the signal at this time is considered to be is the reconstruction sample; (7)：判断第一次迭代与最后一次迭代的残差条件熵差值是否高于设定阈值，如果高于设定阈值，则认为当前帧语音为有声段；如果低于设定阈值，则认为当前帧语音为无声段，获得当前帧语音的端点检测结果；(7): Determine whether the residual conditional entropy difference between the first iteration and the last iteration is higher than the set threshold. If it is higher than the set threshold, it is considered that the current frame of speech is a voiced segment; Then it is considered that the current frame of speech is a silent segment, and the endpoint detection result of the current frame of speech is obtained; (8)：基于端点检测结果，获得重建样本中的有声音的语音片段。(8): Based on the endpoint detection result, obtain a voiced speech segment in the reconstructed sample. 5.如权利要求4所述的带噪语音情感识别方法，其特征是，计算上一次迭代估计值与语音观测值的残差，和残差与传感矩阵的相关系数；具体包括：5. The noisy speech emotion recognition method as claimed in claim 4, is characterized in that, calculates the residual of last iteration estimated value and speech observation value, and the correlation coefficient of residual and sensing matrix; Specifically comprises: 第t次迭代所获得的重建残差rt的计算公式为：The calculation formula of the reconstruction residual r t obtained by the t-th iteration is:
Figure FDA0002996707100000021
Figure FDA0002996707100000021
 其中，At是OMP算法第t次迭代过程中由传感矩阵的原子构成的支撑集，
Figure FDA0002996707100000022
为第t次迭代过程中最小二乘法计算的估计值，y是观测值。where A t is the support set composed of atoms of the sensing matrix during the t-th iteration of the OMP algorithm,
Figure FDA0002996707100000022
is the estimated value calculated by the least squares method during the t-th iteration, and y is the observed value. 6.如权利要求4所述的带噪语音情感识别方法，其特征是，更新残差，计算残差条件熵；具体包括：6. The noisy speech emotion recognition method as claimed in claim 4, is characterized in that, update residual, calculate residual conditional entropy; Specifically comprise: 对每次迭代得到的残差进行存储，更新残差；Store the residual obtained at each iteration and update the residual; 基于更新后的残差，计算残差条件熵；Based on the updated residuals, the residual conditional entropy is calculated; 所述基于更新后的残差，计算残差条件熵；残差条件熵σe的计算公式为：The residual conditional entropy is calculated based on the updated residual; the calculation formula of the residual conditional entropy σ e is:
Figure FDA0002996707100000031
Figure FDA0002996707100000031
 At-1是OMP算法第t-1次迭代过程中由传感矩阵的原子构成的支撑集，
Figure FDA0002996707100000032
为t-1次迭代过程中最小二乘法计算的估计值。A t-1 is the support set composed of the atoms of the sensing matrix during the t-1 iteration of the OMP algorithm,
Figure FDA0002996707100000032
Estimates computed for the least squares method during t-1 iterations. 7.如权利要求4所述的带噪语音情感识别方法，其特征是，稀疏度条件指的是在样本重建过程中每次迭代完之后要判断迭代次数与稀疏度K的大小来判断是否终止迭代；如果迭代次数小于K要继续迭代，否则终止迭代。7. The noisy speech emotion recognition method as claimed in claim 4, wherein the sparsity condition refers to that after each iteration in the sample reconstruction process, the number of iterations and the size of the sparsity K are to be judged to judge whether to terminate. Iteration; if the number of iterations is less than K, continue the iteration, otherwise terminate the iteration. 8.带噪语音情感识别系统，其特征是，包括：8. A noisy speech emotion recognition system, characterized by comprising: 获取模块，其被配置为：获取待识别的带噪语音信号；an acquisition module, which is configured to: acquire a noisy speech signal to be recognized; 端点检测模块，其被配置为：对待识别的带噪语音信号进行端点检测处理；根据端点得到若干个有声音的语音片段；an endpoint detection module, which is configured to: perform endpoint detection processing on the noisy speech signal to be identified; obtain several voiced speech segments according to the endpoint; 特征提取模块，其被配置为：对有声音的语音片段进行特征提取，得到语音特征；a feature extraction module, which is configured to: perform feature extraction on the voice segment with sound to obtain the voice feature; 输出模块，其被配置为：将语音特征输入到训练后的语音情感识别模型中，输出情感类别。The output module is configured to: input the speech feature into the trained speech emotion recognition model, and output the emotion category. 9.一种电子设备，其特征是，包括：一个或多个处理器、一个或多个存储器、以及一个或多个计算机程序；其中，处理器与存储器连接，上述一个或多个计算机程序被存储在存储器中，当电子设备运行时，该处理器执行该存储器存储的一个或多个计算机程序，以使电子设备执行上述权利要求1-7任一项所述的方法。9. An electronic device, characterized in that it comprises: one or more processors, one or more memories, and one or more computer programs; wherein the processor is connected to the memory, and the one or more computer programs are Stored in a memory, when the electronic device is running, the processor executes one or more computer programs stored in the memory to cause the electronic device to perform the method of any one of claims 1-7 above. 10.一种计算机可读存储介质，其特征是，用于存储计算机指令，所述计算机指令被处理器执行时，完成权利要求1-7任一项所述的方法。10. A computer-readable storage medium, characterized by being used for storing computer instructions, which, when executed by a processor, perform the method according to any one of claims 1-7.
CN202110332451.5A 2021-03-29 2021-03-29 Noisy speech emotion recognition method, system, device and storage medium Active CN113053417B (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN202110332451.5A CN113053417B (en) 2021-03-29 2021-03-29 Noisy speech emotion recognition method, system, device and storage medium

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN202110332451.5A CN113053417B (en) 2021-03-29 2021-03-29 Noisy speech emotion recognition method, system, device and storage medium

Publications (2)

Publication Number Publication Date
CN113053417A true CN113053417A (en) 2021-06-29
CN113053417B CN113053417B (en) 2022-04-19

Family

ID=76516320

Family Applications (1)

Application Number Title Priority Date Filing Date
CN202110332451.5A Active CN113053417B (en) 2021-03-29 2021-03-29 Noisy speech emotion recognition method, system, device and storage medium

Country Status (1)

Country Link
CN (1) CN113053417B (en)

Citations (6)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN103345923A (en) * 2013-07-26 2013-10-09 电子科技大学 Sparse representation based short-voice speaker recognition method
CN103474066A (en) * 2013-10-11 2013-12-25 福州大学 Ecological voice recognition method based on multiband signal reconstruction
CN107293302A (en) * 2017-06-27 2017-10-24 苏州大学 A kind of sparse spectrum signature extracting method being used in voice lie detection system
CN107657964A (en) * 2017-08-15 2018-02-02 西北大学 Depression aided detection method and grader based on acoustic feature and sparse mathematics
CN109243493A (en) * 2018-10-30 2019-01-18 南京工程学院 Based on the vagitus emotion identification method for improving long memory network in short-term
CN111081280A (en) * 2019-12-30 2020-04-28 苏州思必驰信息科技有限公司 Text-independent speech emotion recognition method and device and emotion recognition algorithm model generation method

Patent Citations (6)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN103345923A (en) * 2013-07-26 2013-10-09 电子科技大学 Sparse representation based short-voice speaker recognition method
CN103474066A (en) * 2013-10-11 2013-12-25 福州大学 Ecological voice recognition method based on multiband signal reconstruction
CN107293302A (en) * 2017-06-27 2017-10-24 苏州大学 A kind of sparse spectrum signature extracting method being used in voice lie detection system
CN107657964A (en) * 2017-08-15 2018-02-02 西北大学 Depression aided detection method and grader based on acoustic feature and sparse mathematics
CN109243493A (en) * 2018-10-30 2019-01-18 南京工程学院 Based on the vagitus emotion identification method for improving long memory network in short-term
CN111081280A (en) * 2019-12-30 2020-04-28 苏州思必驰信息科技有限公司 Text-independent speech emotion recognition method and device and emotion recognition algorithm model generation method

Non-Patent Citations (3)

* Cited by examiner, † Cited by third party
Title
ANURAG SINGH ET AL.: "Compressed sensing framework of data reduction at multiscale level for eigenspace multichannel ECG signals", 《2015 TWENTY FIRST NATIONAL CONFERENCE ON COMMUNICATIONS (NCC)》 *
P N JAYANTHI ET AL.: "Sparse channel estimation for MIMO-OFDM systems using compressed sensing", 《2016 IEEE INTERNATIONAL CONFERENCE ON RECENT TRENDS IN ELECTRONICS, INFORMATION & COMMUNICATION TECHNOLOGY (RTEICT)》 *
PARVIN AHMADI ET AL.: "A new method for voice activity detection based on sparse representation", 《2014 7TH INTERNATIONAL CONGRESS ON IMAGE AND SIGNAL PROCESSING》 *

Also Published As

Publication number Publication date
CN113053417B (en) 2022-04-19

Similar Documents

Publication Publication Date Title
Giraldo et al. Vocell: A 65-nm speech-triggered wake-up SoC for 10-$\mu $ W keyword spotting and speaker verification
CN108597496B (en) Voice generation method and device based on generation type countermeasure network
CN114038447B (en) Speech synthesis model training method, speech synthesis method, device and medium
CN111785288B (en) Voice enhancement method, device, equipment and storage medium
CN113643693A (en) Acoustic model conditioned on sound features
CN110648655B (en) Voice recognition method, device, system and storage medium
Bryan et al. Autoregressive hidden Markov model and the speech signal
JP2020086434A (en) Denoising variational auto-encoder based integrated training method and apparatus for speech detection
CN112017648A (en) Weighted finite state converter construction method, speech recognition method and device
JP2010078650A (en) Speech recognizer and method thereof
Helali et al. Real time speech recognition based on PWP thresholding and MFCC using SVM
JP4586577B2 (en) Disturbance component suppression device, computer program, and speech recognition system
CN113053356B (en) Voice waveform generation method, device, server and storage medium
Sharma et al. Automatic speech recognition systems: challenges and recent implementation trends
CN105529034A (en) Speech recognition method and device based on reverberation
JP4512848B2 (en) Noise suppressor and speech recognition system
CN113053417B (en) Noisy speech emotion recognition method, system, device and storage medium
Mendiratta et al. Automatic speech recognition using optimal selection of features based on hybrid ABC-PSO
CN110992977B (en) Method and device for extracting target sound source
JP4705414B2 (en) Speech recognition apparatus, speech recognition method, speech recognition program, and recording medium
CN113065449B (en) Face image acquisition method and device, computer equipment and storage medium
Nicolson et al. Sum-product networks for robust automatic speaker identification
CN116758930A (en) Voice enhancement method, device, electronic equipment and storage medium
CN114387949A (en) Confidence estimation method and device for voice recognition and computer equipment
CN114913860A (en) Voiceprint recognition method, voiceprint recognition device, computer equipment, storage medium and program product

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination
GR01 Patent grant
GR01 Patent grant