CN112259113A - Preprocessing system for improving accuracy rate of speech recognition in vehicle and control method thereof - Google Patents
Preprocessing system for improving accuracy rate of speech recognition in vehicle and control method thereof Download PDFInfo
- Publication number
- CN112259113A CN112259113A CN202011060176.8A CN202011060176A CN112259113A CN 112259113 A CN112259113 A CN 112259113A CN 202011060176 A CN202011060176 A CN 202011060176A CN 112259113 A CN112259113 A CN 112259113A
- Authority
- CN
- China
- Prior art keywords
- vehicle
- signal
- neural network
- noise
- model
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
- 238000007781 pre-processing Methods 0.000 title claims abstract description 63
- 238000000034 method Methods 0.000 title claims abstract description 29
- 230000009467 reduction Effects 0.000 claims abstract description 42
- 238000013527 convolutional neural network Methods 0.000 claims abstract description 40
- 230000005236 sound signal Effects 0.000 claims abstract description 33
- 238000003062 neural network model Methods 0.000 claims abstract description 23
- 125000004122 cyclic group Chemical group 0.000 claims abstract description 11
- 238000013507 mapping Methods 0.000 claims abstract description 7
- 230000008054 signal transmission Effects 0.000 claims description 21
- 238000011176 pooling Methods 0.000 claims description 20
- 238000012545 processing Methods 0.000 claims description 14
- 230000000306 recurrent effect Effects 0.000 claims description 13
- 239000011159 matrix material Substances 0.000 claims description 9
- 230000004807 localization Effects 0.000 claims description 7
- 238000012546 transfer Methods 0.000 claims description 3
- 230000005540 biological transmission Effects 0.000 abstract description 5
- 238000013528 artificial neural network Methods 0.000 description 9
- 238000010586 diagram Methods 0.000 description 7
- 230000000694 effects Effects 0.000 description 5
- 230000006870 function Effects 0.000 description 5
- 230000003993 interaction Effects 0.000 description 3
- 210000002569 neuron Anatomy 0.000 description 3
- 230000008569 process Effects 0.000 description 3
- 238000012549 training Methods 0.000 description 3
- 238000013461 design Methods 0.000 description 2
- 238000011946 reduction process Methods 0.000 description 2
- 108010076504 Protein Sorting Signals Proteins 0.000 description 1
- 239000000654 additive Substances 0.000 description 1
- 230000000996 additive effect Effects 0.000 description 1
- 230000009286 beneficial effect Effects 0.000 description 1
- 230000007547 defect Effects 0.000 description 1
- 238000001514 detection method Methods 0.000 description 1
- 239000000284 extract Substances 0.000 description 1
- 238000000605 extraction Methods 0.000 description 1
- 238000012986 modification Methods 0.000 description 1
- 230000004048 modification Effects 0.000 description 1
- 230000001537 neural effect Effects 0.000 description 1
- 238000013179 statistical model Methods 0.000 description 1
- 238000012360 testing method Methods 0.000 description 1
Images
Classifications
-
- G—PHYSICS
- G10—MUSICAL INSTRUMENTS; ACOUSTICS
- G10L—SPEECH ANALYSIS OR SYNTHESIS; SPEECH RECOGNITION; SPEECH OR VOICE PROCESSING; SPEECH OR AUDIO CODING OR DECODING
- G10L21/00—Processing of the speech or voice signal to produce another audible or non-audible signal, e.g. visual or tactile, in order to modify its quality or its intelligibility
- G10L21/02—Speech enhancement, e.g. noise reduction or echo cancellation
- G10L21/0208—Noise filtering
-
- G—PHYSICS
- G10—MUSICAL INSTRUMENTS; ACOUSTICS
- G10L—SPEECH ANALYSIS OR SYNTHESIS; SPEECH RECOGNITION; SPEECH OR VOICE PROCESSING; SPEECH OR AUDIO CODING OR DECODING
- G10L15/00—Speech recognition
- G10L15/22—Procedures used during a speech recognition process, e.g. man-machine dialogue
Abstract
The invention discloses a preprocessing system for improving the accuracy of speech recognition in a vehicle and a control method thereof, wherein the control method comprises the following steps: the method comprises the steps that a vibration sensor collects original vibration signals at different positions outside a current car, and an in-car microphone collects sound signals inside the current car; establishing a transmission channel model through a convolutional neural network noise reduction model and a cyclic neural network model to form a mapping relation between an original vibration signal and an in-vehicle noise signal, and outputting a modeled cancellation signal; and removing the in-vehicle noise signal in the sound signal according to the modeled cancellation signal to obtain a residual signal after noise reduction, wherein the residual signal is used as an input signal of the voice recognition system. The preprocessing system provided by the invention can be used for carrying out noise reduction and enhancement on voice signals at the front end of the voice recognition system, improving the accuracy of voice recognition, and positioning the position of a passenger who sends voice in a vehicle through the arrangement of the microphone, so that the control requirements of different passengers can be met conveniently.
Description
Technical Field
The invention relates to the field of voice noise reduction, in particular to a preprocessing system for improving the accuracy rate of voice recognition in a vehicle and a control method thereof.
Background
The voice interaction is the most convenient and efficient means for controlling the vehicle-mounted system when a person drives a vehicle. The voice recognition and control in the environment inside the vehicle become a hot technical direction for the interaction of people and vehicles in the future. When a vehicle runs, large background noises in the vehicle comprise road noises, engine noises and wind noises, and the non-stable time-varying noises can seriously influence the performance of a voice recognition system and reduce the accuracy of system recognition, so that a corresponding preprocessing system is required to perform noise reduction processing on noisy voice signals. Meanwhile, the existing vehicle-mounted voice recognition system cannot perform positioning distinguishing on voice signals in the vehicle, cannot realize the targeted voice control of passengers in different riding positions, and also reduces the driving experience of the passengers.
The existing noise reduction processing system of vehicle-mounted voice can be divided into single-channel voice enhancement and multi-channel voice enhancement. The single channel enhancement is generally based on a voice activity detection or statistical model method, and the accuracy of the in-vehicle complex environment voice recognition is improved by improving the robustness of a voice recognition algorithm. However, the noise reduction and the distortion of the speech signal are caused, so that the method is usually used for processing stationary noise and has poor control effect on non-stationary noise. Multichannel speech enhancement is through arranging a plurality of microphones in the car, and it carries out speech signal and falls the noise to constitute microphone array and receive sound signal to fall the noise to obtain better recognition effect, can handle non-stationary noise to a certain extent, still exist and fall the unsatisfactory, the poor scheduling problem of noise tracking performance of noise reduction performance.
Disclosure of Invention
In order to overcome the defects in the prior art, the invention provides a pretreatment system for improving the accuracy of speech recognition in a vehicle and a control method thereof, and the technical scheme is as follows:
on one hand, the invention provides a control method of a preprocessing system for improving the accuracy of speech recognition in a vehicle, which comprises the following steps:
s1, collecting original vibration signals at different positions outside the vehicle, and collecting current sound signals inside the vehicle, wherein the sound signals inside the vehicle comprise noise signals inside the vehicle and voice signals inside the vehicle;
s2, performing real-time feature learning on the original vibration signal through a convolutional neural network noise reduction model to obtain a corresponding feature vector and outputting the feature vector to a cyclic neural network model;
s3, establishing a transfer channel model through the recurrent neural network model to form a mapping relation between the original vibration signal and the in-vehicle noise signal, and outputting a modeled cancellation signal;
and S4, removing the in-car noise signal in the in-car sound signal according to the counteracting signal to obtain a voice signal after noise reduction, and using the voice signal as an input signal of a voice recognition system.
Further, in step S2, the convolutional neural network noise reduction model has a six-layer structure, which includes, in order from input to output, a first convolutional layer, a second convolutional layer, a maximum pooling layer, a third convolutional layer, an average pooling layer, and a Dropout layer, and a feature vector output is obtained after inputting the signal data matrix.
Further, in step S3, outputting the modeled cancellation signal includes the following steps:
s31, obtaining a counteracting signal through the recurrent neural network model;
s32, comparing the counteracting signal with the in-vehicle noise signal to obtain a residual error signal;
s33, when the residual error signal exceeds the set value, executing S34; when the residual error signal does not exceed a set value, directly outputting the offset signal;
and S34, updating model parameters in the recurrent neural network model according to the residual error signal, and executing S31-S33.
Further, in step S1, there are a plurality of in-vehicle microphones simultaneously picking up sound signals of different positions in the current vehicle.
Further, before the step S2, acoustic features of the in-vehicle speech signals at different positions in the vehicle are extracted by the convolutional neural network localization model, and the in-vehicle space is divided into a plurality of regions according to the extracted acoustic features.
Further, the acoustic characteristics include intensity and duration.
Furthermore, the convolutional neural network positioning model is of a five-layer structure and sequentially comprises a first convolutional layer, a maximum pooling layer, a second convolutional layer, an average pooling layer and a Softmax layer from input to output, and a vector output is obtained after a signal data matrix is input.
Further, the convolutional neural network positioning model takes the energy and time difference of the sound signals collected by each in-vehicle microphone as learning features, and obtains the probability of the in-vehicle speaker in each in-vehicle area through the time domain signals in the sound signals.
In another aspect, the present invention provides a preprocessing system for improving accuracy of speech recognition in a vehicle, comprising
The system comprises an external vibration sensor, a digital signal transmission line and a signal processing circuit, wherein the external vibration sensor is distributed at different positions outside the vehicle and is connected with the digital signal transmission line;
the system comprises in-vehicle microphones, a digital signal transmission line and a digital signal processing circuit, wherein the in-vehicle microphones are distributed at different positions in the vehicle and are connected to the digital signal transmission line, and the in-vehicle microphones are used for acquiring sound signals in the vehicle;
the voice signal preprocessing module is connected to the digital signal transmission line, the vibration sensor outside the vehicle and the microphone inside the vehicle output the collected signals to the voice signal preprocessing module through the digital signal transmission line, the voice signal preprocessing module carries out noise reduction processing on the sound signals collected by the microphone inside the vehicle through a convolutional neural network noise reduction model and a cyclic neural network model, and the voice signal preprocessing module can judge the position of a speaker according to the sound signals collected by the microphone inside the vehicle through a convolutional neural network positioning model.
Further, the vehicle exterior vibration sensor includes a first vehicle exterior vibration sensor, a second vehicle exterior vibration sensor, and a third vehicle exterior vibration sensor; the first off-board vibration sensor is arranged at an engine compartment of a vehicle for collecting engine noise; the second outside-vehicle vibration sensor is arranged near a vehicle tire and used for collecting road noise generated when the vehicle runs; the third vehicle exterior vibration sensor is arranged at the vehicle exterior rear view mirror and used for collecting wind noise formed at the rear view mirror when the vehicle runs.
The technical scheme provided by the invention has the following beneficial effects:
a. the self-adaptive control of the acquired noise signals is realized, and the noise is reduced before the voice recognition is carried out, so that the recognition rate of a recognition system is improved;
b. realizing the personalized voice recognition control and service of passengers at different riding positions in the vehicle.
Drawings
In order to more clearly illustrate the technical solutions in the embodiments of the present invention, the drawings needed to be used in the description of the embodiments will be briefly introduced below, and it is obvious that the drawings in the following description are only some embodiments of the present invention, and it is obvious for those skilled in the art to obtain other drawings based on these drawings without creative efforts.
FIG. 1 is a schematic structural diagram of a preprocessing system for improving accuracy of speech recognition in a vehicle according to an embodiment of the present invention;
FIG. 2 is a schematic diagram of the distribution of components of the preprocessing system for improving the accuracy of speech recognition in a vehicle according to an embodiment of the present invention;
FIG. 3 is a schematic diagram of a transmission path of a speech noise reduction signal of the preprocessing system according to the embodiment of the present invention for improving the accuracy of speech recognition in a vehicle;
FIG. 4 is a schematic diagram of an online learning and noise reduction process of the preprocessing system for improving the accuracy of speech recognition in the vehicle according to the embodiment of the present invention;
FIG. 5 is a schematic input/output diagram of a neural network noise reduction model of a preprocessing system for improving accuracy of speech recognition in a vehicle according to an embodiment of the present invention;
FIG. 6 is a schematic diagram of a speech positioning process of the preprocessing system for improving accuracy of in-vehicle speech recognition according to the embodiment of the present invention;
FIG. 7 is a schematic diagram of a transmission path of a speech positioning signal of the preprocessing system for improving accuracy of speech recognition in a vehicle according to an embodiment of the present invention;
FIG. 8 is a schematic flow chart of a neural network positioning model of the preprocessing system for improving the accuracy of speech recognition in a vehicle according to the embodiment of the present invention.
The system comprises a first vehicle exterior vibration sensor 11, a second vehicle exterior vibration sensor 12, a third vehicle exterior vibration sensor 13, a vehicle interior microphone 2, a voice signal preprocessing module 3, a digital signal transmission line 4, a voice recognition system 5 and a passenger position 6.
Detailed Description
In order to make the technical solutions of the present invention better understood and more clearly understood by those skilled in the art, the technical solutions of the embodiments of the present invention will be described below in detail and completely with reference to the accompanying drawings. It should be noted that the implementations not shown or described in the drawings are in a form known to those of ordinary skill in the art. Additionally, while exemplifications of parameters including particular values may be provided herein, it is to be understood that the parameters need not be exactly equal to the respective values, but may be approximated to the respective values within acceptable error margins or design constraints. It is to be understood that the described embodiments are merely exemplary of a portion of the invention and not all embodiments. All other embodiments, which can be derived by a person skilled in the art from the embodiments given herein without making any creative effort, shall fall within the protection scope of the present invention. In addition, the terms "comprises" and "comprising," and any variations thereof, in the description and claims of this invention, are intended to cover a non-exclusive inclusion, such that a process, method, apparatus, article, or apparatus that comprises a list of steps or elements is not necessarily limited to those steps or elements expressly listed, but may include other steps or elements not expressly listed or inherent to such process, method, article, or apparatus.
In an embodiment of the present invention, a control method of a preprocessing system for improving accuracy of in-vehicle speech recognition is provided, and with reference to fig. 1 to 4, the method includes the following steps:
s1, collecting original vibration signals at different positions outside the vehicle, and collecting current sound signals inside the vehicle, wherein the sound signals inside the vehicle comprise noise signals inside the vehicle and voice signals inside the vehicle;
s2, performing real-time feature learning on the original vibration signal through a convolutional neural network noise reduction model to obtain a corresponding feature vector and outputting the feature vector to a cyclic neural network model;
s3, establishing a transfer channel model through the recurrent neural network model to form a mapping relation between the original vibration signal and the in-vehicle noise signal, and outputting a modeled cancellation signal;
and S4, removing the in-car noise signal in the in-car sound signal according to the counteracting signal to obtain a voice signal after noise reduction, and using the voice signal as an input signal of a voice recognition system.
In step S3, outputting the modeled cancellation signal includes the following steps:
s31, obtaining a counteracting signal through the recurrent neural network model;
s32, comparing the counteracting signal with the in-vehicle noise signal to obtain a residual error signal;
s33, when the residual error signal exceeds the set value, executing S34; when the residual error signal does not exceed a set value, directly outputting the offset signal;
and S34, updating model parameters in the recurrent neural network model according to the residual error signal, and executing S31-S33.
In step S3, updating model parameters in the recurrent neural network model by using a minimum mean square error algorithm; in step S2, the convolutional neural network noise reduction model has a six-layer structure, which includes, in order from input to output, a first convolutional layer, a second convolutional layer, a maximum pooling layer, a third convolutional layer, an average pooling layer, and a Dropout layer, and a feature vector output is obtained after inputting the signal data matrix.
In an embodiment of the present invention, the control method of the speech signal preprocessing system includes not only a speech noise reduction method but also a speech localization method, and on the basis of the above embodiment, a plurality of in-vehicle microphones simultaneously collect a current in-vehicle sound signal, and before performing speech localization, feature learning needs to be performed through a neural network model. Specifically, acoustic features of in-vehicle voice signals of different positions in the vehicle are extracted through a convolutional neural network positioning model, the in-vehicle space is divided into a plurality of regions according to the extracted acoustic features, wherein the acoustic features comprise intensity and duration, for example, the in-vehicle space is divided into 4 regions according to the distribution of the positions of passengers, so that the signal feature range corresponding to each region received by an in-vehicle microphone is different, and generally one seat corresponds to one region. The convolutional neural network positioning model takes the energy and time difference of sound signals collected by each in-vehicle microphone as learning characteristics, and obtains the probability of the in-vehicle speaker in each in-vehicle seat through time domain signals in the sound signals. The convolutional neural network positioning model is of a five-layer structure and sequentially comprises a first convolutional layer, a maximum pooling layer, a second convolutional layer, an average pooling layer and a Softmax layer from input to output, a vector output is obtained after a signal data matrix is input, and the vector comprises the probability of the occurrence of a speaker in each seat in a vehicle.
In an embodiment of the present invention, a preprocessing system for improving accuracy of speech recognition in a vehicle based on the above control method is provided, and referring to fig. 1 to fig. 3, the system includes a vibration sensor outside the vehicle, preferably a digital microphone, an in-vehicle microphone 2 and a speech signal preprocessing module 3, wherein the vibration sensor outside the vehicle is distributed at different positions outside the vehicle and is connected to a digital signal transmission line 4, and the vibration sensor outside the vehicle is used for collecting original vibration signals outside the vehicle, preferably, the vibration sensor outside the vehicle includes a first vibration sensor outside the vehicle 11, a second vibration sensor outside the vehicle 12 and a third vibration sensor outside the vehicle 13; the first vehicle external vibration sensor 11 is arranged on an engine of a vehicle and can acquire an engine vibration signal; the second off-vehicle vibration sensor 12 is arranged near four tire frames of the vehicle, and one vibration sensor is arranged near each tire frame of the vehicle and can collect road noise generated when the vehicle runs; the third vehicle exterior vibration sensor 13 is disposed at the vehicle exterior rear view mirror, and can collect wind noise generated at the rear view mirror when the vehicle is running.
The in-vehicle microphones 2 are distributed at different positions in the vehicle and are connected to the digital signal transmission line 4, the in-vehicle microphones 2 are used for collecting sound signals in the vehicle, the in-vehicle microphones are preferably digital dual-microphone pairs in the vehicle, the number of the in-vehicle microphones 2 is preferably 4, and one in-vehicle microphone 2 is mounted near each in-vehicle passenger position 6, for example, at the positions of handles above four vehicle doors; the sound signals collected by the in-vehicle microphone 2 comprise in-vehicle voice signals of in-vehicle speakers and in-vehicle noise signals formed by transmitting original vibration outside the vehicle into the vehicle, and the in-vehicle microphones 2 at different positions can simultaneously collect the voice of the same speaker.
The voice signal preprocessing module 3 is connected to the digital signal transmission line 4, the digital signal transmission line 4 is preferably in an A2B connection mode, the vibration sensor outside the vehicle and the in-vehicle microphone 2 output the collected signals to the voice signal preprocessing module 3 through the digital signal transmission line 4, a chip used by the voice signal preprocessing module 3 can be preferably an NXP special neural processing engine i.mx 8M Plus, the voice signal preprocessing module 3 performs noise reduction processing on the sound signals collected by the in-vehicle microphone 2 through a convolutional neural network noise reduction model and a cyclic neural network model, the voice signal preprocessing module 3 can remove noise input into a voice recognition system according to a feedforward signal and the algorithm model, and the left clean voice signals are used for recognition; the voice signal pre-processing module 3 can also judge the position of the speaker through a convolutional neural network positioning model according to the sound signal collected by the in-vehicle microphone 2.
Specifically, the voice noise reduction part of the voice signal preprocessing system adopts a feedforward system structure, feedforward signals of the feedforward system can be collected by three vehicle exterior vibration sensors, original vibration signals collected by a first vehicle exterior vibration sensor, a second vehicle exterior vibration sensor and a third vehicle exterior vibration sensor are respectively recorded as x (n), y (n) and z (n), 4 vehicle interior microphones can collect voice signals at different passenger positions in a vehicle on one hand, and voice signals sent by 4 passenger positions in the vehicle can be respectively recorded as b1(n)、b2(n)、b3(n)、b4(n), on the other hand, an in-vehicle noise signal can be collected, wherein the in-vehicle noise is mainly transmitted into the vehicle by three noise sources outside the vehicle through a transmission channel (including air or a vehicle body) to form, the in-vehicle noise signal is correspondingly composed of three parts, corresponding to x (n), y (n), z (n) collected outside the vehicle, and the three parts of the in-vehicle noise signal are respectively marked as x ' (n), y ' (n), and z ' (n), so that a sound signal collected by an in-vehicle microphone can be regarded as voice plus additive noise and is marked as s (n), the sound signal is the superposition of a pure voice signal and the in-vehicle noise signal and can be expressed by the following formula:
s(n)＝bj(n)+x′(n)+y′(n)+z′(n)
wherein j is 1,2,3, 4.
Referring to fig. 3, noise source signals x (n), y (n), z (n) collected by the exterior vibration sensor are transmitted to the voice signal preprocessing module through a digital signal transmission line, and a mapping relation between interior noise signals x ' (n), y ' (n), z ' (n) and exterior original vibration signals x (n), y (n), z (n) is established by using a voice noise reduction part of the voice signal preprocessing module, so that the time delay influence of a transmission channel is eliminated, the interior noise signals are controlled, the system has noise tracking performance, the noise reduction effect of the preprocessing system is improved, and the damage to voice signals is reduced. The residual clean voice signal after noise reduction and enhancement is recorded as b' (n) to be input into a voice recognition system, so that the voice recognition accuracy can be improved.
The control algorithm of the noise reduction part of the voice signal preprocessing module adopts a mixed model of a one-dimensional convolutional neural network model and a cyclic neural network model, so that the characteristics of a signal transmission channel can be extracted in real time, the signal transmission delay of a processing system is effectively solved, the noise reduction processing of a voice signal is realized, and the one-dimensional convolutional neural network model is the convolutional neural network noise reduction model. The original vibration signals X (n), Y (n), Z (n) collected by the vibration sensor outside the vehicle are respectively (X (1), X (2) · X (n)), (Y (1), Y (2) · Y (n)), (Z (1), Z (2) · Z (n)), the sequences after each corresponding modeling processing are respectively (X (1), X (2) · X (n)), (Y (1), Y (2) · Y (n)), (Z (1), Z (2) · Z (n)), the noise signals X '(n), Y' (n), Z '(n) collected by the microphone inside the vehicle, and the corresponding signal sequences are respectively (X' (1), X '(2) · X' (n)), (Y '(1), Y' (2) · Y '(n)), (Z' (1), z '(2).. z' (n)). The voice signal pre-processing module adopts a minimum mean square error algorithm to adjust a control parameter W of the recurrent neural network model, so that the sum of the squares of the errors of the output signal of the model and the noise signal collected by the microphone in the vehicle is minimum, the noise reduction effect is achieved, and the sum of the squares of the errors is recorded as epsilon (n). The squared error of the noise-reduced residual signal corresponding to the three noise sources after noise reduction can be calculated by the following formula:
εx(n)＝[ex(n)]2＝[X(n)-x′(n)]2
εy(n)＝[ey(n)]2＝[Y(n)-y′(n)]2
εz(n)＝[ez(n)]2＝[Z(n)-z′(n)]2
the updating formula of the model parameter W of the recurrent neural network model in the speech signal preprocessing module is as follows:
in the formula, mux,μy,μzIs the model convergence factor.
Referring to fig. 4, the noise reduction scene of the speech signal preprocessing module can be regarded as an infinite sequence of training tasks, and the real-time online learning and noise reduction process of the noise signal of the hybrid model is as follows:
(1) inputting data: after preprocessing the acquired noise signal, storing the vibration signal values from three noise sources in discrete time intervals, which will produce a 3 × n matrix as the multi-channel input vector of the one-dimensional convolutional neural network.
(2)1D CNN layer: the first layer of one-dimensional convolutional neural network selects 80 convolutional kernels to extract input signal characteristics, an output vector is a weight matrix of the convolutional kernels, and the 1D CNN layer expressed below is a one-dimensional convolutional layer.
(3)1D CNN layer: the output result of the first convolution layer is transmitted to the second convolution layer, and then 50 convolution kernels are selected for further feature extraction.
(4) Maximum pooling layer: to reduce the complexity of the output and prevent data overfitting, the largest pooling layer is chosen after the convolutional layer.
(5)1D CNN layer: the output of the pooling layer is connected with another convolution layer, and 20 convolution kernels are selected to abstract high-dimensional features.
(6) Average pooling layer: to avoid overfitting, the average of the two weights in the neural network is pooled.
(7) Dropout layer: the neurons in the network are randomly assigned a weight of 0, and because of the short-time stationary nature of the noise signal, the selected ratio is preferably 0.4, and the weights for 40% of the neurons in this layer will be set to zero.
(8) RNN structure: inputting the feature vector of the previous layer of output noise signal into a Recurrent Neural Network (RNN) structure, and minimizing the mean square error value between the sequence of the network output vector and the sequence of the noise signal collected by the in-vehicle microphone, as shown in FIG. 5, wherein U, W, V is a model parameter, the model parameter W has the same meaning as the above W and can be adjusted accordingly, the RNN model adopts a parameter sharing mode, s is a hidden state, and o is a model output vector.
(9) Training a model: and updating network model parameters by adopting a real-time online cyclic learning algorithm aiming at the cyclic neural network to achieve the aim of training the model.
In an embodiment of the present invention, the speech signal preprocessing module further has a speech positioning function, for example, the receiving intensity and the time information of the speech signal generated at a passenger position in the vehicle by the 4 in-vehicle microphones with different positions are different, and based on this, a multi-channel convolutional neural network positioning model for speech signal position determination, hereinafter referred to as convolutional neural network positioning model, is established to extract speech signal features at different positions, so as to implement the speech positioning function of the preprocessing system. Referring to fig. 6 and 7, the 4 different in-vehicle microphones are denoted as p (n), q (n), g (n), and h (n) for the voice signals generated by a passenger, and their sequences are (p (1), p (2.. p (n)), (q (1), q (2.. q (n)), (g (1), g (2.. g (n)), (h (1), h (2.. h (n)), respectively.
Referring to fig. 8, the algorithm steps of the convolutional neural network localization model are as follows:
(1) inputting data: after the voice signals collected by the 4 in-vehicle microphones are preprocessed, a 4 x n matrix is generated according to the sequence of the voice signals and is used as a multi-channel input vector of the one-dimensional convolution neural network.
(2)1D CNN layer: the first layer defines a one-dimensional convolution kernel, which can learn one feature in the first layer of the neural network, and for speech signal localization, preferably 10 convolution kernels are defined, thus extracting 10 position features at the first layer of the network.
(3) Maximum pooling layer: to reduce the complexity of the output and prevent data overfitting, a max pooling layer is used after the CNN layer.
(4)1D CNN layer: and then learning the higher-level characteristics through a convolution layer, wherein 6 convolution kernels are defined by the layer.
(5) Average pooling layer: an average pooling layer is employed to further avoid overfitting, taking the average of two weights in the neural network, with each one-dimensional feature convolution kernel leaving only one weight in the neural network of this layer.
(6) An output layer: the fully-connected layer activated by Softmax is adopted, 4 neurons of the output layer correspond to four riding positions in the vehicle, the output value represents the probability of the four positions, the output value can enter the voice recognition system, and a basis is provided for the voice recognition system to perform personalized service, for example, the voice recognition system can open the maximum voice recognition control authority of a driver, a common passenger can only realize some basic voice control, and the output value is filtered by the pre-processing system to output a maximum probability value to the voice recognition system.
It should be noted that, the number and distribution positions of all the in-vehicle microphones and the out-vehicle vibration sensors include, but are not limited to, the above, and other numbers and distribution positions should also be included, for example, the out-vehicle vibration sensors may be additionally arranged at the tail of the vehicle, the formula for calculating the relevant parameters in the above embodiment is also only a preferred embodiment, and also includes a formula for realizing similar functions, and the number of convolution kernels selected in each layer in each neural network model, the number of input data, and the setting of the relevant parameters in the above embodiment are all based on the test preferred values, and other values for realizing similar functions should also be included.
The invention designs a voice signal preprocessing system in a vehicle-mounted environment, which comprises the implementation of carrying a system hardware structure on a vehicle body and the implementation of a corresponding algorithm. When the automobile runs, the system utilizes the vibration sensor outside the automobile to collect non-stable noise source vibration signals in real time in a running state, extracts noise characteristics, establishes a mapping relation between the noise signals received by the voice recognition system inside the automobile and the mapping relation under the condition of system delay, simultaneously adopts a multi-position double-channel microphone structure inside the automobile, and realizes position matching of voice signals on four riding positions inside the automobile through an algorithm. The voice signal preprocessing system needs to reduce noise and enhance the voice signal at the front end of the voice recognition system, so that the accuracy of voice recognition in the vehicle environment is further improved, the position of a passenger sending a voice control signal in the vehicle is located, the voice recognition system is convenient to meet personalized recognition requirements of different passengers, good human-vehicle voice interaction is realized, and the passenger obtains better driving experience.
The noise reduction part in the voice signal pretreatment system provided by the invention adopts a feedforward self-adaptive control structure, vibration sensors near noise sources are used for collecting vibration signals (road noise, engine noise and wind noise) of three noise sources during driving in real time, a one-dimensional convolutional neural network control model is used for carrying out feature learning on the real-time noise signals, a cyclic neural network model is used for realizing signal tracking and control, the accuracy of a voice recognition system is favorably improved, the positioning part is used for collecting voice signals by using microphones of four channels in a vehicle, the acoustic features of the voice signals at different positions in the vehicle are extracted by using the one-dimensional convolutional neural network model, and then the positions of passengers sending the voice signals are calculated and judged.
The voice signal pretreatment system provided by the invention realizes the control of noise in the vehicle, enhances the noise reduction of the voice signal with noise, provides a clean voice signal for the voice recognition system, and positions the position of a passenger sending the voice signal so as to realize the personalized recognition and control of the voice recognition system. The voice signal pre-processing system provided by the invention overcomes the voice signal distortion caused by noise reduction of the traditional voice enhancement system, the in-vehicle noise signal characteristic is extracted in real time in a self-adaptive manner by an algorithm, the noise signal characteristic does not need to be defined manually, noise is filtered without damage to the voice signal, the recognition accuracy of the voice recognition system is improved, a feedforward structure of the system collects a vibration signal in a driving state in real time, a noise reduction model of the pre-processing system is trained, the signal delay problem of the noise reduction system is effectively solved, effective tracking of in-vehicle noise is realized, the system is suitable for complex in-vehicle noise environments in different driving states at different speeds, and the noise reduction performance of the pre-processing system; the in-vehicle microphone also realizes the judgment of the position of the voice signal, provides the source position information of the voice instruction for the voice recognition system and expands the functions of the preprocessing system.
The above description is only for the purpose of illustrating the preferred embodiments of the present invention and is not to be construed as limiting the invention, and any modifications, equivalents, improvements and the like that fall within the spirit and principle of the present invention are intended to be included therein.
Claims (10)
1. A control method of a preprocessing system for improving the accuracy of speech recognition in a vehicle is characterized by comprising the following steps:
s1, collecting original vibration signals at different positions outside the vehicle, and collecting current sound signals inside the vehicle, wherein the sound signals inside the vehicle comprise noise signals inside the vehicle and voice signals inside the vehicle;
s2, performing real-time feature learning on the original vibration signal through a convolutional neural network noise reduction model to obtain a corresponding feature vector and outputting the feature vector to a cyclic neural network model;
s3, establishing a transfer channel model through the recurrent neural network model to form a mapping relation between the original vibration signal and the in-vehicle noise signal, and outputting a modeled cancellation signal;
and S4, removing the in-car noise signal in the in-car sound signal according to the counteracting signal to obtain a voice signal after noise reduction, and using the voice signal as an input signal of a voice recognition system.
2. The method for controlling a preprocessing system for improving accuracy of speech recognition in a vehicle according to claim 1, wherein in step S2, the convolutional neural network noise reduction model has a six-layer structure, and comprises, in order from input to output, a first convolutional layer, a second convolutional layer, a maximum pooling layer, a third convolutional layer, an average pooling layer, and a Dropout layer, and a feature vector output is obtained after inputting the signal data matrix.
3. The control method of the preprocessing system for improving the accuracy of speech recognition in the vehicle according to claim 1, wherein the step of outputting the modeled cancellation signal in S3 includes the following steps:
s31, obtaining a counteracting signal through the recurrent neural network model;
s32, comparing the counteracting signal with the in-vehicle noise signal to obtain a residual error signal;
s33, when the residual error signal exceeds the set value, executing S34; when the residual error signal does not exceed a set value, directly outputting the offset signal;
and S34, updating model parameters in the recurrent neural network model according to the residual error signal, and executing S31-S33.
4. The control method for the preprocessing system according to claim 1, wherein in step S1, a plurality of in-vehicle microphones are present for simultaneously capturing sound signals of different positions in the current vehicle.
5. The control method for the preprocessing system for improving the accuracy of the in-vehicle speech recognition according to claim 4, wherein before the step of S2, the acoustic features of the in-vehicle speech signals at different positions in the vehicle are extracted by the convolutional neural network positioning model, and the in-vehicle space is divided into a plurality of regions according to the extracted acoustic features.
6. The control method of the preprocessing system for improving the accuracy of in-vehicle speech recognition according to claim 5, wherein the acoustic features include intensity and duration.
7. The control method of the preprocessing system for improving the accuracy of the speech recognition in the vehicle according to claim 5, wherein the convolutional neural network positioning model has a five-layer structure, and comprises a first convolutional layer, a maximum pooling layer, a second convolutional layer, an average pooling layer and a Softmax layer from input to output in sequence, and a vector output is obtained after a signal data matrix is input.
8. The control method for the preprocessing system for improving the accuracy of the in-vehicle speech recognition according to claim 7, wherein the convolutional neural network localization model uses the energy and time difference of the sound signal collected by each in-vehicle microphone as the learning feature, and the convolutional neural network localization model obtains the probability of the in-vehicle speaker appearing in each in-vehicle area through the time domain signal in the sound signal.
9. A pre-processing system for improving accuracy of speech recognition in a vehicle is characterized by comprising
The system comprises an external vibration sensor, a digital signal transmission line and a signal processing circuit, wherein the external vibration sensor is distributed at different positions outside the vehicle and is connected with the digital signal transmission line;
the system comprises in-vehicle microphones, a digital signal transmission line and a digital signal processing circuit, wherein the in-vehicle microphones are distributed at different positions in the vehicle and are connected to the digital signal transmission line, and the in-vehicle microphones are used for acquiring sound signals in the vehicle;
the voice signal preprocessing module is connected to the digital signal transmission line, the vibration sensor outside the vehicle and the microphone inside the vehicle output the collected signals to the voice signal preprocessing module through the digital signal transmission line, the voice signal preprocessing module carries out noise reduction processing on the sound signals collected by the microphone inside the vehicle through a convolutional neural network noise reduction model and a cyclic neural network model, and the voice signal preprocessing module can judge the position of a speaker according to the sound signals collected by the microphone inside the vehicle through a convolutional neural network positioning model.
10. The pre-processing system for improving accuracy of in-vehicle speech recognition according to claim 9, wherein the out-of-vehicle vibration sensor comprises a first out-of-vehicle vibration sensor, a second out-of-vehicle vibration sensor, and a third out-of-vehicle vibration sensor; the first off-board vibration sensor is arranged at an engine compartment of a vehicle for collecting engine noise; the second outside-vehicle vibration sensor is arranged near a vehicle tire and used for collecting road noise generated when the vehicle runs; the third vehicle exterior vibration sensor is arranged at the vehicle exterior rear view mirror and used for collecting wind noise formed at the rear view mirror when the vehicle runs.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202011060176.8A CN112259113A (en) | 2020-09-30 | 2020-09-30 | Preprocessing system for improving accuracy rate of speech recognition in vehicle and control method thereof |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202011060176.8A CN112259113A (en) | 2020-09-30 | 2020-09-30 | Preprocessing system for improving accuracy rate of speech recognition in vehicle and control method thereof |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN112259113A true CN112259113A (en) | 2021-01-22 |
Family
ID=74233559
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202011060176.8A Pending CN112259113A (en) | 2020-09-30 | 2020-09-30 | Preprocessing system for improving accuracy rate of speech recognition in vehicle and control method thereof |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN112259113A (en) |
Cited By (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN114464203A (en) * | 2022-01-18 | 2022-05-10 | 小米汽车科技有限公司 | Noise filtering method, device, system, vehicle and storage medium |
| CN115223327A (en) * | 2021-07-14 | 2022-10-21 | 广州汽车集团股份有限公司 | In-vehicle living body protection method and system |
| WO2024028297A1 (en) * | 2022-08-04 | 2024-02-08 | Next.E.Go Mobile SE | Communication method for vehicle occupants via an audio system of a vehicle, and audio system for a vehicle |
Citations (16)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US5404422A (en) * | 1989-12-28 | 1995-04-04 | Sharp Kabushiki Kaisha | Speech recognition system with neural network |
| CN105448303A (en) * | 2015-11-27 | 2016-03-30 | 百度在线网络技术（北京）有限公司 | Voice signal processing method and apparatus |
| CN107944551A (en) * | 2017-12-15 | 2018-04-20 | 福州大学 | One kind is used for electrowetting display screen defect identification method |
| CN108319909A (en) * | 2018-01-29 | 2018-07-24 | 清华大学 | A kind of driving behavior analysis method and system |
| CN108984481A (en) * | 2018-06-26 | 2018-12-11 | 华侨大学 | A kind of homography matrix estimation method based on convolutional neural networks |
| CN109087631A (en) * | 2018-08-08 | 2018-12-25 | 北京航空航天大学 | A kind of Vehicular intelligent speech control system and its construction method suitable for complex environment |
| KR20190006741A (en) * | 2017-07-11 | 2019-01-21 | 현대자동차주식회사 | Connectivity Integration Management Method and Connected Car thereof |
| CN109448726A (en) * | 2019-01-14 | 2019-03-08 | 李庆湧 | A kind of method of adjustment and system of voice control accuracy rate |
| CN110148420A (en) * | 2019-06-30 | 2019-08-20 | 桂林电子科技大学 | A kind of audio recognition method suitable under noise circumstance |
| US20190341057A1 (en) * | 2018-05-07 | 2019-11-07 | Microsoft Technology Licensing, Llc | Speaker recognition/location using neural network |
| CN110459234A (en) * | 2019-08-15 | 2019-11-15 | 苏州思必驰信息科技有限公司 | For vehicle-mounted audio recognition method and system |
| CN110517701A (en) * | 2019-07-25 | 2019-11-29 | 华南理工大学 | A kind of microphone array voice enhancement method and realization device |
| KR102051016B1 (en) * | 2018-11-27 | 2019-12-02 | 오아 주식회사 | Server and method for controlling learning-based speech recognition apparatus |
| US20200035233A1 (en) * | 2019-07-29 | 2020-01-30 | Lg Electronics Inc. | Intelligent voice recognizing method, voice recognizing apparatus, intelligent computing device and server |
| CN111223493A (en) * | 2020-01-08 | 2020-06-02 | 北京声加科技有限公司 | Voice signal noise reduction processing method, microphone and electronic equipment |
| CN116189651A (en) * | 2023-02-24 | 2023-05-30 | 武汉大学 | Multi-speaker sound source positioning method and system for remote video conference |
-
2020
- 2020-09-30 CN CN202011060176.8A patent/CN112259113A/en active Pending
Patent Citations (16)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US5404422A (en) * | 1989-12-28 | 1995-04-04 | Sharp Kabushiki Kaisha | Speech recognition system with neural network |
| CN105448303A (en) * | 2015-11-27 | 2016-03-30 | 百度在线网络技术（北京）有限公司 | Voice signal processing method and apparatus |
| KR20190006741A (en) * | 2017-07-11 | 2019-01-21 | 현대자동차주식회사 | Connectivity Integration Management Method and Connected Car thereof |
| CN107944551A (en) * | 2017-12-15 | 2018-04-20 | 福州大学 | One kind is used for electrowetting display screen defect identification method |
| CN108319909A (en) * | 2018-01-29 | 2018-07-24 | 清华大学 | A kind of driving behavior analysis method and system |
| US20190341057A1 (en) * | 2018-05-07 | 2019-11-07 | Microsoft Technology Licensing, Llc | Speaker recognition/location using neural network |
| CN108984481A (en) * | 2018-06-26 | 2018-12-11 | 华侨大学 | A kind of homography matrix estimation method based on convolutional neural networks |
| CN109087631A (en) * | 2018-08-08 | 2018-12-25 | 北京航空航天大学 | A kind of Vehicular intelligent speech control system and its construction method suitable for complex environment |
| KR102051016B1 (en) * | 2018-11-27 | 2019-12-02 | 오아 주식회사 | Server and method for controlling learning-based speech recognition apparatus |
| CN109448726A (en) * | 2019-01-14 | 2019-03-08 | 李庆湧 | A kind of method of adjustment and system of voice control accuracy rate |
| CN110148420A (en) * | 2019-06-30 | 2019-08-20 | 桂林电子科技大学 | A kind of audio recognition method suitable under noise circumstance |
| CN110517701A (en) * | 2019-07-25 | 2019-11-29 | 华南理工大学 | A kind of microphone array voice enhancement method and realization device |
| US20200035233A1 (en) * | 2019-07-29 | 2020-01-30 | Lg Electronics Inc. | Intelligent voice recognizing method, voice recognizing apparatus, intelligent computing device and server |
| CN110459234A (en) * | 2019-08-15 | 2019-11-15 | 苏州思必驰信息科技有限公司 | For vehicle-mounted audio recognition method and system |
| CN111223493A (en) * | 2020-01-08 | 2020-06-02 | 北京声加科技有限公司 | Voice signal noise reduction processing method, microphone and electronic equipment |
| CN116189651A (en) * | 2023-02-24 | 2023-05-30 | 武汉大学 | Multi-speaker sound source positioning method and system for remote video conference |
Non-Patent Citations (3)
| Title |
|---|
| 肖勇;覃爱娜;: "改进的HMM和小波神经网络的抗噪语音识别", 计算机工程与应用, no. 22 * |
| 肖勇;覃爱娜;: "改进的HMM和小波神经网络的抗噪语音识别", 计算机工程与应用, no. 22, 1 August 2010 (2010-08-01) * |
| 陈玮;陈劭;刘煜;: "某型森林消防运兵车怠速工况下的结构噪声分析", 林业机械与木工设备, no. 06, 10 June 2017 (2017-06-10) * |
Cited By (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN115223327A (en) * | 2021-07-14 | 2022-10-21 | 广州汽车集团股份有限公司 | In-vehicle living body protection method and system |
| CN114464203A (en) * | 2022-01-18 | 2022-05-10 | 小米汽车科技有限公司 | Noise filtering method, device, system, vehicle and storage medium |
| WO2023137912A1 (en) * | 2022-01-18 | 2023-07-27 | 小米汽车科技有限公司 | Noise filtering method, apparatus and system, vehicle, and storage medium |
| WO2024028297A1 (en) * | 2022-08-04 | 2024-02-08 | Next.E.Go Mobile SE | Communication method for vehicle occupants via an audio system of a vehicle, and audio system for a vehicle |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN112259113A (en) | Preprocessing system for improving accuracy rate of speech recognition in vehicle and control method thereof | |
| CN110070868B (en) | Voice interaction method and device for vehicle-mounted system, automobile and machine readable medium | |
| CN104908688B (en) | The method and device of vehicle active noise reduction | |
| CN105931650B (en) | A kind of self-adaptation noise reduction method based on audio feature extraction | |
| JP5913340B2 (en) | Multi-beam acoustic system | |
| CN109087631A (en) | A kind of Vehicular intelligent speech control system and its construction method suitable for complex environment | |
| CN110610715A (en) | Noise reduction method based on CNN-DNN hybrid neural network | |
| CN109545230A (en) | Acoustic signal processing method and device in vehicle | |
| CN110459234A (en) | For vehicle-mounted audio recognition method and system | |
| CN103568998A (en) | Device and method for controlling quality of sounds generated during vehicle door closing | |
| CN110120217B (en) | Audio data processing method and device | |
| JP7186375B2 (en) | Speech processing device, speech processing method and speech processing system | |
| CN114582312B (en) | Active control method and system for anti-interference adaptive road noise in vehicle | |
| CN108538304A (en) | Active Control System for Noise Cancelling in Passenger Compartment of Vehicle | |
| CN111696539A (en) | Voice interaction system and vehicle for actively reducing noise of internal call | |
| CN108189724A (en) | A kind of automobile feedback control active noise reduction neckpillow | |
| CN112216300A (en) | Noise reduction method and device for sound in driving cab of mixer truck and mixer truck | |
| CN107274878A (en) | A kind of Active noise control system in railcar train drivers' cab | |
| CN215815174U (en) | Voice interaction system inside and outside vehicle and vehicle | |
| CN111323753A (en) | Method for positioning voice source in automobile | |
| CN116052630A (en) | Active control system and method for automobile road noise | |
| US20220189450A1 (en) | Audio processing system and audio processing device | |
| CN111833840A (en) | Noise reduction method and device, system, electronic equipment and storage medium | |
| EP1395080A1 (en) | Device and method for filtering electrical signals, in particular acoustic signals | |
| JP4649905B2 (en) | Voice input device |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination |