JP2012155301A - State recognition type speech recognition method - Google Patents

Abstract

PROBLEM TO BE SOLVED: To provide an essential noise resistance technique capable of recognizing a speech by estimating a dynamic state recognition parameter DIP of speech input data through preprocessing of speech recognition on an embedded type system used by a compact information device.SOLUTION: A speech recognition method constitutes an interpreter which calculates the dynamic state recognition parameter DIP for the speech input data as an output of an A/D conversion section by a state recognition preprocessing section S1 in Fig. 1, and determines a next processing part with information on the calculated dynamic state recognition parameter DIP, and a state recognition variable estimation section which calculates a variable noise processing reference parameter RTH. Through a state recognition generation section which generates a state recognition variable IP based upon the dynamic state recognition parameter DIP and a distribution section which compares the variable noise processing reference parameter RTH and the state recognition variable IP with each other, the speech recognition preprocessing is performed using speech extraction processing for extracting a speech section and reducing/removing noise of a part other than the speech section and noise processing for removing the noise and device noise.

Description

  Sound processing technology

  In addition to car navigation systems using voice recognition technology, small information systems in small information devices represented by PDAs and smartphones are in widespread use. In order to solve background noise and noise in a small terminal, it is difficult to solve background noise and noise whose amplitude change with time is unpredictable as software technology. Additional methods such as a method of using and using an operation button, a screen touch, and a method of using a defined device are required.

  In connection with speech recognition methods, especially when performing speech recognition on desktop PCs and portable information recognition devices, there are many cases where conventional speech recognition technology cannot be recognized under environmental noise conditions. By using a cognitive speech recognition system, stable speech recognition is possible.

The present invention relates to a voice recognition system and method, and more particularly to a voice recognition system and method used for small information devices such as mobile phones, smartphones, PDAs, car navigation systems, tablet PCs, notebook PCs, and desktop PCs. In particular, in the case of an OS-based system or an OS-based embedded system that is assumed to be used in an environment with a lot of background noise, the performance depends on a separate external device such as a noise-reducing sound card or a headset. Therefore, it is most suitable for a speech recognition system having such system restrictions.

As a means for accurately recognizing a speech uttered in a noisy environment, first, a speech recognition system preliminarily estimates a dynamic state recognition parameter DIP of speech input data and removes a noise portion from the uttered speech. A high recognition rate is provided by providing clear voice data so that voice can be recognized accurately. In the case of a desktop PC capable of processing a relatively large amount of data, it is possible to acquire data that meets conditions using a high-performance external sound card. However, since embedded systems used in small information devices are more fragile than desktop PCs, in order to put the speech recognition system into practical use, noise reduction technology that can recognize speech uttered in a noisy environment is indispensable. is there.

In order to solve the above problems, a speech recognition method according to the present invention includes a speech input unit that inputs speech spoken to a microphone or wired / wireless headset, and input speech obtained from each speech input unit. A / D converter F1 in FIG. 2 that samples, quantizes and converts the signal into a digital signal, and the situation recognition calculation that calculates the dynamic situation recognition parameter DIP for the voice input data that is output from the A / D converter F1 Part F2, interpreter F3 for determining the next processing part based on the calculated dynamic situation recognition parameter DIP, situation recognition variable estimation part F4 for calculating variable noise processing reference parameter RTH, and dynamic situation recognition parameter The situation recognition generation unit F6 that generates the situation recognition variable IP based on the DIP standard, and compares the variable noise processing reference parameter RTH with the situation recognition variable IP. A distribution unit F7, an audio extraction portion F8 to reduce / eliminate noise portions other than the speech section by extracting speech segments using noise processing unit F9 to remove noise and device noise speech recognition preprocessing performed.

  An object of the present invention is to provide a speech recognition method capable of solving the problem that it is difficult to solve background noise and noise in which the prior art has a change in time vs. amplitude that is unpredictable, It is another object of the present invention to provide a speech recognition system that can further improve speech recognition accuracy even in a noisy environment. In an embedded system represented by a PDA or the like, a microphone built in a main board of the main body is affected by various noises in the surrounding environment. For example, when used outdoors, wind sounds, speaker breath sounds, button and stylus pen operation sounds, and other vibrations adversely affect the input sound signal.

Structure of the invention

In FIG. 1, the situation recognition type pre-processing unit S1 performs voice input data acquired by an unspecified speaker. The feature extraction unit S2 converts the voice input data into the most appropriate information in order to perform voice recognition processing. The recognition processing unit S4 performs speech recognition using the grammar information parameter of the grammar processing unit S5 that includes the vocabulary information parameters of the acoustic model S3 and the vocabulary processing unit S6. The speech recognition post-processing unit performs post-processing of the natural language processing unit S7-A and the phonetic character processing unit S7-B on the speech-recognized result value, and outputs a text type recognition result.

  According to the present invention, since the speech recognition method FIG. 1 is configured as described above, the A / D conversion unit F1 of FIG. 2 performs acoustic processing on analog speech signals respectively input from the microphones under various noises. The acoustic model parameter and the synchronization parameter registered when creating the model are configured, and encoding and quantization are performed based on the parameter, and the digital speech signal is converted. The synchronization parameter is roughly divided into a sampling bit and a sampling frequency.

  The situation recognition variable calculation unit F2 estimates the dynamic situation recognition parameter DIP of the digital audio data using the acquired total bit value when the A / D conversion unit F1 performs quantization. The calculation process of the dynamic situation recognition parameter DIP calculates the noise accuracy of the voice input data, and based on the result, the calculation result is transmitted to the interpreter F3 so that the variable noise processing reference parameter RTH can be created. The situation recognition variable estimation unit F4 determines the variable noise processing reference parameter RTH, and the situation recognition variable estimation unit F4 compares it with the estimation result of the dynamic situation recognition parameter DIP and the variable noise processing reference parameter RTH. By the comparison determination, the situation recognition variable generation unit F6 generates the situation recognition variable IP, and the distribution unit F7 determines which process is branched to the voice extraction unit F8 or the noise processing unit F9.

  FIG. 2 shows detailed processing of the situation recognition type pre-processing unit S1 of FIG. The audio input device supports wired microphones and wireless microphones, so it has two processing modules for A / D conversion and Bluetooth reception. The situation recognition variable calculation unit F2 of FIG. 2 analyzes the stream of voice input data and calculates a dynamic situation recognition parameter DIP. For the analyzed stream, the interpreter F3 determines whether or not audio CODEC compression is possible for the audio input data, and performs distribution processing. For the compressed audio data, the decoding process of the audio CODEC processing unit F5-A and the error concealment processing unit F5-B are performed, and the situation recognition variable estimation unit F4 receives from the data trim information of the dynamic situation recognition parameter DIP. A variable noise processing reference parameter RTH is calculated. The situation recognition variable generation unit F6 calculates the situation recognition variable IP using the variable noise processing reference parameter RTH and the dynamic situation recognition parameter DIP.

  The problem with speech recognition is that when a speech recognition system is used in a noise / noise environment, the recognition accuracy is greatly reduced due to the influence of ambient noise. Moreover, speech recognition of large vocabularies and misrecognition are particularly likely to occur when there are many similar words. In order to solve the problem, the present invention calculates the accuracy of noise / noise itself in the situation recognition type pre-processing unit S1 of FIG. 1 for noise / noise which is one of the major factors that degrade recognition performance. 2 is arranged, and the noise extraction unit F8 of FIG. 2 for converting to extract good audio data and the noise processing unit F9 for reducing / removing noise / noise are arranged, so that noise different from the conventional audio recognition method is provided. A situation recognition type speech recognition system is constructed.

  To achieve the above object, a current frame of input data is generated for the digitized input data. The voice extraction unit F8 generates a voice frame of the voice input data for the digitized voice input data that is the output of the A / D conversion unit F1. In the generated speech frame, a noise estimation module is configured using a method of estimating a noise state from speech input data. Further, the speech data spectrum signal and the noise data spectrum signal are separated in the frequency domain. According to the present invention, the separated speech signal and noise signal are combined after the speech extraction process. At this timing, it is possible to extract a voice section in the voice extraction unit F8 and partially achieve a noise reduction result.

  As described above, the noise processing unit F9 performs noise processing using the noise information of the situation recognition parameter IP, the frame section parameter information, and the variable noise processing reference parameter RTH from the situation recognition variable generation unit F6. In particular, the noise processing unit F9 of the present invention is internally configured by two processing modules, and is compared with the variable noise processing reference parameter RTH of the distribution unit F7 using information on the situation recognition parameter IP in the situation recognition variable generation unit F6. The processing modules are divided according to whether or not the voice section can be extracted based on the situation recognition judgment. First, a result frame taken over from the voice extraction unit F8 is analyzed, and a section in which voice data and noise data are taken in is detected while deleting non-voice section information in the current frame. In the detected and captured section, noise data mixed in the voice data portion is reduced. For the current frame taken over from the distribution unit F7, in order to distinguish from the non-speech section, the noise distribution degree of the section where the voice data and the noise data are mixed is detected while always detecting the information of the section that is not speech. Measurement and noise reduction processing for the section in which the voice data of the current frame is captured so that the voice can be recognized.

  The feature extraction unit S2 in FIG. 1 is a processing unit that obtains an audio feature component from an audio signal. Using the voice feature component, utterance start information, utterance end information, etc. inherited from the current voice frame parameter are acquired and set. Further, the voice analysis in the voice data section is performed, the voice signal is analyzed, and the voice parameter feature is extracted. The speech feature component is a probability distribution of phoneme features, and the speech pattern is converted in units of sound utterances so as to be mapped to the parameters of the acoustic model S3, and is parameterized and input to the recognition processing unit S4.

  The acoustic model S3 is a model used for speech recognition. In the speech recognition system of the present invention, speech audio using the sampling bit value of 8 bits or 16 bits and the sampling frequency of 8 kHz or 16 kHz used in other speech recognition systems is used. Specifically, in the acoustic model S3, the distribution of probability values that the sound table features appear is calculated using speech samples for a total of 100 or more speakers of almost the same number of men and women, and which of the following features appears: This is a record of probability distribution values for transition to a state. Actually, the speech pattern indicating the probability distribution value is converted in units of utterances, and the speech table parameters are converted into information so as to be read by the speech recognition system side, and constructed as an audio database.

  The vocabulary processing unit S6 extracts phoneme parameters from the grammar information, which is a dictionary that collects words, words, and sentences that are speech recognition targets so as to be matched with the acoustic model. Then, phoneme parameters are set together with a grammar processing unit S5 that performs a process of converting a grammar, which is a dictionary that collects words, words, and sentences to be recognized by speech recognition so that the system can recognize them. Specifically, in the vocabulary data registered in the vocabulary processing unit S6 composed of a combination of the sound table parameter information of the acoustic model S3, words, numbers, basic words (nouns, pronouns, verbs, etc.), and basic sentences. The result value selected by the closest approximate word is provided as reference material for outputting the recognition processing result value of the recognition processing unit S4.

  The speech recognition system of the present invention uses a Hidden Markov Model (HMM) capable of dealing with an unspecified speaker who can recognize the voice of any speaker without registering the voice of the specific speaker. ing. The recognition processing unit S4 in FIG. 1 connects the acoustic model S3 and grammar information, collates the speech feature component resulting from the feature extraction unit S2 in units of phonemes, and the grammar processing unit S5 and the vocabulary processing unit S6. Perform the process of selecting the closest word, word, or sentence in the result value.

  As for the processing after recognition in FIG. 1, a natural language processing unit S7-A and a phonetic character processing unit S7-B are configured as a solution for misrecognition by pronunciation similarity, character class, and character notation. Keep it. In order to convert the result of the recognition processing unit S4 into a desired form and obtain the best recognition result, the natural language processing unit S7-A and the phonetic character processing unit S7-B of FIG. Characters are selected and encoded using the phoneme of the recognized recognition result, and output result processing is performed.

FIG. 1 is a block diagram showing the functions of the situation recognition type speech recognition system according to the present invention and the flow of its processing. FIG. 2 shows an embodiment related to a situation recognition type speech recognition method according to the present invention. The following will be described using each functional embodiment of FIG. Under the actual use environment, the A / D conversion unit F1 shown in FIG. 2 corresponds to the microphone audio signal P10 and the A / D conversion P11 shown in FIG. The microphone audio signal P10 acquires the actual audio data and the noise signal corresponding to the environment or the noise incorporated in the audio data, and the A / D conversion P11 converts the analog signal that is the audio signal into a digital signal. . A / D conversion P11 converts analog data to digital data at a sampling frequency of 8 KHz or 16 KHz. In the process of converting the analog data into digital data, the signal-to-noise ratio is examined, the situation recognition degree is estimated, and the dynamic situation recognition parameter DIP is calculated.

  In the A / D conversion P11, audio data which is an analog signal is converted into digital data. Specifically, when quantizing after sampling at 8 KHz or 16 KHz, the dynamic state recognition parameter DIP of the digital audio data is estimated using the acquired total bit value and input stream information. The dynamic situation recognition parameter DIP of FIG. 3 is calculated, and the dynamic situation recognition parameter DIP value is a value obtained by calculating the noise accuracy of the voice input data so that the noise processing reference parameter RTH can be generated. Based on the calculation result, the current frame generation P12 in FIG. 3 generates a current frame so that a state recognition threshold value corresponding to the calculation result can be created, and audio data is transferred to each corresponding frame and recorded in the frame register.

  The variable noise processing reference parameter RTH is created by using the data information of the frame generated by the current frame generation P12 of FIG. 3, the calculated value estimated by the A / D conversion P11, and the dynamic situation recognition parameter DIP. The RTH state recognition threshold is recorded in comparison with DIP. In the situation recognition estimation and calculation P13, the situation recognition threshold value of the dynamic situation recognition estimation parameter DIP information and the variable noise processing reference parameter RTH of the current frame is determined, and the situation recognition judgment P14 makes a comparison judgment. The situation recognition determination P14 corresponds to the distribution unit F7 in FIG.

音声区間抽出Ｐ１５と雑音消去および減少Ｐ１６を行う。In the situation recognition determination P14 of FIG. 3, it is determined by the comparative determination whether the process branches to the speech segment extraction P15 or the noise removal / reduction P16. The speech segment extraction P15 corresponds to the speech extraction unit F8 in FIG. 2, and the noise removal and reduction P16 corresponds to the noise processing unit F9 in FIG. The situation recognition judgment P14 sets the situation recognition threshold value of the variable noise processing reference parameter RTH for the current frame as a reference value, and compares it with the situation recognition parameter IP value indicating the situation recognition degree of the current frame together with the input voice data. To determine the processing path. For example, for the current frame:

Speech segment extraction P15 and noise cancellation and reduction P16 are performed.

ついて処理を行う。図３の現在フレーム生成Ｐ１２によるデジタル化された入力音声データに対し、図２の分配部Ｆ７と図３の状況認知判断Ｐ１４を行い、入力音声データの時間ドメインと入力音声データの周波数ドメインで解析し、解析されたフレームのデータ情報より音声信号分離処理を行う為、処理データの情報を引き抜く。図３の音声区間抽出Ｐ１５には、動的状況認知パラメータＤＩＰを利用して信号の音声区間を再検出し、分離フィルタの係数を計算する。計算された音声区間値は分離フィルタの係数更新によって信号分離処理設定をして音声区間内の音声信号に対して抽出処理を行う。

Process. The distribution unit F7 in FIG. 2 and the situation recognition determination P14 in FIG. 3 are performed on the input voice data digitized by the current frame generation P12 in FIG. 3 and analyzed in the time domain of the input voice data and the frequency domain of the input voice data. In order to perform audio signal separation processing from the analyzed frame data information, the processing data information is extracted. In the speech segment extraction P15 in FIG. 3, the speech segment of the signal is detected again using the dynamic situation recognition parameter DIP, and the coefficient of the separation filter is calculated. The calculated speech section value is subjected to extraction processing for speech signals in the speech section by setting signal separation processing by updating the coefficients of the separation filter.

本段階には簡単に言うと伝統的オーディオフィルターリングアルゴリズムとして音声信号

から引き継いだデータの雑音処理について説明する。分離された入力データストリムについて２Ｎ＋１サイズのウィンドウを持っているフィルタを用いる。結果的に音声信号内の低密度インパルス雑音を除去する。ＩＰ＞ＲＴＨの場合は、引き継いだ状況認知パラメータよりフィルタのウィンドウサイズが決まる。また、フィルタパラメータＮの値を計算および調整して、音声データの推定エラーを検出しながら音声信号内の低密度インパルス雑音を除去する。The noise processing unit F9 in FIG. 2 takes over two processing data values from the situation recognition determination P14 in FIG.

In this stage, the audio signal is the traditional audio filtering algorithm.

The noise processing of data inherited from will be described. A filter having a 2N + 1 size window for the separated input data stream is used. As a result, low density impulse noise in the speech signal is removed. When IP> RTH, the window size of the filter is determined by the inherited situation recognition parameter. Also, the filter parameter N is calculated and adjusted to remove low-density impulse noise in the audio signal while detecting an estimation error of the audio data.

  Hereinafter, the core processing part of the speech recognition system and method according to the present invention will be described with reference to FIGS. In this embodiment, a case where the voice recognition system of the present invention is applied to a patient observation information recording application product by voice input on a business PDA will be described.

  FIG. 1 shows a speech recognition method evolved from a conventional speech recognition method that extracts phoneme features from input speech data, performs speech recognition processing with reference to an acoustic model and a language dictionary, and acquires speech recognition results. The role of the language dictionary is divided into the vocabulary processing unit S6 and the grammar processing unit S5, the speech recognition post-processing unit S7 of FIG. 1 is added before the speech recognition result is acquired, and the natural language processing of FIG. The natural language processing unit S7-A and the phonetic character processing unit S7-B in FIG. P speech signal feature extraction P17 in FIG. 3 corresponds to the feature extraction unit S2 in FIG. 1, and speech recognition P18 in FIG. 3 corresponds to the recognition processing unit S4 in FIG. The process P19 corresponds to the result output of the natural language processing unit S7-A and the phonetic character processing unit S7-B in FIG.

  The speech signal feature extraction P17 of FIG. 3 corresponding to the feature extraction unit S2 of FIG. 1 obtains a speech feature component from the input speech data, performs speech analysis within the input speech data section using the speech feature component, and sets phoneme parameters. Extract feature information. The phoneme parameter feature information resulting from P17 is passed to the speech recognition P18.

  In the speech recognition P18, words, words, and sentences uttered in the patient observation information record by the lifesaving technician in the ambulance are registered in the vocabulary processing unit S6 in FIG. 1, and the phonemes are set so as to match the acoustic model S3 in FIG. Extract parameters. Then, the grammar processing unit S5 performs processing for converting the registered data so that it can be read on the application system side. When the phoneme parameter feature information resulting from P17 is passed, the acoustic model S3 in FIG. 1 and the vocabulary processing unit S6 are collated, and the approximate word list closest to the probability distribution can be selected, and the selected approximate word list is selected. Based on the natural language processing unit S7-A in FIG. 1 and the phonetic character processing unit S7-B in FIG. 1, the result is output as a recognition result value.

As described above in detail, according to the speech recognition method of the present invention, speech recognition is performed by recognizing the sound signal-to-noise level of the surrounding environment. Using a single-instruction multiple-data operation technique of MMX or wireless MMX instruction set, high-speed audio processing is realized by parallel processing of audio data elements on a desktop PC or PDA-based device. Specifically, on the PC, an inline code is configured in the C ++ programming language using the MMX instruction set. In the implementation of the algorithm for PDA, direct addressing is performed using the registers wR0 to wR15 of the processor, and the audio data elements are processed in parallel. A single instruction multiple data processing method is used to store multiple speech data elements and a processor processes multiple data elements simultaneously to optimize the performance of the speech recognition system on the PDA.

The invention's effect

As described above in detail, according to the present invention, an OS-based small information device typified by an embedded system uses a microphone signal pair in order to obtain a high signal-to-noise ratio when using a speech recognition function. Depends on the noise ratio function. As described in detail above, the situation recognition parameter of the input voice data is calculated and then distributed. A noise component corresponding to the type of background noise is removed by the noise removal process, and a speech segment is selected according to the type of background noise. The input speech is recognized by the recognition process with reference to the selected speech segment. Done. Therefore, when speaking in a background noise and various noise environments, high-accuracy speech recognition can be expected without depending on the type and performance of the microphone even if the signal-to-noise ratio due to noise has a low value.

Currently, OS-based small information devices represented by embedded systems rely on high signal-to-noise ratio microphones and noise canceling function chip sets to obtain a high signal-to-noise ratio when using a speech recognition function. . The present invention solves existing problems such as processing power limitation and data parallel processing that could not be implemented in all embedded systems from personal PDAs, professional PDAs to tablet PCs, and only software is optimized. It has the characteristics that can be realized in.

It is a figure which shows the speech recognition procedure in the speech recognition method of the Example of this invention. The situation recognition presumption process probe of this invention is shown. It is a functional block diagram of the speech recognition apparatus for implementing the speech recognition procedure of FIG.

[Figure 1]
S1 ... Situation recognition type pre-processing unit, S2 ... Feature extraction unit,
S3: Acoustic model, S4: Recognition processing unit,
S5: Grammar processing unit, S6: Vocabulary processing unit S7-A ... Natural language processing unit, S7-B ... Phonetic character processing unit [Fig. 2]
F1 ... A / D conversion unit, F2 ... Situation recognition variable calculation unit,
F3 ... interpreter, F4 ... situation recognition variable estimation part,
F5-A ... voice CODEC processing unit, F5-B ... error concealment processing unit,
F6 ... Situation recognition variable generation part, F7 ... Distribution part,
F8 ... voice extraction unit, F9 ... noise processing unit [Fig. 3]
P10: Microphone audio signal, P11: A / D conversion,
P12 ... Current frame generation, P13 ... Situation recognition measurement and calculation,
P14 ... Situation recognition judgment, P15 ... Voice segment extraction,
P16 ... noise removal and reduction, P17 ... voice signal feature extraction,
P18 ... voice recognition, P19 ... recognition result output processing

Claims (6)

Estimate situation awareness in all situations subject to speech recognition on embedded systems, generate dynamic situation recognition parameters, and use it to configure a distribution unit to reduce noise during speech extraction And performing the removal (hereinafter referred to as noise processing algorithm) to display the result obtained from the noise processing algorithm as a speech recognition result or to input the result as speech data. 2. The voice recognition method according to claim 1, wherein dynamic situation recognition parameters and variable noise processing reference parameters are generated when analog data is converted to digital data (hereinafter referred to as A / D conversion), and dynamic situation recognition is performed from the voice input stream. A speech recognition method, comprising: calculating a noise processing reference parameter of a situation recognition parameter under a flowing situation by comparing a parameter and a variable noise processing reference parameter. When generating dynamic situation recognition parameters at a pre-processing site in the situation recognition processing in the voice recognition method of claim 2, the data type is analyzed and classified from the voice input stream, and the noise processing reference parameter has variability. In addition, a voice recognition method characterized in that it can be recorded and used as a variable of a situation recognition parameter in which a noise processing standard is standardized in advance. 4. The speech recognition method according to claim 3, wherein a dynamic situation recognition parameter is generated according to the analysis result of the speech input stream, and a speech section and a non-speech section are determined by performing a comparison operation with the variable noise processing reference parameter according to the analysis result. A speech recognition method characterized by: 4. The speech recognition method according to claim 3, wherein hardware characteristics using dynamic situation recognition parameter information are detected according to the analysis result of the speech input stream, and the interpreter determines that speech CODEC processing is performed. 5. The voice recognition method according to claim 4, wherein the characteristics of the voice input stream taken over by the interpreter are detected, and voice CODEC processing and error concealment processing are performed.

Images