JP2005062572A - Speech recognition apparatus - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To provide a speech recognition apparatus for relatively highly exactly recognizing speech even when large loss is present in a sequence of feature parameters of a speech. <P>SOLUTION: The speech recognition apparatus includes an input buffer 30 which receives a speech data frame, a frame buffer 36 which stores frames in the order of frame numbers, a frame loss detection part 32 which detects the occurrence of frame loss, a feature parameter estimation part 34 which estimates feature data of frames as many as loss frames on the basis of feature data in the frame buffer 36 and frame order information and inserts the estimated feature data into the prescribed position in the frame buffer 36, and a speech recognition part 38 which reads frames out of the feature parameter estimation part 34 in order to perform speech recognition. <P>COPYRIGHT: (C)2005,JPO&NCIPI

Description

  The present invention relates to a speech recognition technology, and more particularly to a speech recognition apparatus capable of performing speech recognition with high accuracy even when packet loss occurs in a speech signal transmitted in a packet format after being converted into a feature vector.

  With the development of voice recognition technology and the spread of mobile terminals such as mobile phones and PDAs (Personal Digital Assistants), it is expected that voice recognition services using mobile terminals will be widely used in the future. On the other hand, available resources (processing capacity, power source) and the like are limited in portable terminals. It is desirable to suppress the power consumption or processing amount in the portable terminal and eliminate the influence on the voice codec processing. For this reason, the Distributed Telecommunications Recognition (DSR) has been standardized by the European Telecommunications Standards Institute (ETSI).

  In the DSR method, acoustic analysis processing is performed on a portable terminal, and analysis data is transmitted to a voice recognition server. The server executes speech recognition processing based on the analysis data.

  The functional configuration of the DSR system is shown in block diagram form in FIG. Referring to FIG. 8, this system is composed of a mobile terminal, and performs analysis of the sound of the input audio signal and transmits the encoded analysis data in packet format, and analysis of this packet format. A voice recognition server 182 that receives and decodes the data, and performs voice recognition on the decoded analysis data. The output of the speech recognition server 182 is given to other services (for example, translation service, automatic response service, etc.).

  The client terminal 180 performs acoustic analysis on the audio signal, extracts a feature parameter (feature data) of a predetermined format, and the feature parameter output from the feature parameter extraction unit 190 The compression unit 192 that performs compression processing, and the feature parameter compressed by the compression unit 192 (hereinafter referred to as “compression feature parameter”) is encoded with an error correction code or the like, and stored in the packet payload. And an encoding processing unit 194 for transmission.

  The speech recognition server 182 decodes the error correction code included in the payload of the received packet to restore the compression feature parameter, and decompresses the compression feature parameter restored by the decoding processing unit 200 Thus, a decompression processing unit 202 that restores the feature parameter of the acoustic analysis result and a recognition processing unit 204 that receives the feature parameter restored by the decompression processing unit 202 as input and performs speech recognition.

  With the recent spread of so-called Internet use, communication from the client terminal 180 to the voice recognition server 182 is often performed via an IP (Internet Protocol) network built on the Internet, and will be more generally used in the future. It seems to be.

  In the DSR service, when a voice conversation between a user and a server is assumed, a short propagation delay is desired. Therefore, it is considered that RTP (Real Time Protocol) / UDP (User Datagram Protocol) / IP which realizes real-time performance is suitable for transmission / reception of audio data in DSR. In the AETF (Audio / Visual Transport) working group of IETF (The Internet Engineering Task Force), which is an Internet technology standardization organization, a recommendation was made regarding the RTP packet configuration for DSR.

  However, in transmission / reception using RTP / UDP / IP, in order to ensure real-time performance, even if the packet does not reach the transmission destination for some reason, the packet is not retransmitted. For example, when a packet is congested, the router may discard the packet. In such a case, the packet is not retransmitted by RTP / UDP / IP. Therefore, packet loss occurs. When RTP / UDP / IP is used for DSR, voice data is lost due to packet loss. It is known that such packet loss occurs in a burst manner.

  Proposals aimed at solving this problem have been made in Non-Patent Documents 1 to 3 below.

Endo and Nakamura, “A Study on Data Complementation in a Distributed Recognition System”, Proc. Of Acoustical Society, 1-4-9, pp. 17-18, March 2003. Milner B.B. Semnani S. "Robust voice recognition on IP network", IEEE ICASSP proceedings, pp. 261-264, June 2000. (Millner, G. and Seminani S., “Robust spec recognition over IP networks”, Proc. IEEE ICASSP, pp. 1791-1794, June 2000. Milner B.B. , “Robust voice recognition in bursty packet loss”, IEEE ICASSP proceedings, pp. 261-264, May 2001 (Millner B., “Robust speech recognition in burst-like packet loss”, Proc. IEEE ICASSP, pp. 261-264, May 2001).

  In Non-Patent Documents 1 to 3, studies and experiments on a data complementing method for performing speech recognition using data obtained by complementing a section in which a packet loss occurs with an alternative value are performed. However, when the packet loss rate is large or the packet loss length is long, these methods cannot sufficiently compensate for the recognition degradation.

  Therefore, an object of the present invention is to provide a speech recognition apparatus capable of performing speech recognition with relatively high accuracy even when a large loss exists in a sequence of speech feature parameters.

  Another object of the present invention is to provide a server type speech recognition apparatus capable of performing speech recognition with relatively high accuracy even when a burst type packet loss occurs.

  A speech recognition apparatus according to a first aspect of the present invention includes a frame receiving unit for receiving a frame including voice feature data and frame order information indicating a temporal order of frames, and the frame receiving unit receives the frame. A frame storage means for storing the frame in association with the frame order information, and a frame receiving means for detecting that a frame loss has occurred based on the frame order information, and for detecting a frame lost due to the frame loss. Frame loss detecting means for detecting the frame position. The speech recognition apparatus further responds to the detection of the occurrence of the frame loss by the frame loss detection means, and stores the feature data of the frames as many as the number of frames detected by the frame loss detection means in the frame storage means. Based on the feature data and the frame order information included in the stored frame, individually estimated, a frame including the estimated feature data is generated, and the frame order information of the generated frame in the frame storage means is used. Feature data estimation means for inserting at a predetermined frame position, and voice recognition means for reading out the frames from the frame storage means in the order according to the frame order information and performing voice recognition on the feature data included in each frame. .

  Preferably, the feature data estimation means is connected to the first frame number storage means for storing the first frame number and the first frame number storage means, and among the frames stored in the frame storage means, A previous frame reading means for reading out the frame feature data of the first frame number before the frame loss detected by the frame loss detecting means from the frame storage means, and a first number read by the previous frame reading means Based on the feature data of the frame, an estimation unit for estimating the feature data included in each frame in the frame loss detected by the frame loss detection unit, and a frame including the estimated feature data are generated. Frame insertion for insertion at a frame position determined by the frame order information of the generated frame in the storage means And a stage.

  The feature data estimation means is further connected to the second frame number storage means for storing the second number of frames and the second frame number storage means. Among the frames stored in the frame storage means, the frame loss And a post-frame reading unit for reading out feature data of the second number of frames after the frame loss detected by the detection unit from the frame storage unit. The estimation means is detected by the frame loss detection means based on the feature data of the first number of frames read by the previous frame reading means and the feature data of the second number of frames read by the subsequent frame reading means. Means may be included for estimating feature data included in each frame in the generated frame loss.

  More preferably, the number of lost frames detected by the frame loss detection means is compared with a predetermined threshold value, and the first frame number stored in the first frame number storage means or the second Update means for updating the second frame number stored in the frame number storage means or both in accordance with a predetermined update method determined according to the comparison result.

  The update means compares the number of lost frames detected by the frame loss detection means with a predetermined threshold value, and compares the first frame number stored in the first frame number storage means, or the second A means for adding a predetermined constant determined according to the comparison result to the second frame number stored in the frame number storage means or both of them may be included.

  Preferably, the predetermined constant is a positive constant if the number of lost frames exceeds a threshold value, and is a negative constant otherwise. Alternatively, the predetermined constant is a negative constant when the number of lost frames exceeds a threshold value, and is a positive constant otherwise.

  Preferably, the means for estimating calculates feature data of the lost frame according to the following equation:

ただしＮ_f及びＮ_bはそれぞれ第１のフレーム数及び第２のフレーム数であり、Ｘ_t'f及びＸ_t'bは、フレームロス検出手段により検出されたフレームロスのそれぞれ前のＮ_f個及び後のＮ_b個の特徴データの平均からなる特徴データであり、t'_f及びｔ'_bはこれらＸ_t'f及びＸ_t'bに対応するフレーム順序情報を示し、Ｘ_t'f及びＸ_t'bは以下の様にして算出され、

However, N _f and N _b are the first frame number and the second frame number, respectively, and X _t′f and X _t′b are N _f frames before the frame loss detected by the frame loss detecting means, respectively. and a characteristic data consisting of the average of the N _b pieces of feature data after, t _'f and t' _b represents the frame sequence information corresponding to these X _T'f and _X t'b, X t'f and X _t'b is calculated as follows:

ただしｔ_f及びｔ_bはそれぞれフレームロスが生じた直前及び直後のフレームに対応する時刻を示す。

However, t _f and t _b indicate times corresponding to the frames immediately before and immediately after the occurrence of the frame loss, respectively.

  More preferably, the feature data estimation means stores the first frame number and the second frame number used for estimation by the feature data estimation means in association with the number of frames included in the frame loss. A first frame number and a second frame number corresponding to the number of frames included in the frame loss detected by the frame loss detection unit, connected to the table and the frame number table storage unit, and read out from the frame number table; Of the frames stored in the storage means, the feature data of the frame having the first frame number before the frame loss detected by the frame loss detecting means and the feature of the frame having the second frame number after the frame loss are detected. Frame reading means for reading data from the frame storage means, and reading by the frame reading means Estimating means for estimating feature data included in each frame in the frame loss detected by the frame loss detecting means based on the feature data of the first number of frames and the second number of frames including.

  The speech recognition apparatus according to the second aspect of the present invention receives a frame including speech feature data and frame order information indicating the temporal order of frames, and received by the frame receiving means. A frame storage means for storing the frame in association with the frame order information, and a frame receiving means for detecting that a frame loss has occurred based on the frame order information, and for detecting a frame lost due to the frame loss. Frame loss detection means for detecting a frame position; and voice recognition means for reading the frames from the frame storage means in the order according to the frame order information and performing voice recognition on the feature data included in each frame, Whether the voice recognition means has detected a frame loss by the frame loss detection means. According to whether, by selecting the method of calculating the output likelihood of each state is calculated the output likelihood, Hidden Markov Models: comprising means for recognizing speech by (Hidden Markov Model HMM).

  Preferably, the means for recognizing the voice by the HMM is when the frame loss is not detected by the frame loss detecting means.

によってＨＭＭの各状態Ｓ_tにおける出力尤度ｐ（Ｘ_t｜Ｓ_t）を算出し、ただしＭはＨＭＭの各ノードを構成するガウス混合分布の混合数を表し、ｗ_jは当該ガウス混合分布の混合要素ｊの混合重みを表し、ｔは順序情報を表し、Ｎ（Ｘ_t；μ_j，σ_j ²）はｔ番目のフレームＸ_tの入力特徴データに対する単変量ガウス分布関数を表し、混合要素ｊは分散σ_j ²及び平均μ_jを持ち、フレームロス検出手段によりフレームロスが検出されているときには

To calculate the output likelihood p (X _t | S _t ) in each state S _t of the HMM, where M represents the number of Gaussian mixture distributions constituting each node of the HMM, and w _j represents the Gaussian mixture distribution. Represents the mixing weight of the mixing element j, t represents order information, N (X _t ; μ _j , σ _j ² ) represents a univariate Gaussian distribution function for the input feature data of the t th frame X _t , and the mixing element j has variance σ _j ² and average μ _j , and when the frame loss is detected by the frame loss detection means

ただしＣは予め定められた定数、によりＨＭＭの各状態Ｓ_tにおける出力尤度ｐ（Ｘ_t｜Ｓ_t）を算出する。

However, C is a predetermined constant, and the output likelihood p (X _t | S _t ) in each state S _t of the HMM is calculated.

  Hereinafter, the first embodiment of the present invention, its modification, and the second embodiment will be described. For each embodiment, the configuration will be described first, and then the operation will be described. In both the first embodiment and the second embodiment, speech recognition is performed based on a theory called Missing Feature Theory (MFT). In the following description, it is assumed that feature parameters (feature data) necessary for speech recognition are transmitted by RTP / UDP. The characteristic parameters are configured in units of frames having a predetermined length (for example, 50 bytes), and a plurality of frames are stored in the UDP payload. A serial number is assigned to each RTP packet. The header of the UDP datagram stores the payload size of the packet.

[First Embodiment]
-Constitution-
FIG. 1 shows a block diagram of a speech recognition server 20 used in the server-client speech recognition system according to the first embodiment of the present invention. The voice recognition server 20 according to the first embodiment performs voice recognition when there is a packet loss using the data interpolation method in the MFT.

  Referring to FIG. 1, a speech recognition server 20 is connected to the Internet network. An input buffer 30 for receiving and temporarily storing packets transmitted with the speech recognition server 20 as a transmission destination, and an input buffer 30 And a frame buffer 36 for storing the frame of the characteristic parameter retrieved from the UDP in association with the frame number. The frame number is order information indicating the temporal order of frames. In the present embodiment, the frame buffer 36 stores frames in the order of frame numbers.

  The voice recognition server 20 further detects whether or not a packet loss has occurred in a series of packets received by the input buffer 30, and further calculates the position and number of frames lost due to the packet loss. And a frame loss detection unit 32 for outputting a frame loss detection signal indicating which frame is lost, and responding to the frame loss detection unit 32 detecting that there is a frame loss. Then, the feature parameter of each frame lost by the data interpolation method using the feature parameter included in the frame stored in the frame buffer 36 is estimated, and it corresponds to the lost frame in the frame buffer 36. A feature parameter for processing to insert a frame consisting of interpolated feature data at a predetermined position. Includes a meter estimator 34, the feature parameter stored in the frame buffer 36 is read sequentially and the speech recognition unit 38 for performing speech recognition. This voice recognition unit 38 may be the same as that used in the prior art.

  FIG. 2 is a detailed block diagram of the frame loss detection unit 32. Referring to FIG. 2, the frame loss detection unit 32 extracts the RTP header included in the UDP datagram temporarily stored in the input buffer 30, and checks the RTP sequence number to determine whether or not a packet loss has occurred. , And a lost packet number detector 50 for detecting the number of lost packets, and a payload size reading unit for reading the payload size from the UDP header in the UDP datagram temporarily stored in the input buffer 30 52.

  The frame loss detection unit 32 further determines the number of frames depending on the number of lost packets detected by the lost packet number detection unit 50, the payload size of the UDP datagram read by the payload size reading unit 52, and a predetermined frame length. And a lost frame number calculation unit 54 for calculating whether the packet has been lost due to packet loss. The loss frame number calculation unit 54 outputs the frame loss detection signal described above according to the calculation result.

  FIG. 3 is a detailed block diagram of the feature parameter estimation unit 34 shown in FIG. Referring to FIG. 3, feature parameter estimation unit 34 includes first and second frame number storage units 80 and 82 that store the number of frames before and after the frame loss, respectively, used for interpolation calculation. The first frame number storage unit 80 stores the number of frames before frame loss and used for interpolation calculation. The second frame number storage unit 82 stores the number of frames after frame loss and used for interpolation calculation.

  The feature parameter estimation unit 34 further receives the frame loss detection signal and the output of the first frame number storage unit 80, and is stored in the first frame number storage unit 80 immediately before the frame loss when a frame loss occurs. The previous frame reading unit 70 for reading the number of frames from the frame buffer 36 and the output of the frame loss detection signal and the second frame number storage unit 82 are also received. And a rear frame reading unit 72 for reading out the same number of frames stored in the second frame number storage unit 82 from the frame buffer 36.

  In response to the detection of the frame loss, the feature parameter estimation unit 34 further outputs the outputs of the first frame number storage unit 80 and the second frame number storage unit 82, and the previous frame reading unit 70 and the subsequent frame reading. An interpolation calculation unit 74 for receiving a frame read from the frame buffer 36 by the unit 72 and estimating a feature parameter of the lost frame by a calculation method described later, and a feature parameter estimated by the interpolation calculation unit 74 And an interpolation frame insertion processing unit 76 for performing processing for inserting the interpolation frame at a predetermined position in the frame buffer 36.

The number of frames stored in the first frame number storage unit 80 is N _f , and the number of frames stored in the second frame number storage unit 82 is N _b . In the present embodiment, both N _f and N _b are updated as follows according to the communication state. Let N _L be the number of frame losses. If this number N _L exceeds a certain threshold value S, a constant is added to both N _f and N _b . In the present embodiment, this constant is a positive constant 1. If N _L is less than or equal to the threshold value S, 1 is subtracted from both N _f and N _b . That is, a negative constant −1 is added. However, the minimum values of N _f and N _b are both 0.

For this purpose, the feature parameter estimation unit 34 includes a threshold value storage unit 84 for storing the threshold value S, a threshold value S stored in the threshold value storage unit 84, and a first frame number storage unit. An update processing unit 78 for updating N _f and N _{b in} accordance with the numbers N _f and N _b stored in the 80 and second frame number storage unit 82 and the number of frame losses represented by the frame loss detection signal; And a threshold value input unit 86 for manually updating the threshold value storage unit 84 stored in the threshold value storage unit 84.

Interpolation calculation performed by the interpolation calculation unit 74 will be described. This frame interpolation is performed for each element of the feature vector transmitted in the packet. In the following description, a vector x at time t _N in the feature vector stream is represented by x = {X _t1 , X _t2 ,..., X _tN }. It is assumed that the mth frame is lost (1 ≦ m ≦ N). m may be a plurality of consecutive ones.

There are many interpolation methods, but it is effective to perform data interpolation based on the received data. In this embodiment, data interpolation is performed by the method shown in FIG. Referring to FIG. 4, feature vector ^ Xtm of the lost frame by using the t _m which satisfies _{t 'f <tm <t'} b are estimated according to the following equation.

ただしＸ_t'f及びＸ_t'bは、失われた特徴ベクトルのそれぞれ前のＮ_f個及び後のＮ_b個の特徴ベクトルの平均ベクトルであり、t'_f及びｔ'_bはこれらＸ_t'f及びＸ_t'bに対応する時刻を示す。

However X _T'f and X _T'b is the mean vector of the N _b-number of feature vectors of the previous N _f-number and after each missing feature vector, t _'f and t' _b These X _{t The} time corresponding to _'f and X _t'b is shown.

X _t′f and X _t′b are calculated as follows.

ただしｔ_f及びｔ_bはそれぞれフレームロスが生じた直前及び直後のフレームに対応する時間を示す。

However t _f and t _b indicates the time corresponding to the immediately preceding and immediately following frame frame loss has occurred, respectively.

The example shown in FIG. 4 shows an example where N _f = N _b = 3. In FIG. 4, a solid line indicates a value of one element of the feature vector, and a cross indicates an average value of three frames before and after the loss frame. A circle indicates an estimated value calculated using equations (2) to (4).

-Operation-
The voice recognition system 10 shown in FIGS. 1 to 3 operates as follows. It is assumed that a predetermined threshold is set in the threshold storage unit 84 in advance. It is also assumed that predetermined values are set in advance in the first frame number storage unit 80 and the second frame number storage unit 82. In many cases, values updated at the time of the previous communication are set in the first frame number storage unit 80 and the second frame number storage unit 82. For example, a predetermined initial value is set in each time the power is turned on. You may make it do.

  The transmitted packet is temporarily stored in the input buffer 30. The payload size reading unit 52 of the frame loss detection unit 32 reads the payload size information from the UDP header and supplies it to the loss frame number calculation unit 54. Usually, the payload size is a fixed value.

  The lost packet number detection unit 50 reads the packet number from the RTP header in the series of UDP payloads, and determines whether or not there is a packet loss based on whether or not these numbers are continuous. When there is a packet loss, the lost packet number detection unit 50 calculates the number of lost packets and gives it to the lost frame number calculation unit 54.

  The loss frame number calculation unit 54 calculates the number of frames included in the payload of one UDP datagram based on the payload size given from the payload size reading unit 52 and a preset frame size. Furthermore, the lost frame number calculation unit 54 calculates the number of lost frames by multiplying the calculated number of frames by the number of lost packets, and provides the result to the feature parameter estimation unit 34 as a frame loss signal.

Referring to FIG. 3, update processing unit 78 of feature parameter estimation unit 34 calculates the number of lost frames N _L specified by the frame loss detection signal and threshold S stored in threshold storage 84. Compare. If N _L > S, the update processing unit 78 performs a process of adding 1 to the values N _f and N _b stored in the first frame number storage unit 80 and the second frame number storage unit 82, respectively. In other cases, the update processing unit 78 performs a process of subtracting 1 from each of the values N _f and N _b .

Based on the values N _f and N _b stored in the first frame number storage unit 80 and the second frame number storage unit 82, respectively, the front frame reading unit 70 and the rear frame reading unit 72 of the feature parameter estimation unit 34, respectively. Frames immediately before and after the lost frame are read from the frame buffer 36 (FIG. 1) in the number specified by N _f and N _b . The read frame is given to the interpolation calculation unit 74. The interpolation calculation unit 74 is given from the values N _f and N _b stored in the first frame number storage unit 80 and the second frame number storage unit 82, the previous frame reading unit 70, and the subsequent frame reading unit 72. Based on the information of the frame immediately before and after the loss frame, each element of the feature vector of the loss frame is calculated according to Equation (1). The interpolation calculation unit 74 gives an estimated loss frame feature vector composed of the calculated elements to the interpolation frame insertion processing unit 76.

  The interpolation frame insertion processing unit 76 updates the contents of the frame buffer 36 so that the feature vector of the loss frame calculated by the interpolation calculation unit 74 is inserted at the frame position specified by the frame loss detection signal.

  The speech recognition unit 38 shown in FIG. 1 performs speech recognition by sequentially reading out feature vectors included in a frame from the frame buffer 36 and giving them to the HMM.

It is assumed that the voice recognition unit 38 is a continuous density HMM (CDHMM). State S _t lost frame is calculated using a feature vector ^ X _t estimated by interpolation calculation unit 74. Therefore, the likelihood function of the node S _t of the HMM is given by the following equation.

ただしＭはガウス混合分布の混合数を表し、ｗ_jは混合要素ｊの混合重みを表し、Ｎ（Ｘ_t；μ_j，σ_j ²）はｔ番目のフレームＸ_tの入力特徴量に対する単変量ガウス分布関数を表し、混合要素ｊは分散σ_j ²及び平均μ_jを持つものとする。

Where M represents the number of mixtures in the Gaussian mixture distribution, w _j represents the mixing weight of the mixing element j, and N (X _t ; μ _j , σ _j ² ) is a univariate with respect to the input feature quantity of the t-th frame X _t. It represents a Gaussian distribution function, and the mixing element j has a variance σ _j ² and an average μ _j .

  According to the apparatus of the first embodiment, even when packet loss occurs and a plurality of frames are lost, the feature vector of the lost frame is estimated by the feature parameter estimation unit 34, and the estimated frame is used as the feature parameter. It is inserted at the position of the loss frame in the estimation unit 34. The voice recognition unit 38 simply reads the frames in order from the frame buffer 36 and performs voice recognition. Therefore, voice recognition can be performed even when packet loss occurs without changing the configuration of the voice recognition unit 38 from the conventional one. Further, as will be described later, the accuracy is high, and voice recognition that is more robust than the conventional one can be realized.

In the system of the embodiment described above, the updating of the value N _f and N _b, the value to be added or subtracted is limited to 1. By doing so, it is possible to prevent the voice recognition from becoming unstable due to the fluctuation of the values N _f and N _b due to the change in the number of lost packets. However, this value is not limited to 1, and an appropriate value may be selected according to the application. The values N _f and N _b may not be updated and may be fixed values (for example, the value N _f = N _b = 1).

In the system according to the above-described embodiment, the loss frame is estimated based on the internal division using the frames before and after the loss frame. However, in this case, since information of the frame after the loss frame is required, it takes time to estimate and delays speech recognition. Therefore, when it is necessary to make speech recognition as fast as possible, it is conceivable to fix the value of N _b to 0. By extrapolating the value of the loss frame from the data of a plurality of frames before the loss frame, the same result as that obtained using Equation (1) can be obtained.

In the above description, if the number N _{L of} lost frames exceeds the threshold S, 1 is added to both N _f and N _b , and if N _L is equal to or less than the threshold S, N _f and 1 is subtracted from both of N _b . This is a method that attaches importance to increasing the accuracy of estimation. However, the method of determining the numbers of N _f and N _b is not limited to this. For example, when the real-time property of processing is more important than the accuracy of estimation, if the number N _{L of} lost frames exceeds the threshold value S, 1 is subtracted from both N _f and N _b , and N _L If N exceeds the threshold value S, 1 may be subtracted from both N _f and N _b .

[Modification of First Embodiment]
In the system according to the first embodiment, the values of N _f and N _b stored in the first frame number storage unit 80 and the second frame number storage unit 82 are updated according to the number of packets. However, the present invention is not limited to such an embodiment, and it may be possible to predetermine the values of N _f and N _b according to the number of packet losses N _L. For this purpose, the values of N _f and N _b for N _L may be stored in advance in a table. A block diagram of the feature parameter estimation unit 120 used in such a system is shown in FIG. The interpolation calculation unit 120 can be used in place of the feature parameter estimation unit 34 shown in FIG.

Referring to FIG. 5, the feature parameter estimation unit 120 receives a table 130 for storing the frame numbers N _f and N _b before and after the frame loss with respect to the packet loss number N _L and the frame loss detection signal. When this occurs, the number N _f corresponding to the number N _L of lost frames is read from the table 130, and the previous frame reading unit 132 for reading the frame immediately before the frame loss corresponding to the number N _f from the frame buffer 36. Similarly, when a frame loss occurs upon receiving a frame loss detection signal output, the number N _b corresponding to the number N _L of lost frames is read from the table 130, and the frame immediately after the frame loss corresponding to the number N _b And a rear frame reading unit 134 for reading from the frame buffer 36.

In response to the detection of the frame loss, the feature parameter estimation unit 120 further reads out the numbers N _f and N _b corresponding to the number N _L of lost frames from the table 130, and further, the previous frame reading unit 132 and the subsequent frame An interpolation calculation unit 136 for receiving a frame read from the frame buffer 36 by the reading unit 134 and estimating a feature parameter of the lost frame by a calculation method similar to Equation (1), and estimation by the interpolation calculation unit 136 And an interpolated frame insertion processing unit 76 for performing processing for inserting an interpolated frame composed of the characteristic parameters into a predetermined position in the frame buffer 36.

The feature parameter estimation unit 120 operates in the same manner as in the first embodiment except for how to determine the values of N _f and N _b .

In this modification, the relationship between the number N _L of lost packets and the values N _f and N _b for internal division is fixed. Therefore, the relationship itself cannot be changed dynamically as in the first embodiment. However, it is effective when it is possible to predict in advance the relationship between the occurrence of packet loss and the values N _f and N _b . Further, in this modified example, since the response time is constant, it is effective for stably performing speech recognition with a constant accuracy.

[Second Embodiment]
In the system according to the first embodiment, the feature vector included in the loss frame is estimated from the feature vectors of the frames before and after the loss frame group. Another method of speech recognition when there is a frame loss is called a marginalization method. The system according to the second embodiment of the present invention uses a marginalization method.

  In the marginalization method, when a part of audio data is lost, recognition is performed by manipulating the output likelihood in the HMM without using the lost data. In order to realize this, the point that the voice recognition server needs to have a function of detecting a frame loss is the same as in the system of the first embodiment.

  FIG. 6 shows a block diagram of a speech recognition server 140 used in the server-client speech recognition system according to the second embodiment. With reference to FIG. 6, the speech recognition server 140 includes an input buffer 30, a frame loss detection unit 32, and a frame buffer 36 similar to the speech recognition server 20 of the first embodiment. Furthermore, unlike the speech recognition server 20 of the first embodiment, the speech recognition server 140 directly receives the frame loss detection signal output from the frame loss detection unit 32, and without estimating the feature vector of the lost frame. A speech recognition unit 150 that performs speech recognition using a marginalization method is included.

  In FIG. 6, the same components as those in FIG. 1 are denoted by the same reference numerals. Their names and functions are also the same. Therefore, detailed description thereof will not be repeated here.

In speech recognition by merging internalization technique, the output likelihood p nodes S _t of the HMM by the following equation | Request (X _t S _t).

ただしＭはガウス混合分布の混合数を表し、ｗ_jは混合要素ｊの混合重みを表し、Ｎ（Ｘ_t；μ_j，σ_j ²）はｔ番目のフレームＸ_tの入力特徴量に対する単変量ガウス分布関数を表し、混合要素ｊは分散σ_j ²及び平均μ_jを持つものとする。

Where M represents the number of mixtures in the Gaussian mixture distribution, w _j represents the mixing weight of the mixing element j, and N (X _t ; μ _j , σ _j ² ) is a univariate with respect to the input feature quantity of the t-th frame X _t. It represents a Gaussian distribution function, and the mixing element j has a variance σ _j ² and an average μ _j .

When there is no frame loss, the output likelihood of each state of the HMM is calculated using the first equation above Equation (5). If there is a frame loss, there is no _{Xt in} the first equation, so that the output likelihoods in all states are set to the same value “C” as indicated by the second equation in equation (5). Thus, when there is a frame loss, the state transition depends only on the state transition probability learned in advance.

  A configuration of the speech recognition unit 150 is schematically shown in FIG. Referring to FIG. 7, the speech recognition unit 150 includes an HMM 160 and an output likelihood calculation unit 164 that calculates the output likelihood of each state of the HMM 160 using the first equation of the equation (5) shown above. The constant storage unit 166 that stores the constant C, and the output likelihood calculation unit 164 when there is no packet loss, and the output C of the constant storage unit 166 when there is a packet loss, each output likelihood of each state. And a selection unit 162 that controls the HMM 160 so as to calculate the degree.

  When the frame loss detection signal is a value indicating frame loss detection, the selection unit 162 sets the output of the constant storage unit 166 as each output likelihood of the HMM 160. When no frame loss is detected, the selection unit 162 uses the calculation result of the output likelihood calculation unit 164 as the output likelihood of each state of the HMM 160.

  The voice recognition unit 150 enables voice recognition by the above-described marginalization.

[Experimental result]
An experiment was conducted to examine the result of speech recognition when a frame loss occurred using the system of the first embodiment and the system of the second embodiment. In the experiment, the random loss model assumed that packet loss occurs randomly, and the Gilbert loss model obtained by determining the transition probability between the normal state and the loss state, the packet loss rate and the average burst loss length The tendency of word recognition rate was investigated. In order to simplify the experiment, it was assumed that one frame was stored in one packet.

In the experiment of the first embodiment, for the sake of simplicity, an experiment in which N _f = 1 and N _b = 0 were fixed and an experiment in which N _f = 1 and N _b = 1 were fixed were performed.

  For comparison, in the first embodiment, instead of calculating a feature vector, an average of data used at the time of HMM learning is obtained in advance, and this average vector is used as lost frame data as an HMM. We also conducted an experiment to perform speech recognition. We consider the experimental results using this as a baseline.

  As a result, as the average burst length increased, the word recognition rate decreased in any of the experiments described above. However, the word recognition rate when word recognition according to the present invention was performed was significantly higher than the baseline result in all cases. The difference increases as the packet loss rate increases. Moreover, the word recognition rate by the marginalization method (2nd Embodiment) exceeded all the others. Therefore, it is considered that the marginalization method is more robust against burst packet loss than other methods.

  The embodiment disclosed herein is merely an example, and the present invention is not limited to the above-described embodiment. The scope of the present invention is indicated by each claim in the claims after taking into account the description of the detailed description of the invention, and all modifications within the meaning and scope equivalent to the wording described therein are intended. Including.

It is a block diagram of the speech recognition server which concerns on the 1st Embodiment of this invention. FIG. 2 is a detailed block diagram of a frame loss detection unit 32 shown in FIG. FIG. 2 is a detailed block diagram of a feature parameter estimation unit 34 shown in FIG. 1. It is a figure for demonstrating the estimation process of the feature vector performed in the feature parameter estimation part. It is a block diagram of the modification of the speech recognition server of 1st Embodiment. It is a block diagram of the speech recognition server which concerns on the 2nd Embodiment of this invention. FIG. 7 is a detailed block diagram of the voice recognition unit 150 shown in FIG. 6. It is a block diagram which shows the structure of the conventional server-client type | mold speech recognition system.

Explanation of symbols

  20, 140, 182 Speech recognition server, 30 input buffer, 32 frame loss detection unit, 34, 120 feature parameter estimation unit, 36 frame buffer, 38, 150 speech recognition unit, 50 lost packet number detection unit, 52 payload size reading unit , 54 Loss frame number calculation unit, 70, 132 Previous frame reading unit, 72, 134 Post frame reading unit, 74, 136 Interpolation calculation unit, 76 Interpolation frame insertion processing unit, 78 Update processing unit, 80 First frame number storage Part 80, 82 second frame number storage part, 130 table

Claims (11)

Frame receiving means for receiving a frame including voice feature data and frame order information indicating a temporal order of frames;
Frame storage means for storing the frame received by the frame receiving means in association with frame order information;
Frame loss detecting means connected to the frame receiving means, detecting that a frame loss has occurred based on the frame order information, and detecting a frame position of a frame lost due to the frame loss;
In response to the occurrence of frame loss detected by the frame loss detection means, frame feature data corresponding to the number of frames detected by the frame loss detection means is stored in the frame storage means. A frame determined based on the frame order information of the generated frame in the frame storage means, which is estimated based on the feature data included in the existing frame and the frame order information, generates a frame including the estimated feature data Feature data estimation means for insertion at a position;
A speech recognition apparatus comprising: speech recognition means for reading out frames from the frame storage means in an order according to frame order information and performing speech recognition on feature data included in each frame. The feature data estimation means includes:
First frame number storage means for storing the first frame number;
Of the frames stored in the frame storage means and connected to the first frame number storage means, the feature data of the frame having the first frame number before the frame loss detected by the frame loss detection means is obtained. Previous frame reading means for reading from the frame storage means;
Estimation means for estimating feature data included in each frame in the frame loss detected by the frame loss detection means based on the feature data of the first number of frames read by the previous frame reading means When,
The speech recognition apparatus according to claim 1, further comprising: a frame insertion unit for inserting the estimated frame at a frame position determined by frame order information in the frame storage unit. The feature data estimation means further includes
Second frame number storage means for storing a second frame number;
Of the frames connected to the second frame number storage means and stored in the frame storage means, the characteristics of the second number of frames after the frame loss detected by the frame loss detection means Post-frame reading means for reading data from said frame storage means,
The estimating means is based on the feature data of the first number of frames read by the previous frame reading means and the feature data of the second number of frames read by the subsequent frame reading means. The speech recognition apparatus according to claim 2, comprising means for estimating feature data included in each frame in the frame loss detected by the frame loss detection means. The number of lost frames detected by the frame loss detection means is compared with a predetermined threshold value, and the first frame number stored in the first frame number storage means or the second The speech recognition apparatus according to claim 3, further comprising an updating unit configured to update the second frame number stored in the frame number storage unit or both according to a predetermined updating method determined according to the comparison result. The update means compares the number of lost frames detected by the frame loss detection means with a predetermined threshold value, and the first frame number stored in the first frame number storage means Or a means for adding a predetermined constant determined according to a comparison result to the second frame number stored in the second frame number storage means, or both, and updating it. The speech recognition apparatus according to the description. The speech recognition apparatus according to claim 5, wherein the predetermined constant is a positive constant when the number of lost frames exceeds the threshold value, and is a negative constant otherwise. . The speech recognition apparatus according to claim 5, wherein the predetermined constant is a negative constant when the number of lost frames exceeds the threshold value, and is a positive constant otherwise. . The means for estimating calculates feature data of a lost frame by the following equation:

However, N _f and N _b are the first frame number and the second frame number, respectively, and X _t′f and X _t′b are respectively before the frame loss detected by the frame loss detecting means. The feature data is an average of N _f feature data and the subsequent N _b feature data, t ′ _f and t ′ _b indicate frame order information corresponding to these X _t′f and X _t′b , and X _{t 'f} and X _t'b are calculated as follows:

However t _f and t _b indicates time corresponding to the immediately preceding and immediately following frame frame loss has occurred, respectively, the speech recognition apparatus according to claim 3. The feature data estimation means includes:
A frame number table for storing the first frame number and the second frame number used for estimation by the feature data estimation unit in association with the number of frames included in a frame loss;
The first frame number and the second frame number corresponding to the number of frames included in the frame loss detected by the frame loss detection unit and connected to the frame number table storage unit are read from the frame number table. , Out of the frames stored in the frame storage means, feature data of the first number of frames before the frame loss detected by the frame loss detection means, and the second feature data after the frame loss Frame reading means for reading frame feature data of the number of frames from the frame storage means;
Included in each frame in the frame loss detected by the frame loss detecting means based on the feature data of the first number of frames and the second number of frames read by the frame reading means The speech recognition apparatus according to claim 1, comprising estimation means for estimating feature data. Frame receiving means for receiving a frame including voice feature data and frame order information indicating a temporal order of frames;
Frame storage means for storing the frame received by the frame receiving means in association with frame order information;
Frame loss detecting means connected to the frame receiving means, detecting that a frame loss has occurred based on the frame order information, and detecting a frame position of a frame lost due to the frame loss;
Voice recognition means for reading out frames from the frame storage means in an order according to frame order information and performing voice recognition on feature data included in each frame;
The speech recognition unit selects a method for calculating an output likelihood of each state according to whether or not a frame loss is detected by the frame loss detection unit, and calculates an output likelihood. A hidden Markov model (HMM) A speech recognition apparatus comprising means for recognizing speech by means of Means for recognizing speech by the HMM are:
When no frame loss is detected by the frame loss detection means

The output likelihood p in each state S _t of the HMM by | calculates (X _t S _t), where M represents the number of mixtures Gaussian mixture that constitutes each node of the HMM, w _j is the Gaussian mixture Represents the mixing weight of the mixing element j of the distribution, t represents order information, N (X _t ; μ _j , σ _j ² ) represents a univariate Gaussian distribution function for the input feature data of the t th frame X _t , The mixing element j has a variance σ _j ² and an average μ _j ,
When a frame loss is detected by the frame loss detection means

However C is predetermined constant, the output likelihood p in each state S _t of the HMM | calculates the (X _t S _t), the speech recognition apparatus according to claim 10.

Images