KR20100073178A - Speaker adaptation apparatus and its method for a speech recognition - Google Patents

Abstract

PURPOSE: A speaker adaptation apparatus and a method for speech recognition are provided to remarkably improve the performance of a speaker by estimating the answer of actual speech data in high possibility through an N-best recognition result screen output function. CONSTITUTION: A voice data verification unit(202) obtains measurement data for each phoneme with regard to accumulation data through reliability evaluation. The accumulation data includes voice data and N-best recognition result data. A sound model speaker adapting unit(204) performs speaker adaptation by measurement data for each acquired per-phoneme. A sound model updating unit(206) updates a sound model by a new speaker-subordinate sound model through performed speaker adaptation.

Description

SPEAKER ADAPTATION APPARATUS AND ITS METHOD FOR A SPEECH RECOGNITION}

The present invention relates to a speaker adaptation technique for speech recognition. More particularly, the present invention relates to a speaker adaptation apparatus and method for speech recognition suitable for performing speaker adaptation using speech data input from a navigation device having a speech recognition function. It is about.

The present invention is derived from research conducted as a part of the core technology development project of IT growth engines of the Ministry of Knowledge Economy and ICT. [Task management number: 2006-S-036-03, Task name: Large-capacity interactive for new growth engine industry] Development of distributed processing voice interface technology].

As is well known, since a speech recognition system performs speech recognition for an unspecified speaker, it collects voice data from many training speakers and trains a speaker-independent acoustic model, which is called a speaker-independent speech recognition system.

In general, however, a specific speaker continues to use the speech recognition system. A speaker-dependent speech recognition system using a speaker-dependent acoustic model trained with speech data of a specific speaker is relatively superior to that of a speaker-independent speech recognition system.

Thus, there is a need for a speaker adaptation method that converts a speaker independent acoustic model from a speaker independent acoustic model to a speaker dependent acoustic model using a specific speaker's voice.

In particular, in case of a voice recognition service that does not know which speaker to use, the speaker adapts using the speaker's voice signal in the service situation. In this case, the user does not know what the voice is spoken. Speaker adaptation is performed, which results in a slight decrease in speaker adaptation performance.

On the other hand, if a predetermined speaker, such as a navigation device having a speech recognition function, uses the speech recognition system, the speaker adaptation can be performed to obtain an improved performance of the speech recognition system.

Conventionally, methods for performing speaker adaptation for speech recognition include MAP (maximum a posteriori), MLLR (maximum likelihood linear regression), Eigenvoice, and the like. Depending on the amount of data, the appropriate speaker adaptation method should be selected.

Here, the Eigenvoice speaker adaptation method is a fast speaker adaptation method that adapts closely to the speaker-dependent acoustic model even if the user uses less speaker-adapted speech data that is not inconvenient for the user. In many cases, the MAP speaker adaptation method is known to be close to the speaker-dependent acoustic model, and the MLLR speaker adaptation method is expected to achieve optimal performance when using intermediate observation data between the two methods. That's how it can be.

As described above, speaker adaptation methods have been proposed according to the amount of observation data for improving speech recognition performance, and various speaker adaptation techniques have been continuously developed to further improve the speech recognition performance.

Accordingly, the present invention stores the speech data and the corresponding recognition result data as cumulative data, verifies the reliability thereof, accumulates and stores the observation data for each phoneme, and performs speaker adaptation using the observation data for each phoneme. The present invention provides a speaker adaptation apparatus and method for speech recognition.

According to an aspect of the present invention, there is provided a voice data verification unit which obtains observation data for each phoneme of the cumulative data by verifying the reliability of the cumulative data including the voice data and the N best recognition result data corresponding thereto. Speech recognition includes a sound model speaker adaptor for performing speaker adaptation using the acquired phoneme-specific observation data, and an acoustic model updater for updating the acoustic model with a new speaker-dependent acoustic model generated through the speaker adaptation. It provides a speaker adaptation device.

In another aspect, the present invention, the step of extracting the cumulative data including the voice data and the N-best recognition result data corresponding to the verification, by verifying the reliability of the extracted cumulative data by the phoneme observation of the cumulative data Acquiring data, extracting the cumulative data, and acquiring observation data for each phoneme by repeating all the cumulative data, accumulating the observation data for each phoneme, and accumulating the observation data for each phoneme. It provides a speaker adaptation method for speech recognition comprising the step of performing a speaker adaptation using the; and updating the acoustic model with a new speaker-dependent acoustic model generated by the speaker adaptation.

According to the present invention, it is possible to estimate the correct answer of actual speech data with high possibility through N-best best recognition result display function according to speech recognition, thereby greatly improving the performance of non-speaker adaptation. It can increase.

In addition, it is possible to verify the case where there is a difference between the recognition result and the actual voice data determined by the user selection process of the N-Best recognition result through the reliability verification process for the voice data.

In this way, the speaker who is closer to the performance of the supervised mode speaker adaptation method than the unsupervised mode speaker adaptation method through N-Best recognition result screen output and user selection and verification of the voice data. Adaptive performance improvements can be obtained.

According to the present invention, in a state where cumulative data including voice data and N-best recognition result data corresponding thereto are stored, reliability verification for each cumulative data is performed, and accordingly, the observation data for each phoneme is accumulated and stored accordingly. Speaker adaptation is performed using star observation data to generate a new speaker-dependent acoustic model, and replace and update the current acoustic model with the generated new speaker-dependent acoustic model. Can be.

Hereinafter, embodiments of the present invention will be described in detail with reference to the accompanying drawings.

1 is a block diagram of a navigation device suitable for providing a navigation service corresponding to input voice data using a voice recognition function according to an embodiment of the present invention, which includes an input unit 102, a control unit 104, and a GPS receiver ( 106, a sensor 108, a storage 110, an image processor 112, a display 114, a voice processor 116, a speaker 118, a voice input unit 120, a voice recognition unit 122, and the like. It may include.

Referring to FIG. 1, the input unit 102 generates various operation signals for controlling the operation of the navigation apparatus by using a keypad, a touch screen, and the like. A request operation signal is provided to the control unit 104, and an input operation signal such as voice input mode activation, destination input, and route search is generated and provided to the control unit 104.

In addition, the control unit 104 performs overall operation control of the navigation device including a microprocessor and the like, and is equipped with a speaker adaptation device for speech recognition, and according to various operation signals provided from the input unit 102, the GPS receiver Map the position value of the navigation device (i.e., the mobile device on which the navigation device is mounted) provided from the 106 with the map data stored in the storage unit 110, and the navigation device (or the navigation device provided from the sensor unit 108) The sensing value including the rotation angle and the speed of the mounted moving object) is mapped to the map information data to provide the mapped navigation data (navigation information) to the image processing unit 112, and to display the mapped navigation data on the screen. Providing a control signal for outputting a signal, a voice guide signal through the speaker 118, to the voice processing unit 116, and the like. Performance.

In addition, when the navigation service request operation signal is provided from the input unit 102 in the normal mode, the controller 104 activates each component to operate the navigation service, thereby generating and displaying a menu screen for inputting a destination. When the control signal is provided to the image processor 112, and a destination input manipulation signal is provided from the input unit 102, the GPS receiver may provide a control signal for searching for an optimum path (that is, a destination path) between the current position of the moving object and the destination. 106, and maps the position value of the moving object provided from the GPS receiver 106 with the map information data stored in the storage unit 110 to search for and display an optimum path from the current position to the destination. Provided to the image processing unit 112 and displaying a driving route on an optimal route searched according to a sensing value from the sensor unit 108. The navigation data is provided to the image processor 112.

On the other hand, in the case of the voice input mode, when the voice input mode activation operation signal is provided from the input unit 102, the controller 104 activates the voice input mode and then operates the navigation function according to the voice input. The control signal for generating and displaying a menu screen for the display is provided to the image processing unit 112. When the voices input from the voice input unit 120 are recognized through the voice recognition unit 122 and N recognition results are transmitted, After detecting N (N is an integer) recognition result candidates corresponding to the voice in the storage unit 110, a control signal for displaying the result is provided to the image processing unit 112, and the recognition result candidates are input from the input unit 102. If a specific candidate selection operation signal is provided during the operation, the selected speech result and the selected recognition result data are stored in the storage unit 110 as cumulative data, and then the selected in According to the result destination, the GPS receiver 106 provides a control signal for searching for the position of the current moving object and the optimum path (i.e., the destination path) between the destinations, and stores the position value of the moving object provided from the GPS receiver 106. Mapping with the map information data stored in the unit 110 to provide a control signal to the image processing unit 112 to search for and display the optimal path from the current location to the destination, according to the sensing value from the sensor unit 108 The navigation data indicating the driving route is provided to the image processor 112 on the found optimal route. Here, the storing of the accumulated data may be performed for a predetermined number of times (for example, 20 times, 30 times, etc.).

When the cumulative data is stored a predetermined number of times, the controller 104 extracts each cumulative data using the on-board speaker adaptation apparatus, and verifies the accumulated cumulative data based on the reliability evaluation. After accumulating the observation data for each phoneme, the speaker adaptation is performed using the accumulated phoneme observation data, and a new speaker adaptation acoustic model is generated accordingly. The new speaker adaptation acoustic model is used as an acoustic model for speech recognition. By updating the acoustic model by applying, it is then controlled to perform the voice input mode using speech recognition through the updated new speaker adaptive acoustic model.

Next, the GPS receiver 106 calculates a position value of the navigation device by using satellite signals received through an antenna from a plurality of satellites, and transmits the position value to the controller 104.

The sensor unit 108 includes a gyro sensor, a speed sensor, and the like, and senses and calculates a rotation angle, a speed, and the like of the navigation device (or a moving body equipped with the navigation device), and senses the rotation angle, the speed, and the like. It passes to the control unit 104.

In addition, the storage unit 110 is a storage medium including a hard disk drive (HDD), and the like, various map information for providing a route search and navigation service, various voice guide information for providing a voice guidance, and voice. Various application programs for operation of the speaker adaptation apparatus for speech recognition including a recognition module, an acoustic model, and the like are stored and managed, and these are extracted and provided to the controller 104 as necessary.

Meanwhile, the image processing unit 112 performs digital signal processing on an image signal including an MPEG codec and the like. When the navigation data indicating an optimum path or a driving path is provided from the control unit 104, the image processing unit processes the image signal to display the LCD. The display unit 114 displays a selection menu screen for the N recognition result candidates provided from the control unit 104 and displays the selection menu screen through the display unit 114.

In addition, the voice processing unit 116 performs digital signal processing on a voice signal including an MPEG codec, and the like, and when the voice guidance data is provided from the control unit 104 at a point of time during the navigation service, the voice signal processing is performed. To output through the speaker 118.

Next, the voice input unit 120 inputs a voice including a microphone and the like, and inputs a user voice in the voice input mode to transfer the voice data to the voice recognition unit 122.

The voice recognition unit 122 recognizes the voice data transmitted from the voice input unit 120 using the voice recognition module and transmits the recognition result to the control unit 104.

Next, in the navigation device having the above-described configuration, in the state in which the input voice data and the corresponding recognition result data are accumulated for a predetermined number of times, each accumulated data is extracted, and for each of the extracted accumulated data, After accumulating the observation data for each phoneme by verifying it based on the reliability evaluation, the speaker adaptation is performed using the accumulated phoneme observation data, and a new speaker adaptation acoustic model is generated accordingly. A speaker adaptation apparatus for speech recognition, which is applied as an acoustic model for recognition and updates the acoustic model, will now be described.

FIG. 2 is a block diagram of a speaker adaptation apparatus for speech recognition suitable for generating and updating a cumulative data including speech data and recognition result data into a new speaker-dependent acoustic model according to an embodiment of the present invention. The verification unit 202, the acoustic model speaker adaptor 204, and the acoustic model updater 206 may be included.

Referring to FIG. 2, the speech data verification unit 202 verifies whether the accumulated speech data is correct by accumulating the (voice) recognition result data. After extracting the voice data and the corresponding recognition result data from the storage 110, reliability measurement is performed using the accumulated voice data and the recognition result data. Here, the reliability measurement is generally evaluated as the recognition result phoneme string data obtained by converting the given speech data and the recognition result word into a phoneme string. The reliability of the voice data and the phoneme may be defined as reliability and calculated.

The voice data verification unit 202 compares the reliability measured through the reliability measurement with a preset threshold. The reason for performing this comparison process is that even if the user selects a correct answer among N recognition results (ie, N-best recognition results), the selected destination name may not be the same as the actual voice. For example, the user spoke with "Seoul National University" because the user could select "Seoul National University" among N recognition results. In this case, the speaker adaptation using "Seoul National University" as the correct reference for the "Seoul National University" speech was performed. In this case, the performance may be deteriorated because it is adapted to the wrong data. Therefore, by performing the above verification operation on the accumulated speech data and the recognition result data, the speech recognition performance may be improved.

Here, the voice data verification unit 202 does not perform speaker adaptation on the corresponding data when the measured reliability is smaller than the preset threshold (that is, when the voice data and the recognition result data do not completely match), and accumulate the speaker. The reliability measurement is performed on the next voice data among the received voice data (ie, the reliability measurement process is performed on the next cumulative data).

In addition, when the measured reliability is greater than a preset threshold, the voice data verification unit 202 acquires and accumulates the observation data for each phoneme using phoneme strings corresponding to the corresponding voice data and the recognition result data. That is, observation data for each voice section may be detected by performing forced alignment on a given phoneme string (ie, a phoneme string obtained through a comparison of reliability), and the corresponding data may be accumulated for each phoneme.

Next, when the accumulation process of the phoneme-based observation data is completed, the voice data verification unit 202 determines whether the reliability measurement and the phoneme-based observation data accumulation process have been completed for all the accumulated data. If this is not completed, the reliability measurement is performed on the next cumulative data. When all the cumulative data verification is completed, the accumulated phoneme-specific observation data is transmitted to the acoustic model speaker adaptor 204.

In addition, the acoustic model speaker adaptor 204 performs speaker adaptation using speech data having high reliability according to the speech recognition result data as correct reference data, and performs speaker adaptation using the observed phonemic data. Create a new speaker-dependent acoustic model.

Next, the acoustic model updating unit 206 replaces and updates the current acoustic model with the new speaker dependent acoustic model, and updates the acoustic model by replacing the current acoustic model with the generated new speaker dependent acoustic model.

Next, after recognizing the voice input in the voice input mode of the navigation device having the above-described configuration, a plurality of voice recognition result candidates are detected and displayed, and among the voice recognition data selected among the voice recognition result data. A process of storing cumulative data by repeating the above-described process a predetermined number of times will be described.

3 is a flowchart illustrating a process of storing cumulative data for speaker adaptation according to an embodiment of the present invention.

Referring to FIG. 3, in the voice input mode of the navigation device (step 302), the navigation device operates a navigation function according to a voice input, and thus, for example, a menu screen for inputting a destination is displayed. The input unit 120 checks whether a voice for inputting a destination is input, for example (step 304).

As a result of the check in the step 304, if a voice is input, the voice input unit 120 transmits the input voice to the voice recognition unit 122, and the voice recognition unit 122 recognizes the input voice and N Two recognition results (for example, text data corresponding to voice data, etc.) are transmitted to the control unit 104 (step 306).

In addition, the controller 104 detects N (N is an integer) recognition result candidates corresponding to the N recognition results from the storage 110, and then provides a control signal for displaying them to the image processor 112. In addition, the image processor 112 constructs a recognition result menu screen including the recognition result candidate and displays the same through the display unit 114 (step 308). As an example, FIG. 5 is a diagram illustrating a recognition result menu screen including a recognition result candidate corresponding to input voice data according to an embodiment of the present invention, in which various recognition result candidates are displayed in response to the input voice. It can be seen that.

Next, the control unit 104 checks from the input unit 102 whether a specific candidate selection operation signal is provided among these recognition result candidates (step 310).

As a result of the check in step 310, if a specific candidate selection operation signal is provided, the controller 104 stores the corresponding voice data and the selected recognition result data in the storage 110 as cumulative data (step 312).

In addition, the controller 104 controls the GPS receiver 106 to search for the optimum path (i.e., the destination path) between the current moving object and the destination according to the destination of the selected recognition result, and the moving object from the GPS receiver 106. Maps the location value with the map information data stored in the storage unit 110 to search for an optimal path from the current location to the destination and provide the navigation service such as displaying through the image processing unit 112 and the display unit 114 (Step 314).

Next, the controller 104 checks whether the storage of the accumulated data has been performed a predetermined number of times (for example, 20 times, 30 times, etc.) (step 316).

As a result of the check in step 316, if the accumulation of the accumulated data has not been performed a predetermined number of times, the controller 104 repeats the processes of steps 302 to 314 until the accumulation of the accumulated data is performed a predetermined number of times. To perform.

Therefore, in the voice input mode of the navigation device, the recognition result candidate according to the voice input may be detected, and the selected recognition result data may be accumulated and stored together with the voice data to effectively store accumulated data for speaker adaptation.

Next, while accumulating data including voice data and recognition result data corresponding thereto is stored as described above, reliability verification of each cumulative data is performed, and accordingly, the observation data for each phoneme is accumulated and stored accordingly. A process of performing speaker adaptation by using phoneme-specific observation data to generate a new speaker-dependent acoustic model and replacing and updating the current acoustic model with the generated new speaker-dependent acoustic model will be described.

4 is a flowchart illustrating a process of updating a sound model by generating a new speaker dependent acoustic model through cumulative data according to another embodiment of the present invention.

Referring to FIG. 4, for example, when the navigation device is not operated in the controller 104 in a state where cumulative data including voice data and recognition result data corresponding thereto are stored in the storage unit 110 of the navigation device. When the navigation device is upgraded, the speaker adaptation device for speech recognition is activated, and the speech data verification unit 202 of the speaker adaptation device extracts the accumulated data stored in the storage unit 110 (step 402). .

The speech data verification unit 202 of the speaker adaptation apparatus measures reliability using the speech data included in the extracted accumulated data and the recognition result data (step 404). In this case, the reliability measurement may be performed by calculating reliability using the recognition result phoneme string data obtained by converting the voice data and the recognition result word into a phoneme string.

Next, the voice data verification unit 202 compares the reliability measured through the reliability measurement with a preset threshold (step 406).

On the other hand, the voice data verification unit 202 checks whether the measured reliability has a value greater than the preset threshold (step 408).

As a result of the check in the step 408, if the measured reliability has a value smaller than the preset threshold value, the voice data verification unit 202 extracts the next cumulative data (step 410) and then the steps 404 to 408. Rerun the process.

On the other hand, when the result of the check in the step 408, the measured reliability has a value larger than the predetermined threshold value, the voice data verification unit 202 uses the phoneme string corresponding to the corresponding speech data and the recognition result data phoneme Star observation data is accumulated (step 412). That is, observation data for each voice section may be found by forcibly sorting the voice data into a given phoneme string, and the corresponding data may be accumulated for each phoneme.

Next, the voice data verification unit 202 checks whether the cumulative storage of observation data for each phoneme for all cumulative data is completed (step 414).

As a result of the check in the step 414, if the cumulative storage of observation data for each phoneme for all cumulative data is not completed, the voice data verification unit 202 extracts the next cumulative data (step 410), and then starts from step 404. Rerun the process.

On the other hand, when the cumulative storage of the phoneme observation data for all the cumulative data is completed as a result of the check in step 414, the acoustic model speaker adaptor 204 performs speaker adaptation by using the stored and stored phoneme-specific observation data. A new speaker dependent acoustic model is created (steps 416 and 418).

Next, the acoustic model updating unit 206 updates the acoustic model by replacing the current acoustic model with the generated new speaker dependent acoustic model (step 420).

Accordingly, after verifying reliability of each cumulative data, accumulate and store observation data for each phoneme according to the accumulator, and perform speaker adaptation using the accumulated phoneme observation data to generate a new speaker-dependent acoustic model. By replacing and updating the model with the generated new speaker-dependent acoustic model, it is possible to effectively perform speaker adaptation for speech recognition in the navigation device.

In the foregoing description, various embodiments of the present invention have been described and described, but the present invention is not necessarily limited thereto. It will be readily appreciated that the substitutions, modifications and variations are possible.

1 is a block diagram of a navigation device suitable for providing a navigation service corresponding to input voice data using a voice recognition function according to an embodiment of the present invention;

2 is a block diagram of a speaker adaptation apparatus for speech recognition suitable for generating and updating a new speaker-dependent acoustic model by verifying cumulative data including speech data and recognition result data according to an embodiment of the present invention;

3 is a flowchart illustrating a process of storing cumulative data for speaker adaptation according to an embodiment of the present invention;

4 is a flowchart illustrating a process of updating a sound model by generating a new speaker dependent acoustic model through cumulative data according to another embodiment of the present invention;

5 is a diagram illustrating a recognition result menu screen including a recognition result candidate corresponding to input voice data according to an exemplary embodiment of the present invention.

<Description of the symbols for the main parts of the drawings>

102 input unit 104 control unit

106: GPS receiver 108: sensor

110: storage unit 112: image processing unit

114: display unit 116: voice processing unit

118: speaker 120: voice input unit

122: speech recognition unit 202: speech data verification unit

204: Acoustic model speaker adaptor 206: Acoustic model update unit

Claims (10)

A voice data verification unit verifying the accumulated data including the voice data and the N-best recognition result data corresponding thereto and acquiring observation data for each phoneme of the accumulated data by verifying the reliability data; An acoustic model speaker adaptor for performing speaker adaptation using the acquired phoneme-specific observation data; Acoustic model updating unit for updating an acoustic model with a new speaker dependent acoustic model generated by the speaker adaptation performed above Speaker adaptation device for speech recognition comprising a. The method of claim 1, And the voice data verification unit is configured to perform the reliability evaluation using the recognition result phoneme string data and the voice data obtained by converting a word of the recognition result data into a phoneme string. The method of claim 2, And the speech data verification unit calculates the similarity between the phoneme string data and the speech data as a result of the recognition, and compares the measured reliability with a preset threshold to perform the reliability evaluation. The method of claim 3, wherein And the speech data verification unit obtains the observation data for each phoneme by using a phoneme string obtained when the measured reliability has a value greater than the preset threshold. The method of claim 4, wherein And the speech data verification unit detects observation data for each speech section by forcibly sorting the speech data according to the obtained phoneme sequence, and accumulates the speech data for each phoneme to obtain observation data for each phoneme. Extracting cumulative data including voice data and N best recognition result data corresponding thereto; Obtaining observation data for each phoneme of the cumulative data by verifying the reliability of the extracted cumulative data; Accumulating the observation data for each phoneme by repeatedly extracting the cumulative data and acquiring observation data for each phoneme for all the cumulative data; Performing speaker adaptation using the accumulated phoneme-specific observation data; Updating an acoustic model with a new speaker dependent acoustic model generated through the speaker adaptation; Speaker adaptation method for speech recognition comprising a. The method of claim 6, Extracting the observation data for each phoneme, Measuring a reliability of the extracted cumulative data; Comparing the measured reliability with a preset threshold value; Selectively acquiring observation data for each phoneme according to the comparison result Speaker adaptation method for speech recognition comprising a. The method of claim 7, wherein Selecting and obtaining the observation data for each phoneme, If the measured reliability has a value smaller than the preset threshold, performing the measuring and comparing with respect to the next cumulative data; Acquiring observation data for each phoneme when the measured reliability has a value greater than the preset threshold; Speaker adaptation method for speech recognition comprising a. 9. The method according to claim 7 or 8, The measuring of the reliability may be performed by using a recognition result phoneme string data and the voice data of converting a word of the recognition result data into a phoneme string. The method of claim 8, The acquiring observation data for each phoneme may detect the observation data for each voice section by forcibly sorting the voice data according to a phoneme string obtained when the measured reliability has a value greater than the preset threshold. Speaker adaptation method for speech recognition to accumulate by the phone to obtain the observation data for each phoneme.

Images