DE60026106T2 - System and method for speech recognition by means of language models - Google Patents

Description

The The present invention relates to a method and a system for Authentication of a speaker. Specifically, the present invention relates The invention relates to a method with multiple-pass speech recognition and speaker authentication for user authentication in a voice portal and an improved Voice portal.

State of technology

Voice portals, for example, interactive voice response systems in which information can be stored and to those of several users through spoken applications, the directly expressed in a microphone or transmitted through a telephone network and are interpreted in the voice portal, accessed can, are already known. Voice portals have the advantage of being on the information without prior knowledge and without special equipment from any phone in the world can be accessed.

voice portals can often provide access to confidential information, such as banking or medical information. The level of confidentiality and the type of information stored in the voice portal will hang Therefore, much of the reliable identification of authorized Users to authorize them or not.

In Many conventional systems require users to have their identity with one User identification and / or a password, which as DTMF Enter the code on the user device keyboard or confirm. In many situations, for example in cars, this proves to be solution as impractical. additionally User IDs and passwords are hard to remember and become common near of the user device written down, which results in a weak security.

It has already been proposed to users with a spoken identity which at the entry point is voiced by the dialogue with the voice portal to authenticate. Speech recognition techniques were used around the claimed identity and speaker verification techniques for verification the claimed identity. In the next Section of the dialog, the user is accepted or rejected, if he is not registered in the portal or if claimed identity can not be verified.

Currently exists the big trend in speech recognition and speech verification technology in Hidden Markov Models (Hidden Markov Models). This static Method describes speech through a sequence of states (Markov States). These are due to transition probabilities connected to move from state to state and every state gives out a speech vector with a certain probability. Among the many weaknesses of HMMs you could the low robustness against a typical offset situation mention (training and test language material are different). In recent years, research in the field Of speech recognition, many researchers have concluded that the robustness from present Process is very far away from what people can afford.

Various Methods have already been described to increase the robustness of speech recognition to improve. It is well known that speaker verification is much it is better if more language material is available. In addition reach some systems through the use of speaker-dependent language models, which usually learned in an input session, better performance. However, most users would do not accept to accept the necessary time for this session. In addition, speaker-dependent Detection can only be used once the speaker has been identified was, but not, the identity that was created by the user during the The first section of the dialog is claimed to recognize.

Instead of to learn a complete set of language models for each user Various adaptation techniques have been described to one sentence from good speaker-independent Adapt models to a new speaker. If the adjustment data available in a block are, e.g. from a speaker's enrollment session, one could use static methods. Alternatively, an adjustment could be incremental happen when fitting data becomes available (incremental adjustment). In the case where the true transcription of adaptation data is known, she is supervised Adaptation called, however, if the models with unmarked Language material, it is called unmonitored adaptation. Known customization techniques include a linear technique Maximum Likelihood Regression (MLLR) and an adjustment with a posteriori maximum (MAP).

It There are other adaptation techniques to models with the current noise that are out the management has been accepted. Many systems use a little moment of silence to do the modeling. This special model is then combined with the original. A two-pass system is described in the patent application EP-A2-0 880 126 which the models of breastfeeding first by segmentation of the speech utterances and then the breastfeeding models by an unmonitored adjustment adapts.

Speaker-dependent language models were already alike for used the task of speaker recognition. Sungjoo Ahn et al. in "Effective speaker adaptations for speaker verification ", 2000 IEEE International Conference on Acoustics, speech and signal processing, Istanbul, 5-9 June 2000, pp. 1081-1084, for example, describes effective adjustment method with only one limited amount of available Enrollment data. Sadoki Furui, in "Recent advantages in speaker recognition ", Pattern Recognition letters, September 1, 1997, pp. 859-872, is a general article, which different advantages in speaker recognition techniques, including speaker-dependent and contextual Schemes, describes. Both documents still rely on one rather cumbersome Input session to customize the user's language models.

In a voice portal All of these customization techniques are used to model the language while the entrance session and / or during to improve the whole dialogue; the adapted models can even stored in the system and for the speech recognition in future Sessions are used. But you can not be in everyone Meeting as long as the identity the speaker was not recognized and / or verified used become.

Of the first section of the dialogue is therefore very critical, because only speaker independent Detection techniques can be used and because the complexity (number the branches) of the identity input is at least proportional to the number of users. Vice versa has speech recognition in the rest of the spoken dialog one easier task; the complexity is lower.

Summary the invention

It An object of the invention is a voice portal system and method with improved speaker authentication performance than existing To create processes and systems.

One Another goal is a speaker-dependent verification process in a voice portal system providing accurate speaker recognition and verification on the basis of a statement made during the the first section of the dialog was spoken by the user, allowed.

In accordance with an embodiment The present invention will be a speaker authentication method created in which a special dialogue was constructed, to collect more data and to understand the language and breastfeeding models that for the first dialog section used to customize.

According to one In another aspect of the invention, a multipass speech recognition scheme is used to to improve the adaptation of language models.

• (1) Speichern von mindestens einer Äusserung von einem Benutzer des besagten Portals im besagten Portal;
• (2) darauf folgend während des Dialogs Abfragen des besagten Benutzers, Auswahl in einem Sprachmenü zu treffen,
• (3) darauf folgend Anpassen der Sprach- und Stillemodelle, welches für die Erkennung der Äusserungen des besagten Benutzers verwendet werden, mit besagter Auswahl, die von dem Benutzer geäussert wurden,
• (4) Verwenden der besagten Sprach- und Stillemodelle, um besagten Benutzer zu authentifizieren, wobei die Äusserung, die in dem besagten Sprachportal gespeichert sind, verwendet werden.

According to another aspect of the invention, there is provided a method of authenticating a speaker in a voice portal in which the identity is recognized by said user, the method comprising the following successive steps in a dialogue:

• (1) storing at least one utterance from a user of said portal in said portal;
• (2) subsequently during the dialog querying said user to make selections in a speech menu,
• (3) subsequently adapting the speech and breastfeeding models used to recognize the utterances of said user with said selection voiced by the user,
• (4) using said speech and still models to authenticate said user, using the utterance stored in said voice portal.

According to another aspect of the invention, a spoken dialogue is constructed which allows a delayed response after the user's speech input. It is a fact that in most Voice portal applications require reliable user authentication only for the delivery of security-related information or for special transactions that are normally executed only after some selection in the language menu. Therefore, user authentication can be postponed until later in this dialog. In addition, user authentication can be easily suppressed if the critical dialog part is not reached.

According to one Another aspect of the invention is the language models used for speaker authentication to be used while adapted to the spoken dialogue. This allows the task of Speaker authentication with new speaker-dependent and session-dependent voice and breastfeeding models and therefore improves an increased Authentication accuracy.

According to one In another aspect of the invention, the first utterance is stored in the system its detection and verification of the identity of the user until enough language material has been gathered to make one speakerdependent Perform speech recognition and speaker verification. The first statement can be normalized, multipass applications can be applied to it and the language models can be adapted to speaker authentication and the quality of speech recognition to improve.

Short description the figures

The The invention will be explained in more detail with reference to the accompanying figures, wherein

1 a schematic representation of a possible embodiment of a voice portal according to the invention shows.

2 a flowchart illustrating a dialogue with the inventive voice portal shows.

Detailed description

1 shows a diagram of a telecommunication system, which is a voice portal hosting system 3 contains. A variety of users can have a voice connection with the voice portal 3 via a public telecommunications network 2 , such as a publicly interconnected telecommunications network (PSTN), an integrated serial data network (ISDN), a mobile telecommunications network, eg a GSM network, or a voice-enabled IP network (VoIP). The connection is made by selecting the telephone number of the voice portal 3 on a user terminal device 1 , as on a telephone terminal, built. Preferably, the telephone number of the voice portal is a free business number in area 0800 or in another embodiment a charge number in area 0900. The voice portal 3 can access information and services from one or a variety of service providers 13 permanently or occasionally over a second telecommunications network 12 connected to the portal.

The voice portal hosting system includes a dialog manager 11 to have a spoken dialogue with the user 1 build. The dialogue manager 11 responds to commands or requests made by the user, with the requested information or service, or by establishing a new connection to an external server.

Speech requests of the user are made by a speech recognizer 10 which preferably uses HMM (Hidden Markov Models), adaptive neural networks or hybrid networks. In a preferred embodiment, the speech recognizer module includes Hidden Markov Models which are constructed with a toolkit, such as a HTK toolkit.

A selector 9 can speaker-independent language models 7 or once the user has been identified, speaker-dependent language models 8th select. Speaker-dependent speech models usually allow faster and more reliable speech recognition than speaker-independent speech models, but obviously can only be used after user identification.

At least the first utterance spoken by the user during the session, but preferably all utterances, will be in temporary memory 5 stored and by the speech adaptation module 60 used to improve the quality of speech recognition and speaker authentication, each through the two modules 6 and 10 be carried out to improve. As explained later At least some utterances are stored to be recognized or to serve for authentication, using enhancements from the language models obtained from the subsequent utterances.

The speaker is identified and preferably authenticated by a speaker authentication module 6 , The speaker authentication module preferably uses HMM (Hidden Markov Models), but could possibly use adaptive neural networks or hybrid networks. In a preferred embodiment, the speaker authentication module includes Hidden Markov Models, which are constructed with a toolkit, such as a HTK toolkit. In a preferred embodiment, user identification of text that corresponds to a spoken utterance is first made using speaker-independent language models, for example, through the use of language models 7 , recognized. An active verification signal Verif is provided as soon as the identity is known by a calling user. Speaker-dependent models 8th can through the selector 9 be selected and used by the speech recognizer 10 used as soon as the signal Verif is active. The user identity is then verified to authenticate the user with increased reliability; an authentication signal Auth is activated when the user has been authenticated with a required level of confidentiality.

Instead of speaker-dependent language models in the speaker-dependent database 8th During the initial session, customization techniques can be applied to enhance the models available for speech recognition and speaker authentication. In this case, by using only a small amount of new user data, the speaker-independent model can 7 be adjusted by the use of a maximum likelihood linear regression (MLLR). This adjustment can be incremental as fitting data becomes available (incremental adjustment).

The speech models used for each utterance for recognition and / or authentication, as will be explained later, during each session with the speech material stored in the memory 5 saved, adjusted. A second pass may be applied to the first utterances to enhance recognition and / or authentication reliability through the use of subsequently spoken material. The segmentation of utterances can be improved with a subsequent language selection. A normalization process of speech segments can also be performed. New language models, including, for example, new breastfeeding models that are better adapted to the channel used, can be generated from subsequent utterances and the model used to recognize the first utterances and to authenticate the user of such utterances, to be added.

The speaker authentication module 6 and the speech recognizer 10 can be combined in a single module. The speaker authentication module may utilize additional information, such as the CLI in a landline, the IP address in an Internet network, or the IMSI (International Mobile Subscriber Identification) in a mobile network to facilitate the task of user identification / authentication.

The level of confidentiality required to authenticate the user may preferably be determined by the dialogue manager and / or by the operator of the voice portal, eg, the service provider 13 , depending on the required security. A USER signal unambiguously indicates that the user is through the module 6 has been identified and / or authenticated.

Various Procedures can independently or at the same time be applied to the authentication and Recognition of the first statement based on subsequent statements to improve:

1. Subsequent Normalization techniques

Various Normalization techniques (CMN, energy normalization) can be found in the invention Dialog design are applied. This normalization technique works only if there is sufficient language material to the required value of energy resources or cepstral coefficients.

According to the invention, the first utterance of the user is normalized using subsequent speech material, eg, a selection in a menu voiced by a user. This allows for more accurate speech recognition and a more accurate speaker identification / verification algorithm.

2. Multiple Pass Adaptation Techniques

2 illustrates a specially adapted dialog between the user 1 and the voice portal according to the invention 3 , The dialog has been adapted in a sense that the dialog manager will only activate the final pass of the speech recognition and the authentication method when the user reaches a critical part of the dialogue.

Of the Specialist will understand that in addition to the improved user interface, in which this dialog design is applied, very efficient Speech recognition techniques can be applied to the perceived quality improve the language interaction.

• • Der Benutzer möchte auf einen sicheren Teil des Sprachmenus zugreifen und
• • Die Sprachmodelle wurden mit einem nachfolgenden Sprachmaterial angepasst und/oder
• • Genug Sprachmaterial wurde von dem Benutzer gesammelt, die eine zuverlässige Sprecherverifizierung erlauben.

1. During the first step at the entry point of the personal voice portal, the user dials the telephone number of the voice portal 3 in the network 2 and the call is being set up. The user is then asked to provide his identification and / or password. The speech material and the channel sounds are collected and stored in the temporary memory 5 during step 22 saved. At this point in the dialog, no decision is made on the user's rejection. The claimed identity is later verified with known speaker verification methods only if the following conditions are met:

• • The user wants to access a secure part of the voice menu and
• • The language models have been adapted with a subsequent language material and / or
• • Enough speech material has been collected by the user allowing for reliable speaker verification.

2. Two procedures will then be parallel in the voice portal 3 carried out. In a first method, the user is granted access to a free part of the speech menu (step 24 ). This free part can, for example, access non-confidential information from the service provider 13 contain. The user can then make a selection in a language menu (step 26 ); Speech material is stored in the memory 5 collected. The dialog continues with subsequent questions and answers using speaker independent speech recognition (step 30 ) as long as the user has not been identified.

In a preferred embodiment, the user becomes during the steps 26 to 30 using speaker-independent recognition of text corresponding to the first utterance or recognized by non-voice elements such as CLI, IMSI or IP terminal identification. This provisional identity is used by the user profile database, which supports the language and / or language models 8th which is used to improve the speech recognition accuracy during the dialogue, and this will later be verified, if necessary, using known speaker verification techniques to authenticate the user. test 28 indicates a check of the Verif signal indicating whether the user has already been identified and whether speaker-dependent speech algorithms are being used during step 32 can be used for speech recognition.

In another embodiment an incremental fitting technique is used to model the speech models, the for every section of the dialog used to improve. Monitored Adaptation techniques are preferably used when the answer the user can be expected (for example, for a selection in a closed menu list).

3. In a concurrent task within the system, the authentication flag Auth is initially reset (step 42 ). During step 44 then the authentication module tries to identify the user using a first utterance spoken during a first exchange (step 440 ) and to authenticate (step 442 ). Besides the user's language, other elements such as terminal identification (CLI, IMSI, IP-address, etc.) may be used to aid in the identification and authentication process.

At least the first statement, but preferably all voice inputs are first by means of a speaker independent speech recognizer segmented. The speech segmentation and the non-speech signals become then used the speech and breastfeeding models with a fitting technique how to adapt to MLLR for example.

4th For language models Only the segments with an acceptable level of confidentiality used to for a supervised Adaptation of model parameters. Unmonitored techniques can as well be applied.

5. Subsequent recognition / customization adjustments may be made during step 50 (i) to further improve the level of confidentiality of the recognition and authentication system, (ii) segmentation, and (iii) the adaptation of speech and silence models. In the case where the user number is composed of a character string, the subsequent recognition pass is stopped when the error recognition code gives permission.

6. When the dialog enters the critical safety point (step 34 ) in the menu where user authentication is required, for example, to access more sensitive information or to perform financial transactions, the final recognition and verification pass is performed and the user can be rejected by the service at that point.

If the user has already been authenticated at this point, he is denied access to the secure parts of the voice menu (step 40 ). Otherwise, the access is denied or the user may be given a second attempt or he is prompted to authenticate using another method, for example, by uttering a password or by entering a DTMF PIN code on the keyboard of his device (step 38 ).

In the preferred embodiment, which is in 2 is illustrated, a sequence of authentication pass 44 Iteratively, as the user navigates through the language menu until a preset level of confidentiality is reached (Test 46 ). The authentication flag is set when this level has been reached (step 48 ); otherwise, another adjustment pass is made (step 50 ). There is no access to safe parts from this menu until the user has been authenticated reliably. It is also possible to construct a language menu with a plurality of parts with different security requirements, with different levels of user authentication confidentiality needed to access different parts.

Therefore can according to the invention a very reliable User verification and a very accurate recognition of the first utterance based on more speech material and using language models, which are adapted to the current speaker.

7th There is no step in this dialog to explicitly prompt the user to accept. He is accepted by the presets.

The Method uses the known fact that accuracy by speaker-dependent Speech recognition higher as with speaker independent Systems is.

In the case of personal Voice portals use the present invention various spoken utterances (as many as possible), to adapt the language models to the speaker and therefore shifts it is the decision until the user on the secured part of the Want to access language menus. In a sense, it's just like a fast online enrollment session.

The Technique 1 (subsequent normalization) and 2 (multiple pass adjustments) can combined to an increased Achieve performance.

3. Improved speaker verification

During the elapsed time between the first section of the dialogue and The critical safety point can use different command words be the rejection / acceptance decision from the server system to complete. To this extra To implement the verification scheme must be the spoken command words of each session in the authentication system and the authentication system extracts the model from them, that for the speaker verification procedure is needed. In fact, will be these command words used to extra language material to generate speaker verification. It is known that because improved speaker customization with the availability of more data speaker verification will be in the same order as rising quality improved by the language material.

The Combination of all three methods contributes to the overall perceived quality to improve the service.

As mentioned above and from the perspective of dialogue, there is an obvious one Advantage in improving the dialogue because the rejection or acceptance dialogue simply skipped in most cases becomes. The dialogue is shorter and therefore more efficient.

In some cases and for some special model parts was having a higher adjustment speed with it Measured easier models. The technique becomes, for example, for customization used by breastfeeding models.

implementation example

At least some of the voice applications that are performed by the dialog manager contain language proficient e-commerce applications. The following implementation example is from a tele-banking service. The user wants to know the amount of money which is in the account.

The voice portal 3 may comprise a standard computer with internal memory (not shown) in which a computer program product can be directly loaded to perform the method steps of the invention when said program runs on said computer.

Claims (27)

Method for authenticating a speaker in a voice portal ( 3 ), in which method the identity is recognized by said user, the method comprising the following successive steps in a dialog: (1) storing at least one utterance from a user of said portal in said portal ( 22 ); (2) subsequently during the dialog queries of said user, selection in a language menu ( 26 (3) subsequently adapting the speech and breastfeeding models used to recognize the utterances of said user with said selection voiced by the user (4) using said speech and breastfeeding models, to authenticate said user, the statement made in said voice portal ( 442 ) are used. The method according to claim 1, further comprising Repeat steps 2 through 4 until a predefined level of confidentiality for the Authentication of the said user has been achieved. Method according to one of the preceding Claims, in which the identity said user is first recognized and then verified, while during the verification of the said adapted speech and breastfeeding models for the Authentication of the user is used. Method according to one of the preceding Claims, in which the identity said user of the text, with the said statement corresponds, is detected. Method according to the preceding claim, in which the identity said user of the text, with the said statement is recognized, wherein speaker-independent speech recognition algorithms be used. Method according to claim 4, in which the identity said user of the text, with the said statement is recognized, wherein the said adapted speech and Breast models are used. Method according to one of the preceding Claims, in which the identity said user is recognized, with text-independent speaker verification is used. Method according to claim 3, in which the identity of the said user, in which data is transmitted via a Transmit signal channel will be used. Method according to the preceding claim, in which said data contain a DLI terminal identification. Method according to one of the preceding claims, in which user-dependent language models ( 8th ) are stored in said portal and, once the user is recognized, used to enhance subsequent speech recognition and / or verification of user identity. Method according to the preceding claim, in which the said speaker-dependent language models are adapted being the selection made by said user were used. Method according to one of the preceding Claims, in which the identity the user using the said customized language models is verified. Method according to the preceding claim, in which the user using monitored adaptation algorithms is authenticated by language models. Method according to one of the preceding Claims, in which the language models using an adaptation technique be adapted by hidden Markov models. Method according to the preceding claim, in which said language models under Use of MLLR adaptation techniques adapted. Method according to the preceding claim, in which the said selection is used to interpret said speech and breastfeeding models to adapt and thus the segmentation of said statement, which is stored in said portal to improve. Method according to one of the preceding Claims, in which the utterance of the user with subsequent speech materials of said User is normalized. Method according to one of the preceding Claims, in what new speech and breastfeeding models of the said on it following expressed User selection are generated. Method according to the preceding claim, in which said new speech and breastfeeding models new breastfeeding models contain. Method according to one of the preceding Claims, in which said user immediately after the establishment a connection to the voice portal is asked to express his identity, the said identity as said statement is stored. Method according to one of the preceding Claims, in which said user is only authenticated when he access a secured part of the said language menu want. Method according to one of the preceding Claims, in which a variety of parts with different safety regulations is defined in the said language menu, and in which different Confidentiality Levels for the said user authentication is defined to be on the said different ones To access parts of said language menu. A method of speech recognition in a voice portal, the method comprising the following successive steps in a dialogue: (1) storing at least one utterance from a user of said system in said portal; (2) subsequently, during the dialog querying said user to make selections in a speech menu, (3) subsequently adjusting the speech and breastfeeding models used to recognize the utterances of said user with said selection made by (4) using said speech and silence models to authenticate said user, the utterance contained in said voice portal ( 442 ) are used. Computer program product, which directly into the internal Memory of a digital computer is loadable, comprising parts of a Software codes for carrying out the steps of any one of the preceding claims, when said product on an interactive voice response system, which is connected to a first telecommunication network is executed. Voice portal ( 3 ), comprising: a speaker authentication module ( 6 ), which contains Hidden Markov Models and Breast Models, to authenticate the users of the said portal, a dialog manager ( 11 ) to establish a dialogue between said system and users ( 1 ) of said system, said dialogue allowing access to at least one secure part in said system, an adaptation module ( 60 ) to adapt said Hidden Markov Models, said dialog manager being adapted to enable a dialogue in which said user authentication occurs only when said user requests access to said secured portion based on at least a first utterance Beginning of said dialogue and stored in said portal, and wherein the authentication with said first utterance is performed using said voice and silence models based on the subsequent voice material from said user. Voice portal according to Claim 25, wherein said at least one first utterance with subsequent speech material normalized by said user becomes. Voice portal according to Claim 25, in which said language material is used to adapt the speech and breastfeeding models and thus the segmentation said statement, which is stored in said portal, to improve.