KR20200100332A - Speech recognition device, method and computer program for updating speaker model - Google Patents

Abstract

A device for updating a speaker model comprises: a voice input unit receiving a voice uttered from a user; an extraction unit extracting a feature vector value from the received voice; a similarity calculation unit calculating similarity by comparing the extracted feature vector value and a plurality of previously registered speaker models; a speaker model selection unit selecting one of the plurality of previously registered speaker models based on the calculated similarity; and a speaker model update unit updating the selected speaker model based on the similarity between the extracted feature vector value and the selected speaker model. Therefore, the speaker model may be automatically updated.

Description

Speech recognition device, method, and computer program for updating speaker models {SPEECH RECOGNITION DEVICE, METHOD AND COMPUTER PROGRAM FOR UPDATING SPEAKER MODEL}

The present invention relates to a speech recognition apparatus, method and computer program for updating a speaker model.

The intelligent personal assistant is a software agent that handles tasks requested by users and provides specialized services to users. The intelligent personal assistant collects and provides customized information to the user based on artificial intelligence (AI) engine and voice recognition, and performs various functions such as scheduling, sending email, and restaurant reservation according to the user's voice command. It has the advantage of improving convenience.

These intelligent personal assistants are mainly provided in the form of personalized personal services on smartphones, and include Apple's Siri, Google's Now, and Samsung's Bixby. In this regard, Korean Patent Publication No. 2016-0071111, which is a prior art, discloses a method of providing a personal assistant service in an electronic device.

In recent years, by registering as a speaker in an intelligent personal assistant for each user, it is possible to receive customized services for each user. At this time, the process of registering the user's voice with the intelligent personal assistant takes place for a very short time. For this reason, it has a disadvantage that the speaker recognition performance deteriorates as the time when the voice is registered passes.

An object of the present invention is to provide a speech recognition apparatus, method, and computer program for updating a speaker model that automatically updates a speaker model without re-registration of a previously registered speaker model when the speaker is not recognized.

An object of the present invention is to provide a speech recognition apparatus, method, and computer program for updating a speaker model to flexibly update the speaker model by adjusting the importance between the speaker model and the spoken voice input from the user by weight.

In the case of modeling a speaker model using a small amount of data, the present invention intends to provide a speech recognition apparatus, a method, and a computer program for updating a speaker model to supplement the speaker model by additionally using a voice input from a user.

However, the technical problem to be achieved by the present embodiment is not limited to the technical problems as described above, and other technical problems may exist.

As a means for achieving the above-described technical problem, an embodiment of the present invention provides a voice input unit for receiving a speech uttered from a user, an extraction unit for extracting a feature vector value from the received voice, and the extracted feature vector value. A similarity calculation unit that compares a plurality of previously registered speaker models to calculate a similarity, a speaker model selection unit that selects any one speaker model among the plurality of previously registered speaker models based on the calculated similarity, and the extraction It is possible to provide a speech recognition apparatus including a speaker model update unit that updates the selected speaker model based on a similarity between the selected feature vector value and the selected speaker model.

In another embodiment of the present invention, the step of receiving a voice spoken from a user, extracting a feature vector value from the received voice, and comparing the extracted feature vector value with a plurality of pre-registered speaker models to determine the similarity. Calculating, selecting any one speaker model among the plurality of previously registered speaker models based on the calculated similarity, and the selected speaker model based on a similarity between the extracted feature vector value and the selected speaker model It is possible to provide a speaker model update method including the step of updating.

In another embodiment of the present invention, when a computer program is executed by a computing device, the computer program receives a speech uttered from a user, extracts a feature vector value from the received voice, and extracts the extracted feature vector value and the previously registered voice. A similarity is calculated by comparing a plurality of speaker models, and a speaker model is selected from among the plurality of previously registered speaker models based on the calculated similarity, and the similarity between the extracted feature vector value and the selected speaker model It is possible to provide a computer program stored in a medium including a sequence of instructions for updating the selected speaker model based on.

The above-described problem solving means are merely exemplary and should not be construed as limiting the present invention. In addition to the above-described exemplary embodiments, there may be additional embodiments described in the drawings and detailed description of the invention.

According to any one of the above-described problem solving means of the present invention, when a speaker is not recognized, a speech recognition apparatus, method, and computer for updating a speaker model to automatically update a speaker model without re-registration of a previously registered speaker model. Program can be provided.

It is possible to provide a speech recognition apparatus, a method, and a computer program for updating a speaker model to flexibly update the speaker model by adjusting the importance between a speaker model and a spoken speech input from a user with a weight.

When a speaker model is modeled using a small amount of data, a speech recognition apparatus, a method, and a computer program for updating a speaker model to supplement the speaker model by additionally using a voice input from a user can be provided.

1 is a block diagram of a speech recognition apparatus for updating a speaker model according to an embodiment of the present invention.
2 is an exemplary diagram showing an environmental element according to an embodiment of the present invention.
3 is a flowchart of a method of updating a speaker model based on whether the speaker model is updated in a speech recognition device for updating a speaker model according to an embodiment of the present invention.
4 is a flowchart of a method of updating a speaker model in a speech recognition apparatus for updating a speaker model according to an embodiment of the present invention.

Hereinafter, embodiments of the present invention will be described in detail with reference to the accompanying drawings so that those of ordinary skill in the art can easily implement the present invention. However, the present invention may be implemented in various different forms and is not limited to the embodiments described herein. In the drawings, parts irrelevant to the description are omitted in order to clearly describe the present invention, and similar reference numerals are assigned to similar parts throughout the specification.

Throughout the specification, when a part is said to be "connected" to another part, this includes not only "directly connected" but also "electrically connected" with another element interposed therebetween. . In addition, when a part "includes" a certain component, it means that other components may be further included, and one or more other features, not excluding other components, unless specifically stated to the contrary. It is to be understood that it does not preclude the presence or addition of any number, step, action, component, part, or combination thereof.

In the present specification, the term "unit" includes a unit realized by hardware, a unit realized by software, and a unit realized using both. Further, one unit may be realized using two or more hardware, or two or more units may be realized using one hardware.

In this specification, some of the operations or functions described as being performed by the terminal or device may be performed instead by a server connected to the terminal or device. Likewise, some of the operations or functions described as being performed by the server may also be performed by a terminal or device connected to the server.

Hereinafter, exemplary embodiments of the present invention will be described in detail with reference to the accompanying drawings.

1 is a block diagram of a speech recognition apparatus for updating a speaker model according to an embodiment of the present invention. Referring to FIG. 1, a speech recognition apparatus 100 for updating a speaker model includes a speech input unit 110, a vector extraction unit 120, a similarity calculation unit 130, a speaker model selection unit 140, and an update determination unit ( 150) and a speaker model update unit 160.

The voice input unit 110 may receive a voice spoken by a user.

The extraction unit 120 may extract a feature vector value from the input voice. For example, the extraction unit 120 may generate a plurality of frames from an input voice and extract a feature vector value from the generated plurality of frames.

The extraction unit 120 may select an effective speech frame based on the extracted feature vector value, and extract an I-Vector from the selected valid speech frame. I-Vector is a Gaussian mixed model (GMM, Gaussian) that expresses the distribution of frame-by-frame features such as frequency characteristics at a specific time or MFCC (Mel Frequency Cepstral Coefficient) to recognize a speaker from the input voice It is a feature vector that expresses various variability of (Mixture Model) as a small vector. Eye vector is a feature vector that is widely used in speaker recognition. It expresses various variability due to speaker, recording state, noise, etc. existing in the input speech as a small-dimensional vector, and has high performance in speaker recognition.

The similarity calculation unit 130 may calculate a similarity by comparing the extracted feature vector value with a plurality of previously registered speaker models. In this case, the similarity calculation unit 130 may calculate the similarity by comparing the extracted eye vector with the eye vectors of a plurality of pre-registered speaker models. For example, the similarity calculation unit 130 may perform probabilistic linear discriminative analysis (PLDA), within class covariance normalization (WCCN), deep neural network (DNN), or a combination thereof and t-norm and z- Similarity can be calculated by comparing the eye vectors of a plurality of previously registered speaker models through a normalization method such as norm.

The similarity calculation unit 130 calculates the similarity based on the likelihood that the child vector extracted using the following Equation 1 and the child vectors of each of the plurality of previously registered speaker models are the same speaker and the likelihood of different speakers. Can be calculated.

는 추출된 아이 벡터와 기등록된 화자 모델의 아이 벡터가 동일 화자일 가능성(likelihood)을 나타내고,

는 추출된 아이 벡터와 기등록된 화자 모델의 아이 벡터가 다른 화자일 가능성을 나타낸다. 여기서, 유사도(λ)는 이 두 가능성(확률)에 log를 취함으로써 도출되며, 추출된 아이 벡터와 기등록된 화자 모델의 아이 벡터가 동일 화자라고 판단되는 경우 양수 값이 도출될 수 있다. Referring to Equation 1, λ represents the degree of similarity,

Represents the likelihood that the extracted child vector and the child vector of the pre-registered speaker model are the same speaker,

Represents the possibility that the extracted child vector and the child vector of the previously registered speaker model are different speakers. Here, the similarity (λ) is derived by taking a log of these two possibilities (probability), and a positive value can be derived when it is determined that the extracted eye vector and the eye vector of the previously registered speaker model are the same speaker.

The speaker model selection unit 140 may select any one speaker model from among a plurality of previously registered speaker models based on the calculated similarity. For example, the speaker model selection unit 140 may select any one speaker model from among a plurality of previously registered speaker models based on the calculated priority of similarity. In this case, the speaker model selection unit 140 may select a speaker model with the highest similarity among a plurality of previously registered speaker models based on the calculated priority of the similarity.

The update determination unit 150 may determine whether to perform the update of the speaker model selected through steps 1 and 2. The update determination unit 150 may determine whether to perform the update of the speaker model selected in step 1 by using Equation 2.

Referring to Equation 2, the update determination unit 150 determines whether to perform the update of the selected speaker model based on whether the similarity (λ) between the extracted eye vector and the selected speaker model exceeds the speaker threshold value (TH _spk ). I can judge. At this time, if the similarity (λ) exceeds the speaker threshold (TH _spk ), proceed to step 2 to determine whether to update the selected speaker model, and the similarity (λ) exceeds the speaker threshold (TH _spk ). If not, it is possible not to perform the update on the selected speaker model.

The update determination unit 150 may determine whether to perform the update of the speaker model selected in step 2 using Equation (3).

)이 업데이트 문턱값(TH_dec)을 초과하는지 여부에 기초하여 선택된 화자 모델의 업데이트 수행 여부를 판단할 수 있다. 이 때, 추출된 아이 벡터가 선택된 화자 모델에서 생성되는지를 판단하여 화자 모델의 업데이트 수행 여부가 결정될 수 있다. The update determination unit 150 determines the likelihood that the extracted child vector and the child vector of the selected speaker model are the same speaker.

It may be determined whether to perform the update of the selected speaker model based on whether) exceeds the update threshold value TH _dec . In this case, it may be determined whether or not the speaker model is updated by determining whether the extracted eye vector is generated in the selected speaker model.

)을 재이용하는 이유는 유사도의 산출 시,

가 매우 작고(예를 들어, 0.0001),

이 작은 경우(예를 들어, 0.01)에도, 유사도(λ = 100)가 높게 산출되며, 이로 인해, 동일 화자일 가능성이 적더라도 화자 문턱값(TH_spk)을 통과할 수 있기 때문이다. Here, the possibility that the extracted eye vector used to calculate the similarity and the child vector of the previously registered speaker model are the same speaker (

The reason for reusing) is when calculating the similarity,

Is very small (for example, 0.0001),

This is because even in this small case (for example, 0.01), the similarity (λ = 100) is calculated high, and thus, the speaker threshold TH _spk can be passed even if the probability of being the same speaker is small.

) 및 환경 요소 중 적어도 하나 이상을 가중치(α, β)로 설정하여 선택된 화자 모델을 업데이트할 수 있다. The speaker model update unit 160 may update the selected speaker model based on the similarity between the extracted feature vector value and the selected speaker model using Equation 4. At this time, the speaker model update unit 160 may determine that the extracted child vector and the child vector of the selected speaker model are the same speaker (

) And at least one of the environmental elements as weights (α, β) to update the selected speaker model.

은 업데이트할 화자 모델을 나타내고,

은 입력된 음성으로부터 추출한 아이 벡터를 나타낸다. Referring to Equation 4,

Represents the speaker model to be updated,

Represents the eye vector extracted from the input voice.

)을 가중치(α)로하여 선택된 화자 모델을 업데이트할 수 있다. 또한, 화자 모델 업데이트부(160)는 음성 이외의 변수를 제어하는 환경 요소를 가중치(β)로 하여 선택된 화자 모델을 업데이트할 수 있다. 환경 요소에 대해서는 도 2를 통해 상세히 설명하도록 한다. The speaker model update unit 160 has the possibility that the extracted eye vector and the eye vector of the selected speaker model are the same speaker (

) As a weight (α) to update the selected speaker model. In addition, the speaker model update unit 160 may update the selected speaker model by using an environmental factor that controls variables other than speech as a weight β. The environmental factors will be described in detail with reference to FIG. 2.

2 is an exemplary diagram showing an environment element set as a weight according to an embodiment of the present invention. Referring to FIG. 2, the environmental element may include a signal to noise ratio (SNR) 200 and weights β and 210. Here, the weights β and 210 may indicate the reliability of the input voice as to whether the input voice is input in a quiet environment.

The speaker model update unit 160 estimates the SNR 200 for the speech to which the weights (β, 210) are input and the noise included in the speech, and varies the weights (β, 210) according to the range of the SNR 200. Can be applied. In this case, the closer to an unclean environment, the speaker model update may not be performed based on the input voice by using a value closer to '0'.

The speech recognition apparatus 100 for updating the speaker model may be executed by a computer program stored in a medium including a sequence of instructions for updating the speaker model. When the computer program is executed by a computing device, it receives speech uttered from the user, extracts a feature vector value from the received speech, and calculates a similarity by comparing the extracted feature vector value with a plurality of pre-registered speaker models. And a sequence of instructions for selecting any one speaker model from among a plurality of previously registered speaker models based on the calculated similarity, and updating the selected speaker model based on the similarity between the extracted feature vector value and the selected speaker model. can do.

FIG. 3 is a flowchart of a method of updating a speaker model based on whether the speaker model is updated in a speech recognition device that updates a speaker model according to an embodiment of the present invention. The method of updating the speaker model based on whether the speaker model is updated in the speech recognition apparatus 100 for updating the speaker model shown in FIG. 3 is processed in time series according to the embodiments shown in FIGS. 1 and 2. Includes steps. Therefore, even if omitted below, the method of updating the speaker model based on whether or not the speaker model is updated in the speaker model update apparatus 100 according to the exemplary embodiment illustrated in FIGS. 1 and 2 is also applied.

In step S310, the speech recognition apparatus 100 may receive a voice spoken by a user.

In step S320, the speech recognition apparatus 100 may generate a plurality of frames from the received speech, and extract a feature vector value from the generated plurality of frames.

In operation S330, the speech recognition apparatus 100 may select a valid speech frame based on the extracted feature vector value, and extract an eye vector from the selected valid speech frame.

In step S340, the speech recognition apparatus 100 calculates a similarity by comparing the extracted eye vector with the eye vectors of a plurality of pre-registered speaker models, and based on the calculated similarity, the speech recognition apparatus 100 You can choose a speaker model.

In operation S350, the speech recognition apparatus 100 may determine whether the similarity between the extracted eye vector and the eye vector of the selected speaker model exceeds the speaker threshold.

For example, when the similarity between the extracted eye vector and the eye vector of the selected speaker model does not exceed the speaker threshold (S351), the speech recognition apparatus 100 may not perform speaker model update. For another example, when the similarity between the extracted eye vector and the child vector of the selected speaker model exceeds the speaker threshold (S352), the speech recognition apparatus 100 may determine whether the update threshold is exceeded through step S360. I can.

In step S360, the speech recognition apparatus 100 may determine whether the likelihood that the extracted eye vector and the eye vector of the selected speaker model are the same speaker is equal to or greater than the update threshold.

For example, when the likelihood that the extracted eye vector and the eye vector of the selected speaker model are the same speaker does not exceed the update threshold (S361), the speech recognition apparatus 100 may not perform speaker model update. For another example, when the likelihood that the extracted eye vector and the eye vector of the selected speaker model are the same speaker exceeds the update threshold (S362), the speech recognition apparatus 100 may update the selected speaker model through step S370. I can.

In the above description, steps S310 to S370 may be further divided into additional steps or combined into fewer steps, according to an embodiment of the present invention. In addition, some steps may be omitted as necessary, and the order between steps may be switched.

4 is a flowchart of a method of updating a speaker model in a speech recognition apparatus for updating a speaker model according to an embodiment of the present invention. A method of updating a speaker model in the speech recognition apparatus 100 shown in FIG. 4 includes steps that are processed in time series according to the embodiment shown in FIGS. 1 to 3. Accordingly, even if omitted below, it is also applied to a method of updating a speaker model in the speech recognition apparatus 100 according to the exemplary embodiment illustrated in FIGS. 1 to 3.

In step S410, the speech recognition apparatus 100 may receive a voice spoken by a user.

In step S420, the speech recognition apparatus 100 may extract a feature vector value from the received speech.

In step S430, the speech recognition apparatus 100 may calculate a similarity by comparing the extracted feature vector value with a plurality of previously registered speaker models.

In step S440, the speech recognition apparatus 100 may select any one speaker model from among a plurality of previously registered speaker models based on the calculated similarity.

In operation S450, the speech recognition apparatus 100 may update the selected speaker model based on the similarity between the extracted feature vector value and the selected speaker model.

In the above description, steps S410 to S450 may be further divided into additional steps or may be combined into fewer steps, according to an embodiment of the present invention. In addition, some steps may be omitted as necessary, and the order between steps may be switched.

The method of updating the speaker model in the speech recognition apparatus described with reference to FIGS. 1 to 4 may be implemented in the form of a computer program stored in a medium executed by a computer or a recording medium including instructions executable by a computer. . In addition, the method of updating the speaker model in the speech recognition apparatus described with reference to FIGS. 1 to 4 may be implemented in the form of a computer program stored in a medium executed by a computer.

Computer-readable media can be any available media that can be accessed by a computer, and includes both volatile and nonvolatile media, removable and non-removable media. Further, the computer-readable medium may include a computer storage medium. Computer storage media includes both volatile and nonvolatile, removable and non-removable media implemented in any method or technology for storage of information such as computer readable instructions, data structures, program modules or other data.

The above description of the present invention is for illustrative purposes only, and those of ordinary skill in the art to which the present invention pertains will be able to understand that it can be easily modified into other specific forms without changing the technical spirit or essential features of the present invention. will be. Therefore, it should be understood that the embodiments described above are illustrative and non-limiting in all respects. For example, each component described as a single type may be implemented in a distributed manner, and similarly, components described as being distributed may also be implemented in a combined form.

The scope of the present invention is indicated by the claims to be described later rather than the detailed description, and all changes or modified forms derived from the meaning and scope of the claims and their equivalent concepts should be interpreted as being included in the scope of the present invention. do.

100: speech recognition device
110: voice input unit
120: extraction unit
130: similarity calculation unit
140: speaker model selection unit
150: update determination unit
160: speaker model update unit

Claims (17)

In the speech recognition device for updating a speaker model,
A voice signal input unit receiving a voice spoken from a user;
A vector extractor for extracting a feature vector value from the input voice;
A similarity calculation unit for calculating a similarity by comparing the extracted feature vector value with a plurality of previously registered speaker models;
A speaker model selection unit for selecting any one speaker model among the plurality of previously registered speaker models based on the calculated similarity; And
A speaker model update unit that updates the selected speaker model based on the similarity between the extracted feature vector value and the selected speaker model
Containing, speech recognition device.
The method of claim 1,
The vector extraction unit generates a plurality of frames from the input speech, extracts a feature vector value from the generated plurality of frames, and selects an effective speech frame based on the extracted feature vector values. .
The method of claim 2,
The vector extraction unit extracts an I-Vector from the selected valid speech frame.
The method of claim 3,
The similarity calculation unit calculates the similarity by comparing the extracted eye vector with the pre-registered eye vectors of the plurality of speaker models.
The method of claim 4,
The similarity calculation unit calculates the similarity based on a likelihood that the extracted child vector and the child vectors of each of the plurality of pre-registered speaker models are the same speaker and different speakers,
Wherein the speaker model selection unit selects any one speaker model from among the plurality of previously registered speaker models based on the calculated priority of the similarity.
The method of claim 5,
An update determination unit that determines whether to update the selected speaker model based on whether the similarity between the extracted eye vector and the selected speaker model exceeds a speaker threshold
That further comprises a voice recognition device.
The method of claim 6,
The update determination unit determines whether to perform the update of the selected speaker model based on whether the extracted child vector and the child vector of the selected speaker model are the same speaker exceeds an update threshold. Speech recognition device.
The method of claim 7,
The speaker model update unit updates the selected speaker model by setting at least one of a probability that the extracted eye vector and the eye vector of the selected speaker model are the same speaker and environment factors as weights.
In a method for updating a speaker model in a speech recognition device,
Receiving a voice spoken by a user;
Extracting a feature vector value from the input speech;
Calculating a similarity by comparing the extracted feature vector value with a plurality of previously registered speaker models;
Selecting any one speaker model from among the plurality of previously registered speaker models based on the calculated similarity; And
Updating the selected speaker model based on the similarity between the extracted feature vector value and the selected speaker model
Including, speaker model update method.
The method of claim 9,
The step of extracting the feature vector value,
Generating a plurality of frames from the received voice;
Extracting feature vector values from the generated plurality of frames;
And selecting a valid speech frame based on the extracted feature vector value.
The method of claim 10,
The step of extracting the feature vector value,
And extracting an I-Vector from the selected valid speech frame.
The method of claim 11,
The step of calculating the similarity,
And calculating the similarity by comparing the extracted eye vector with the eye vectors of the plurality of previously registered speaker models.
The method of claim 12,
Calculating the similarity based on a likelihood that the extracted eye vector and the eye vectors of each of the plurality of pre-registered speaker models are the same speaker and different speakers; And
And selecting any one speaker model from among the plurality of previously registered speaker models based on the calculated priority of the similarity.
The method of claim 13,
The method further comprising determining whether to update the selected speaker model based on whether a similarity between the extracted eye vector and the selected speaker model exceeds a speaker threshold.
The method of claim 14,
The step of determining whether to update the speaker model is performed,
Determining whether to update the selected speaker model based on whether the extracted eye vector and the child vector of the selected speaker model are the same speaker exceeds an update threshold, How to update speaker model.
The method of claim 15,
Updating the speaker model,
And updating the selected speaker model by setting at least one of a probability that the extracted eye vector and the eye vector of the selected speaker model are the same speaker and environment factors as weights.
A computer program stored in a computer-readable medium comprising a sequence of instructions for updating a speaker model, comprising:
When the computer program is executed by a computing device,
Receive the voice spoken by the user,
Extracting a feature vector value from the received voice,
A similarity is calculated by comparing the extracted feature vector value with a plurality of previously registered speaker models,
Selecting any one speaker model from among the plurality of previously registered speaker models based on the calculated similarity,
A computer program stored in a medium comprising a sequence of instructions for updating the selected speaker model based on the similarity between the extracted feature vector value and the selected speaker model.

Images