KR20200114697A - Speaker authentication method and system using cross validation - Google Patents

Abstract

The present invention relates to a speaker authentication method and device using a cross validation. According to one embodiment of the present invention, the voice authentication method may comprise the steps of: determining the number of combinations for classifying voice data collected from an uttered voice of a speaker into a training dataset or a validation dataset; performing cross validation of authentication models generated based on the training dataset according to the determined number of combinations; determining the maximum and minimum values among the results of the cross validation and estimating a threshold interval value for voice authentication of the speaker based on the maximum and minimum values; and identifying the speaker based on the estimated threshold interval value.

Description

Voice authentication method and device using cross verification {SPEAKER AUTHENTICATION METHOD AND SYSTEM USING CROSS VALIDATION}

The present invention relates to a voice authentication method and device thereof using cross-validation, and more particularly, to a method for distinguishing a change in a speaker's vocalization with a small amount of data through cross-validation of models for voice authentication and to identify a speaker, and It relates to the device.

The conventional voice authentication apparatus determines whether or not the person to be spoken is a true speaker by modeling the characteristics of the voice uttered by a speaker. The user authentication process through such a conventional device is performed by determining whether the result of analyzing the speech spoken by the speaker exceeds a set threshold.

Since the conventional voice authentication apparatus can increase the accuracy of authentication through feature learning using accumulated voice data, a large amount of voice data is required for accurate speaker identification (i.e. identity authentication). In other words, in the conventional apparatus, when the amount of learning data for modeling is small, the accuracy of judgment for self-authentication is inevitably degraded, and there is a problem in that the change of speaker's speech cannot be accurately distinguished. In order to overcome this problem, a method of collecting a large amount of voice data may be considered, but in reality, a lot of time and cost are required.

Korean Registered Patent Publication No. 10-1154011 (2012.06.08)

The present invention is to solve the above-described problems, and an object of the present invention is to provide a voice authentication method and apparatus that enables accurate speaker identification and confirmation even with a small amount of data through cross-validation of models for voice authentication. .

In addition, the change in the speaker's vocalization can be appropriately reflected in the speaker identification process by setting the threshold section value of voice authentication using the result of cross-validation. An object of the present invention is to provide a voice authentication method and apparatus that can be used.

In the voice authentication method using cross validation according to an embodiment of the present invention, determining the number of combinations for classifying voice data collected from the spoken voice of a speaker into a training data set or a verification data set , Performing cross-validation of authentication models generated based on the training data set according to the determined number of combinations, determining the maximum and minimum values among the results of the cross-validation, and authenticating the speaker's speech based on the maximum and minimum values It may include estimating a threshold section value for and identifying a speaker based on the estimated threshold section value.

The step of estimating the threshold interval value according to an embodiment of the present invention may include setting an authentication model from which a maximum value is derived from among the results of cross-validation as a reference model for voice authentication of a speaker.

The step of estimating a threshold interval value according to an embodiment of the present invention includes estimating an average value of the maximum value and the minimum value or the average value of the result values of the cross-validation excluding the maximum value and the minimum value, the average value of the maximum value and the minimum value, or It may include estimating a deviation based on the average value of the result values of the cross-validation other than the maximum value and the minimum value, and estimating a threshold interval value based on the maximum value by using the deviation.

In the step of identifying a speaker according to an embodiment of the present invention, the speaker may be authenticated as the speaker only when the characteristic value of the voice data collected from the spoken voice for identification of the speaker corresponds to the threshold interval value.

A voice authentication apparatus using cross-validation according to an embodiment of the present invention includes a data classification unit for determining the number of combinations for classifying voice data collected from a speaker's spoken voice into a training data set or a verification data set, and A cross-validation unit that performs cross-validation of authentication models generated based on the training data set according to the number of combinations, determines the maximum and minimum values among the results of the cross-validation, and authenticates the speaker's speech based on the maximum and minimum values It may include a threshold section estimating unit for estimating a critical section value for and a speaker identification section for identifying a speaker based on the estimated threshold section value.

The threshold section estimator according to an embodiment of the present invention estimates the average value of the maximum value and the minimum value, or the average value of the remaining cross-validation result values excluding the maximum value and the minimum value, and calculates the average value of the maximum value and the minimum value or the maximum value and the minimum value. The deviation is estimated based on the average value of the other cross-validation result values, and the critical interval value based on the maximum value can be estimated using the deviation.

The speaker identification unit according to an embodiment of the present invention may authenticate that the speaker is himself/herself only when the characteristic value of the speech data collected from the speech spoken for the identification of the speaker corresponds to the threshold interval value.

Meanwhile, according to an embodiment of the present invention, a computer-readable recording medium in which a program for executing the above-described method on a computer is recorded may be provided.

According to the voice authentication method and apparatus provided as an embodiment of the present invention, by performing efficient authentication model training with a small amount of voice data, the accuracy of voice authentication is greatly improved at a relatively low cost and time compared to the prior art. I can make it.

In addition, by setting the threshold section based on the analyzed result through cross-validation rather than identifying the speaker according to a uniform threshold standard as in the prior art, it is possible to estimate the change in speaker's vocalization or determine digital forgery of the voice. , It is possible to significantly improve the performance of the device for voice authentication compared to the prior art.

1 is a flowchart of a voice authentication method using cross verification according to an embodiment of the present invention.
2 is a flowchart illustrating a step of estimating a threshold value of a voice authentication method using cross-validation according to an embodiment of the present invention.
3 is a conceptual diagram illustrating a process of generating a reference model using cross-validation according to an embodiment of the present invention.
4 is a conceptual diagram illustrating a voice authentication process for a speaker's confirmed speech according to an embodiment of the present invention.
5 is a block diagram of a voice authentication apparatus using cross verification according to an embodiment of the present invention.

The terms used in the present specification will be briefly described, and the present invention will be described in detail.

The terms used in the present invention have been selected from general terms that are currently widely used while considering functions in the present invention, but this may vary depending on the intention or precedent of a technician working in the field, the emergence of new technologies, and the like. In addition, in certain cases, there are terms arbitrarily selected by the applicant, and in this case, the meaning of the terms will be described in detail in the description of the corresponding invention. Therefore, the terms used in the present invention should be defined based on the meaning of the term and the overall contents of the present invention, not a simple name of the term.

When a part of the specification is said to "include" a certain component, it means that other components may be further included rather than excluding other components unless otherwise stated. In addition, terms such as "... unit" described in the specification mean a unit that processes at least one function or operation, which may be implemented as hardware or software, or a combination of hardware and software.

Hereinafter, embodiments of the present invention will be described in detail with reference to the accompanying drawings so that those of ordinary skill in the art can easily implement the present invention. However, the present invention may be implemented in various forms and is not limited to the embodiments described herein. In the drawings, parts irrelevant to the description are omitted in order to clearly describe the present invention, and similar reference numerals are assigned to similar parts throughout the specification.

Hereinafter, the present invention will be described in detail with reference to the accompanying drawings.

FIG. 1 is a flowchart of a voice authentication method using cross-validation according to an embodiment of the present invention, and FIG. 2 is a detailed estimating step 300 of a voice authentication method using cross-validation according to an embodiment of the present invention. It is a flow chart.

In addition, FIG. 3 is a conceptual diagram showing a process of generating a reference model using cross-validation according to an embodiment of the present invention, and FIG. 4 is a conceptual diagram showing a process of voice authentication for confirming speech of a speaker according to an embodiment of the present invention.

Referring to FIG. 1, in a voice authentication method using cross validation according to an embodiment of the present invention, a combination for classifying voice data collected from a speaker's spoken voice into a training data set or a verification data set Determining the number of (S100), performing cross-validation of authentication models generated based on the training data set according to the determined number of combinations (S200), determining a maximum value and a minimum value among the result values of the cross-validation , Estimating a threshold value for voice authentication of the speaker based on the maximum value and the minimum value (S300), and identifying the speaker based on the estimated threshold value (S400).

In the voice authentication method according to an embodiment of the present invention, a cross-validation method is used in order to improve the performance of the overall system for voice authentication even with a small amount of voice data. Cross-validation is a method for statistically evaluating a model, and it refers to a verification method in which the performance of the model is evaluated even when the amount of data is insufficient to systematically select a better model. For example, if the data are A, B, C, D, E, train with some of them, A, B, C, and D to create a model, then train the remaining E to verify the generated model. By using it as input data for doing so, cross-validation can be performed. In other words, cross-validation does not use all of the collected data to generate the training model, but uses some of it as input data to verify the model, so that one system evaluates and improves the performance of the model even with a small amount of data. It is characterized by providing an index to predict the direction.

In this case, the evaluation index derived as a result of the above-described cross-validation may vary depending on the number of input data for verifying the model. For example, if the data are A, B, C, D, E, if only E is used as the input data to verify the model, a total of 5 evaluation indicators can be generated, but D and E are used as the model. When used as input data for verification, 10 evaluation indicators can be generated. In this way, the number of cross-validation results can vary depending on how the data set is configured.

According to an embodiment of the present invention, in order to apply the above-described cross-validation to voice authentication, first determining the number of combinations for classifying voice data into a training data set or a verification data set by the data classification unit 200 (S100) may be performed. Determining the number of combinations means determining the number of results obtained by performing cross-validation. That is, the number of combinations may be determined according to how many of the collected speech data are classified into an input data set for cross-validation of models (i.e. a verification data set) and used.

For example, the data classification unit 200 may determine the number of combinations as 7 in order to classify 6 of the 7 speech data obtained as the speaker speaks 7 times as a training data set and the other as a verification data set. have. In addition, the data classification unit 200 may determine the number of combinations as 21 in order to classify five of the seven voice data into a training data set and the remaining two into a verification data set. According to the number of combinations determined in this way, the 7 speech data may be classified into 6 training data sets and 1 verification data set, or 5 training data sets and 2 verification data sets.

The number of combinations according to an embodiment of the present invention may be determined based on a result of analyzing the number, quality, and feature values of voice data by the data classifier 200. For example, when 10 pieces of audio data are collected by the audio data collection unit, but 7 of them are of a quality that makes it difficult to analyze a feature value, the data classification unit 200 converts 3 pieces of audio data into 2 pieces of training data. The number of combinations can be determined to be three to classify the set and one validation data set. In addition, it may be determined by an external input authorized by the user according to various purposes for additional performance improvement of the device or creation of a new model.

Referring to FIG. 3, when the number of combinations is determined according to an embodiment of the present invention and classified into a training data set or a verification data set according to the determined number of combinations (S120), the training data is determined by the data classification unit 200. An authentication model based on the set may be generated (S130). The authentication model may be generated based on a result of analyzing feature values of speech data classified as a training data set.

For example, assuming that 7 speech data are collected as the speaker speaks 7 times, when the data classification unit 200 determines the number of combinations to be 7, 7 authentication models based on 6 training data sets Can be generated. In addition, when the data classification unit 200 determines the number of combinations to be 21, 21 authentication models based on five training data sets may be generated. That is, when the number of combinations is determined by the data classification unit 200, a number of authentication models corresponding to the determined number of combinations may be generated.

1 and 3, when authentication models are generated based on a training data set according to the number of combinations determined according to an embodiment of the present invention (S130), the cross validation unit 300 crosses the authentication models. Verification may be performed (S200). This cross-validation may be performed by inputting a verification data set to each of the authentication models.

For example, when 7 authentication models are generated by the data classification unit 200 determining the number of combinations as 7 for 7 voice data, one verification data set is input to each of the 7 authentication models, and the degree of similarity Can be analyzed. Therefore, when the number of combinations is 7, a total of 7 cross-validation results (i.e. similarity analyzed through each of the 7 authentication models) can be derived.

In addition, when 21 authentication models are generated by the data classification unit 200 determining the number of combinations as 21 for 7 voice data, two verification data sets are input one by one to each of the 21 authentication models to analyze the similarity. Can be. That is, when the number of combinations is 21, a total of 42 cross-validation result values may be derived as two cross-validation result values (i.e. similarity analyzed through each of 21 authentication models) are derived per one authentication model.

1 and 3, when cross-validation of authentication models is performed according to an embodiment of the present invention and a verification result value is derived (S200), among the verification result values by the threshold section estimator 400 The maximum and minimum values are determined, and a threshold section value for voice authentication of the speaker may be estimated based on the maximum and minimum values (S300).

2 and 3, in the estimation of a threshold interval value according to an embodiment of the present invention, determining a maximum value and a minimum value among the result values of cross-validation (S310), and the maximum value among the result values of the cross-validation is Setting the derived authentication model as a reference model for the speaker's voice authentication (S320), estimating the average value of the maximum and minimum values or the average value of the results of cross-validation excluding the maximum and minimum values (S330), Estimating a deviation based on the average value of the maximum value and the minimum value or the average value of the cross-validation result values excluding the maximum and minimum values (S340) and the step of estimating a threshold interval value based on the maximum value using the deviation It can be performed through (S350).

In the step (S310) of determining the maximum and minimum values among the cross-validation result values according to an embodiment of the present invention, when the number of cross-validation result values derived from one authentication model is two or more (ie, when there are two or more verification data sets), the average value of the cross-validation result values is estimated, and the maximum and minimum values may be determined by comparing the estimated average values.

For example, when the data classification unit 200 determines the number of combinations as 7 for 7 voice data, a total of 7 cross-validation result values are derived (ie, the cross-validation result values derived from one authentication model If the number is one), the critical section estimating unit 400 may compare the sizes of the seven cross-validation result values and determine a maximum value and a minimum value among them without additional analysis of the cross-validation result value.

On the other hand, when the data classifier 200 determines the number of combinations as 21 for 7 voice data, a total of 42 cross-validation result values are derived (ie, the number of cross-validation result values derived from one authentication model is two. In case), the critical section estimating unit 400 may first estimate an average value of the two cross-validation result values derived through each of the 21 authentication models. That is, the threshold section estimating unit 400 may estimate an average value of a total of 21 cross-validation result values, and may determine a maximum value and a minimum value among them by comparing the sizes of the estimated 21 average values.

2 and 3, the threshold section estimating unit 400 according to an embodiment of the present invention sets the authentication model from which the maximum value is derived from the result values of cross-validation as a reference model for speech authentication of the speaker. The step S320 may be performed by the threshold section estimating unit 400 after the step S310 of determining the maximum and minimum values. However, the setting of the reference model may be performed after the step (S350) of estimating a threshold interval value based on the maximum value by using a deviation in order to optimize the data analysis process (i.e. an operation process).

Referring to FIG. 3, in the step (S330) of estimating the average value of the maximum value and the minimum value or the average value of the remaining cross-validation result values excluding the maximum value and the minimum value, the threshold section estimating unit 400 is first derived from one authentication model. It may be determined whether the number of cross-validation result values is two or more (ie, two or more verification data sets) (S331). In this case, if the number of cross-validation result values is one (i.e., if there is one verification data set) according to the determination result, the threshold section estimating unit 400 may estimate the average value of the maximum value and the minimum value (S332). On the other hand, if the number of cross-validation result values is two or more according to the determination result (ie, if there are two or more verification data sets), the threshold section estimating unit 400 estimates the average value of the remaining cross-validation result values excluding the maximum and minimum values. It can be done (S333).

For example, when the data classification unit 200 determines the number of combinations to be 7 for 7 voice data, a total of 7 cross-validation result values are derived (ie, when there is one verification data set), the threshold section is added. The government 400 may estimate an average value of the maximum value and the minimum value determined in the above-described process (S310).

On the other hand, when the data classification unit 200 determines the number of combinations to be 21 for 7 speech data, a total of 42 cross-validation result values are derived (ie, when there are two verification data sets), the critical section estimator 400 ) May estimate the average value of the remaining cross-validation result values excluding the maximum value and the minimum value (ie, the maximum value and the minimum value among the average values of the cross-validation result values) determined in the above-described process S310.

In the step (S340) of estimating a deviation based on the average value of the maximum value and the minimum value or the average value of the cross-validation result values excluding the maximum value and the minimum value according to an embodiment of the present invention, the threshold section estimation unit 400 A deviation may be estimated from an estimated average value through determination of the number of one verification data set, and an operation of multiplying the estimated deviation by a predetermined constant value may be performed. In this case, the predetermined constant value may be understood as a correction value for the estimated deviation, and the predetermined constant value may be preset and stored in the database 600 before the operation of the device 1000 is started.

Referring to FIG. 3, in the step of estimating a threshold section value based on a maximum value using a deviation (S350), the threshold section estimator 400 applies the estimated threshold section value to the reference model, so that the reference model is critical. It may be set to identify the speaker through the section value (S351). In this case, the threshold section value estimated by the threshold section estimating unit 400 may be expressed as shown in [Equation 1] below.

[Equation 1]

In [Equation 1], M represents a maximum value and δ represents a deviation. In addition, A denotes the minimum value of the critical section, B denotes the maximum value of the critical section, and X denotes an input variable (i.e. analysis result of speech data generated by speech for speaker confirmation).

As described above, when a reference model is generated in the voice authentication apparatus 1000 according to an embodiment of the present invention and a threshold interval value for speaker identification is set, the speaker identification unit 500 is spoken by the speaker for voice authentication. Based on the voice, it is possible to determine whether or not the object uttered corresponds to the speaker himself. The determination process (S400) by the speaker identification unit 500 will be described later in more detail with reference to FIG. 4.

Referring to FIG. 4, according to an embodiment of the present invention, when a speaker utters a confirmation voice for his/her voice authentication, the voice data collection unit 100 may collect confirmation voice data (S410). When the collection of the confirmation speech data is completed by the speech data collection unit 100, the speaker identification unit 500 may analyze the confirmation speech data to extract a confirmation speech feature value (S420).

Referring to FIG. 4, the speaker identification unit 500 according to an embodiment of the present invention may analyze the similarity by inputting the extracted feature value of the confirmed voice into the reference model (S430). In this case, the speaker identification unit 500 may determine whether the result value (i.e. similarity) analyzed through the reference model corresponds to the critical interval value (S440).

According to an embodiment of the present invention, when the analyzed result value is included in the threshold section value, the speaker identification unit 500 may confirm that the speaker himself/herself is the speaker (S451). On the other hand, when the analyzed result value exceeds the threshold section value, the speaker identification unit 500 may confirm that the object of speech is not the speaker himself, but another person (S452). That is, in the step (S400) of identifying a speaker according to an embodiment of the present invention, it is determined that the speaker is himself/herself only when the analysis result value of the speech data collected from the speech spoken for the identification of the speaker falls within the threshold section value. Can be authenticated.

For example, if the critical section value is set to 85% or more and 90% or less on the reference model, when the analysis result of the confirmation speech feature value by the reference model is estimated to be 87% similarity value, the speaker identification unit 500 The speaker's own voice can be verified. On the other hand, when the analysis result of the confirmed speech feature value by the reference model is estimated to be 80% or 99% of the similarity value, it is determined by the speaker identification unit 500 to be someone else's voice, not the speaker itself. As a result, voice authentication may not be performed.

If voice authentication is performed only when the threshold value is set as described above, the speaker can be identified by accurately discriminating not only whether it is the voice of another person but also whether the voice has been manipulated through digital forgery. Conventional voice authentication devices identify a speaker according to whether or not a threshold is exceeded, so when the speaker's own voice is completely manipulated through digital forgery and the analysis result value is estimated too high (ex. when the similarity value is estimated to be 99%) ) Is also judged by the speaker's own voice. However, since the voice authentication apparatus 1000 according to an embodiment of the present invention confirms that the speaker is the speaker only when it corresponds to the threshold value, the speaker's voice is completely manipulated through digital forgery and the analysis result value is estimated to be too high. If it is (ex. if the similarity value is estimated to be 99%), it is judged as the voice of someone other than the speaker and does not perform voice authentication. Accordingly, according to an embodiment of the present invention, if voice authentication is performed only when the threshold value is satisfied, the accuracy of speaker identification can be greatly improved compared to the prior art, and the security of voice authentication can be further enhanced.

5 is a block diagram of a voice authentication apparatus 1000 using cross verification according to an embodiment of the present invention.

Referring to FIG. 5, the voice authentication apparatus 1000 using cross-validation according to an embodiment of the present invention comprises a combination for classifying voice data collected from the spoken voice of a speaker into a training data set or a verification data set. A data classification unit 200 that determines the number, a cross-validation unit 300 that performs cross-validation of authentication models generated based on the training data set according to the determined number of combinations, the maximum and minimum values among the result values of the cross-validation And a threshold section estimating unit 400 that estimates a threshold section value for voice authentication of a speaker based on the maximum and minimum values, and a speaker identification section 500 that identifies the speaker based on the estimated threshold section value. Can include.

In addition, referring to FIG. 5, the voice authentication apparatus 1000 according to an embodiment of the present invention includes a voice data collection unit 100 that collects voice data for voice uttered from a speaker, and collects and analyzes through each component. Alternatively, it may include a database 600 that stores the estimated data.

In relation to the apparatus 1000 according to an embodiment of the present invention, the above-described method may be applied. Accordingly, description of the same contents as those of the above-described method with respect to the apparatus 1000 has been omitted.

Meanwhile, according to an embodiment of the present invention, a computer-readable recording medium in which a program for executing the above-described method on a computer is recorded may be provided. In other words, the above-described method can be written as a program that can be executed on a computer, and can be implemented in a general-purpose digital computer operating the program using a computer-readable medium. Further, the structure of the data used in the above-described method may be recorded on a computer-readable medium through various means. A recording medium for recording executable computer programs or codes for performing various methods of the present invention should not be understood as including temporary objects such as carrier waves or signals. The computer-readable medium may include a storage medium such as a magnetic storage medium (eg, ROM, floppy disk, hard disk, etc.), and an optical reading medium (eg, CD-ROM, DVD, etc.).

The above description of the present invention is for illustrative purposes only, and those of ordinary skill in the art to which the present invention pertains will be able to understand that other specific forms can be easily modified without changing the technical spirit or essential features of the present invention will be. Therefore, it should be understood that the embodiments described above are illustrative and non-limiting in all respects. For example, each component described as a single type may be implemented in a distributed manner, and similarly, components described as being distributed may also be implemented in a combined form.

The scope of the present invention is indicated by the claims to be described later rather than the detailed description, and all changes or modified forms derived from the meaning and scope of the claims and their equivalent concepts should be interpreted as being included in the scope of the present invention. .

100: voice data collection unit 200: data classification unit
300: cross verification unit 400: critical section estimation unit
500: speaker identification unit 600: database
1000: voice authentication device using cross verification

Claims (8)

In the voice authentication method using cross validation,
Determining a number of combinations for classifying speech data collected from the spoken speech of the speaker into a training data set or a verification data set;
Performing cross-validation of authentication models generated based on the training data set according to the determined number of combinations;
Determining a maximum value and a minimum value among the result values of the cross-validation, and estimating a threshold interval value for voice authentication of the speaker based on the maximum value and the minimum value; And
And identifying the speaker based on the estimated threshold interval value.
The method of claim 1,
The step of estimating the threshold interval value,
And setting an authentication model from which a maximum value is derived from among the cross-validation result values as a reference model for voice authentication of the speaker.
The method of claim 1,
The step of estimating the threshold interval value,
Estimating an average value of the maximum value and the minimum value or an average value of a result of cross-validation other than the maximum value and the minimum value;
Estimating a deviation based on an average value of the maximum value and the minimum value or an average value of the result values of cross-validation other than the maximum value and the minimum value; And
And estimating a threshold interval value based on the maximum value by using the deviation.
The method of claim 1,
In the step of identifying the speaker,
Voice authentication method using cross-validation, characterized in that the speaker is authenticated only when the analysis result value of the voice data collected from the voice spoken for verification of the speaker corresponds to the threshold section value.
In the voice authentication device using cross validation,
A data classification unit that determines the number of combinations for classifying speech data collected from the spoken speech of the speaker into a training data set or a verification data set;
A cross-verification unit performing cross-validation of authentication models generated based on the training data set according to the determined number of combinations;
A threshold section estimating unit that determines a maximum value and a minimum value among the result values of the cross-validation, and estimates a threshold section value for voice authentication of the speaker based on the maximum value and the minimum value; And
And a speaker identification unit for identifying the speaker based on the estimated threshold interval value.
The method of claim 5,
The critical section estimation unit,
Estimates the average value of the maximum and minimum values or the average value of the cross-validation result values excluding the maximum and minimum values, and the average value of the maximum and minimum values or the average value of the remaining cross-validation results excluding the maximum and minimum values A voice authentication apparatus using cross verification, characterized in that estimating a deviation based on the deviation, and estimating a threshold section value based on the maximum value by using the deviation.
The method of claim 5,
The speaker identification unit,
A voice authentication apparatus using cross-verification, characterized in that the speaker is authenticated only when the analysis result value of the voice data collected from the voice spoken for verification of the speaker corresponds to the threshold value.
A computer-readable recording medium in which a program for implementing the method of any one of claims 1 to 4 is recorded.

Images