JP6738867B2 - Speaker authentication method and voice recognition system - Google Patents

Description

The present invention relates to a speaker authentication method and a speaker recognition system, and more particularly to a method for authenticating a speaker in a voice recognition system including a voice recognition device and a voice recognition server.

Artificial intelligence speaker devices equipped with a voice recognition function are on the market. The artificial intelligence speaker device plays a role of an artificial intelligence secretary by recognizing a user's voice, extracting a command included in the voice, executing an action according to the command, and outputting the result as a voice. .. The artificial intelligence speaker device exceeds the level of simply answering the voice question and outputting the question and answer result as voice, and in order to be used in fields requiring security such as financial transactions and shopping, it is necessary to accurately talk the speaker. Must be recognized and identified. However, since the artificial intelligence speaker device has no choice but to identify the user based on the voice, the accuracy is lower than that of the user identification method or the user authentication method using biometric information such as fingerprint recognition and iris recognition.

The problem to be solved by the present invention is to solve the above-mentioned problem, and to additionally authenticate a speaker after correctly recognizing the content of the voice and the speaker from the voice of the speaker. It is to provide a method that can.

As a technical means for achieving the above technical problem, the first aspect of the present disclosure provides a speaker authentication method in a voice recognition system including a voice recognition device and a voice recognition server. In the present speaker authentication method, the voice recognition server receives a voice signal including a voice of a first speaker from the voice recognition device, and the voice recognition server performs voice recognition on the voice signal. And executing a step of generating a first voice recognition result, wherein the voice recognition server extracts a first speaker feature vector from the voice signal, and registers the first speaker feature vector and the registered speaker feature vector. Calculating a degree of similarity to the voice recognition server, and determining that the voice recognition server is a registered second speaker if the voice recognition server is greater than or equal to a first reference value. And a step in which the voice recognition server requests an authentication voice from the first speaker or the second speaker, and the voice recognition server outputs an authentication voice signal from the first speaker or the second speaker. Receiving, the voice recognition server authenticates the identity of the second speaker and the first speaker based on the authentication voice signal, and if the identity is authenticated, The voice recognition server performs an operation corresponding to the first voice recognition result.

A second aspect of the present disclosure provides a voice recognition server including a communication module that communicates with a voice recognition device, and a processor. The processor receives the voice signal including the voice of the first speaker from the voice recognition device using the communication module, performs voice recognition on the voice signal, and generates a first voice recognition result. Then, a first speaker feature vector is extracted from the voice signal, the similarity between the first speaker feature vector and the registered speaker feature vector is calculated, and the similarity is equal to or greater than a first reference value. If it is, the speaker of the voice signal is determined to be the registered second speaker, and the first speaker or the second speaker is requested to authenticate voice, and the first speaker or the second speaker is requested. An authentication voice signal is received from two speakers, the identity between the second speaker and the first speaker is authenticated based on the authentication voice signal, and when the identity is authenticated, the first One voice recognition result is configured to be executed.

A third aspect of the present disclosure detects a voice signal including a voice of a first speaker from the audio module, a communication module that communicates with a voice recognition server according to the second aspect, a microphone that generates an audio signal, and the voice signal. A voice recognition device comprising: a processor configured to transmit a signal to the voice recognition server and receive a synthetic voice signal from the voice recognition server; and a speaker that reproduces a synthetic voice corresponding to the synthetic voice signal. provide.

A fourth aspect of the present disclosure provides a voice recognition system including a voice recognition server and a voice recognition device. The voice recognition device detects a voice signal including a voice of a first speaker from the audio signal, a first communication module that communicates with the voice recognition server, a microphone that generates an audio signal, and outputs the voice signal. A first processor configured to transmit to a voice recognition server and receive a synthesized voice signal from the voice recognition server, and a speaker for playing back a synthesized voice corresponding to the synthesized voice signal. The voice recognition server includes a second processor and a second communication module that communicates with the voice recognition device. The second processor receives the voice signal from the voice recognition device, performs voice recognition on the voice signal, generates a first voice recognition result, and outputs a first speaker feature vector from the voice signal. Is calculated and the similarity between the first speaker feature vector and the registered speaker feature vector is calculated. If the similarity is equal to or higher than a first reference value, the speaker of the voice signal is registered. The second speaker, requesting an authentication voice from the first speaker or the second speaker, receiving an authentication voice signal from the first speaker or the second speaker, and Based on the authentication voice signal, the identity of the second speaker and the first speaker is authenticated, and when the identity is authenticated, the operation corresponding to the first voice recognition result is executed. Composed.

A fifth aspect of the present disclosure provides a program that causes a processor of a voice recognition server of a voice recognition system to execute the speaker authentication method according to the second aspect.

A sixth aspect of the present disclosure provides a computer-readable recording medium in which the program according to the fifth aspect is recorded.

According to the embodiments of the present invention, since the speaker can be accurately identified through the speaker authentication procedure, the voice recognition system can be used as a speaker without fear of speaker misrecognition or malfunction due to voice theft. The instructions of can be executed safely and accurately.

FIG. 1 is an exemplary network configuration diagram of a voice recognition system according to an embodiment. It is a block diagram for explaining an internal configuration of a voice recognition speaker device according to an embodiment. It is a block diagram for explaining an internal configuration of a voice recognition server according to an embodiment. It is a block diagram for explaining an internal configuration of a processor of a voice recognition server according to an embodiment. It is a block diagram for explaining an internal configuration of a processor of a voice recognition server according to another embodiment. 3 is an exemplary flowchart illustrating a speaker authentication method of a voice recognition system according to an exemplary embodiment. 9 is an exemplary flowchart illustrating a speaker authentication method of a voice recognition system according to another exemplary embodiment. 9 is an exemplary flowchart illustrating a speaker authentication method of a voice recognition system according to another exemplary embodiment.

Hereinafter, exemplary embodiments of the present invention will be described in detail with reference to the accompanying drawings so that those skilled in the art can easily implement the present invention. However, the invention is embodied in various different forms and is not limited to the embodiments described herein. In the drawings, in order to clearly describe the present invention, parts unrelated to the description are omitted, and like parts are denoted by like reference numerals throughout.

Throughout the specification, when a part is “connected/connected” to another part, it means not only when it is “directly connected/connected” but also in the middle with another element interposed. It also includes the case of being “electrically connected/connected”. In addition, when a certain part "includes" a certain constituent element, it does not exclude other constituent elements, and may further include other constituent elements unless there is a specific opposite statement. Means

The appearances of the phrase "in some embodiments" or "in one embodiment" in various places in this specification are not necessarily all referring to the same embodiment.

One embodiment is presented in functional block organization and various processing stages. Some or all of such functional blocks may also be embodied by a variable number of hardware and/or software components that perform particular functions. For example, the functional blocks of the present disclosure may be embodied by one or more microprocessors or circuit configurations for predetermined functions. Further, for example, the functional blocks of the present disclosure may be embodied in various programming languages or script languages. Such functional blocks may be embodied by algorithms executed by one or more processors. Also, the present disclosure may employ conventional techniques for electronic configuration, signal processing and/or data processing, and the like. Terms such as module and configuration are general, mechanical and not limited to physical configuration.

In addition, the connection/connection lines or connection/connection members between the components illustrated in the drawings may be functional connection/connection and/or physical or circuit connection/connection. Not too much. In an actual device, various functional interconnections/connections, physical or circuit interconnections/connections, which may be substituted or added, represent interconnections/connections between components.

In the present disclosure, the voice recognition function refers to converting a voice signal including a user's voice into a character string (or text). A character string (or text) obtained by converting a voice signal by such a voice recognition function is also called a voice recognition result. The voice signal of the user includes a voice command, and such a voice recognition result may include a command corresponding to the voice command. Such a voice command may perform a specific function of the voice recognition speaker device or the voice recognition server. On the other hand, in the present disclosure, the voice synthesis function refers to conversion of a character string (or text) into a voice signal, as opposed to the voice recognition function. A voice signal in which a character string (or text) is converted by such a voice synthesizing function is also called a synthetic voice signal.

In the present disclosure, the expression "registered" means registered in the voice recognition system as a user or related information thereof. “Registered user” means a user who has completed user registration in the voice recognition system. One person can register as a user in the voice recognition system according to the present disclosure, and when registering as a user, the voice of the person who speaks a sentence presented by such a voice recognition system can be input. Such a voice recognition system can extract a speaker feature vector from a voice signal of a voice input at the time of user registration and store it as related information of the registered user. As such, the speaker feature vector stored in the speech recognition system may be referred to as the registered speaker feature vector. In addition, at the time of user registration, the identification number of the mobile device owned by the user can be stored together.

The user-related information stored in the voice recognition system may include a cipher used for user authentication. In addition, at the time of user registration, the user can input the encrypted voice uttering the unique encryption to the voice recognition system. Such a voice recognition system can store the encrypted voice signal of the encrypted voice, and can store the voice recognition result of the encrypted voice signal, that is, the encrypted character string or the speaker feature vector extracted from the encrypted voice signal. .. The encrypted voice signal, the encrypted character string, and the speaker feature vector extracted from the encrypted voice signal stored in such a voice recognition system are the registered encrypted voice signal, the registered encrypted character string, and the registered speech, respectively. It is also called the person feature vector.

A plurality of users are registered in such a voice recognition system. In the present disclosure, the first speaker means a person who actually utters the voice of the voice signal, and the registered second speaker is a voice recognition system among a plurality of users registered in the voice recognition system. , Means a user who recognizes or determines that the voice of the voice signal is uttered. The registered second speaker is generally the same as the first speaker, but if speaker misrecognition or voice theft occurs in the voice recognition system, the registered second speaker is the first speaker. Different from the speaker.

In the present disclosure, keywords can have word forms or phrase forms. In the present disclosure, a voice command spoken after a wake-up keyword may have a natural language form of sentence form, word form or phrase form.

Hereinafter, the present disclosure will be described in detail with reference to the accompanying drawings.

FIG. 1 is an exemplary network configuration diagram of a voice recognition system according to an embodiment. Referring to FIG. 1, the network environment of the voice recognition system is illustrated as including a voice recognition speaker device 100, a voice recognition server 200, a mobile device 300, and a network 400. Such a voice recognition system includes a voice recognition speaker device 100 and a voice recognition server 200.

The voice recognition speaker device 100 is an example of a voice recognition device, and is a speaker device that is equipped with a voice control function and executes a specific function. The voice recognition speaker device 100 is also called a smart speaker device or an artificial intelligence speaker device. When the voice recognition speaker device 100 receives a voice of a speaker, the voice recognition speaker device 100 recognizes the voice and the speaker, extracts a command included in the voice, executes an operation according to the command, and outputs the result as a voice. You can The specific function that the voice recognition speaker device 100 can perform is, for example, voice information provision, music reproduction, internet shopping, financial transaction, telephone call, message transmission, alarm setting, and voice recognition speaker device via a network. It may include control of connected electronic or mechanical devices.

For example, when the voice recognition speaker device 100 is connected to a smart TV via a network, the specific function may include channel viewing, channel search, video playback, program search, and the like. For example, when the voice recognition speaker device 100 is connected to a home electric appliance such as a smart refrigerator, the specific function may include a check of a refrigerating state and a freezing state, a temperature setting, and the like. However, in the present disclosure, such a specific function is not limited to the above-described function.

The voice recognition speaker device 100 can communicate with the voice recognition server 200 via the network 400 via wireless communication or wired communication.

The communication method of the network 400 is not limited, and is not limited to a communication method that utilizes a communication network (for example, a mobile communication network, a wired internet, a wireless internet, a broadcasting network) included in the network 400, and a voice recognition speaker. Near field communication with the device 100 may be included. For example, the network 400 includes a PAN (personal area network), a LAN (local area network), a CAN (campus area network), a MAN (metropolitan area network), a WAN (wide area network), a BBN (broadband network), the Internet, and the like. It may include any one or more of the networks. Network 400 may include any one or more of network topologies including, but not limited to, bus networks, star networks, ring networks, mesh networks, star bus networks, tree or hierarchical networks, and the like. Absent.

The voice recognition server 200 communicates with the voice recognition speaker device 100 via the network 400, and may be embodied by at least one computer device. The voice recognition server 200 is distributed in a cloud form and can provide instructions, codes, files, contents and the like.

The voice recognition server 200 can convert the voice signal received from the voice recognition speaker device 100 into a character string (or text) and generate a voice recognition result. The voice recognition server 200 can synthesize voices reproduced by the voice recognition speaker device 100, generate a synthesized voice signal, and can transmit the synthesized voice signal to the voice recognition speaker device 100.

The voice recognition server 200 can actually perform the specific function that the voice recognition speaker device 100 can perform. For example, in the case of the voice information providing function, the voice recognition server 200 recognizes the information request included in the voice signal received from the voice recognition speaker device 100, generates a result according to the information request, and outputs the synthesized voice. It can be transmitted to the voice recognition speaker device 100 in the form of a signal. In the case of the telephone call function, the voice recognition server 200 recognizes the telephone call request included in the voice signal received from the voice recognition speaker device 100, executes the telephone call according to the request, and transmits when the telephone call is made. The signal and the received signal can be relayed. The voice recognition server 200 is also connected to a home electric appliance via the network 400, and the voice recognition server 200 controls the home electric appliance by a control command included in a voice signal received from the voice recognition speaker device 100. be able to.

The voice recognition server 200 is also connected to the mobile device 300 via the network 400. The network that connects the voice recognition server 200 and the voice recognition speaker device 100 and the network that connects the voice recognition server 200 and the mobile device 300 may be of different types. For example, the network connecting the voice recognition server 200 and the voice recognition speaker device 100 may be a LAN or the Internet, and the network connecting the voice recognition server 200 and the mobile device 300 may be a mobile communication network. According to one embodiment, the voice recognition server 200 is not connected to the mobile device 300.

The mobile device 300 is an electronic device that supports wireless communication and can be carried around by a user. For example, the mobile device 300 may be a mobile phone, a smartphone, a tablet PC (personal computer), a notebook PC, or the like. The mobile device 300 may have a telephone function, a message function, or a messenger function. The mobile device 300 can convert a user's voice into a voice signal and provide the voice signal to the voice recognition server 200. In addition, the mobile device 300 can reproduce the audio signal or the video signal received from the voice recognition server 200. The mobile device 300 is also generally an electronic device used by one individual.

FIG. 1 illustrates that the voice recognition speaker device 100 is connected to the voice recognition server 200 that performs the voice recognition function via the network 400, but it is only an example. The recognition speaker device 100 can also independently perform a voice recognition function or a voice synthesis function.

FIG. 2 is a block diagram for explaining the internal configuration of the voice recognition speaker device 100 according to the embodiment. Referring to FIG. 2, the voice recognition speaker device 100 may include a processor 110, a microphone 120, a speaker 130, and a communication module 140. The voice recognition speaker device 100 may include more components than those shown in FIG. For example, the voice recognition speaker device 100 may further include a memory. The voice recognition speaker device 100 is connected to the network 400 of FIG. 1 via the communication module 140 and can communicate with the voice recognition server 200.

The microphone 120 can directly generate an audio signal by converting surrounding audio into electrical acoustic data. Further, the voice recognition speaker device 100 may include a plurality of microphones 120, and the plurality of microphones 120 can be used to find the input direction of the audio signal. According to another example, the voice recognition speaker device 100 can also receive an audio signal transmitted from an external device via the communication module 140. The speaker 130 can convert an audio signal into voice and output the voice.

Processor 110 is configured to process instructions of a computer program by performing basic arithmetic, logic and input/output operations. Such instructions may be provided to processor 110 from memory or received via communication module 140 and provided to processor 110. For example, the processor 110 may be configured to execute instructions according to program code stored in a recording device such as a memory.

The processor 110 can detect a voice signal corresponding to the voice of the speaker from the audio signal generated by the microphone 120, and can transmit the detected voice signal to the voice recognition server 200 via the communication module 140. .. The processor 110 can detect a voice signal from an audio signal by using the keyword. The processor 110 can identify a voice signal received subsequent to the keyword voice signal by extracting a keyword voice signal corresponding to the keyword from the audio signal.

The processor 110 can receive a synthetic sound signal from the voice recognition server 200 and reproduce a synthetic sound corresponding to the synthetic sound signal via the speaker 130.

FIG. 3 is a block diagram for explaining the internal configuration of the voice recognition server 200 according to the embodiment. Referring to FIG. 3, the voice recognition server 200 includes a processor 210, a memory 220 and a communication module 230. Speech recognition server 200 may include more components than those illustrated in FIG. For example, the voice recognition server 200 may further include an input/output device.

The communication module 230 can provide a function for the voice recognition server 200 to communicate with the voice recognition speaker device 100 via the network 400. The voice recognition server 200 is connected to the network 400 of FIG. 1 via the communication module 230 and can communicate with the voice recognition speaker device 100. According to one embodiment, the voice recognition server 200 may also communicate with the mobile device 300 via the communication module 230.

The memory 220 is a computer-readable recording medium, and may include a permanent mass storage device such as a random access memory (RAM), a read-only memory (ROM), and a disk drive. The memory 220 stores an operating system and at least one program code (eg, code for a voice recognition application, a voice synthesis application, etc. installed and executed in the voice recognition server 200). Such software components may be loaded into memory 220 via communications utilizing communications module 230. For example, at least one program is loaded into the memory 220 based on a program installed by a file provided by a file distribution system that distributes an installation file of a developer or an application via the network 400.

The processor 210 performs basic arithmetic, logic and input/output operations and may be configured to process computer program instructions. Processor 210 may be configured to execute instructions according to program code stored in memory 220.

The processor 210 may be configured to receive a voice signal including the voice of the first speaker from the voice recognition speaker device 100, perform voice recognition on the voice signal, and generate a first voice recognition result. For example, the processor 210 may perform frequency recognition of a voice signal, extract frequency characteristics of the voice signal, and utilize the acoustic model and the language model to perform voice recognition. Such frequency characteristics mean a distribution of frequency components of the acoustic input extracted by analyzing the frequency spectrum of the acoustic input. The acoustic model and the language model are stored in the memory 220. However, the voice recognition method is not limited to this, and various techniques for converting a voice signal into a character string (or text) are used. In the present disclosure, the first voice recognition result means a result of performing voice recognition on a voice signal including the voice of the first speaker.

The processor 210 can analyze the voice signal to determine who is the speaker who uttered the voice contained in the voice signal. The processor 210 extracts the first speaker feature vector from the voice signal, compares the first speaker feature vector with the registered speaker feature vector, and registers the speaker of the voice signal according to the result of this comparison. May be configured to determine to be a second speaker. For example, the processor 210 identifies the speaker of the voice signal by calculating the similarity between the first speaker feature vector and the registered speaker feature vector and comparing the calculated similarity with a reference value. be able to. In the present specification, the first speaker feature vector means a speaker feature vector extracted from a voice signal including the voice of the first speaker. The registered second speaker is one of the users registered in the voice recognition system, and means a person who is registered in advance so as to normally use the voice recognition speaker device 100.

The registered speaker feature vector is stored in the memory 220 in advance as related information of the second speaker at the time of user registration. A plurality of speakers are registered in the voice recognition server 200, and in that case, a plurality of registered speaker feature vectors are stored in the memory 220. The registered speaker feature vector is related information of the registered speaker and corresponds to each registered speaker. The second speaker is one of the users registered in advance in the voice recognition server 200.

The processor 210 uses at least one of states posteriors extracted from the acoustic model, general background models, and global variability transformation information to determine the speaker of the speech signal. A speaker feature vector can be generated from the frequency characteristic. The memory 220 stores at least one of posterior information, general background model, global variability conversion information, and registered speaker information.

The processor 210 may determine whether the speaker of the audio signal is a registered speaker based on the first speaker feature vector and the registered speaker feature vector stored in the memory 220. it can. The processor 210 can compare the first speaker feature vector with each of the registered speaker feature vectors. The processor 210 can select the registered speaker feature vector having the highest similarity to the first speaker feature vector as the registered second speaker feature vector. If the highest similarity is greater than or equal to the first reference value, the processor 210 may determine the registered second speaker of the registered second speaker feature vector to be the speaker of the audio signal. it can. If the highest similarity is less than the first reference value, the processor 210 can determine that the speaker of the audio signal is an unregistered speaker.

The processor 210 may be configured to request an authenticated voice from the first speaker or the second speaker and receive an authenticated voice signal from the first speaker or the second speaker. The processor 210 additionally confirms whether the second speaker is the same as the first speaker based on the received authentication voice signal, so that the first speaker and the second speaker are the same. You can authenticate your sex.

The processor 210 may be configured to perform an action corresponding to the first speech recognition result if the identities of the first speaker and the second speaker are verified. If the similarity between the first speaker feature vector and the registered second speaker feature vector is greater than or equal to the second reference value that is higher than the first reference value, processor 210 identifies the registered second speaker as It may be configured to regard as the speaker of the voice signal, omit the speaker authentication process, and immediately perform the operation corresponding to the first voice recognition result.

The processor 210 can determine the function corresponding to the first speech recognition result and execute the function. The processor 210 may be configured to generate a synthetic sound signal for reporting the performance result of the operation. The processor 210 may be configured to send the synthetic sound signal to the voice recognition speaker device 100.

The voice recognition server 200 may further include a microphone or a speaker that is an input/output device. The voice recognition server 200 can also directly generate a voice signal and directly reproduce a synthetic voice.

FIG. 4A is a block diagram illustrating an internal configuration of a processor of the voice recognition server according to the exemplary embodiment. Referring to FIG. 4A, the processor 210 of the voice recognition server 200 includes a voice signal receiving unit 211, a voice recognizing unit 212, a speaker recognizing unit 213, a speaker authenticating unit 214, a functional unit 215, and a synthetic voice signal generating unit 216. Including. The speaker recognition unit 213 includes a speaker feature vector extraction unit 213a, a speaker feature vector comparison unit 213b, and a registered speaker determination unit 213c.

The voice signal receiving unit 211 receives a voice signal including the voice of the first speaker from the voice recognition speaker device 100.

The voice recognition unit 212 performs voice recognition on the voice signal received by the voice signal reception unit 211 and generates a first voice recognition result. The voice recognition unit 212 can perform voice recognition on the voice signal and convert the voice of the speaker into a character string (or text). The voice recognition unit 212 can perform natural language processing on the converted character string (or text) to extract a speaker command included in the voice signal. The first voice recognition result includes the command of the first speaker, and the action corresponding to the voice recognition result means the action according to the command of the first speaker.

The speaker recognition unit 213 determines that the speaker of the audio signal received by the audio signal reception unit 211 is the second speaker. For example, the speaker feature vector extraction unit 213a extracts the speaker feature vector from the voice signal received by the voice signal reception unit 211. The speaker feature vector extraction unit 213a extracts a speaker feature vector by converting a time-domain-based speech signal into a frequency-domain signal and transforming the converted signals so that the frequency energies of the signals are different from each other. You can For example, such a speaker feature vector is extracted based on the mel frequency cepstrum coefficient or the filter bank energy, but is not limited thereto, and the speaker feature vector is extracted from the audio data by various methods. can do. The speaker feature vector extracted from the voice signal including the voice of the first speaker is called the first speaker feature vector.

The speaker feature vector comparison unit 213b compares the first speaker feature vector extracted by the speaker feature vector extraction unit 213a with the registered speaker feature vector stored in the memory 220. For example, the speaker feature vector comparison unit 213b calculates the similarity between the first speaker feature vector and the registered speaker feature vector.

There are a plurality of registered speaker feature vectors in the memory 220, and the speaker feature vector comparison unit 213b compares the first speaker feature vector with each of the registered speaker feature vectors, and A registered speaker feature vector with high similarity is determined. The registered speaker feature vector with the highest degree of similarity is called the second speaker feature vector.

The registered speaker determination unit 213c determines that the speaker of the voice signal is the registered second speaker based on the result of the comparison by the speaker feature vector comparison unit 213b. For example, when the degree of similarity between the first speaker feature vector and the second speaker feature vector is greater than or equal to the first reference value and less than the second reference value, the registered speaker determination unit 213c talks about the voice signal. The speaker can be determined to be the second speaker corresponding to the second speaker feature vector. The second reference value is higher than the first reference value. The second speaker is one of the users registered in the voice recognition system or the voice recognition server 200. In such an aspect, the second speaker is also referred to as a registered second speaker.

The registered speaker determination unit 213c considers that the speaker of the audio signal is the second speaker when the similarity between the first speaker feature vector and the second speaker feature vector is equal to or greater than the second reference value. be able to. In that case, since the similarity between the first speaker feature vector and the second speaker feature vector is very high, the authentication for the second speaker is omitted.

When the degree of similarity between the first speaker feature vector and each of the registered speaker feature vectors does not exceed a preset reference value, that is, the first speaker feature vector and the second speaker feature vector If the degree of similarity with the is less than the first reference value, the registered speaker determination unit 213c determines that the speaker of the voice signal is not among the registered users, that is, the speaker of the voice signal is registered. It can be determined that there is no user. In that case, the processor 210 does not perform the action corresponding to the voice recognition result, or only when the action corresponding to the voice recognition result is set to the action that anyone can perform. Can perform various actions.

The speaker authenticator 214 may be configured to request an authentication voice from the first speaker or the second speaker and receive an authentication voice signal from the first speaker or the second speaker. According to an example, the speaker authentication unit 214 can transmit an authentication voice request to the voice recognition speaker device 100. The first speaker using the voice recognition speaker device 100 can receive the authentication voice request. The first speaker speaks an authentication voice to the voice recognition speaker device 100, and the voice recognition speaker device 100 can send an authentication voice signal corresponding to the authentication voice to the processor 210.

According to another example, the speaker authenticator 214 requests the second speaker's mobile device 300 for an authentication voice, and the second speaker can speak the authentication voice in response to the authentication voice request. .. The second speaker can speak the authentication voice to the voice recognition speaker device 100 or the portable device 300 of the second speaker. The identification information of the mobile device 300 of the second speaker can be input when the second speaker registers as a user in the voice recognition system and can be stored in the memory 220 in advance. Here, the speaker authentication unit 214 can transmit the voice recognition result and the voice signal together with the request for the authentication voice to the portable device 300 of the second speaker. The mobile device 300 of the second speaker is also referred to as the mobile device 300 with which the second speaker is registered.

The speaker authentication unit 214 additionally confirms whether the second speaker is the same as the first speaker based on the received authentication voice signal, and determines whether the second speaker is the same as the first speaker. Identity can be verified.

The functional unit 215 executes an operation corresponding to the first voice recognition result generated by the voice recognition unit 213 when the identity between the first speaker and the second speaker is authenticated by the speaker authentication unit 214. To do. When the similarity between the first speaker feature vector and the second speaker feature vector is greater than or equal to the second reference value, the function unit 215 generates the first speaker generated by the voice recognition unit 213 without a speaker authentication procedure. The operation corresponding to the voice recognition result can be executed.

In the function unit 215, the registered speaker determination unit 213c determines that the speaker of the voice signal is an unregistered speaker, or the speaker authentication unit 214 determines the first speaker and the second speaker. If the identity with the person is not authenticated, the operation corresponding to the first voice recognition result is not executed.

When the functional unit 215 executes the operation, the synthetic sound signal generation unit 216 generates a synthetic sound signal for reporting the execution result of the operation. The synthesized voice signal generation unit 216 determines that the speaker of the voice signal is an unregistered user, and the operation corresponding to the first voice recognition result is not executed, or the speaker authentication unit 214 If the identity is not authenticated and the operation corresponding to the first voice recognition result is not executed, a synthetic sound signal for reporting that the operation is not executed can be generated.

According to another embodiment, the processor 210 may further include a speaker vector improver. The similarity between the first speaker feature vector and the second speaker feature vector is greater than or equal to the second reference value, or the speaker authenticating unit 214 determines that the first speaker and the second speaker have the same identity. If it is authenticated, it is confirmed that the speaker of the voice signal is the second speaker. Therefore, such a speaker vector improving unit uses the first speaker feature vector. However, the second speaker feature vector of the second speaker stored in the memory 220 can be improved. The speaker feature vector improving unit generates a second speaker feature vector of the second speaker through the adaptive training method using the first speaker feature vector extracted from the voice signal, and newly generates the second speaker feature vector. When the adaptive training performance of the generated second speaker feature vector is higher than that of the second speaker feature vector before adaptive training, the newly generated second speaker feature vector is stored in the memory 220. As a result, the second speaker feature vector can be improved.

FIG. 4B is a block diagram for explaining the internal configuration of the processor of the voice recognition server according to another embodiment. Referring to FIG. 4B, the processor 210a of the voice recognition server 200 includes a voice signal receiving unit 211, a voice recognizing unit 212, a speaker recognizing unit 213, an authentication decision unit 217, a speaker authenticating unit 214, a functional unit 215, and a synthetic voice. The signal generation unit 216 is included.

Since the similarity between the first speaker feature vector and the second speaker feature vector is greater than or equal to the first reference value and less than the second reference value, the registered speaker determination unit 213c determines that the speaker of the audio signal is If it is determined that the speaker is the second speaker, the authentication determination unit 217 determines the speaker authenticator based on at least one of the operation corresponding to the first voice recognition result and the setting of the second speaker. It is determined whether to execute the operation of 214.

According to an example, when the operation based on the first voice recognition result is included in the pre-approved operation list preset by the second speaker, the authentication decision unit 217 performs the operation of the speaker authenticating unit 214. Can be determined. The pre-approved operation list is stored in the memory 220, and some operations that can be performed by the voice recognition speaker device 100 or the voice recognition server 200 may be included in the pre-approved operation list set in advance. .. For example, operations such as financial transactions, internet shopping, and message sending may be included in the preset pre-approved operation list. The operations included in the pre-approved operation list are set differently for each registered speaker.

According to another example, when the operation based on the first voice recognition result is included in the posterior notification operation list preset by the second speaker, the authentication decision unit 217 causes the function unit 215 to perform the first voice recognition result. Then, the operation corresponding to is first performed, and it may be determined to transmit the voice recognition result and the voice signal to the mobile device 300 of the second speaker. The post notification operation list is stored in the memory 220, and some of the operations that can be performed by the voice recognition speaker device 100 may be included in the post notification operation list set in advance. For example, operations such as making a call and changing settings may be included in the post-notification operation list that is set in advance. The operation included in the post notification operation list is set to be different for each registered speaker.

According to still another example, the authentication determination unit 217 determines the operation of the speaker authenticator 214 when at least one of the position of the portable device 300 of the second speaker and the current time matches the pre-approval condition. You can decide to do it. For example, when the mobile device 300 of the second speaker is located near the position of the voice recognition speaker device 100, for example, the mobile device 300 of the second speaker and the voice recognition speaker device 100 are the same wireless Wi-Fi. When connected to a (registered trademark) access point, or when the GPS (global position system) position of the portable device 300 of the second speaker or the wireless network connection position substantially matches the position of the voice recognition speaker device 100. , It is highly possible that the second speaker actually uttered the voice included in the voice signal received by the voice signal receiving unit 211. Therefore, the authentication decision unit 217 causes the speaker authenticating unit 214 to operate. It can be omitted. The registered speaker can set whether or not the operation of the speaker authenticating unit 214 is omitted.

The authentication determination unit 217 can determine to execute the operation of the speaker authentication unit 214 at the time set by the second speaker, for example, weekday daytime. For example, since the second speaker who is an office worker is likely not to be at home during the daytime on weekdays, the voice recognition speaker device 100 located at the home is unlikely to receive the voice of the second speaker. Low. In such a case, the authentication decision unit 217 can decide to execute the operation of the speaker authentication unit 214. The registered speaker can set whether to execute the operation of the speaker authenticating unit 214 based on time.

The pre-approval condition is preset by the second speaker and is also stored in the memory 220. Further, when it is determined that the operation of the speaker authentication unit 214 is performed based on the current time, the pre-approval condition is also determined based on the behavior pattern of the second speaker. The action pattern of the second speaker is generated based on the position of the portable device 300 of the second speaker and the execution action. For example, the authentication determination unit 217 can collect the position of the portable device 300 of the second speaker for a long time. The authentication determination unit 217 can analyze the position of the mobile device 300 and determine a time period in which the second speaker is not located near the voice recognition speaker device 100. The authentication determination unit 217 can automatically determine to perform the operation of the speaker authentication unit 214 when the current time falls within this time zone.

FIG. 5 is an exemplary flowchart for explaining a speaker authentication method of a voice recognition system according to an exemplary embodiment. Referring to FIG. 5, the voice recognition system includes a voice recognition speaker device 100 and a voice recognition server 200.

The voice recognition speaker device 100 can use the microphone 120 (FIG. 2A) to electrically convert surrounding sounds and generate an audio signal (S101).

The voice recognition speaker device 100 can detect a voice signal from the audio signal (S102). Such a voice signal may include the voice of the user. Here, the user is the first speaker. Such voice may include a voice command of the first speaker. Such voice commands may include voice information retrieval, phone calls, message transmissions, financial transactions, internet shopping, food delivery, peripheral appliance controls, smart home controls, and the like. In this example, it is assumed that the voice command relates to a financial transaction, and the voice of the first speaker is "Send 1 million won to B". The voice of the first speaker may include a trigger keyword for waking up the voice recognition speaker device 100. The voice recognition speaker device 100 can detect the voice signal from the audio signal by recognizing the trigger keyword.

The voice recognition speaker device 100 transmits a voice signal to the voice recognition server 200, and the voice recognition server 200 receives the voice signal from the voice recognition speaker device 100 (S103).

The voice recognition server 200 performs voice recognition on the voice signal and generates a first voice recognition result (S104). The voice recognition server 200 can perform the voice recognition by extracting the frequency characteristic of the voice signal and using the acoustic model and the language model. The voice recognition server 200 can generate a voice recognition result by converting a voice signal into a character string and subjecting the character string to natural language processing. The voice recognition result may include the voice command of the first speaker.

The voice recognition server 200 extracts the first speaker feature vector from the voice signal (S105). The voice recognition server 200 uses at least one of the posterior information extracted from the acoustic model, the general background model, and the global variability conversion information to generate the first speaker feature vector from the frequency characteristic of the voice signal. can do.

The voice recognition server 200 compares the first speaker feature vector with the registered speaker feature vector (S106). The registered speaker feature vector is stored in the memory 220 (FIG. 3) and is generated in advance based on the user's voice input when the user registers with the voice recognition system. The voice recognition server 200 can calculate the similarity between the first speaker feature vector and the registered speaker feature vector. The memory 220 stores a plurality of speaker feature vectors corresponding to the authorized users of the voice recognition speaker device 100, and the voice recognition server 200 stores the first speaker feature vector and each of the registered speaker feature vectors. Can be calculated, and the highest similarity among the calculated similarities can be determined. Here, among the registered speaker feature vectors, the registered speaker feature vector having the highest similarity to the first speaker feature vector is called the second speaker feature vector, and the highest similarity is It is called the first similarity.

The voice recognition server 200 compares the first similarity with the first reference value ref1 (S107). The first reference value ref1 is also determined by the speaker recognition performance of the voice recognition server 200. When the first similarity is equal to or higher than the first reference value ref1, the voice recognition server 200 determines that the speaker of the voice signal is the second speaker corresponding to the second speaker feature vector (S108). .. Here, the second speaker is one of the users of the voice recognition speaker device 100 registered in the voice recognition system, and is also called the registered second speaker.

When the first similarity is less than the first reference value ref1, the voice recognition server 200 determines that the speaker of the voice signal is an unregistered user, and executes the operation corresponding to the first voice recognition result. No (S115). In that case, the voice recognition server 200 generates a synthetic voice signal corresponding to a synthetic voice, for example, "The speaker of the voice was not identified and no operation was performed. Please input the voice again". Can be performed (S116).

When it is determined that the speaker of the voice signal is the second speaker, the voice recognition server 200 compares the first similarity with the second reference value ref2 (S109). The second reference value ref2 is higher than the first reference value ref1 and is also determined by the speaker recognition performance of the voice recognition server 200. The order of the steps (S107 to S109) illustrated in FIG. 5 is exemplary, and the order is variable.

When the first similarity is equal to or higher than the second reference value ref2, the similarity between the first speaker feature vector and the second speaker feature vector is very high, and thus the voice recognition server 200 determines that the speaker of the voice signal is the speaker. Can be considered to be the second speaker. In that case, the voice recognition server 200 can perform the operation corresponding to the first voice recognition result without further authentication procedure (S114).

If the first similarity is less than the second reference value ref2, the voice recognition server 200 performs an authentication procedure to further confirm whether the first speaker matches the second speaker. You can The second speaker is generally the same as the first speaker. However, due to the error in the speaker recognition function of the voice recognition server 200, the second speaker is different from the first speaker.

For example, the first speaker utters a voice command, but the voice recognition server 200 recognizes that the voice command is uttered by a second speaker different from the first speaker due to an error in the speaker recognition function. Will end up. In that case, the voice recognition server 200 recognizes that the second speaker uttered “Send 1 million won to B”, and the voice recognition server 200 sends 1 million won to B from the account of the second speaker. The problem of sending money occurs.

As another example, the first speaker utters a voice command imitating the voice of the second speaker, and the voice recognition server 200 recognizes that the second speaker utters this voice command. .. In that case, it is a case of using voice theft. Even in that case, the voice recognition server 200 recognizes that the second speaker uttered “Send 1 million won to B”, and the voice recognition server 200 receives 1 million from the second speaker's account to B. There is a problem of sending won.

In order to prevent such a problem, according to the present embodiment, the voice recognition server 200 can request the voice recognition speaker device 100 to perform encrypted utterance. The encrypted voice request is transmitted from the voice recognition server 200 to the voice recognition speaker device 100 (S110). For example, the voice recognition server 200 can generate a synthetic voice signal corresponding to a synthetic voice of “Please tell me the code” and transmit the synthetic voice signal to the voice recognition speaker device 100. The voice recognition speaker device 100 can receive the synthesized sound signal and can reproduce the synthesized sound using the speaker 130 (FIG. 2).

The first speaker using the voice recognition speaker device 100 can speak a cipher in response to the cipher voice request. The voice recognition speaker device 100 can detect the encrypted voice signal including the encrypted voice uttered by the first speaker (S111). The encrypted voice signal is also transmitted from the voice recognition speaker device 100 to the voice recognition server 200 (S112). The encrypted voice signal is also called an authentication voice signal.

The voice recognition server 200 can authenticate the identity of the second speaker and the first speaker based on the received encrypted voice signal (S113). If the voice recognition server 200 cannot receive the encrypted voice signal from the voice recognition speaker device 100 within the preset time, it is determined that the speaker of the voice signal is not the second speaker, and the identity is determined. Can be denied.

According to an example of the step (S113), the voice recognition server 200 may perform voice recognition on the received encrypted voice signal to generate a second voice recognition result. The voice recognition for the encrypted voice signal and the voice recognition for the voice signal of the first speaker are performed by the same method. The second voice recognition result may include the encrypted voice uttered by the first speaker. The voice recognition server 200 can detect the part corresponding to the encryption from the second voice recognition result.

The voice recognition server 200 can compare the second voice recognition result with the registered cipher of the second speaker. The registered cipher of the second speaker is previously registered by the second speaker when the second speaker registers as a user of the voice recognition speaker device 100 in the voice recognition system, and is stored in the memory 220. Has been done.

If the second voice recognition result and the second speaker's cipher are substantially the same, for example, if the second voice recognition result includes the second speaker's cipher, It is determined that the first speaker and the second speaker are the same, and the identity between the first speaker and the second speaker is verified. In that case, the voice recognition server 200 can execute the operation based on the first voice recognition result (S114). In that case, the voice recognition server 200 can generate a synthetic sound signal corresponding to the synthetic sound, for example, "Sent 1 million won to B to the account of the second speaker" (S116).

The voice recognition server 200 determines that the first speaker and the second speaker are different from each other when the second voice recognition result and the cipher of the second speaker are not substantially the same, and determines that the first speaker is the same as the first speaker. It can be determined that the identity authentication with the second speaker has failed. In that case, the voice recognition server 200 does not execute the operation corresponding to the first voice recognition result (S115). In that case, the voice recognition server 200 can generate a synthetic voice signal corresponding to the synthetic voice, for example, “the code does not match and the operation was not executed” (S116).

According to another example of step S113, the voice recognition server 200 may extract the third speaker feature vector from the received encrypted voice signal. The voice recognition server 200 can compare the third speaker feature vector with the fourth speaker feature vector extracted from the registered encrypted voice signal of the second speaker. The encrypted voice signal registered by the second speaker is generated in advance based on the encrypted voice uttered by the second speaker when the second speaker registers as a user of the voice recognition speaker device 100 in the voice recognition system. , Are stored in the memory 220. The fourth speaker feature vector is also extracted from the encrypted voice signal registered when the encrypted voice signal registered with the second speaker is generated, and is also stored in the memory 220. Only the fourth speaker feature vector is stored in the memory 220, and the registered encrypted voice signal of the second speaker is not stored.

When the similarity between the third speaker feature vector and the fourth speaker feature vector is higher than a preset reference value, the voice recognition server 200 determines that the first speaker and the second speaker are the same. If there is, the identity between the first speaker and the second speaker is verified. In that case, the voice recognition server 200 can execute the operation based on the first voice recognition result (S114). The third-speaker feature vector and the fourth-speaker feature vector are extracted from the voice signal including the voice uttered with the same cipher, and thus the reliability of the similarity result is high.

The voice recognition server 200 determines that the first speaker and the second speaker are different from each other when the similarity between the third speaker feature vector and the fourth speaker feature vector is lower than a preset reference value. It is determined that the identity authentication between the first speaker and the second speaker has failed, and the operation corresponding to the first voice recognition result is not executed (S115).

According to yet another example of operation S113, the voice recognition server 200 may extract the third speaker feature vector from the received encrypted voice signal. The voice recognition server 200 can compare the third speaker feature vector with the fourth speaker feature vector extracted from the registered encrypted voice signal of the second speaker.

The voice recognition server 200 can perform voice recognition on the received encrypted voice signal and generate a second voice recognition result. The second voice recognition result may include the encrypted voice uttered by the first speaker, and the voice recognition server 200 can detect the portion corresponding to the cipher from the second voice recognition result. The voice recognition server 200 can compare the second voice recognition result with the third voice recognition result of the encrypted voice signal registered by the second speaker. The third voice recognition result is a result of voice recognition performed on the encrypted voice signal registered by the second speaker when the encrypted voice signal registered by the second speaker is generated. Stored in advance. The third voice recognition result may also include the encrypted voice uttered by the second speaker, and the voice recognition server 200 can detect the portion corresponding to the cipher from the third voice recognition result.

In the voice recognition server 200, the similarity between the third speaker feature vector and the fourth speaker feature vector is higher than a preset reference value, and the second voice recognition result and the third voice recognition result are substantially the same. , The identities of the first and second speakers are verified. In that case, the voice recognition server 200 can execute the operation based on the first voice recognition result (S114).

The voice recognition server 200 determines whether the degree of similarity between the third speaker feature vector and the fourth speaker feature vector is lower than a preset reference value, or the second voice recognition result and the third voice recognition result. Are not substantially the same, it is determined that the first speaker and the second speaker are different from each other, it is determined that the identity authentication between the first speaker and the second speaker has failed, and the first voice The operation corresponding to the recognition result may not be executed (S115).

If the identity between the first speaker and the second speaker is verified in step S113, or if the first similarity is greater than or equal to the second reference value ref2 in step S109, the voice recognition server 200 , The operation corresponding to the first voice recognition result can be executed (S114). The voice recognition server 200 can generate a synthetic voice signal for reporting the result of executing the operation corresponding to the first voice recognition result (S116).

If the identity between the first speaker and the second speaker is not authenticated in step S113, or if the first similarity is less than the first reference value ref1 in step S107, the voice recognition server 200 determines that The operation corresponding to the first voice recognition result is not executed (S115). The voice recognition server 200 may generate a synthesized voice signal for reporting that the operation corresponding to the first voice recognition result has not been executed (S116).

The voice recognition server 200 can transmit the generated synthetic voice signal to the voice recognition speaker device 100 (S117). The voice recognition speaker device 100 can reproduce the synthetic sound corresponding to the synthetic sound signal (S118). Therefore, the first speaker who speaks the voice of the voice signal can directly confirm the execution result of his voice command.

According to another embodiment, in step S114, since the first speaker and the second speaker are confirmed to be the same, the voice recognition server 200 performs the operation corresponding to the first voice recognition result while performing the operation. By using the one-speaker feature vector, the registered second-speaker feature vector of the second speaker can be improved.

FIG. 6 is an exemplary flowchart illustrating a speaker authentication method of a voice recognition system according to another exemplary embodiment. Referring to FIG. 6, the voice recognition system includes a voice recognition speaker device 100 and a voice recognition server 200. The portable device 300 of the second speaker is connected to the voice recognition server 200 via the network.

The steps (S201-S209) and the steps (S216-S220) illustrated in FIG. 6 are substantially the same as the steps (S101-S109) and the steps (S114-S118) described with reference to FIG. 5, respectively. As such, they will not be repeated. Hereinafter, the steps (S210 to S215) different from the embodiment of FIG. 5 will be mainly described.

In steps S207 to S209, when the first similarity is greater than or equal to the first reference value ref1 and less than the second reference value ref2, the voice recognition server 200 determines that the speaker of the voice signal is the second speaker. While deciding that, the authentication procedure can be performed to further confirm whether the first speaker matches the second speaker.

The voice recognition server 200 may generate a one-time password (OTP) for the authentication procedure (S210), and may transmit the generated one-time password to the mobile device 300 of the second speaker (S211). .. As described above, the identification number of the mobile device 300 is also input when the second speaker registers as a user in the voice recognition system and is stored in the memory 220. The one-time password is transmitted from the voice recognition server 200 to the mobile device 300 by a method such as a text message, a text message of a chat application, and an image including the one-time password. For example, such a one-time password can be a single or multiple digit number. According to another example, such a one-time password may be a text word, a text sentence or an object image.

The mobile device 300 can receive the one-time password and display it on the display window (S212). The second speaker who is the owner of the portable device 300 can confirm the one-time password displayed in the display window (S212a). For example, in the display window of the mobile device 300, a notification window including a message "The authentication number is XXX. Please press the authentication button and say the authentication number in front of the voice recognition speaker device" is activated. It In the notification window, the remaining time is displayed along with the authentication button.

After confirming the message in the notification window and pressing the authentication button, the second speaker can speak the authentication number specified in the one-time password in front of the voice recognition speaker device (S212a). The voice recognition speaker device 100 can detect the authentication voice signal including the voice of the authentication number uttered by the second speaker (S213). The authentication voice signal is also transmitted from the voice recognition speaker device 100 to the voice recognition server 200 (S214).

The voice recognition server 200 can authenticate the identity between the second speaker and the first speaker based on the received authentication voice signal (S215). When the voice recognition server 200 cannot receive the authentication voice signal from the voice recognition speaker device 100 within the preset time, it is determined that the speaker of the voice signal is not the second speaker, and the identity is determined. Can be denied.

According to an example of step S215, the voice recognition server 200 may perform voice recognition on the received authentication voice signal and generate a second voice recognition result. The second voice recognition result may include the voice of the one-time password or the authentication number uttered by the first speaker. The voice recognition server 200 can detect the part corresponding to the one-time password or the authentication number from the second voice recognition result. The voice recognition for the authentication voice signal and the voice recognition for the voice signal of the first speaker are performed by the same method. According to another example, the one-time password can be a single digit or multiple digit number. In that case, the authentication voice signal includes a voice that speaks a number, and when performing voice recognition on the authentication voice signal, a language model specialized for the number is used. In contrast, speech recognition for the first speaker's speech signal is also performed using a character-specific language model.

The voice recognition server 200 may compare the second voice recognition result with the one-time password or the designated number generated by the voice recognition server 200 in step S210.

When the second voice recognition result is substantially the same as the one-time password or the designated number, for example, when the second voice recognition result includes the one-time password or the designated number, the voice recognition server 200 It is determined that the first speaker and the second speaker are the same, and the identity between the first speaker and the second speaker is verified. In that case, the voice recognition server 200 can execute the operation based on the first voice recognition result (S216). In that case, the voice recognition server 200 can generate a synthetic voice signal for reporting the execution of the operation (S218).

When the second voice recognition result and the one-time password are not substantially the same, for example, when the second voice recognition result does not include the one-time password or the designated number, the voice recognition server 200 determines the first talk. It can be determined that the speaker and the second speaker are different from each other, and it can be determined that the identity authentication between the first speaker and the second speaker has failed. In that case, the voice recognition server 200 does not execute the operation corresponding to the first voice recognition result (S217). In that case, the voice recognition server 200 can generate a synthetic voice signal for reporting the non-execution of the operation (S218).

According to another example of step S215, the voice recognition server 200 may perform voice recognition on the received authentication voice signal to generate a second voice recognition result. The voice recognition server 200 can compare the second voice recognition result with the one-time password.

The voice recognition server 200 can extract the third speaker feature vector from the received authentication voice signal. The voice recognition server 200 can compare the third speaker feature vector with the registered speaker feature vector of the second speaker, that is, the second speaker feature vector. According to an example, when the one-time password is composed of numbers, the voice recognition server 200 may compare the second speaker feature vector and the third speaker feature vector with each other based on the feature vector specialized for the numbers. it can. Also, when the second speaker registers as a user, the second speaker speaks a specific sentence presented by the speech recognition system in order to generate a second speaker feature vector. It is also selected as a sentence in which numbers are well recognized.

In the voice recognition server 200, the second voice recognition result and the one-time password are substantially the same, and the similarity between the third speaker feature vector and the second speaker feature vector is a preset reference value. If it is higher, it is determined that the first speaker and the second speaker are the same, and the identity between the first speaker and the second speaker is authenticated. In that case, the voice recognition server 200 can execute the operation based on the first voice recognition result (S216).

In the voice recognition server 200, the second voice recognition result and the one-time password are not substantially the same, or the similarity between the third speaker feature vector and the second speaker feature vector is set in advance. If it is lower than the reference value, it is determined that the first speaker and the second speaker are different from each other, and it is determined that the identity authentication between the first speaker and the second speaker has failed, and the first voice recognition result. The operation corresponding to is not executed (S217).

If the identity between the first speaker and the second speaker is verified in step S215, or if the first similarity is greater than or equal to the second reference value ref2 in step S209, the voice recognition server 200 determines that , The operation corresponding to the first voice recognition result can be executed (S216). If the identity between the first speaker and the second speaker is not authenticated in step S215, or if the first similarity is less than the first reference value ref1 in step S207, the voice recognition server 200 determines that The operation corresponding to the first voice recognition result is not executed (S217).

FIG. 7 is an exemplary flowchart illustrating a speaker authentication method of a voice recognition system according to another exemplary embodiment. Referring to FIG. 7, the voice recognition system includes a voice recognition speaker device 100 and a voice recognition server 200. The portable device 300 of the second speaker is connected to the voice recognition server 200 via the network.

The steps (S301-S309) and the steps (S314-S318) illustrated in FIG. 7 are substantially the same as the steps (S101-S109) and the steps (S114-S118) described with reference to FIG. 5, respectively. As such, they will not be repeated. Hereinafter, the steps (S310 to S313) different from the embodiment of FIG. 5 will be mainly described.

In step (S307 to S309), if the first similarity is equal to or more than the first reference value ref1 and less than the second reference value ref2, the voice recognition server 200 determines that the speaker of the voice signal is the second speaker. While determining that there is, an authentication procedure can be performed to further confirm whether the first speaker matches the second speaker.

The voice recognition server 200 may request the portable device 300 of the second speaker, that is, the second speaker, to speak the same content as the voice of the first speaker for the authentication procedure (S310). ). Here, the voice of the first speaker means the voice included in the voice signal detected by the voice recognition speaker device in step S302. If the second speaker and the first speaker are the same, the second speaker can know the content of the voice signal received in step S302, which is the first speaker, that is, the second speaker. Only the person who knows it has confidentiality.

The mobile device 300 can receive the same content utterance request and display it externally, for example, in a display window. For example, in the display window of the mobile device 300, a notification window including a message "A command was executed from the voice recognition speaker device under the name of the second speaker. Please speak this command again" is active. Has been converted. The second speaker who is the owner of the portable device 300 can speak the voice of the same content in response to the request of the same content. The mobile device 300 can detect the authentication voice signal including the voice of the same content (S311). The authentication voice signal is transmitted from the mobile device 300 to the voice recognition server 200 (S312).

The voice recognition server 200 can authenticate the identity between the second speaker and the first speaker based on the received authentication voice signal (S313). When the voice recognition server 200 cannot receive the authentication voice signal from the mobile device 300 within the preset time, it is determined that the speaker of the voice signal is not the second speaker, and the identity is denied. can do.

According to an example of step S313, the voice recognition server 200 may compare the received authentication voice signal with the voice signal of the first speaker received in step S303. The authentication voice signal and the voice signal of the first speaker may have similar waveforms because they are generated at similar times and contain voices of substantially the same content. The comparison between the authentication voice signal and the voice signal of the first speaker is performed on the waveform, frequency spectrum, and the like. The method of comparing the authentication voice signal and the voice signal is not limited. As a result of the comparison between the authentication voice signal and the voice signal, the similarity between the authentication voice signal and the voice signal is calculated.

If the similarity calculated as a result of the comparison exceeds a reference value set in advance, the voice recognition server 200 determines that the first speaker and the second speaker are the same, and the first talk The identity between the person and the second speaker is verified. In that case, the voice recognition server 200 can execute the operation based on the first voice recognition result (S314). If the similarity is lower than a preset reference value, the voice recognition server 200 determines that the first speaker and the second speaker are different from each other, and the first speaker and the second speaker are the same. It can be determined that the sex authentication has failed. In that case, the voice recognition server 200 does not execute the operation corresponding to the first voice recognition result (S315).

According to another example of step S313, the voice recognition server 200 may perform voice recognition on the received authentication voice signal and generate a second voice recognition result. The voice recognition for the authentication voice signal and the voice recognition for the voice signal of the first speaker are performed by the same method. The voice recognition server 200 can compare the second voice recognition result with the first voice recognition result for the voice signal of the first speaker.

In the voice recognition server 200, when the second voice recognition result and the first voice recognition result are substantially the same, for example, when the second voice recognition result and the first voice recognition result are semantically the same. , It is determined that the first speaker and the second speaker are the same, and the identity between the first speaker and the second speaker is authenticated. In that case, the voice recognition server 200 can execute the operation based on the first voice recognition result (S314).

When the second voice recognition result and the first voice recognition result are not substantially the same, the voice recognition server 200 determines that the first speaker and the second speaker are different from each other, and determines that the first speaker and the first speaker are different from each other. It can be determined that the identity verification with the two speakers has failed. In that case, the voice recognition server 200 does not execute the operation corresponding to the first voice recognition result (S315).

According to still another example of operation S313, the voice recognition server 200 may perform voice recognition on the received authentication voice signal and generate a second voice recognition result. The voice recognition server 200 can compare the second voice recognition result with the first voice recognition result for the voice signal of the first speaker.

The voice recognition server 200 can extract the third speaker feature vector from the received authentication voice signal. The voice recognition server 200 can compare the third speaker feature vector with the first speaker feature vector. Since the authentication voice signal of the second speaker and the voice signal of the first speaker have the same contents and are generated at similar times, if the second speaker is the same as the first speaker, The speaker feature vector and the first speaker feature vector are easily compared with each other and a high degree of similarity is calculated.

In the voice recognition server 200, the second voice recognition result and the first voice recognition result are substantially the same, and the similarity between the third speaker feature vector and the first speaker feature vector is set in advance. When it is higher than the reference value, it is determined that the first speaker and the second speaker are the same, and the identity between the first speaker and the second speaker is authenticated. In that case, the voice recognition server 200 can execute the operation based on the first voice recognition result (S314).

In the voice recognition server 200, the second voice recognition result and the first voice recognition result are not substantially the same, or the similarity between the third speaker feature vector and the first speaker feature vector is determined in advance. If it is lower than the set reference value, it is determined that the first speaker and the second speaker are different from each other, it is determined that the identity authentication between the first speaker and the second speaker has failed, and the first voice The operation corresponding to the recognition result is not executed (S315).

If the identity between the first speaker and the second speaker is verified in step S313, or if the first similarity is greater than or equal to the second reference value ref2 in step S309, the voice recognition server 200 determines , The operation corresponding to the first voice recognition result can be executed (S314). If the identity between the first speaker and the second speaker is not verified in step S313, or if the first similarity is less than the first reference value ref1 in step S307, the voice recognition server 200 determines that 1 The operation corresponding to the voice recognition result is not executed (S315).

The embodiments of the present invention described above are embodied in the form of a computer program executed on various components of a computer, and the computer program may be recorded on a computer-readable medium. Here, such a medium may be one that continuously stores computer-executable programs, or one that temporarily stores them for execution or download. Further, such a medium may be various recording means or storage means in a form in which a single piece or a plurality of pieces of hardware are combined, but is not limited to the medium directly connected to a certain computer system. Instead, it may exist in a distributed manner on the network. Examples of such media are magnetic media such as hard disks, floppy disks and magnetic tapes; optical recording media such as compact disc read only memory (CD-ROM) and digital versatile disc (DVD); Some include magneto-optical media, such as floptical disks; and ROM, RAM, flash memory, etc., configured to store program instructions. Examples of other media include an application store that distributes applications, a site that supplies and distributes various other software, and a recording medium that is managed by a server or the like.

As used herein, “parts”, “modules”, etc. may be hardware components such as processors or circuits, and/or software components executed by hardware components such as processors. For example, "part", "module", etc. are software components, object-oriented software components, components such as class components and task components; and processes, functions, attributes, procedures, subroutines, program code segments, drivers, firmware. , Microcode, circuits, data, databases, data structures, tables, arrays and variables.

The above description of the present invention is for the purpose of illustration, and a person skilled in the art to which the present invention belongs does not need to change the technical idea or essential features of the present invention to obtain other specific examples. It will be understood that various forms can be easily modified. Therefore, it should be understood that the embodiments described above are illustrative in all aspects and not restrictive. For example, each component described as a single type may be implemented in a distributed manner, and similarly, components described as distributed may be implemented in a combined form. Sometimes.

The scope of the present invention is shown not by the above detailed description but by the scope of the claims, and all modifications or modified forms derived from the meaning of the scope of the claims, the scope and the equivalent concept thereof are the scope of the present invention. Should be construed as included in.

The speaker authentication method and the voice recognition system according to the embodiment of the present invention can be effectively applied to, for example, a technical field related to security.

100 voice recognition speaker device 110 processor 120 microphone 130 speaker 140 communication module 200 voice recognition server 210, 210a processor 211 voice signal receiving unit 212 voice recognition unit 213 speaker recognition unit 213a speaker feature vector extraction unit 213b speaker feature vector comparison unit 213c Registered speaker determination unit 214 Speaker authentication unit 215 Functional unit 216 Synthetic sound signal generation unit 217 Authentication decision unit 220 Memory 230 Communication module 300 Mobile device 400 Network

Claims (16)

A speaker authentication method in a voice recognition system including a voice recognition device and a voice recognition server, comprising:
The voice recognition server receives a voice signal including a voice of a first speaker from the voice recognition device;
The voice recognition server performs voice recognition on the voice signal to generate a first voice recognition result;
The voice recognition server extracts a first speaker feature vector from the voice signal, and calculates a similarity between the first speaker feature vector and the registered speaker feature vector;
If the similarity is greater than or equal to a first reference value, the voice recognition server determines that the first speaker of the voice signal is a second speaker who has completed user registration with the voice recognition server. ,
The voice recognition server requests the second speaker's mobile device to speak the same content as the first speaker's voice;
The voice recognition server receives an authentication voice signal including an authentication voice from the portable device of the second speaker;
The voice recognition server, based on the voice signal received from the voice recognition device and the authentication voice signal received from the portable device of the second speaker, the second speaker and the first speaker. Certifying the identity of
If the identity is authenticated, the voice recognition server performs an operation corresponding to the first voice recognition result;
Speaker authentication method including. The voice recognition server requesting the voice recognition device to speak a code;
The voice recognition server receives an encrypted voice signal containing encrypted voice from the voice recognition device;
The voice recognition server compares the encrypted voice signal received from the voice recognition device with a registered password speech signal of the second speaker, and compares the encrypted voice signal with the first speaker. The method for authenticating a speaker according to claim 1, further comprising a step of determining identity with two speakers. The step of determining the identity includes
The voice recognition server extracting a second speaker feature vector from the encrypted voice signal;
The speech recognition server comparing the second speaker feature vector with a speaker feature vector extracted from the registered password speech signal of the second speaker;
The voice recognition server determines the identity of the first speaker and the second speaker based on at least the result of the comparison;
The speaker authentication method according to claim 2, further comprising: The voice recognition server generating a one-time password,
The voice recognition server transmitting the one-time password to the portable device of the second speaker;
The voice recognition server further comprises a step of receiving from the voice recognition device a one-time password voice signal including a voice uttered the one-time password. Speaker authentication method. The voice recognition server performs voice recognition on the one-time password voice signal to generate a second voice recognition result;
The voice recognition server comparing the second voice recognition result with the one-time password;
The voice recognition server determines the identity of the first speaker and the second speaker based on at least the result of the comparison;
The speaker authentication method according to claim 4, further comprising: The voice recognition server performs voice recognition on the one-time password voice signal to generate a second voice recognition result;
The voice recognition server extracting a second speaker feature vector from the one-time password voice signal;
The voice recognition server determines the identity between the second voice recognition result and the one-time password and the similarity between the second speaker feature vector and the registered speaker feature vector of the second speaker. Determining the identity of the first speaker and the second speaker based on the
The speaker authentication method according to claim 4, further comprising: The step of verifying the identity includes
The voice recognition server comparing the voice signal with the authentication voice signal;
The voice recognition server determines the identity of the first speaker and the second speaker based on at least the result of the comparison;
The speaker authentication method according to claim 1, further comprising: The step of verifying the identity includes
The voice recognition server performs voice recognition on the authentication voice signal to generate a second voice recognition result;
The voice recognition server comparing the second voice recognition result with the first voice recognition result;
The voice recognition server determines the identity of the first speaker and the second speaker based on at least the result of the comparison;
The speaker authentication method according to claim 1, further comprising: After the voice recognition server determines that the first speaker of the voice signal is the registered second speaker when the similarity is equal to or higher than a second reference value that is higher than the first reference value. Corresponding to the first voice recognition result without performing the steps of requesting to speak the same content as the voice of the first speaker, receiving the authentication voice signal, and authenticating the identity. 9. The speaker authentication method according to claim 1, further comprising a step of performing the operation. When the degree of similarity is less than the first reference value, the voice recognition server determines that the first speaker of the voice signal is a user who is not registered, and is the same as the voice of the first speaker. Requesting to speak content, receiving the authentication voice signal, and authenticating the identity, and not performing an action corresponding to the first voice recognition result. 10. The speaker authentication method according to any one of items 1 to 9. The speaker authentication method according to any one of claims 1 to 10, wherein the registered second speaker is one of a plurality of users registered in the voice recognition system. A program for causing a processor of a voice recognition server of a voice recognition system to execute the speaker authentication method according to any one of claims 1 to 11. A communication module for communicating with the voice recognition device and the mobile device;
Using the communication module, a voice signal including the voice of the first speaker is received from the voice recognition device, voice recognition is performed on the voice signal, a first voice recognition result is generated, and the voice is generated. A first speaker feature vector is extracted from the signal, a similarity between the first speaker feature vector and the registered speaker feature vector is calculated, and when the similarity is equal to or higher than a first reference value, It is determined that the first speaker of the voice signal is the second speaker who has completed the user registration in the voice recognition server, and the same content as the voice of the first speaker is uttered to the portable device of the second speaker. To receive an authentication voice signal including an authentication voice from the portable device of the second speaker, and to receive the voice signal received from the voice recognition device and the portable device of the second speaker. On the basis of the authentication voice signal, the identity between the second speaker and the first speaker is authenticated, and when the identity is authenticated, an operation corresponding to the first voice recognition result is executed. A processor configured to
Speech recognition server including. The processor is
Performing voice recognition on the authentication voice signal to generate a second voice recognition result,
Comparing the second speech recognition result with the first speech recognition result,
The voice recognition server according to claim 13, wherein the voice recognition server is configured to determine the identity between the first speaker and the second speaker based on at least the result of the comparison. A voice recognition system including a voice recognition server and a voice recognition device,
The voice recognition device detects a voice signal including a voice of a first speaker from the first communication module that communicates with the voice recognition server, a microphone that generates an audio signal, and the voice signal, and detects the voice signal. A first processor configured to transmit to a voice recognition server and receive a synthetic voice signal from the voice recognition server; and a speaker that reproduces a synthetic voice corresponding to the synthetic voice signal,
The voice recognition server includes a second processor and a second communication module that communicates with the voice recognition device and the mobile device,
The second processor is
Receiving the voice signal from the voice recognition device,
Performing voice recognition on the voice signal to generate a first voice recognition result,
A first speaker feature vector is extracted from the voice signal, and a similarity between the first speaker feature vector and the registered speaker feature vector is calculated,
When the similarity is equal to or higher than the first reference value, it is determined that the first speaker of the voice signal is the second speaker who has completed the user registration in the voice recognition server,
Requesting the portable device of the second speaker to speak the same content as the voice of the first speaker,
Receiving an authentication voice signal including an authentication voice from the portable device of the second speaker,
The identity of the second speaker and the first speaker is authenticated based on the voice signal received from the voice recognition device and the authentication voice signal received from the portable device of the second speaker. ,
A voice recognition system configured to perform an action corresponding to the first voice recognition result if the identity is verified. The second processor is
Performing voice recognition on the authentication voice signal to generate a second voice recognition result,
Comparing the second speech recognition result with the first speech recognition result,
The voice recognition system according to claim 15, wherein the voice recognition system is configured to determine the identity between the first speaker and the second speaker based on at least the result of the comparison.