JP2019028465A - Speaker verification method and speech recognition system - Google Patents

Abstract

To provide a speaker verification method and a speech recognition system.SOLUTION: With respect to a speech recognition system capable of accurately recognizing a speaker and a speaker verification method for the same, the speaker verification method includes: a step for a speech recognition server to receive from a speech recognition unit a speech signal including speech of a first speaker; a step for the speech recognition server to perform speech recognition of the speech signal and to generate a speech recognition result; a step for the speech recognition server to extract a speaker feature vector from the speech signal, to compare the speaker feature vector with a registered speaker feature vector, and to determine according to a result of the comparison that a speaker of the speech signal is a registered second speaker; a step for the speech recognition server to transmit to a portable device of the second speaker a speech recognition result and the speech signal; a step for the speech recognition server to receive from the portable device recognition input; and a step for the speech recognition server to perform operation corresponding to the speech recognition result.SELECTED DRAWING: Figure 4A

Description

  The present invention relates to a speaker verification method and a speech recognition system, and more particularly to a method for verifying a speaker in a speech recognition system including a speech recognition device and a speech recognition server.

  Artificial intelligence speaker devices equipped with a voice recognition function are on the market. The artificial intelligence speaker device recognizes the user's voice, extracts a command included in the voice, executes an operation according to the command, and outputs the result as a voice, thereby playing a role like an artificial intelligence secretary. Can fulfill. In order for such an artificial intelligence speaker device to exceed the level of simply answering voice questions and outputting the results of the questions as voice, and to be used in areas where security is required, such as financial transactions and shopping, it is accurate. It must be able to recognize and identify the speaker. However, the artificial intelligence speaker device identifies a user based on a voice, and is inferior in accuracy as compared with a user identification method or an authentication method using biometric information such as fingerprint or iris recognition.

  The problem to be solved by the present invention is to solve the above-mentioned problem, and to provide a method capable of accurately recognizing and identifying a speaker.

  As technical means for achieving the above technical problem, the first aspect of the present disclosure provides a method for verifying a speaker in a speech recognition system including a speech recognition device and a speech recognition server. In the speaker verification method, the voice recognition server receives a voice signal including the voice of the first speaker from the voice recognition device; and the voice recognition server performs voice recognition on the voice signal. Executing and generating a speech recognition result, wherein the speech recognition server extracts a speaker feature vector from the speech signal, compares the speaker feature vector with a registered speaker feature vector, and The result of determining that the speaker of the voice signal is a registered second speaker, and the voice recognition server transmits the voice recognition result and the voice signal to the portable device of the second speaker And a step in which the voice recognition server receives an approval input from the portable device, and a step in which the voice recognition server performs an operation corresponding to the voice recognition result.

  A second aspect of the present disclosure provides a voice recognition server including a communication module that communicates with a voice recognition device and a portable device, and a processor. The processor uses the communication module to receive a voice signal including the voice of the first speaker from the voice recognition device, perform voice recognition on the voice signal, and generate a voice recognition result. A speaker feature vector is extracted from the speech signal, the speaker feature vector is compared with a registered speaker feature vector, and a speaker of the speech signal is registered as a result of the comparison. Determining that there is, using the communication module, transmitting the voice recognition result and the voice signal to the portable device of the second speaker, using the communication module, receiving an approval input from the portable device; An operation corresponding to the voice recognition result is executed.

  A third aspect of the present disclosure provides a voice recognition system including a voice recognition server and a voice recognition device. The voice recognition device detects a voice signal from the first communication module that communicates with the voice recognition server, a microphone that generates an audio signal, and the audio signal, transmits the voice signal to the voice recognition server, and A first processor that receives the synthesized sound signal from the speech recognition server; and a speaker that reproduces the synthesized sound corresponding to the synthesized sound signal. The voice recognition server includes a second processor and a second communication module capable of communicating with the voice recognition device and the portable device. The second processor receives a voice signal including the voice of the first speaker from the voice recognition device, performs voice recognition on the voice signal, generates a voice recognition result, and speaks from the voice signal. A speaker feature vector is extracted, the speaker feature vector is compared with a registered speaker feature vector, and the speaker of the voice signal is recognized as a registered second speaker according to a result of the comparison; The voice recognition result and the voice signal are transmitted to the portable device of the second speaker, an approval input is received from the portable device, and an operation corresponding to the voice recognition result is executed.

  The fourth aspect of the present disclosure can provide a program that causes a processor of a speech recognition server of a speech recognition system to execute the speaker verification method according to the first aspect.

  The fifth aspect of the present disclosure can provide a computer-readable recording medium in which the program according to the fourth aspect is recorded.

  According to various embodiments of the present disclosure, since a speaker can be accurately identified and recognized through a speaker verification procedure, a speaker's command can be executed without fear of theft. Further, even if a speaker verification error or voice stealing occurs, such a fact can be grasped by the speaker party.

1 is an exemplary network configuration diagram of a voice recognition system according to an embodiment. FIG. It is a block diagram for demonstrating the internal structure of the speech recognition speaker apparatus by one Embodiment. It is a block diagram for demonstrating the internal structure of the speech recognition speaker apparatus by other embodiment. It is a block diagram for demonstrating the internal structure of the speech recognition server by one Embodiment. It is a block diagram for demonstrating the internal structure of the processor of the speech recognition server by one Embodiment. It is a block diagram for demonstrating the internal structure of the processor of the speech recognition server by one Embodiment. 6 is an exemplary flowchart for explaining a speaker verification method of the speech recognition system according to an embodiment; 6 is an exemplary flowchart for explaining a speaker verification method of the speech recognition system according to an embodiment; 6 is a diagram illustrating an exemplary screen of a second speaker's mobile device connected to a voice recognition system, according to one embodiment. 6 is an exemplary flowchart for explaining a speaker verification method of a speech recognition system according to another embodiment.

  DESCRIPTION OF EMBODIMENTS Hereinafter, embodiments of the present invention will be described in detail with reference to the accompanying drawings so that those skilled in the art can easily implement them in the technical field to which the present invention belongs. However, the present invention may be embodied in various different forms and is not limited to the embodiments described herein. In the drawings, in order to clearly describe the present invention, parts not related to the description are omitted, and like parts are denoted by like reference numerals throughout.

  Throughout the specification, when a part is “coupled / connected” to another part, it is not only “directly coupled / connected”, but also other elements in between. This includes cases where they are “electrically coupled / connected”. In addition, when a certain component “includes” a component, it means that it does not exclude other components, and may include other components unless otherwise stated to the contrary. Means.

  In this specification, phrases such as “in some embodiments” or “in one embodiment” appearing in various places do not necessarily indicate the same embodiment.

  Some embodiments are also illustrated by functional block configurations and various processing steps. Some or all of the functional blocks may be embodied by various numbers of hardware components and / or software components that execute specific functions. For example, the functional block of the present disclosure may be realized by one or more microprocessors, or may be realized by a circuit configuration for a predetermined function. Further, for example, the functional blocks of the present disclosure may be embodied by various programming languages or script languages. Such functional blocks may be embodied by algorithms executed by one or more processors. In addition, the present disclosure can employ conventional techniques for electronic environment setting, signal processing and / or data processing. Terms such as “module” and “configuration” are general purpose, mechanical and not limited to physical configurations.

  Also, the connection / connection lines or connection / connection members between the components illustrated in the drawings are illustrative of functional connection / connection and / or physical or circuit connection / connection. Not too much. In actual devices, various functional connections / connections that can be substituted or added, physical / circuit connections / connections, indicate connections / connections between components.

  In the present disclosure, the voice recognition function refers to converting a voice signal including a user's voice into a character string (or text). A character string (or text) in which a voice signal is converted by such a voice recognition function is also called a voice recognition result. The user's voice signal includes a voice command, and such a voice recognition result may also include a command corresponding to the voice command. Such a voice command can execute a specific function of the voice recognition speaker device or the voice recognition server. On the other hand, in the present disclosure, the speech synthesis function refers to converting a character string (or text) into a speech signal, contrary to the speech recognition function. A voice signal in which a character string (or text) is converted by such a voice synthesis function is also called a synthesized voice signal.

  In the present disclosure, the expression “registered” means that it is registered as a user or related information in the speech recognition system. “Registered user” means a user who has finished user registration in the speech recognition system. One person can register as a user in the speech recognition system according to the present disclosure, and can input his / her voice when registering as a user. Such a speech recognition system can extract a speaker feature vector from a speech signal of speech input at the time of user registration, and can store it as information related to the registered user. Thus, the speaker feature vectors stored in the speech recognition system are called registered speaker feature vectors. In addition, at the time of user registration, the identification number of the self-owned portable device can be stored together.

  A plurality of users may be registered in such a voice recognition system. In the present disclosure, the first speaker means a person who actually utters the voice of the voice signal, and the registered second speaker is a voice recognition system among a plurality of users registered in the voice recognition system. Means a user who recognizes or determines that the voice of the voice signal is spoken. The registered second speaker is generally the same as the first speaker. However, if a speaker misrecognition or voice stealing occurs in the speech recognition system, the registered second speaker is the first speaker. Can be different from the speaker.

  In the present disclosure, a keyword can have a word form or a phrase form. In the present disclosure, the voice command uttered after the wakeup keyword can have a sentence form, a word form, or a phrase form in a natural language form.

  Hereinafter, the present disclosure will be described in detail with reference to the accompanying drawings.

  FIG. 1 is an exemplary network configuration diagram of a speech recognition system according to an embodiment.

  Referring to FIG. 1, the network environment of the voice recognition system is illustratively shown as including a voice recognition speaker device 100, a voice recognition server 200, a portable device 300, and a network 400. Such a speech recognition system includes a speech recognition speaker device 100 and a speech recognition server 200.

  The voice recognition speaker device 100 is an example of a voice recognition device, and is a speaker device that is equipped with a voice control function and executes a specific function. The speech recognition speaker device 100 is also called a smart speaker device or an artificial intelligence speaker device. Upon receiving the voice of the speaker, the voice recognition speaker device 100 recognizes the voice and the speaker, extracts a command included in the voice, executes an operation based on the command, and outputs the result as a voice. Can do. Specific functions that can be executed by the voice recognition speaker device 100 include, for example, voice information provision, music playback, Internet shopping, financial transaction, telephone call, message transmission, alarm setting, and voice recognition speaker device via a network. Control of connected electronic devices or mechanical devices may be included.

  For example, when the voice recognition speaker device 100 is connected to a smart TV via a network, the specific function may include channel viewing, channel search, video playback, program search, and the like. For example, when the voice recognition speaker device 100 is connected to a household electrical appliance such as a smart refrigerator, the specific function may include inspection of a refrigerated state and a frozen state, temperature setting, and the like. However, in the present disclosure, such specific functions are not limited to the functions described above.

  The voice recognition speaker device 100 can communicate with the voice recognition server 200 via the network 400 via wireless communication or wired communication.

  The communication method of the network 400 is not limited, and not only a communication method using a communication network included in the network 400 (for example, a mobile communication network, a wired Internet, a wireless Internet, and a broadcast network), but also a voice recognition speaker. Short-range wireless communication with the device 100 is also included. For example, the network 400 includes a PAN (personal area network), a LAN (local area network), a CAN (campus area network), a MAN (metropolitan area network), a WAN (wide area network), a BBN (broadband network), the Internet, etc. One or more arbitrary networks may be included. The network 400 may include any one or more of network topologies including, but not limited to, bus networks, star networks, ring networks, mesh networks, star bus networks, tree or hierarchical networks, and the like. Absent.

  The voice recognition server 200 communicates with the voice recognition speaker device 100 via the network 400 and can be embodied by at least one computer device. The voice recognition server 200 is distributed in a cloud form and can provide commands, codes, files, contents, and the like.

  The voice recognition server 200 can convert a voice signal received from the voice recognition speaker device 100 into a character string (or text) and generate a voice recognition result. The speech recognition server 200 can synthesize the speech reproduced by the speech recognition speaker device 100, generate a synthesized sound signal, and transmit the synthesized sound signal to the speech recognition speaker device 100.

  The voice recognition server 200 can actually execute a specific function that can be executed by the voice recognition speaker device 100. For example, in the case of the voice information providing function, the voice recognition server 200 recognizes an information request included in the voice signal received from the voice recognition speaker device 100, generates a result related thereto, and in the form of a synthesized voice signal. Can be transmitted to the speech recognition speaker device 100. In the case of the telephone call function, the voice recognition server 200 recognizes the telephone call request included in the voice signal received from the voice recognition speaker device 100, executes the telephone call in response to the request, and transmits when the telephone call is made. Signals and received signals can be relayed. The voice recognition server 200 is also connected to home appliances via the network 400, and the voice recognition server 200 controls the home appliances by a control command included in the voice signal received from the voice recognition speaker device 100. be able to.

  The voice recognition server 200 can also be connected to the mobile device 300 via the network 400. The network that connects the voice recognition server 200 and the voice recognition speaker device 100 and the network that connects the voice recognition server 200 and the portable device 300 may be of different types. For example, the network that connects the voice recognition server 200 and the voice recognition speaker device 100 may be a LAN or the Internet, and the network that connects the voice recognition server 200 and the portable device 300 may be a mobile communication network.

  The portable device 300 is an electronic device that supports wireless communication that can be carried around by a user. For example, the mobile device 300 may be a mobile phone, a smartphone, a tablet PC (personal computer), a notebook PC, or the like. The portable device 300 can have a telephone function, a message function, or a messenger function, and can reproduce an audio signal or a video signal received from the voice recognition server 200. In addition, the mobile device 300 can provide a voice signal to the voice recognition server 200. In general, the portable device 300 is also an electronic device used by one individual.

  In FIG. 1, the voice recognition speaker device 100 is illustrated as being connected to the voice recognition server 200 that performs the voice recognition function via the network 400, but this is exemplary. The voice recognition speaker device 100 can also independently execute a voice recognition function or a voice synthesis function.

  FIG. 2A is a block diagram for explaining the internal configuration of the speech recognition speaker device 100 according to an embodiment.

  Referring to FIG. 2A, the voice recognition speaker device 100 may include a processor 110, a microphone 120, a speaker 130, and a communication module 140. The speech recognition speaker device 100 may include more components than those illustrated in FIG. 2A. For example, the voice recognition speaker device 100 may further include a memory. The voice recognition speaker device 100 is connected to the network 400 of FIG. 1 via the communication module 140 and can communicate with the voice recognition server 200.

  The microphone 120 can directly generate an audio signal by converting surrounding audio into electrical acoustic data. The voice recognition speaker device 100 may include a plurality of microphones 120, and the input direction of the audio signal can be found using the plurality of microphones 120.

  According to another example, the voice recognition speaker device 100 can also receive an audio signal transmitted from an external device via the communication module 140. The speaker 130 can convert an audio signal into sound and output the sound.

  The processor 110 may be configured to process computer program instructions by performing basic arithmetic, logic, and input / output operations. Such instructions are provided from memory to the processor 110 or received via the communication module 140 and provided to the processor 110. For example, the processor 110 is configured to execute instructions by program code stored in a recording device such as a memory.

  The processor 110 can detect a voice signal corresponding to the voice of the speaker from the audio signal generated by the microphone 120 and transmit the detected voice signal to the voice recognition server 200 via the communication module 140. . The processor 110 can detect the voice signal from the audio signal using the keyword. The processor 110 can identify a voice signal received subsequent to the keyword voice signal by extracting a keyword voice signal corresponding to the keyword from the audio signal.

  The processor 110 can receive the synthesized sound signal from the speech recognition server 200 and reproduce the synthesized sound corresponding to the synthesized sound signal via the speaker 130.

  FIG. 2B is a block diagram for explaining an internal configuration of a voice recognition speaker device 100a according to another embodiment.

  Referring to FIG. 2B, the voice recognition speaker device 100a further includes a camera 150 as compared to the voice recognition speaker device 100 of FIG. 2A.

  The camera 150 is controlled by the processor 110 and can generate a video signal corresponding to the video at the time when the audio signal is detected in the audio signal received from the microphone 120. The processor 110 can transmit the video signal to the voice recognition server 200 via the communication module 140. The camera 150 may be, for example, a 360 ° camera that can capture any 360 ° image.

  According to another example, the voice recognition speaker device 100a can adjust the shooting direction of the camera 150. The voice recognition speaker device 100a may include a sensor that can sense the direction in which the voice is generated. The processor 110 can sense the direction in which the audio signal is detected and adjust the shooting direction of the camera 150 to the direction in which the audio signal is detected. At this time, the video signal transmitted from the processor 110 to the voice recognition server 200 may include a video in a direction in which the voice of the speaker is generated.

  According to another example, the voice recognition speaker device 100a may include a plurality of cameras 150 arranged at regular intervals so as to photograph the periphery 360. The voice recognition speaker device 100a may include a sensor that can sense the direction in which the voice is generated. The processor 110 senses the direction in which the audio signal is detected, and uses the camera 150 that can capture the direction in which the audio signal is detected among the plurality of cameras 150 to determine the direction in which the speaker's audio is generated. A video signal corresponding to the video can be acquired.

  FIG. 3 is a block diagram for explaining the internal configuration of the speech recognition server 200 according to an embodiment.

  Referring to FIG. 3, the voice recognition server 200 includes a processor 210, a memory 220, and a communication module 230. The speech recognition server 200 may include more components than those illustrated in FIG. For example, the voice recognition server 200 may further include an input / output device.

  The communication module 230 can provide a function for the voice recognition server 200 to communicate with the voice recognition speaker device 100 and the portable device 300 via the network 400. The voice recognition server 200 is connected to the network 400 of FIG. 1 via the communication module 230 and can communicate with the voice recognition speaker device 100 and the portable device 300.

  The memory 220 is a computer-readable recording medium, and may include a permanent mass storage device such as a random access memory (RAM), a read-only memory (ROM), and a disk drive. The memory 220 stores an operating system and at least one program code (for example, a code for a speech recognition application or a speech synthesis application installed and executed in the speech recognition server 200). Such software components are loaded into the memory 220 via communication using the communication module 230. For example, at least one program is loaded into the memory 220 based on a program installed by a developer or a file distribution system that distributes an installation file of an application via a file provided by the network 400.

  The processor 210 may be configured to process computer program instructions by performing basic arithmetic, logic and input / output operations. The processor 210 may be configured to execute instructions according to program code stored in the memory 220.

  The processor 210 may be configured to receive a voice signal including the voice of the first speaker from the voice recognition speaker device 100, perform voice recognition on the voice signal, and generate a voice recognition result. For example, the processor 210 may extract a frequency characteristic of the voice signal and perform voice recognition using an acoustic model and a language model in order to perform voice recognition on the voice signal. Such a frequency characteristic means a distribution of frequency components of the sound input extracted by analyzing the frequency spectrum of the sound input. The acoustic model and the language model are also stored in the memory 220. However, the voice recognition method is not limited to this, and various techniques for converting a voice signal into a character string (or text) are used.

  The processor 210 can analyze the audio signal and determine who is the speaker who spoke the audio contained in the audio signal. For example, a speaker feature vector is extracted from such a speech signal, the speaker feature vector is compared with a registered speaker feature vector, and the speaker of the speech signal is registered as a result of this comparison. It may be configured to determine that there are two speakers. The registered speaker feature vectors are stored in the memory 220 in advance. In the voice recognition server 200, a plurality of speakers may be registered as users of the voice recognition speaker device 100. In this case, a plurality of registered speaker feature vectors are stored in the memory 220. Each registered speaker feature vector corresponds to each registered speaker.

  The processor 210 determines at least one of states posteriors extracted from the acoustic model, general background model, and global variability conversion information to determine that the speaker of the audio signal is the second speaker. Can be used to generate a speaker feature vector from the frequency characteristics of the speech signal. The processor 210 determines whether or not the speaker of the audio signal is a registered speaker based on the generated speaker feature information and the registered speaker feature vector stored in the memory 220. Can be determined. The memory 220 stores at least one of posterior information, general background model, global variability conversion information, and registered speaker information.

  The processor 210 may be configured to send a speech recognition result and a voice signal to the second speaker's mobile device 300 and receive an approval input from the second speaker's mobile device 300. Information about the mobile device 300 in which the second speaker is registered is stored in the memory 220 in advance. When the second speaker registers with the speech recognition server 200 as a user of the speech recognition speaker device 100 via the speech recognition speaker device 100, information about his own mobile device 300 (for example, an identification number of the mobile device 300). ) Can be registered together.

  The processor 210 may be configured to perform an operation corresponding to the speech recognition result. The processor 210 can determine a function corresponding to the speech recognition result and execute such a function. The processor 210 may be configured to generate a synthesized sound signal for reporting the execution result of the operation. The processor 210 may be configured to transmit the synthesized sound signal to the speech recognition speaker device 100.

  The voice recognition server 200 may further include a microphone or a speaker as an input / output device. The voice recognition server 200 can directly generate a voice signal and directly reproduce the synthesized sound.

  FIG. 4A is a block diagram for explaining the internal configuration of the processor of the speech recognition server according to the embodiment.

  Referring to FIG. 4A, the processor 210 of the voice recognition server 200 includes a voice signal receiving unit 211, a voice recognition unit 212, a speaker recognition unit 213, a speaker verification unit 214, and a function unit 215. The speech recognition server 200 may further include a synthesized sound signal generation unit 216. The speaker recognition unit 213 includes a speaker feature vector extraction unit 213a, a speaker feature vector comparison unit 213b, and a registered speaker determination unit 213c.

  The voice signal receiving unit 211 receives a voice signal including the voice of the first speaker from the voice recognition speaker device 100.

  The voice recognition unit 212 performs voice recognition on the voice signal received by the voice signal reception unit 211 and generates a voice recognition result. The voice recognition unit 212 can perform voice recognition on the voice signal and convert the voice of the speaker into a character string (or text). The speech recognition unit 212 can perform natural language processing on the converted character string (or text) and extract a speaker's command included in the speech signal. The speech recognition result includes a speaker's command, and the operation corresponding to such a speech recognition result means an operation based on the speaker's command.

  The speaker recognizing unit 213 determines that the speaker of the audio signal received by the audio signal receiving unit 211 is the second speaker. The second speaker is a speaker registered as a user of the voice recognition speaker device 100 in the voice recognition system. For example, the speaker feature vector extraction unit 213a extracts a speaker feature vector from the voice signal received by the voice signal reception unit 211. The speaker feature vector extraction unit 213a extracts a speaker feature vector by converting a time domain-based speech signal into a frequency domain signal and transforming the frequency energy of the converted signal to be different from each other. Can do. For example, such speaker feature vectors are extracted based on mel frequency cepstrum coefficients or filter bank energy, but are not limited thereto, and speaker feature vectors are extracted from audio data in various ways. can do.

  The speaker feature vector comparison unit 213 b compares the speaker feature vector extracted by the speaker feature vector extraction unit 213 a with the registered speaker feature vector stored in the memory 220. There are a plurality of registered speaker feature vectors in the memory 220, and the speaker feature vector comparison unit 213b compares the extracted speaker feature vectors with a plurality of registered speaker feature vectors. A registered speaker feature vector having a high degree of similarity is determined. The registered speaker determination unit 213c determines that the speaker of the audio signal is the registered second speaker based on the comparison result of the speaker feature vector comparison unit 213b. The registered speaker determination unit 213c determines the speaker of the registered speaker feature vector having the highest similarity as the speaker of the audio signal. If none of the registered speaker feature vectors has a similarity that exceeds a preset reference value, the registered speaker determination unit 213c is not a speaker in which the speaker of the audio signal is registered. Can be determined. At this time, the processor 210 does not execute the operation corresponding to the speech recognition result, or only when the operation corresponding to the speech recognition result is set to an operation that can be executed by anyone. Can be executed.

  The speaker verification unit 214 sends the voice recognition result generated by the voice recognition unit 213 and the voice signal received by the voice signal reception unit 211 to the portable device of the second speaker determined by the speaker recognition unit 213. , Send. The identification number of the mobile device of the second speaker is stored in the memory 220 in advance. The speaker verification unit 214 can receive an approval input from the portable device of the second speaker.

  Upon receiving the approval input from the speaker verification unit 214, the function unit 215 performs an operation corresponding to the voice recognition result generated by the voice recognition unit 213. If it is determined that the speaker of the voice signal is not a registered speaker, the function unit 215 does not perform an operation corresponding to the voice recognition result. When the function unit 215 receives the rejection input from the speaker verification unit 214, the function unit 215 does not execute the operation corresponding to the voice recognition result.

  When the function unit 215 executes an operation, the synthetic sound signal generation unit 216 generates a synthetic sound signal for reporting the execution result of the operation. The synthesized sound signal generation unit 216 determines that the speaker of the speech signal is not a registered speaker, and when the operation corresponding to the speech recognition result is not executed, or the approval input is received by the speaker verification unit 214. First, when the operation corresponding to the speech recognition result is not executed, a synthesized sound signal for reporting that the operation is not executed can be generated.

  According to another embodiment, the processor 210 may further include a video signal receiving unit that receives a video signal from the voice recognition speaker device 100. At this time, the speaker verification unit 214 can transmit the video signal received by the video signal receiving unit to the mobile device of the second speaker.

  According to other embodiments, the processor 210 is configured to further include a speaker vector refiner. When the speaker verification unit 214 receives the approval input from the portable device of the second speaker, it is confirmed whether or not the speaker of the received voice signal is the second speaker. The speaker vector improvement unit can improve the registered speaker feature vector of the second speaker stored in the memory 220 using the speaker feature vector extracted from the received speech signal. The speaker feature vector improvement unit generates a speaker feature vector registered for the second speaker through an adaptive training method using the speaker feature vector extracted from the speech signal, and newly generates it. In the case where the adaptive speaker performance increases compared with the registered speaker feature vector before the adaptive training, the newly generated speaker feature vector is registered by storing the newly generated speaker feature vector in the memory 220. The improved speaker feature vector can be improved.

  FIG. 4B is a block diagram for explaining an internal configuration of a processor of the speech recognition server according to the embodiment.

  Referring to FIG. 4B, the processor 210a of the voice recognition server 200 includes a voice signal reception unit 211, a voice recognition unit 212, a speaker recognition unit 213, a verification determination unit 217, a speaker verification unit 214, a function unit 215, and a synthesized sound. A signal generation unit 216 is included.

  The verification determination unit 217 determines whether to execute the operation of the speaker verification unit 214 based on at least one of the operation corresponding to the speech recognition result and the setting of the second speaker.

  According to an example, the verification determination unit 217 determines to execute the operation of the speaker verification unit 214 when the operation based on the speech recognition result is included in the pre-approval operation list set in advance by the second speaker. can do. Such a pre-approval operation list is stored in the memory 220, and some of the operations or functions that can be executed via the voice recognition speaker device 100 are also included in the pre-approval operation list set in advance. For example, operations such as financial transactions, Internet shopping, and message transmission are included in the pre-approval operation list set in advance. The operations included in such a pre-approval operation list are set so as to be different for each registered speaker.

  According to another example, when the operation based on the speech recognition result is included in the post-notification operation list set in advance by the second speaker, the verification unit 217 uses the function unit 215 to perform an operation corresponding to the speech recognition result. Is first executed and it can be decided to transmit the speech recognition result and the speech signal to the second speaker's mobile device. Such a post notification operation list is stored in the memory 220, and some of the operations that can be executed by the speech recognition speaker device 100 are also included in the post notification operation list set in advance. For example, operations such as making a call and changing settings are also included in the post notification operation list set in advance. The actions included in such a post-notification action list are also set so as to differ for each registered speaker.

  According to another example, the verification determination unit 217 executes the operation of the speaker verification unit 214 when at least one of the position and the current time of the second speaker's mobile device meets the pre-approval condition. You can decide to do that. For example, when the second speaker's portable device is located near the position of the voice recognition speaker device 100, for example, the second speaker's portable device and the same wireless Wi-Fi (registered trademark) as the voice recognition speaker device 100 are used. ) When connected to the access point, or when the GPS (global position system) position or wireless network connection position of the mobile device of the second speaker substantially matches the position of the voice recognition speaker device 100, the second talk Since it is highly likely that the person has actually uttered the voice corresponding to the voice signal received by the voice signal receiving unit 211, the verification determination unit 217 can omit the operation of the speaker verification unit 214. The registered speaker can set the omission of the operation of the speaker verification unit 214 as described above.

  The verification determination unit 217 can determine to execute the operation of the speaker verification unit 214 at a time set by the second speaker, for example, at daytime on weekdays. For example, since the second speaker who is a company employee is not likely to be at home during weekday daytime, the voice recognition speaker device 100 located at home may receive the voice of the second speaker. Is low. In such a case, the verification determination unit 217 can determine to execute the operation of the speaker verification unit 214. The registered speakers can set whether or not to execute the operation of the speaker verification unit 214 based on time.

  The pre-approval condition is set in advance by the registered speaker and is also stored in the memory 220. Such pre-approval conditions are also determined based on the registered speaker behavior patterns. The registered speaker's behavior pattern is generated based on the position of the registered speaker's portable device. For example, the verification determination unit 217 can collect the position of the registered speaker's portable device for a long time. The verification determination unit 217 can analyze the collected position of the portable device and determine a time zone in which the registered speaker is not located near the voice recognition speaker device 100. The verification determination unit 217 can automatically determine to execute the operation of the speaker verification unit 214 when the current time falls within this time zone.

  5A and 5B are exemplary flowcharts for explaining a speaker verification method of a speech recognition system according to an embodiment.

  Referring to FIGS. 5A and 5B, the voice recognition system includes a voice recognition speaker device 100 and a voice recognition server 200. The second speaker's portable device 300 is connected to the voice recognition server 200 via a network.

  The voice recognition speaker device 100 can use the microphone 120 (FIG. 2A) to electrically convert surrounding sounds and generate an audio signal (S101).

  The voice recognition speaker device 100 can detect the voice signal from the audio signal (S102). Such an audio signal may include the user's voice. Here, the user is the first speaker. Such voice may include user voice commands. Such voice commands may include voice information retrieval, telephone call, message transmission, financial transaction, Internet shopping, food delivery, peripheral home appliance control, smart home control, and the like. In this example, it is assumed that the voice command is related to a financial transaction, and the voice of the first speaker is “send 1 million won to B”. The voice of the first speaker may include a trigger keyword for causing the voice recognition speaker device 100 to wake up. The voice recognition speaker device 100 can detect the voice signal from the audio signal by recognizing the trigger keyword.

  The voice recognition speaker device 100 transmits a voice signal to the voice recognition server 200 (S103), and the voice recognition server 200 receives the voice signal from the voice recognition speaker device 100 (S201).

  The voice recognition server 200 performs voice recognition on the voice signal and generates a voice recognition result (S202). The speech recognition server 200 can extract speech signal frequency characteristics and execute speech recognition using an acoustic model and a language model. The voice recognition server 200 can generate a voice recognition result by converting a voice signal into a character string and subjecting the character string to natural language processing. Such a speech recognition result may include a speech command.

  The speech recognition server 200 extracts speaker feature vectors from the speech signal (S203). The speech recognition server 200 can generate a speaker feature vector from the frequency characteristics of the speech signal by using at least one of the posterior information extracted from the acoustic model, the general background model, and the global variability conversion information. .

  The speech recognition server 200 compares the extracted speaker feature vector with the registered speaker feature vector (S204). The registered speaker feature vectors are also stored in the memory 220 (FIG. 3), and are generated in advance based on the input user's voice when the user registers in the voice recognition system.

  The speech recognition server 200 determines that the speaker of the speech signal is the registered second speaker based on the comparison result in step S204 (S205). In step S204, the speech recognition server 200 can compare the extracted speaker feature vector with each of the registered speaker feature vectors. As a result of the comparison, a registered speaker feature vector having the highest similarity with the extracted speaker feature vector is determined among the registered speaker feature vectors. The speech recognition server 200 determines that the user of the registered speaker feature vector having the highest similarity is the speaker of the speech signal, and corresponds to the registered speaker feature vector having the highest similarity. The user who performs this operation is the second speaker. Such a second speaker is generally the same as the first speaker. However, the second speaker may be different from the first speaker due to an error in the speaker recognition function of the voice recognition server 200.

  For example, the first speaker utters a voice command, but the voice recognition server 200 recognizes that this voice command is uttered by a second speaker different from the first speaker due to an error in the speaker recognition function. In this case, the voice recognition server 200 recognizes that the second speaker has uttered “Remit 1 million won to B”, and the voice recognition server 200 sends 100 to B from the account of the second speaker. The problem of remitting 10,000 won occurs.

  As another example, the first speaker utters a voice command by imitating the voice of the second speaker, and the voice recognition server 200 recognizes that this voice command is uttered by the second speaker. In that case, it is a case by voice stealing. Also in this case, the voice recognition server 200 recognizes that the second speaker has uttered “Remit one million won to B”, and the voice recognition server 200 recognizes the B from the account of the second speaker. The problem of remitting 1 million won occurs.

  The voice recognition server 200 can execute a further speaker verification procedure to solve such a problem. The voice recognition server 200 can transmit the voice recognition result and the voice signal to the portable device 300 of the second speaker. When the second speaker registers in the voice recognition system as a user of the voice recognition speaker device 100, the identification number of the portable device 300 of the second speaker can be input, and the identification number of the portable device 300 of the second speaker is , Stored in the memory 220.

  On the other hand, the voice recognition server 200 can determine that the speaker of the voice signal is not a user registered in advance based on the comparison result in step S204. If the similarity between the extracted speaker feature vector and the registered speaker feature vector does not exceed the preset reference similarity, the speech recognition server 200 registers the speaker of the speech signal in advance. It is determined that it has not been performed, and the operation based on the voice command is not executed. In that case, the speech recognition server 200 can immediately proceed to step S209 and generate a synthesized sound for reporting that the operation has not been executed. Even in this case, if the content of the voice command is set so that anyone can use it, the voice recognition server 200 can execute the voice command even if the speaker of the voice signal is not registered in advance. The operation by can be executed. In that case, the speech recognition server 200 can skip the speaker verification procedure (S206, S301-S305, S207), and can immediately proceed to step S208 to execute an operation based on the voice command. According to one embodiment, the voice recognition server 200 does not perform steps S203 to S205 when the operation based on the voice command is an operation that anyone can perform.

  The mobile device 300 of the second speaker receives the voice recognition result and the voice signal from the voice recognition server 200 (S301). As described above, the second speaker may or may not be the same as the first speaker who actually uttered the voice included in the voice signal.

  The portable device 300 can display the voice recognition result (S302). FIG. 6 illustrates an exemplary screen of a second speaker's mobile device connected to the speech recognition system, according to one embodiment.

  As illustrated in FIG. 6, the wording 301 “The following command has been executed by the voice recognition speaker device” is displayed on the display window 310 of the portable device 300. The received voice recognition result or a voice command included in the voice recognition result is displayed in an area 302 below the wording 301. In this example, in the area 302, a word “Remit 1 million won to B” is displayed.

  The display window 310 displays a reproduction button 303 that can reproduce an audio signal detected by the voice recognition speaker device 100. When the second speaker touches the play button 303, the voice signal received from the voice recognition server 200 is played.

  The display window 310 displays a word 304 “Is it approved?” An approval button 305 and a rejection button 306 are displayed below the wording 304. The second speaker confirms the received voice recognition result or the reproduced voice, and then touches the approval button 305 so that the voice recognition server 200 performs an operation based on the voice command of the voice recognition result. Can be approved. If the second speaker is not speaking or if the received voice recognition result is different from the voice command that he / she spoke, the second speaker touches the reject button 306 to cause the voice recognition server 200 to The operation corresponding to the voice recognition result is not executed. If there is an error in the speaker recognition function of the voice recognition server 200 or the second speaker steals his / her voice, the second speaker uses the mobile device 300 to respond to the voice recognition result. The damage can be prevented by not executing.

  An offer button 307 is displayed on the display window 310. When the second speaker touches the play button 303 and hears the reproduced voice, and determines that his / her voice has been stolen, the second speaker touches the offer button 307 and the voice stealing fact is related to the related financial company, government office, Or you can apply to an external organization such as the manufacturer of a speech recognition system. In that case, the portable device 300 can transmit the voice signal received from the voice recognition server 200 to the external organization together with the information related to voice theft.

  In the display window 310, an audio feedback sending button 308 is displayed. When the second speaker touches the voice feedback feed button 308, the voice output from the voice recognition speaker device 100 can be input. After the second speaker touches the voice feedback sending button 308, the second speaker can utter a response voice to be transmitted to a voice stealer or the like. The portable device 300 can generate a response voice signal corresponding to the response voice and transmit the response voice signal to the voice recognition server 200. The voice recognition server 200 can receive the response voice signal and transmit it to the voice recognition speaker device 100. The voice recognition speaker device 100 can receive the response voice signal transmitted from the voice recognition server 200 and output the response voice of the second speaker corresponding to the response voice signal.

  According to another embodiment, a voice communication button is displayed on the display window 310, and voice is transmitted / received between the voice recognition speaker device 100 and the portable device 300 when the second speaker touches the voice communication button. A session is established that can be The second speaker can exchange voice with a person located around the voice recognition speaker device 100 through the established session. For example, when the first speaker is the family of the second speaker, the first speaker explains to the second speaker why the action based on the voice recognition result is necessary, and the second speaker After listening to the speaker's explanation, the operation can be approved. The portable device 300 is directly connected to the voice recognition speaker device 100 or is connected to the voice recognition speaker device 100 under the relay of the voice recognition server 200.

  Referring to FIGS. 5A and 5B again, the mobile device 300 may display the voice recognition result on the area 302 of the display window 310 (S302). In addition, the portable device 300 can reproduce the audio signal when the second speaker touches the reproduction button 303 (S303).

  The mobile device 300 may receive an approval input when the second speaker touches the approval button 305 or may receive a rejection input when the second speaker touches the rejection button 306. Yes (S304). The portable device 300 can transmit an approval input or a rejection input to the voice recognition server 200 (S305). If the approval input is not received for a preset time, the portable device 300 can consider that the rejection input has been received and transmit the rejection input to the voice recognition server 200.

  The voice recognition server 200 can receive an approval input or a rejection input from the portable device 300 (S207). The voice recognition server 200 executes an operation corresponding to the voice recognition result when receiving the approval input, and does not execute an operation corresponding to the voice recognition result when receiving the rejection input (S208). In this example, when receiving the approval input, the voice recognition server 200 can remit 1 million won to B from the account of the second speaker. When the voice recognition server 200 receives the rejection input, the voice recognition server 200 does not remit 1 million won to B.

  After executing the operation corresponding to the speech recognition result, the speech recognition server 200 generates a synthesized sound signal for reporting the operation execution result (S209). At this time, such a synthesized sound signal corresponds to, for example, a synthesized sound “1 million won has been transferred to B from the account of the second speaker”. If the operation corresponding to the speech recognition result is not executed, the speech recognition server 200 generates a synthesized sound signal for reporting the non-execution of the operation (S209). At this time, such a synthesized sound signal corresponds to, for example, a synthesized sound such that “1 million won has not been transferred to B due to the disapproval of the second speaker”. The speech recognition server 200 can transmit the generated synthesized sound signal to the speech recognition speaker device 100 (S210).

  The speech recognition speaker device 100 can receive the synthesized sound signal (S104) and reproduce the synthesized sound corresponding to the synthesized sound signal (S105). Therefore, the first speaker who utters the voice of the voice signal can directly check the execution result of his voice command.

  According to another embodiment, when the speech recognition speaker device 100 includes the camera 150 (FIG. 2B), the speech recognition speaker device 100 includes the video at the time when the speech signal is detected using the camera 150 in step S102. A video signal can be generated. In step S <b> 103, the voice recognition speaker device 100 can transmit the video signal together with the voice signal to the voice recognition server 200.

  In step S201, the voice recognition server 200 receives the video signal from the voice recognition speaker device 100. In step S206, the voice recognition server 200 can transmit the video signal together with the voice recognition result and the voice signal to the mobile device 300 of the second speaker. .

  In that case, the mobile device 300 may have an interface capable of displaying a video signal, for example, a video display button. If the second speaker touches the video display button to display the video signal, the portable device 300 displays the video signal video, so that the second speaker can recognize the voice recognition result that he / she does not speak. , The video containing the person who spoke the voice can be confirmed. Such a video signal may include a video in a direction in which the voice of the first speaker is generated. Such a video signal is also a moving image signal. The mobile device 300 may have an interface that can play back a moving image signal, for example, a video playback button. If the second speaker touches the video playback button, the mobile device 300 can play back the moving image included in the moving image signal.

  According to another embodiment, when the voice recognition server 200 receives an approval input from the mobile device 300 in step S207, it is confirmed that the identity of the first speaker and the second speaker is confirmed. The recognition server 200 can use the extracted speaker feature vector to improve the registered speaker feature vector of the second speaker.

  FIG. 7 is an exemplary flowchart for explaining a speaker verification method of a speech recognition system according to another embodiment.

  Referring to FIG. 7, according to the speaker verification method according to another embodiment, after the step S205 of FIG. 5 is executed, the speech recognition server 200 performs the operation corresponding to the speech recognition result and the second speaker's Based on at least one of the settings, execution of the verification procedure (S206, S301-S305, S207) using the mobile device 300 of the second speaker can be determined (S211).

  If it is determined in step S211 that the verification procedure is to be executed, the speech recognition server 200 can proceed to step S206 and transmit a speech recognition result and a speech signal. However, if it is determined in step S211 that the verification procedure is not to be executed, the speech recognition server 200 can omit the verification procedure and proceed to step S208 to execute an operation corresponding to the speech recognition result.

  According to an example, when the action based on the voice recognition result is included in the pre-approval action list set in advance by the second speaker, the voice recognition server 200 determines whether or not the verification procedure (S206, S301-S305, S207) in step S211. ) Can be determined to perform. The pre-approved action list can be stored in the memory 220 (FIG. 3). Such a pre-approval operation list includes some of the operations that can be executed via the voice recognition speaker device 100 and selected in advance by the second speaker. For example, the second speaker may select an action such as financial transaction, Internet shopping, or message transmission and include it in such a pre-approved action list.

  According to another example, in step S211, the speech recognition server 200 determines whether the second speaker's mobile device 300 has a location and / or current time that matches the pre-approval condition (S206, S301). -S305, S207) can be determined to be executed. For example, when the second speaker's mobile device 300 is located away from the speech recognition speaker device 100, the second speaker is unlikely to actually speak the voice corresponding to the voice signal. The voice recognition server 200 can determine to execute the verification procedure (S206, S301-S305, S207). However, if the mobile device 300 of the second speaker is located near the speech recognition speaker device 100, the speech recognition server 200 may omit the verification procedure (S206, S301-S305, S207) in step S211. it can. Such pre-approval conditions are determined based on the time zone set by the second speaker in addition to the position of the portable device 300.

  For example, in step S211, the voice recognition server 200 executes a verification procedure (S206, S301-S305, S207) when the current time corresponds to a time set by the second speaker, for example, a weekday daytime. Can be determined. For example, the second speaker can set in advance a time zone when he / she will not be at home. Such pre-approval conditions are set differently for each registered speaker.

  Such pre-approval conditions are set in advance by a registered speaker and stored in the memory 220. For example, such pre-approval conditions are also determined based on a registered speaker's behavior pattern generated based on the position of the registered speaker's portable device 300. The voice recognition server 200 can collect the position of the registered speaker's portable device 300. The voice recognition server 200 can analyze the collected position of the mobile device 300 and determine a time zone in which the registered speaker is located far from the voice recognition speaker device 100. The voice recognition server 200 can automatically determine to execute the verification procedure (S206, S301-S305, S207) when the current time falls within such a time zone.

  According to another example, when the action based on the voice recognition result is included in the post-notification action list set in advance by the second speaker, the voice recognition server 200 first executes step S209 to perform a post-notification procedure ( S206, S301-S303) can be determined to be executed later. Such a post-notification action list is stored in the memory 220, and the post-notification action list includes a part of the actions that can be executed by the voice recognition speaker device 100 selected by the second speaker. Behavior is included. For example, operations such as making a call and changing settings are included in the post notification operation list set in advance. The actions included in such a post-notification action list are set differently for each registered speaker.

  The embodiment according to the present invention described above is embodied in the form of a computer program executed via various components on a computer, and such a computer program is recorded on a computer-readable medium. . Such a medium may continuously store a computer-executable program or may be temporarily stored for execution or download. Such a medium may be a variety of recording means or storage means in a form in which a single piece or a plurality of pieces of hardware are combined, but is not limited to a medium that is directly connected to a computer system. It may be distributed on the network. Examples of such media are: magnetic media such as hard disks, floppy disks and magnetic tape; optical recording media such as compact disc read only memory (CD-ROM) and digital versatile disc (DVD); Some include magneto-optical media such as optical disks; ROM, RAM, flash memory, etc., and are configured to store program instructions. Examples of other media include an application store that distributes applications, a site that supplies and distributes various other software, and a recording medium that is managed by a server.

  In this specification, the term “unit”, “module”, and the like may be a hardware component such as a processor or a circuit, and / or a software component executed by a hardware component such as a processor. For example, “part”, “module”, etc. are components such as software components, object-oriented software components, class components and task components; processes, functions, attributes, procedures, subroutines, segments of program code, drivers, firmware, micros It can be embodied by codes, circuits, data, databases, data structures, tables, arrays and variables.

  The above description of the present invention is given for the purpose of illustration, and those skilled in the art to which the present invention pertains can be applied to other specific examples without changing the technical idea and essential features of the present invention. It will be understood that the form can be easily modified. Accordingly, it should be understood that the embodiments described above are illustrative in all aspects and not limiting. For example, each component described as a single type may be implemented in a distributed manner, and similarly, a component described as being distributed may be implemented in a combined form. There is also.

  The scope of the present invention is defined by the terms of the claims, rather than the foregoing detailed description, and is intended to include all modified and modified forms derived from the meaning and scope of the claims and the equivalents thereof. Should be construed as being included within the scope of the present invention.

  The speaker verification method and the speech recognition system according to the embodiment of the present invention can be effectively applied to, for example, a technical field related to security.

100, 100a Speech recognition speaker device 110 Processor 120 Microphone 130 Speaker 140 Communication module 150 Camera 200 Speech recognition server 210, 210a Processor 211 Speech signal receiving unit 212 Speech recognition unit 213 Speaker recognition unit 213a Speaker feature vector extraction unit 213b Speaker Feature vector comparison unit 213c Registered speaker determination unit 214 Speaker authentication unit 215 Function unit 216 Synthetic sound signal generation unit 217 Verification determination unit 220 Memory 230 Communication module 300 Portable device 400 Network

Claims (19)

A speaker verification method in a voice recognition system including a voice recognition device and a voice recognition server,
The voice recognition server receiving a voice signal including the voice of the first speaker from the voice recognition device;
The voice recognition server performing voice recognition on the voice signal and generating a voice recognition result;
The speech recognition server extracts a speaker feature vector from the speech signal, compares the speaker feature vector with a registered speaker feature vector, and a speaker of the speech signal is registered according to the comparison result. Determining to be a second speaker
The voice recognition server transmitting the voice recognition result and the voice signal to the portable device of the second speaker;
The voice recognition server receiving an approval input from the portable device;
The voice recognition server performing an operation corresponding to the voice recognition result;
Speaker verification method. The speech recognition server further includes a step of determining whether to perform a verification procedure using the portable device based on at least one of an operation corresponding to the speech recognition result and a setting of the second speaker. ,
The verification procedure is:
The voice recognition server transmitting the voice recognition result and the voice signal to the portable device;
The voice recognition server receiving an approval input from the portable device;
The speaker verification method according to claim 1, comprising:   The step of determining whether or not to execute the verification procedure using the portable device is performed when the voice recognition server executes the verification procedure when at least one of the position and the current time of the portable device meets a pre-approval condition. The speaker verification method according to claim 2, comprising a step of determining.   The speaker verification method according to claim 3, wherein the pre-approval condition is set in advance by the second speaker or is determined based on an action pattern of the second speaker.   The step of determining whether to execute the verification procedure using the portable device includes the operation based on the speech recognition result included in the pre-approved operation list set in advance by the second speaker, The speaker verification method according to claim 2, comprising the step of determining execution of the verification procedure. When the operation based on the speech recognition result is included in the post-notification operation list set in advance by the second speaker, the mobile phone after the speech recognition server executes the operation corresponding to the speech recognition result. The speaker verification method according to claim 2, further comprising: transmitting the voice recognition result and the voice signal to a device. The portable device receiving the voice recognition result and the voice signal from the voice recognition server;
The portable device displaying the voice recognition result on the portable device;
The portable device playing the audio signal;
The portable device receiving an approval input or rejection input of the second speaker for approving or rejecting execution of an operation corresponding to the voice recognition result;
The portable device transmitting the approval input or rejection input to the voice recognition server;
The speaker verification method according to claim 1, further comprising: The voice recognition server receiving a rejection input from the portable device;
The voice recognition server not performing an operation corresponding to the voice recognition result;
The speaker verification method according to claim 1, further comprising: The voice recognition server receiving a response voice signal including a response voice of the second speaker from the portable device;
The voice recognition server transmitting the response voice signal to the voice recognition device such that the voice recognition device reproduces the response voice of the second speaker;
The speaker verification method according to claim 8, further comprising: When an approval input is received from the portable device, the speech recognition server uses the speaker feature vector extracted from the speech signal to improve the registered speaker feature vector of the second speaker The speaker verification method according to any one of claims 1 to 9, further comprising: The voice recognition device uses a camera to generate a video signal including a video at a time when the voice signal is generated, and transmits the video signal to the voice recognition server;
The voice recognition server transmitting the video signal together with the voice recognition result and the voice signal to the portable device;
The speaker verification method according to claim 1, further comprising:   The speaker verification method according to claim 11, wherein the video signal includes a video in a direction in which the voice of the first speaker is generated.   The speaker verification method according to claim 1, wherein the registered second speaker is one of a plurality of users registered in the voice recognition system. The speech recognition server further includes a step of not performing an operation corresponding to the speech recognition result when the speech recognition server determines that a speaker of the speech signal is an unregistered user based on the comparison result. 14. The speaker verification method according to any one of items 13 to 13.   The program which makes the processor of the speech recognition server of a speech recognition system perform the speaker verification method of any one of Claims 1 thru | or 6. A communication module for communicating with the voice recognition device and the portable device;
Using the communication module, receiving a voice signal including the voice of the first speaker from the voice recognition device, performing voice recognition on the voice signal, generating a voice recognition result, and generating a voice recognition result from the voice signal A speaker feature vector is extracted, the speaker feature vector is compared with a registered speaker feature vector, and the speaker of the speech signal is determined to be a registered second speaker based on the comparison result. , Using the communication module, transmitting the voice recognition result and the voice signal to the portable device of the second speaker, using the communication module, receiving an approval input from the portable device, and performing the voice recognition A processor configured to perform an action corresponding to the result;
Voice recognition server including The processor is
Receiving a response voice signal including a response voice of the second speaker together with a rejection input from the portable device;
17. The device according to claim 16, configured to transmit the response voice signal to the voice recognition device using the communication module such that the voice recognition device reproduces the response voice of the second speaker. Voice recognition server. A communication module for communicating with the voice recognition server according to claim 16 or 17,
A microphone that generates an audio signal;
A processor configured to detect a voice signal including the voice of the first speaker from the audio signal, send the voice signal to the voice recognition server, and receive a synthesized voice signal from the voice recognition server;
A speaker for reproducing a synthesized sound corresponding to the synthesized sound signal;
A speech recognition device. A speech recognition system including a speech recognition server and a speech recognition device,
The voice recognition device detects a voice signal from the first communication module that communicates with the voice recognition server, a microphone that generates an audio signal, and the audio signal, transmits the voice signal to the voice recognition server, and A first processor that receives a synthesized sound signal from a speech recognition server; and a speaker that reproduces the synthesized sound corresponding to the synthesized sound signal;
The voice recognition server includes a second processor and a second communication module that communicates with the voice recognition device and the portable device,
The second processor is
Receiving a voice signal including the voice of the first speaker from the voice recognition device;
Performing voice recognition on the voice signal to generate a voice recognition result;
A speaker feature vector is extracted from the speech signal, the speaker feature vector is compared with a registered speaker feature vector, and a speaker of the speech signal is registered as a result of the comparison. Recognize that there is
Transmitting the voice recognition result and the voice signal to the portable device of the second speaker;
A speech recognition system configured to perform an operation corresponding to the speech recognition result when an approval input is received from the portable device.