JP2019133182A - Speech control apparatus, speech control method, computer program, and recording medium - Google Patents

Abstract

To provide a speech control apparatus for preventing false detection of a keyword and a method of operating the same.SOLUTION: The method according to the present disclosure includes the steps of: receiving an audio signal corresponding to a surrounding sound; generating audio stream data; determining a first interval in which a candidate keyword corresponding to a predetermined keyword is detected, from the audio stream data; extracting a first speaker feature vector for identifying a speaker in the first interval and a second speaker feature vector for identifying a speaker in a second interval adjacent to the first interval from the audio stream data; and determining whether or not the predetermined keyword is included in the first interval based on similarity between the first speaker feature vector and the second speaker feature vector.SELECTED DRAWING: Figure 2

Description

  The present invention relates to a voice control device, and more particularly to a voice control device capable of preventing erroneous keyword recognition, a method for operating the voice control device, a computer program, a recording medium, and the like.

  In order to improve the operability of computer devices such as portable communication devices, desktop PCs (personal computers), tablet PCs, and entertainment systems, voice recognition functions are installed and controlled by voice. Electronic devices are on the market. The voice recognition function has an advantage that the apparatus can be easily controlled by recognizing the voice of the user regardless of a separate button operation or touch module touch.

  According to such a voice recognition function, for example, in a portable communication device such as a smartphone, a call function can be performed or a text message can be created without pressing a separate button. Various functions such as route guidance, Internet search, and alarm setting can be easily set. However, if such a voice control device misrecognizes the user's voice, there may occur a problem of performing an unintentional operation.

Korean Patent Publication No. 10-2017-0028628

  The problem to be solved by the present invention is to provide a voice control device, an operation method of the voice control device, a computer program, a recording medium, and the like that can prevent erroneous keyword recognition.

  As technical means for achieving the above-described technical problem, a first aspect of the present disclosure includes an audio processing unit that receives an audio signal corresponding to ambient sound and generates audio stream data, and the audio stream data. A keyword detecting unit that detects a candidate keyword corresponding to a predetermined keyword, and determines a start point and an end point of a first section corresponding to the first audio data in which the candidate keyword is detected in the audio stream data; A first speaker feature vector related to one audio data is extracted, and in the audio stream data, a second speaker feature vector related to second audio data corresponding to a second section whose end point is the start point of the first section is obtained. A speaker feature vector extraction unit to extract, the first speaker feature vector and the previous Based on the similarity between the second speaker feature vectors, the first audio data, a wake-up determination section that determines whether contains the keyword, it is possible to provide a voice control system comprising a.

  According to a second aspect of the present disclosure, an audio signal corresponding to ambient sound is received and audio stream data is generated, a candidate keyword corresponding to a predetermined keyword is detected from the audio stream data, and the audio Determining a start point and an end point of a first section corresponding to the first audio data in which the candidate keyword is detected in the stream data; and extracting a first speaker feature vector related to the first audio data; Extracting, from the audio stream data, a second speaker feature vector related to second audio data corresponding to a second section whose end point is the starting point of the first section; the first speaker feature vector; Based on the similarity to the two-speaker feature vector, the first audio data is added to the keyword. Determines whether contains de, it can provide a method of operating a voice control system comprising the steps of determining whether to wake up, the.

  The third aspect of the present disclosure can provide a computer program including an instruction word that causes a processor of the voice control device to execute the operation method according to the second aspect.

  The fourth aspect of the present disclosure can provide a computer-readable recording medium in which the computer program according to the third aspect is recorded.

  According to various embodiments of the present invention, since the possibility of erroneously recognizing a keyword is reduced, malfunction of the voice control device is prevented.

1 is a diagram illustrating an example of a network environment according to an embodiment. It is a block diagram for demonstrating the internal structure of an electronic device and a server by one Embodiment. 2 is a diagram illustrating an example of functional blocks that can be included in a processor of a voice control device according to an embodiment; 6 is a flowchart illustrating an example of an operation method that can be performed by a voice control apparatus according to an exemplary embodiment; 6 is a flowchart illustrating an example of an operation method that can be performed by a voice control apparatus according to another embodiment. 6 is a diagram illustrating an example in which a single command keyword is spoken when the voice control apparatus according to an embodiment executes the operation method of FIG. 5. 7 is a diagram illustrating an example in which a general dialogue voice is spoken when the voice control apparatus according to an embodiment executes the operation method of FIG. 6. 6 is a flowchart illustrating an example of an operation method that can be performed by a voice control apparatus according to another embodiment; 8 is a diagram illustrating an example in which a wakeup keyword and a natural language voice command are uttered when the voice control apparatus according to an embodiment executes the operation method of FIG. 7. 8 is a diagram illustrating an example in which a general dialogue voice is spoken when the voice control apparatus according to an embodiment executes the operation method of FIG. 7.

  Hereinafter, embodiments of the present invention will be described in detail with reference to the accompanying drawings so that those skilled in the art can easily implement them in the technical field to which the present invention belongs. However, the present invention may be embodied in various different forms and is not limited to the embodiments described herein. In the drawings, in order to clearly describe the present invention, portions not related to the description are omitted, and similar portions are denoted by similar drawing symbols throughout the specification.

  Throughout the specification, when a part is “connected” to another part, it is not only “directly connected”, but other parts are sandwiched between them and “electrical” It is also included when it is connected to. Also, when a part “includes” a component, it means that it does not exclude other components and may further include other components unless otherwise specified. .

  In this specification, phrases such as “in some embodiments” or “in one embodiment” appearing in various places do not necessarily indicate the same embodiment.

  Some embodiments are shown with functional block configurations and various processing stages. Some or all of such functional blocks may be embodied by various numbers of hardware configurations and / or software configurations that perform specific functions. For example, the functional block of the present disclosure may be embodied by one or more microprocessors or a circuit configuration for a predetermined function. Further, for example, the functional blocks of the present disclosure are embodied by various programming languages or scripting languages. The functional block is also embodied by an algorithm executed by one or more processors. In addition, the present disclosure can employ conventional techniques for electronic environment setting, signal processing, and / or data processing. Terms such as “module” and “configuration” are generic and mechanical and are not limited to physical configurations.

  Also, the connecting lines or connecting members between the components shown in the drawings are merely illustrative of functional connections and / or physical or circuit connections. In actual devices, the connections between components are indicated by various functional, physical or circuit connections that may be substituted or added.

  In the present disclosure, the keyword refers to voice information that can wake up a specific function of the voice control device. The keyword is both a single command keyword and a wake-up keyword based on the user's voice signal. The wake-up keyword is a keyword based on voice that can switch the voice control device in the sleep mode to the wake-up mode, and is also a voice keyword such as “clover” or “high computer”, for example. After the user utters the wakeup keyword, the user can utter a command for instructing a function or operation desired to be performed by the voice control device in a natural language form. It should be noted that “clover” (Cloba), which appears as an example of a wakeup keyword in the following description, is a registered trademark, and differs from “clover” in “four-leaf clover”. In this case, the voice control device can recognize a voice command in a natural language form and perform a function or operation corresponding to the result of the voice recognition. The single command keyword is also a voice keyword that can directly control the operation of the voice control device such as “stop” when music is being played. Wake-up keywords referred to in this disclosure are referred to by terms such as wake-up word, hot word, and trigger word.

  In this disclosure, candidate keywords include words that are similar in pronunciation to the keyword. For example, when the keyword is “clover”, the candidate keyword may be “clover”, “global”, “club”, and the like. The candidate keyword is defined as a keyword detected by the keyword detection unit of the voice control device as a keyword from the audio data. The candidate keyword is the same as the keyword, but is another word having a similar pronunciation to the keyword. In general, when the user utters a sentence including a term corresponding to a candidate keyword, the voice control device may erroneously recognize the keyword and wake it up. The voice control device according to the present disclosure reacts even when the candidate keyword as described above is detected from the voice signal, but can prevent the candidate keyword from being woken up.

  In the present disclosure, the voice recognition function refers to converting a user's voice signal into a character string (or text). The user's voice signal may include a voice command. The voice command can perform a specific function of the voice control device.

  In the present disclosure, a voice control device refers to an electronic device equipped with a voice control function. An electronic device equipped with a voice control function is also an independent electronic device such as a smart speaker or an artificial intelligence speaker. In addition, an electronic device equipped with a voice control function is not only a computer device equipped with a voice control function, such as a desktop PC (personal computer) or a notebook computer, but also a portable computer device, such as It is also a smartphone. In this case, a program or application for performing a voice control function is installed in the computer device. An electronic device equipped with the voice control function is an electronic product mainly performing a specific function, such as a smart TV, a smart refrigerator, a smart air conditioner, a smart navigation, and the like, and is also an automobile infotainment system. Not only that, but also the Internet devices of things controlled by voice.

  In the present disclosure, the specific function of the voice control device may include, for example, executing an application installed in the voice control device, but is not limited thereto. For example, when the voice control device is a smart speaker, specific functions of the voice control device may include music playback, Internet shopping, voice information provision, control of an electronic device or a mechanical device connected to the smart speaker, and the like. For example, when the voice control device is a smartphone, the application execution may include making a call, searching for a way, searching the Internet, or setting an alarm. For example, when the voice control device is a smart TV, the application execution may include a program search or a channel search. When the voice control device is a smart oven, the application execution may include cooking method search and the like. When the voice control device is a smart refrigerator, the application execution may include checking a refrigerated state and a frozen state, or setting a temperature. When the voice control device is a smart vehicle, the application execution may include automatic start, autonomous driving, automatic parking, and the like. Application execution in the present disclosure is not limited to the foregoing.

  In the present disclosure, the keywords can have a word form or a sphere form. In the present disclosure, the voice command uttered after the wake-up keyword may have a natural language form sentence form, a word form, or a sphere form.

  Hereinafter, the present disclosure will be described in detail with reference to the accompanying drawings.

  FIG. 1 is a diagram illustrating an example of a network environment according to an embodiment. The network environment illustrated in FIG. 1 is exemplarily illustrated to include a plurality of electronic devices 100a to 100f, a server 200, and a network 300.

  The electronic devices 100a to 100f are exemplary electronic devices controlled by voice. Each of the electronic devices 100a to 100f can perform a specific function in addition to the voice recognition function. Examples of the electronic devices 100a to 100f include smart speakers or artificial intelligence speakers, smartphones, mobile phones, navigation, computers, notebook computers, digital broadcasting terminals, PDA (personal digital assistants), PMP (portable multimedia players), There are tablet PCs and smart electronic products. The electronic devices 100a to 100f can communicate with the server 200 and / or other electronic devices 100a to 100f via the network 300 using a wireless or wired communication method. However, the present invention is not limited to this, and each of the electronic devices 100a to 100f is not connected to the network 300 and can operate independently. The electronic devices 100a to 100f are also collectively referred to as the electronic device 100.

  The communication method of the network 300 is not limited, and is not limited to a communication method using a communication network (for example, a mobile communication network, a wired Internet, a wireless Internet, or a broadcast network) that can be included in the network 300, but also the electronic device 100a. Or near field communication between 100f may be included. For example, the network 300 includes a PAN (personal area network), a LAN (local area network), a CAN (campus area network), a MAN (metropolitan area network), a WAN (wide area network), a BBN (broadband network), the Internet, and the like. One or more arbitrary networks may be included. The network 300 may include any one or more of network topologies including a bus network, a star network, a ring network, a mesh network, a star bus network, a tree network, a hierarchical network, and the like. It is not limited to.

  The server 200 is also embodied by a computer device or a plurality of computer devices that communicate with the electronic devices 100a to 100f via the network 300 and perform a voice recognition function. The server 200 is distributed in a cloud form and can provide instructions, codes, files, contents, and the like.

  For example, the server 200 receives an audio file provided from the electronic devices 100a to 100f, converts an audio signal in the audio file into a character string (or text), and converts the converted character string (or text). The electronic devices 100a to 100f can be provided. In addition, the server 200 can provide an application installation file for performing a voice control function to the electronic devices 100a to 100f connected via the network 300. For example, the second electronic device 100b can install an application using a file provided from the server 200. The second electronic device 100b is connected to the server 200 under the control of the installed operating system (OS) and / or at least one program (for example, the installed voice control application), and the voice recognition service provided by the server 200 Provided.

  FIG. 2 is a block diagram for explaining an internal configuration of an electronic device and a server according to an embodiment.

  The electronic device 100 is one of the electronic devices 100a to 100f of FIG. 1, and the electronic devices 100a to 100f may have at least the internal configuration illustrated in FIG. Although the electronic device 100 is illustrated as being connected to a server 200 that performs a voice recognition function via the network 300, it is exemplary, and the electronic device 100 independently has a voice recognition function. It can also be accomplished. The electronic device 100 is an electronic device that is controlled by voice, and is also referred to as a voice control device 100. The voice control apparatus 100 is embodied by being included in a smart speaker or an artificial intelligence speaker, a computer apparatus, a portable computer apparatus, a smart home appliance, or the like, or connected to them by wire and / or wireless.

  The electronic device 100 and the server 200 may include memories 110 and 210, processors 120 and 220, communication modules 130 and 230, and input / output interfaces 140 and 240. The memories 110 and 210 are computer-readable recording media, and non-destructive mass storage devices such as random access memory (RAM), read-only memory (ROM), and disk drives. May be included. The memories 110 and 210 store an operation system and at least one program code (for example, a code for a voice control application, a voice recognition application, and the like installed and driven in the electronic device 100). Such software components are loaded into the memories 110 and 210 via the communication modules 130 and 230 that are not computer-readable recording media. For example, at least one program is loaded into the memories 110 and 210 based on a program installed by a developer or a file distribution system that distributes an installation file of an application via a file provided via the network 300.

  The processors 120 and 220 are configured to process computer program instructions by performing basic arithmetic, logic and input / output operations. The instructions are also provided to the processors 120 and 220 by the memories 110 and 210 or the communication modules 130 and 230. For example, the processors 120 and 220 are also configured to execute instructions received by program code stored in a recording device such as the memories 110 and 210.

  The communication modules 130 and 230 can provide a function for the electronic device 100 and the server 200 to communicate with each other via the network 300, and provide a function for communicating with the other electronic devices 100b to 100f. be able to. As an example, a request (for example, a voice recognition service request) generated by the processor 120 of the electronic device 100 using a program code stored in a recording device such as the memory 110 is transmitted via the network 300 under the control of the communication module 130. Is transmitted to the server 200. In contrast, a character string (text) that is a voice recognition result provided by the control of the processor 220 of the server 200 passes through the communication module 230 and the network 300, and then via the communication module 130 of the electronic device 100. Received. For example, the speech recognition result of the server 200 received via the communication module 130 is transmitted to the processor 120 and the memory 110. The server 200 can transmit a control signal, a command, content, a file, and the like to the electronic device 100, and the control signal, the command, etc. received via the communication module 130 are transmitted to the processor 120 and the memory 110, and the content is transmitted. And the file are also stored in a separate recording medium that the electronic device 100 can further include.

  The input / output interfaces 140 and 240 are also means for interfacing with the input / output device 150. For example, the input device includes not only the microphone 151 but also a device such as a keyboard or a mouse, and the output device displays not only the speaker 152 but also a status display LED (light emitting diode) indicating the status, and a communication session of the application. Or a device such as a display. As another example, the input / output device 150 may include a device that integrates functions for input and output, such as a touch screen.

  The microphone 151 can convert ambient sound into an electrical audio signal. The microphone 151 is not directly attached to the electronic device 100 but is attached to an external device (for example, a smart watch) that is communicably connected, and the generated external signal is transmitted to the electronic device 100 by communication. In FIG. 2, the microphone 151 is illustrated as being included in the electronic device 100, but according to another embodiment, the microphone 151 is included in a separate device. The present invention is also embodied in a form connected by wired communication or wireless communication.

  In other embodiments, electronic device 100 and server 200 may include more components than the components of FIG. For example, the electronic device 100 may be configured to include at least a part of the input / output device 150 described above, or may be a transceiver, a GPS (global position system) module, a camera, various sensors, a database, or the like. Other components may be further included.

  FIG. 3 is a diagram illustrating examples of functional blocks that can be included in the processor of the voice control device according to an embodiment. FIG. 4 is an example of an operation method that can be performed by the voice control device according to the embodiment. It is an illustrated flowchart.

  As illustrated in FIG. 3, the processor 120 of the voice control device 100 includes an audio processing unit 121, a keyword detection unit 122, a speaker feature vector extraction unit 123, a wakeup determination unit 124, a voice recognition unit 125, and a function unit 126. Can be included. At least a part of the processor 120 and the functional blocks 121 to 126 may control the voice control apparatus 100 to perform the steps (S110 to S190) included in the operation method illustrated in FIG. . For example, the processor 120 and at least a part of the functional blocks 121 to 126 of the processor 120 execute an operation system code included in the memory 110 of the voice control device 100 and an instruction based on at least one program code. Is also embodied.

  A part or all of the functional blocks 121 to 126 illustrated in FIG. 3 may be embodied in a hardware configuration and / or a software configuration that performs a specific function. The functions performed by the function blocks 121 to 126 shown in FIG. 3 are implemented by one or more microprocessors or a circuit configuration for the functions. A part or all of the functional blocks 121 to 126 illustrated in FIG. 3 may be software modules configured by various programming languages or script languages executed by the processor 120. For example, the audio processing unit 121 and the keyword detection unit 122 are implemented by a digital signal processor (DSP), and the speaker feature vector extraction unit 123, the wakeup determination unit 124, and the speech recognition unit 125 are also implemented by software modules. Is done.

  The audio processing unit 121 receives an audio signal corresponding to ambient sound and generates audio stream data. The audio processing unit 121 can receive an audio signal corresponding to ambient sound from an input device such as the microphone 151. The microphone 151 is included in a peripheral device that is connected to the voice control device 100 by communication, and the audio processing unit 121 can receive the audio signal generated by the microphone 151 by communication. The ambient sounds include background sounds as well as voices spoken by the user. Therefore, the audio signal includes not only the audio signal but also the background sound signal. The background sound signal corresponds to noise in keyword detection and speech recognition.

  The audio processing unit 121 can generate audio stream data corresponding to continuously received audio signals. The audio processing unit 121 can filter and digitize the audio signal to generate audio stream data. The audio processing unit 121 filters the audio signal to remove the noise signal, and can amplify the audio signal compared to the background sound signal. The audio processing unit 121 can also remove an echo of the audio signal from the audio signal.

  The audio processing unit 121 can always operate to receive the audio signal even when the audio control device 100 operates in the sleep mode. The audio processing unit 121 operates at a low operating frequency when the audio control device 100 operates in the sleep mode, and can operate at a high operating frequency when the audio control device 100 operates in the normal mode.

  The memory 110 can temporarily store the audio stream data generated by the audio processing unit 121. The audio processing unit 121 can buffer the audio stream data using the memory 110. The memory 110 stores not only audio data including a keyword but also audio data before the keyword is detected. In order to store recent audio data in the memory 110, the audio data that was most recently stored in the memory 110 is deleted. If the size allocated to the memory 110 is the same, audio data of the same period is always stored. The period corresponding to the audio data stored in the memory 110 is preferably longer than the time for speaking a keyword.

  According to another embodiment of the present invention, the memory 110 may extract and store speaker feature vectors related to the audio stream generated by the audio processing unit 121. At that time, the speaker feature vector is extracted and stored for an audio stream having a specific length. As described above, in order to store the speaker feature vectors related to the recently generated audio stream, the most recently stored speaker feature vectors are deleted.

  The keyword detection unit 122 detects candidate keywords corresponding to predefined (that is, predetermined) keywords from the audio stream data generated by the audio processing unit 121. The keyword detection unit 122 can detect candidate keywords corresponding to the predefined keywords from the audio stream data temporarily stored in the memory 110. There may be a plurality of predefined keywords, and the plurality of predefined keywords are stored in the keyword storage 110a. The keyword storage 110 a may be included in the memory 110.

  The candidate keyword means a keyword detected from the keyword detection unit 122 as a keyword in the audio stream data. The candidate keyword may be the same as the keyword, or may be another word that is pronounced similar to the keyword. For example, when the keyword is “clover”, the candidate keyword may be “global”. That is, when the user utters a sentence including “global”, the keyword detection unit 122 may mistakenly detect “global” as “clover” and detect it from the audio stream data. The detected “global” corresponds to the candidate keyword.

  The keyword detection unit 122 can compare the audio stream data with known keyword data, and calculate the possibility that the audio stream data includes voice corresponding to the keyword. The keyword detection unit 122 may extract audio features such as filter bank energy or Mel-frequency cepstram coefficients (MFCC) from the audio stream data. The keyword detection unit 122 can process such audio features using a classifying window, for example, a support vector machine or a neural network. . Based on the processing of the audio feature, the keyword detection unit 122 can calculate the possibility that the keyword is included in the audio stream data. The keyword detection unit 122 detects a candidate keyword by determining that the keyword is included in the audio stream data when the possibility is higher than a preset reference value (that is, a predetermined reference value). Can do.

  The keyword detection unit 122 generates an artificial neural network (for example, a neural network for artificial intelligence) using a voice sample corresponding to the keyword data, and uses the generated neural network to generate a keyword from the audio stream data. To be trained to detect. The keyword detection unit 122 can calculate the probability of the phoneme constituting the keyword or the overall probability of the keyword for each frame in the audio stream data. The keyword detection unit 122 can output the probability sequence corresponding to each phoneme or the probability of the keyword itself from the audio stream data. Based on the sequence or probability, the keyword detection unit 122 can calculate the possibility that the keyword is included in the audio stream data. If the possibility is equal to or greater than the preset reference value, the candidate keyword is detected. Can be judged. The above-described method is exemplary, and the operation of the keyword detection unit 122 may be implemented through various methods.

  Further, the keyword detection unit 122 extracts an audio feature for each frame in the audio stream data, so that the audio data of the frame may correspond to a human voice and a background sound. Can be calculated. The keyword detection unit 122 compares the possibility of corresponding to human voice with the possibility of corresponding to background sound, and can determine that the audio data of the frame corresponds to human voice. For example, when the possibility that the audio data of the frame corresponds to human speech is higher than the preset reference value than the possibility that the audio data of the frame corresponds to background sound, the keyword detection unit 122 determines that the audio data of the frame is human. It can be determined that it corresponds to the voice.

  The keyword detection unit 122 can specify a section in which the candidate keyword is detected from the audio stream data, and can determine a start point and an end point of the section in which the candidate keyword is detected. The section in which the candidate keyword is detected from the audio stream data is the keyword detection section, the current section, or the first section. The audio data corresponding to the first section in the audio stream data is the first audio data. The keyword detection unit 122 can determine the end of the section in which the candidate keyword is detected as the end point. According to another example, the keyword detection unit 122 waits until a silent sound is generated for a preset time (for example, 0.5 seconds) after a candidate keyword is detected, and then the silent section is included in the first section. The end point of the first section can be determined so as to be included, or the end point of the first section can be determined so that the silent period is not included.

  The speaker feature vector extraction unit 123 reads the second audio data corresponding to the second section from the memory 110 in the audio stream data temporarily stored in the memory 110. The second section is a section before the first section, and the end point of the second section is the same as the start point of the first section. The second section is the previous section. The length of the second section is also variably set depending on the keyword corresponding to the detected candidate keyword. According to another example, the length of the second section is also fixedly set. According to another example, the length of the second interval is adaptively varied so that the keyword detection performance is optimized. For example, when the audio signal output from the microphone 151 is “four-leaf clover” and the candidate keyword is “clover”, the second audio data corresponds to the voice “four-leaf”.

  The speaker feature vector extraction unit 123 extracts a first speaker feature vector of the first audio data corresponding to the first section and a second speaker feature vector of the second audio data corresponding to the second section. . The speaker feature vector extraction unit 123 can extract speaker feature vectors that are robust to speaker recognition from audio data. The speaker feature vector extraction unit 123 converts a time domain speech signal into a frequency domain signal, and transforms the frequency energy of the converted signal to be different from each other. Speaker feature vectors can be extracted. For example, the speaker feature vectors are extracted based on Mel Frequency Cepstrum Coefficients (MFCC) or filter bank energy, but are not limited to them. Speaker feature vectors can be extracted from audio data in various ways. Can be extracted.

  The speaker feature vector extraction unit 123 can generally operate in a sleep mode. When the keyword detection unit 122 detects a candidate keyword from the audio stream data, the keyword feature vector extraction unit 123 can wake up. When the keyword detecting unit 122 detects a candidate keyword from the audio stream data, the keyword detecting unit 122 can transmit a wakeup signal to the speaker feature vector extracting unit 123. The speaker feature vector extraction unit 123 is woken up in response to a wakeup signal indicating that a candidate keyword has been detected by the keyword detection unit 122.

  According to one embodiment, the speaker feature vector extraction unit 123 extracts a frame feature vector for each frame of audio data, normalizes and averages the extracted frame feature vector, and speaks representing audio data. A person feature vector can be extracted. L2 normalization is used to normalize the extracted frame feature vector. Averaging of the extracted frame feature vectors is accomplished by calculating an average of the normalized frame feature vectors generated by normalizing the extracted frame feature vectors for each of all frames in the audio data. The

  For example, the speaker feature vector extraction unit 123 extracts a first frame feature vector for each frame of the first audio data, normalizes and averages the extracted first frame feature vector, and converts the first audio data into The representative first speaker feature vector can be extracted. Further, the speaker feature vector extraction unit 123 extracts a second frame feature vector for each frame of the second audio data, normalizes and averages the extracted second frame feature vector, and converts the second audio data into the second audio data. A representative second speaker feature vector can be extracted.

  According to another embodiment, the speaker feature vector extraction unit 123 does not extract frame feature vectors for all frames in audio data, but extracts frame feature vectors for some frames in audio data. Can be extracted. The partial frame is selected as an audio frame in a frame in which the audio data of the frame is highly likely to be user audio data. Such a voice frame is selected by the keyword detection unit 122. The keyword detection unit 122 can calculate a first probability that is a human voice and a second probability that is a background sound for each frame of the audio stream data. The keyword detection unit 122 can determine, as a speech frame, a frame in which the first probability that the audio data of each frame is human speech exceeds the preset reference value than the second probability that the background sound is human speech. The keyword detecting unit 122 can store a flag or a bit indicating whether or not the frame is an audio frame in the memory 110 in association with each frame of the audio stream data.

  When the speaker feature vector extraction unit 123 reads the first audio data and the second audio data from the memory 110, the speaker feature vector extraction unit 123 knows whether or not the frame is an audio frame by reading both the flag and the bit. it can.

  The speaker feature vector extraction unit 123 extracts a frame feature vector for each frame determined to be a speech frame from the frames in the audio data, normalizes and averages the extracted first frame feature vector, and converts the audio data into A representative speaker feature vector can be extracted. For example, the speaker feature vector extraction unit 123 extracts a first frame feature vector for each frame determined to be a speech frame from the frames in the first audio data, and normalizes the extracted first frame feature vector. And averaging the first speaker feature vectors representative of the first audio data. In addition, the speaker feature vector extraction unit 123 extracts a second frame feature vector for each frame determined to be a speech frame from the frames in the second audio data, and normalizes the extracted second frame feature vector. And a second speaker feature vector representative of the second audio data can be extracted.

  Whether the wake-up determination unit 124 includes the keyword in the first audio data based on the similarity between the first speaker feature vector and the second speaker feature vector extracted by the speaker feature vector extraction unit 123. Whether or not the keyword is included in the audio signal of the first section is determined. The wake-up determination unit 124 compares the similarity between the first speaker feature vector and the second speaker feature vector with a preset reference value, and when the similarity is equal to or less than the reference value, the first audio in the first section It can be determined that the keyword is included in the data.

  A typical case in which the voice control apparatus 100 misrecognizes a keyword is a case where a word having a pronunciation similar to the keyword is located in the middle of the voice in the user's voice. For example, when the keyword is “clover”, the voice control device 100 is waked up in response to “clover” when the user says “how to find a four-leaf clover” to others, and the user intends You might end up doing something you didn't do. Furthermore, when an announcer in TV news says "JN Global's market capitalization is ...", voice control device 100 may be woken up in response to "Global". In order to prevent such misrecognition of a keyword from occurring, according to an embodiment, the voice control device 100 may detect a word with a pronunciation similar to the keyword only when it is positioned first in the voice. react. Also, in an environment where there is a lot of background noise in the surroundings, or in an environment where other people are talking, even if the user speaks the voice that corresponds to the keyword first, It may not be detected that the voice corresponding to the keyword is spoken first. According to one embodiment, the voice control device 100 extracts a first speaker feature vector of a section in which a candidate keyword is detected and a second speaker feature vector of a previous section, and the first speaker feature vector When the second speaker feature vectors are different from each other, it can be determined that the user has spoken the voice corresponding to the keyword first.

  For this determination, the wake-up determination unit 124 determines that the user corresponds to the keyword when the similarity between the first speaker feature vector and the second speaker feature vector is equal to or less than a preset reference value. Can be determined to have been uttered first. That is, the wakeup determination unit 124 can determine that the keyword is included in the first audio data in the first section, and can wake up a part of the function of the voice control device 100. The high degree of similarity between the first speaker feature vector and the second speaker feature vector means that the person who emitted sound corresponding to the first audio data and the person who emitted sound corresponding to the second audio data Are likely to be the same.

  When the second audio data corresponds to silence, the speaker feature vector extraction unit 123 can extract a second speaker feature vector corresponding to silence from the second audio data. Since the speaker feature vector extraction unit 123 extracts the first speaker feature vector corresponding to the user's voice from the first audio data, the similarity between the first speaker feature vector and the second speaker feature vector is ,Low.

  The voice recognition unit 125 can receive the third audio data corresponding to the third section in the audio stream data generated by the audio processing unit 121, and can recognize the third audio data. According to another example, the voice recognition unit 125 transmits the third audio data to the outside and receives the voice recognition result so that the third audio data is recognized by the outside (for example, the server 200). be able to.

  The function unit 126 can perform a function corresponding to the keyword. For example, when the voice control device 100 is a smart speaker, the function unit 126 includes a music playback unit, a voice information providing unit, a peripheral device control unit, and the like, and can perform a function corresponding to the detected keyword. When the voice control device 100 is a smartphone, the function unit 126 includes a telephone connection unit, a character transmission / reception unit, an Internet search unit, and the like, and can perform a function corresponding to the detected keyword. The functional unit 126 is variously configured depending on the type of the voice control device 100. The functional unit 126 comprehensively shows functional blocks for performing various functions that the voice control apparatus 100 can perform.

  The voice control device 100 illustrated in FIG. 3 is illustrated as including the voice recognition unit 125, but this is exemplary, and the voice control device 100 does not include the voice recognition unit 125. The server 200 shown in FIG. 5 can perform the voice recognition function instead. In that case, as shown in FIG. 1, the voice control device 100 is connected via a network 300 to a server 200 that performs a voice recognition function. The voice control device 100 can provide a voice file including a voice signal that needs voice recognition to the server 200, and the server 200 performs voice recognition on the voice signal in the voice file and corresponds to the voice signal. A string can be generated. The server 200 can transmit the generated character string to the voice control device 100 via the network 300. However, the following description assumes that the voice control device 100 includes a voice recognition unit 125 that performs a voice recognition function.

  The processor 120 can load the program code stored in the program file for the operation method into the memory 110. For example, a program is installed in the voice control device 100 by a program file. At that time, when the program installed in the voice control device 100 is executed, the processor 120 can load the program code into the memory 110. At that time, at least some of the audio processing unit 121, the keyword detection unit 122, the speaker feature vector extraction unit 123, the wakeup determination unit 124, the speech recognition unit 125, and the function unit 126 included in the processor 120 are stored in the memory 110. The present invention may be realized by executing an instruction using a corresponding code among the loaded program codes and executing the steps (S110 to S190) of FIG.

  Thereafter, the fact that the function blocks 121 to 126 of the processor 120 control the voice control device 100 is understood to mean that the processor 120 controls other components of the voice control device 100. For example, the processor 120 controls the communication module 130 included in the voice control apparatus 100, and can control the voice control apparatus 100 so that the voice control apparatus 100 communicates with the server 200, for example.

  In step (S110), the processor 120, for example, the audio processing unit 121 receives an audio signal corresponding to the ambient sound. The audio processing unit 121 can continuously receive an audio signal corresponding to the ambient sound. The audio signal is also an electric signal generated by an input device such as the microphone 151 corresponding to the ambient sound.

  In step (S <b> 120), the processor 120, for example, the audio processing unit 121 generates audio stream data based on the audio signal from the microphone 151. The audio stream data corresponds to an audio signal that is continuously received. The audio stream data is also data generated by filtering and digitizing an audio signal.

  In step (S130), the processor 120, for example, the audio processing unit 121 temporarily stores the audio stream data generated in step (S120) in the memory 110. The memory 110 has a limited size, and a part of the audio stream data corresponding to the audio signal for a certain period of time from the present is temporarily stored in the memory 110. When new audio stream data is generated, the oldest data among the audio stream data stored in the memory 110 is deleted, and the new audio stream data is stored in a space that is freed by the deletion in the memory 110. Is done.

  In step (S140), the processor 120, for example, the keyword detection unit 122 detects candidate keywords corresponding to the predefined keywords from the audio stream data generated in step (S120). The candidate keyword is a word having a pronunciation similar to that of the predefined keyword, and indicates a word detected as a keyword by the keyword detection unit 122 in step (S140).

  In step (S150), the processor 120, for example, the keyword detection unit 122 identifies a keyword detection section in which a candidate keyword is detected from the audio stream data, and determines a start point and an end point of the keyword detection section. The keyword detection section is the current section. The data corresponding to the current section in the audio stream data is the first audio data.

  In step (S160), the processor 120, for example, the speaker feature vector extraction unit 123 reads the second audio data corresponding to the previous section from the memory 110. The previous section is a section immediately before the current section, and the end point of the previous section is also the same as the start point of the current section. The speaker feature vector extraction unit 123 can also read the first audio data from the memory 110 together.

  In step (S170), the processor 120, for example, the speaker feature vector extraction unit 123 extracts the first speaker feature vector from the first audio data, and extracts the second speaker feature vector from the second audio data. The first speaker feature vector is an index for identifying the speaker of the speech corresponding to the first audio data, and the second speaker feature vector identifies the speaker of the speech corresponding to the second audio data. It is an indicator for. The processor 120, for example, the wakeup determination unit 124 determines whether or not a keyword is included in the first audio data based on the similarity between the first speaker feature vector and the second speaker feature vector. be able to. When the wakeup determination unit 124 determines that the keyword is included in the first audio data, the wakeup determination unit 124 can wake up some components of the voice control device 100.

  In step (S180), the processor 120, for example, the wakeup determination unit 124 compares the similarity between the first speaker feature vector and the second speaker feature vector with a preset reference value.

  When the similarity between the first speaker feature vector and the second speaker feature vector is equal to or less than a preset reference value, the wakeup determination unit 124 determines whether the speaker of the first audio data in the current section is the second in the previous section. Since the speakers of the audio data are different from each other, it can be determined that a keyword is included in the first audio data. In that case, as in step (S190), the processor 120, for example, the wake-up determination unit 124, can wake up some components of the voice control device 100.

  However, if the similarity between the first speaker feature vector and the second speaker feature vector is higher than the preset reference value, the wakeup determination unit 124 determines whether the speaker of the first audio data in the current section is Since the two audio data speakers are the same as each other, it is determined that the keyword is not included in the first audio data, and wakeup cannot proceed. In this case, the process proceeds to step (S110), and an audio signal corresponding to the ambient sound is received. In step (S110), audio signal reception is continued when performing steps (S120 to S190).

  A plurality of predefined keywords are stored in the keyword storage location 110a in FIG. Such keywords are both wake-up keywords and single command keywords. The wakeup keyword is used to wake up a part of the function of the voice control device 100. Generally, a user utters a desired natural language voice command after speaking a wakeup keyword. The voice control device 100 can recognize a natural language voice command and perform operations and functions corresponding to the natural language voice command.

  The single command keyword is for the voice control device 100 to directly perform a specific operation or function, and is also a simple simple word such as “play” or “stop”. When the single command keyword is received, the voice control device 100 can wake up the function corresponding to the single command keyword and perform the function.

  Hereinafter, a case where a candidate keyword corresponding to a single command keyword is detected from audio stream data and a case where a candidate keyword corresponding to a wakeup keyword is detected from audio stream data will be described.

  FIG. 5 is a flowchart illustrating an example of an operation method that can be performed by a voice control apparatus according to another embodiment.

  6A illustrates an example in which a single command keyword is uttered when the voice control device according to the embodiment executes the operation method of FIG. 5, and FIG. 6B illustrates the voice control device according to the embodiment. In the case where the operation method is executed, an example in which a general dialogue voice is spoken is illustrated.

  The operation method of FIG. 5 includes steps that are substantially the same as the operation method of FIG. Of the steps of FIG. 5, those steps that are substantially the same as the steps of FIG. 4 will not be described in detail. 6A and 6B illustrate an audio signal corresponding to audio stream data and a user's voice corresponding to the audio signal. 6A illustrates an audio signal corresponding to the voice “stop”, and FIG. 6B illustrates an audio signal corresponding to the voice “stop here”.

  Referring to FIG. 5 together with FIGS. 6A and 6B, in step (S210), the processor 120, for example, the audio processing unit 121 receives an audio signal corresponding to the ambient sound.

  In the step (S220), the processor 120, for example, the audio processing unit 121 generates audio stream data based on the audio signal from the microphone 151.

  In step (S230), the processor 120, for example, the audio processing unit 121 temporarily stores the audio stream data generated in step (S220) in the memory 110.

  In step (S240), the processor 120, for example, the keyword detection unit 122 detects candidate keywords corresponding to the predefined single command keyword from the audio stream data generated in step (S220). The single command keyword is also a voice keyword that can directly control the operation of the voice control device 100. For example, the single command keyword is also a word such as “stop” as illustrated in FIG. 6A. In that case, the audio control device 100 reproduces music or a moving image, for example.

  In the example of FIG. 6A, the keyword detection unit 122 can detect the candidate keyword “stop” from the audio signal. In the example of FIG. 6B, the keyword detection unit 122 can detect a candidate keyword “stop”, which is a word having a pronunciation similar to the keyword “stop”, from the audio signal.

  In step (S250), the processor 120, for example, the keyword detection unit 122 identifies a keyword detection section in which a candidate keyword is detected from the audio stream data, and determines a start point and an end point of the keyword detection section. The keyword detection section is the current section. In the audio stream data, the data corresponding to the current section is the first audio data.

  In the example of FIG. 6A, the keyword detection unit 122 can identify the section in which the candidate keyword “stop” is detected as the current section, and can determine the start point and end point of the current section. The audio data corresponding to the current section is the first audio data AD1.

  In the example of FIG. 6B, the keyword detection unit 122 can identify the section in which the candidate keyword “stop” is detected as the current section, and can determine the start point and end point of the current section. The audio data corresponding to the current section is the first audio data AD1.

  In step (S250), the processor 120, for example, the keyword detection unit 122 determines whether the detected candidate keyword is a candidate keyword corresponding to any one of the wakeup keyword and the single command keyword. be able to. 6A and 6B, the keyword detection unit 122 can determine that the detected candidate keywords, that is, “stop” and “stop” are candidate keywords corresponding to the single command keywords.

  In step (S260), the processor 120, for example, the speaker feature vector extraction unit 123 reads the second audio data corresponding to the previous section from the memory 110. The previous section is a section immediately before the current section, and the end point of the previous section is also the same as the start point of the current section. The speaker feature vector extraction unit 123 can also read the first audio data from the memory 110 together.

  In the example of FIG. 6A, the speaker feature vector extraction unit 123 can read from the memory 110 the second audio data AD2 corresponding to the previous section that is the section immediately before the current section. In the example of FIG. 6B, the speaker feature vector extraction unit 123 can read from the memory 110 the second audio data AD2 corresponding to the previous section that is the section immediately before the current section. In the example of FIG. 6B, the second audio data AD2 corresponds to the voice “kode”. The length of the previous section is also variably set according to the detected candidate keyword.

  In step (S <b> 270), the processor 120, for example, the speaker feature vector extraction unit 123 receives third audio data corresponding to the next section after the current section from the audio processing unit 121. The next section is the section immediately following the current section, and the start point of the next section is the same as the end point of the current section.

  In the example of FIG. 6A, the speaker feature vector extraction unit 123 can receive the third audio data AD3 corresponding to the next section immediately after the current section from the audio processing unit 121. In the example of FIG. 6B, the speaker feature vector extraction unit 123 can receive the third audio data AD3 corresponding to the next section immediately after the current section from the audio processing unit 121. In the example of FIG. 6B, the third audio data AD3 corresponds to the voice “do”. The length of the next section is also variably set according to the detected candidate keyword.

  In step (S280), the processor 120, for example, the speaker feature vector extraction unit 123 extracts the first speaker feature vector to the third speaker feature vector from the first audio data 1 to the third audio data, respectively. Each of the first speaker feature vector to the third speaker feature vector is an index for identifying a speaker of speech corresponding to the first audio data 1 to the third audio data. Based on the similarity between the first speaker feature vector and the second speaker feature vector and the similarity between the first speaker feature vector and the third speaker feature vector. It can be determined whether or not a single command keyword is included in the first audio data. When determining that the first audio data includes the single command keyword, the wakeup determination unit 124 can wake up some components of the voice control device 100.

  In the example of FIG. 6A, the first speaker feature vector corresponding to the first audio data AD1 is an index for identifying the speaker who uttered the voice “stop”. Since the second audio data AD2 and the third audio data AD3 are substantially silent, the second speaker feature vector and the third speaker feature vector can have vectors corresponding to the silence. Therefore, the similarity between the first speaker feature vector, the second speaker feature vector, and the third speaker feature vector is low.

  As another example, when a non-speaker who uttered the speech “stop” in the previous section and the next section utters speech, the second speaker feature vector and the third speaker feature vector are Since the vector corresponding to the other person is included, the similarity between the first speaker feature vector, the second speaker feature vector, and the third speaker feature vector is low.

  In the example of FIG. 6B, one person uttered “Stop here”. Therefore, the first speaker feature vector extracted from the first audio data AD1 corresponding to “stop”, the second speaker feature vector extracted from the second audio data AD2 corresponding to “kode”, and “shi Since the third speaker feature vector extracted from the third audio data AD3 corresponding to “t” is a vector for identifying speakers that are substantially the same, the first speaker feature vector through the first speaker feature vector. The similarity to the three-speaker feature vector is high.

  In step (S290), the processor 120, for example, the wake-up determination unit 124 compares the similarity between the first speaker feature vector and the second speaker feature vector with the preset reference value, and determines the first speaker feature vector. And the third speaker feature vector are compared with a preset reference value. The wake-up determination unit 124 has a similarity between the first speaker feature vector and the second speaker feature vector equal to or less than a preset reference value, and the similarity between the first speaker feature vector and the third speaker feature vector is Since the speaker of the first audio data in the current section is different from the speaker of the second audio data in the previous section and the speaker of the third audio data in the next section if it is less than the preset reference value, It can be determined that a single command keyword is included in one audio data. In this case, as in step (S300), the processor 120, for example, the wakeup determination unit 124 provides the single command keyword to the function unit 126, and the function unit 126 adds the first audio data to the first audio data by the wakeup determination unit 124. In response to a determination that a single command keyword is included, a function corresponding to the single command keyword can be performed.

  In the example of FIG. 6A, the first speaker feature vector is a vector corresponding to the speaker who has uttered “stop”, and the second speaker feature vector and the third speaker feature vector are vectors corresponding to the silence. Therefore, the similarity between the first speaker feature vector, the second speaker feature vector, and the third speaker feature vector is lower than the preset reference value. In that case, the wakeup determination unit 124 can determine that the first audio data AD1 includes the single command keyword “stop”. In this case, the function unit 126 is waked up in response to the determination, and can perform an operation or function corresponding to the single command keyword “stop”. For example, if the voice control device 100 is playing music, the function unit 126 can stop the music playback in response to the single command keyword “stop”.

  However, the wakeup determination unit 124 determines whether the similarity between the first speaker feature vector and the second speaker feature vector is higher than a preset reference value, or the first speaker feature vector and the third speaker feature vector. Is higher than the preset reference value, the speaker of the first audio data in the current section is the same as the speaker of the second audio data of the previous section or the third audio data of the next section. Therefore, it is determined that no keyword is included in the first audio data, and the wakeup cannot proceed. In that case, the process proceeds to step (S210), and an audio signal corresponding to the ambient sound is received.

  In the example of FIG. 6B, since one person uttered “Stop here”, the similarity between the first speaker feature vector and the third speaker feature vector is high. In the example of FIG. 6B, “stop here” does not include a keyword for controlling the voice control device or causing the wake-up to occur, so the wake-up determination unit 124 determines the first audio data AD1. Are not included in the single command keyword, so that the function unit 126 does not perform the function or operation corresponding to “stop” or “stop”.

  According to a general technique, the voice control device is technically capable of detecting a voice of “stop” from utterances of “stop here” and performing a function or operation corresponding to “stop”. Is possible. Such functions and operations are not intended by the user, and the user feels inconvenience when using the voice control device. However, according to one embodiment, since the voice control device 100 can accurately recognize the single command keyword from the user's voice, unlike the general technique, the voice control device 100 does not perform a malfunction.

  FIG. 7 is a flowchart illustrating an example of an operation method that can be performed by a voice control apparatus according to still another embodiment.

  FIG. 8A illustrates an example in which a wake-up keyword and a natural language voice command are spoken when the voice control apparatus according to one embodiment executes the operation method of FIG. 7, and FIG. 8B illustrates a voice control apparatus according to one embodiment. However, when the operation method of FIG. 7 is executed, an example in which a general dialogue voice is spoken is illustrated.

  The operation method of FIG. 7 includes substantially the same steps as the operation method of FIG. Of the steps in FIG. 7, steps that are substantially the same as the steps in FIG. 4 will not be described in detail. 6A and 6B illustrate an audio signal corresponding to audio stream data and a user's voice corresponding to the audio signal. FIG. 8A illustrates an audio signal corresponding to the wakeup keyword “clover” and the natural language voice command “tell me tomorrow's weather”, and FIG. An audio signal corresponding to speech is illustrated.

  Referring to FIG. 7 together with FIGS. 8A and 8B, in step (S410), the processor 120, for example, the audio processing unit 121 receives an audio signal corresponding to the ambient sound. In the step (S420), the processor 120, for example, the audio processing unit 121 generates audio stream data based on the audio signal from the microphone 151. In step (S430), the processor 120, for example, the audio processing unit 121 temporarily stores the audio stream data generated in step (S120) in the memory 110.

  In step (S440), the processor 120, for example, the keyword detection unit 122 detects candidate keywords corresponding to the predefined wakeup keyword from the audio stream data generated in step (S420). The wake-up keyword is a keyword based on voice that can switch the voice control device in the sleep mode to the wake-up mode. For example, the wake-up keyword is also a voice keyword such as “clover” or “high computer”.

  In the example of FIG. 8A, the keyword detection unit 122 can detect a candidate keyword “clover” from the audio signal. In the example of FIG. 8B, the keyword detection unit 122 can detect a candidate keyword “clover” that is a word having a pronunciation similar to the keyword “clover” from the audio signal.

  In step (S450), the processor 120, for example, the keyword detection unit 122 identifies a keyword detection section in which a candidate keyword is detected from the audio stream data, and determines a start point and an end point of the keyword detection section. The keyword detection section is the current section. The data corresponding to the current section in the audio stream data is the first audio data.

  In the example of FIG. 8A, the keyword detection unit 122 can identify the section in which the candidate keyword “crowbar” is detected as the current section, and can determine the start point and end point of the current section. The audio data corresponding to the current section is the first audio data AD1. In the example of FIG. 8B, the keyword detection unit 122 can identify the section in which the candidate keyword “clover” is detected as the current section, and can determine the start point and the end point of the current section. The audio data corresponding to the current section is the first audio data AD1.

  In step (S450), the processor 120, for example, the keyword detection unit 122 determines whether the detected candidate keyword is a candidate keyword corresponding to any one of the wakeup keyword and the single command keyword. Can do. 8A and 8B, the keyword detection unit 122 can determine that the detected candidate keywords, that is, “clover” and “clover” are candidate keywords corresponding to the wakeup keyword.

  In step (S <b> 460), the processor 120, for example, the speaker feature vector extraction unit 123 reads the second audio data corresponding to the previous section from the memory 110. The previous section is a section immediately before the current section, and the end point of the previous section is also the same as the start point of the current section. The speaker feature vector extraction unit 123 can also read the first audio data from the memory 110 together.

  In the example of FIG. 8A, the speaker feature vector extraction unit 123 can read from the memory 110 the second audio data AD2 corresponding to the previous section that is the section immediately before the current section. In the example of FIG. 8B, the speaker feature vector extraction unit 123 can read from the memory 110 the second audio data AD2 corresponding to the previous section that is the section immediately before the current section. In the example of FIG. 8B, the second audio data AD2 corresponds to the voice “, Yotsuba”. The length of the previous section is also variably set according to the detected candidate keyword.

  In step (S470), the processor 120, for example, the speaker feature vector extracting unit 123 extracts the first speaker feature vector and the second speaker feature vector from the first audio data and the second audio data, respectively. The processor 120, for example, the wakeup determination unit 124, determines whether or not the wakeup keyword is included in the first audio data based on the similarity between the first speaker feature vector and the second speaker feature vector. Judgment can be made. When the wakeup determination unit 124 determines that the wakeup keyword is included in the first audio data, the wakeup determination unit 124 can wake up some components of the voice control device 100.

  In the example of FIG. 8A, the first speaker feature vector corresponding to the first audio data AD1 is an index for identifying the speaker who uttered the voice “clover”. Since the second audio data AD2 is substantially silent, the second speaker feature vector may have a vector corresponding to the silence. Therefore, the degree of similarity between the first speaker feature vector and the second speaker feature vector is low.

  As another example, when another person who is not the speaker who uttered the voice “clover” in the previous section utters the voice, the second speaker feature vector has a vector corresponding to the other person. The degree of similarity between the speaker feature vector and the second speaker feature vector is low.

  In the example of FIG. 8B, one person said, "How can I find a four-leaf clover?" Therefore, the first speaker feature vector extracted from the first audio data AD1 corresponding to “clover” and the second speaker feature vector extracted from the second audio data AD2 corresponding to “four leaves” Are vectors for identifying speakers that are substantially the same, the degree of similarity between the first speaker feature vector and the second speaker feature vector is high.

  In step (S480), the processor 120, for example, the wakeup determination unit 124 compares the similarity between the first speaker feature vector and the second speaker feature vector with a preset reference value. When the similarity between the first speaker feature vector and the second speaker feature vector is higher than the preset reference value, the wakeup determination unit 124 and the speaker of the first audio data in the current section and the second audio in the previous section Since the data speakers are the same as each other, it is determined that the keyword is not included in the first audio data, and wakeup cannot proceed. In that case, proceeding to step (S410), the processor 120, for example, the audio processing unit 121 receives an audio signal corresponding to the ambient sound.

  In the example of FIG. 8B, since one person uttered “four-leaf clover ...”, the similarity between the first speaker feature vector and the second speaker feature vector is high. In the example of FIG. 8B, the person who utters “four-leaf clover” determines that there is no intention to wake up the voice control device 100, and the wakeup determination unit 124 includes the wakeup keyword in the first audio data AD1. Therefore, the voice control device 100 is not woken up.

  When the similarity between the first speaker feature vector and the second speaker feature vector is equal to or less than a preset reference value, the wakeup determination unit 124 determines whether the speaker of the first audio data in the current section is the first speaker data in the previous section. Since the two audio data speakers are different from each other, it can be determined that a keyword is included in the first audio data. In that case, the wake-up determination unit 124 can wake up some components of the voice control device 100. For example, the wakeup determination unit 124 can wake up the voice recognition unit 125.

  In the example of FIG. 8A, the first speaker feature vector is a vector corresponding to the speaker who uttered “clover”, and the second speaker feature vector is a vector corresponding to the silent sound. The similarity between the speaker feature vector and the second speaker feature vector is lower than the preset reference value. In this case, the wakeup determination unit 124 can determine that the wakeup keyword “clover” is included in the first audio data AD1. In that case, the voice recognition unit 125 is woken up to recognize a natural language voice command.

  In step (S <b> 490), the processor 120, for example, the speech recognition unit 125 receives third audio data corresponding to the next section after the current section from the audio processing unit 121. The next section is the section immediately following the current section, and the start point of the next section is the same as the end point of the current section.

  The voice recognition unit 125 can determine the end point of the next section when a preset length of silence is detected in the third audio data. The voice recognition unit 125 can recognize the third audio data. The speech recognition unit 125 can recognize the third audio data by various methods. According to another example, the voice recognition unit 125 may send third audio data to an external device, for example, the server 200 having the voice recognition function illustrated in FIG. Can be transmitted. The server 200 receives the third audio data, recognizes the third audio data by voice, generates a character string (text) corresponding to the third audio data, and converts the generated character string (text) into the voice. The recognition result can be transmitted to the voice recognition unit 125.

  In the example of FIG. 8A, the third audio data in the next section is a natural language voice command such as “Tell me about tomorrow's weather”. The voice recognition unit 125 can directly recognize the third audio data and generate a voice recognition result, or transmit the third audio data to the outside (for example, the server 200) so that the third audio data is recognized. .

  In step S500, the processor 120, for example, the functional unit 126 may perform a function corresponding to the voice recognition result of the third audio data. In the example of FIG. 8A, the function unit 126 is also an audio information providing unit that searches for tomorrow's weather and provides the results. The function unit 126 searches for tomorrow's weather using the Internet, and sends the results to the user. Can be provided. The function unit 126 can also provide a search result of tomorrow's weather as audio using the speaker 152. The functional unit 126 is woken up in response to the voice recognition result of the third audio data.

  The embodiment according to the present invention described above is embodied in the form of a computer program that is executed on a computer via various components, and such a computer program is recorded on a computer-readable medium. . At that time, the medium may be a program that continuously stores a computer-executable program or a temporary storage for execution or download. The medium may be a variety of recording means or storage means in a form in which a single piece or several pieces of hardware are combined. However, the medium is not limited to a medium that is directly connected to a computer system. It is also distributed on top. Examples of the medium include magnetic media such as hard disks, floppy disks and magnetic tapes; optical recording media such as compact disc read only memory (CD-ROM) and digital versatile discs (DVD); floptical disks (floptical) including a magneto-optical medium such as disk; and read-only memory (ROM), random access memory (RAM), flash memory, etc., configured to store program instructions It is also a thing. Examples of other media include an application store that distributes applications, a site that supplies and distributes various other software, and a recording medium or recording medium that is managed by a server.

  In this specification, “unit”, “module” and the like may be a hardware component such as a processor or a circuit and / or a software component executed by a hardware configuration such as a processor. is there. For example, “part”, “module”, etc. are software components, object-oriented software components, components such as class components and task components, processes, functions, attributes, procedures, subroutines, segments of program code, It is also embodied by drivers, firmware, microcode, circuits, data, databases, data structures, tables, arrays and variables.

  The above description of the present invention is for illustrative purposes only, and those skilled in the art to which the present invention pertains can be applied to other specific examples without changing the technical idea and essential features of the present invention. It will be understood that it can be easily transformed into a specific form. Accordingly, it should be understood that the embodiments described above are illustrative in all aspects and not limiting. For example, each component described as a single type may be implemented in a distributed manner, and similarly, a component described as being distributed may be implemented in a combined form.

  The scope of the present invention is defined by the terms of the claims, rather than the foregoing detailed description, and all modifications or variations derived from the meaning and scope of the claims and the equivalents thereof are It should be construed as being included in the scope of the invention.

  The voice control device and its operation method for preventing erroneous keyword recognition of the present invention can be effectively applied to, for example, a technical field related to voice recognition.

(Appendix 1)
An audio processing unit that receives an audio signal corresponding to ambient sound and generates audio stream data;
Keyword detection for detecting a candidate keyword corresponding to a predetermined keyword from the audio stream data, and determining a start point and an end point of a first section corresponding to the first audio data in which the candidate keyword is detected in the audio stream data And
A first speaker feature vector related to the first audio data is extracted, and in the audio stream data, a second speaker feature related to the second audio data corresponding to the second section whose end point is the start point of the first section. A speaker feature vector extraction unit for extracting a vector;
A wakeup determination unit that determines whether or not the keyword is included in the first audio data based on a similarity between the first speaker feature vector and the second speaker feature vector; Control device.
(Appendix 2)
The wakeup determination unit includes the keyword in the first audio data when the similarity between the first speaker feature vector and the second speaker feature vector is equal to or less than a preset reference value. The voice control device according to appendix 1, characterized in that:
(Appendix 3)
A keyword storage for storing a plurality of keywords including the predetermined keyword;
The voice control device according to appendix 1, wherein each of the keywords is a wakeup keyword or a single command keyword.
(Appendix 4)
When the keyword detection unit detects the candidate keyword corresponding to the single command keyword from the audio stream data,
The speaker feature vector extraction unit receives third audio data corresponding to a third section starting from an end point of the first section in the audio stream data, and a third speaker feature of the third audio data. Extract the vector,
The wake-up determination unit is based on the similarity between the first speaker feature vector and the second speaker feature vector and the similarity between the first speaker feature vector and the third speaker feature vector. 4. The voice control apparatus according to appendix 3, wherein the first audio data is determined whether or not the single command keyword is included.
(Appendix 5)
The wakeup determination unit has a similarity between the first speaker feature vector and the second speaker feature vector equal to or less than a predetermined reference value, and the first speaker feature vector and the third speaker feature vector. 5. The voice control device according to appendix 4, wherein the first command data is determined to be included in the first audio data when the similarity to the first audio data is equal to or less than a predetermined reference value.
(Appendix 6)
When the keyword detection unit detects the candidate keyword corresponding to the wakeup keyword from the audio stream data,
In response to the determination by the wake-up determination unit that the wake-up keyword is included in the first audio data, the audio stream data includes a third section starting from the end point of the first section. And an audio recognition unit for receiving the corresponding third audio data and recognizing the third audio data or transmitting the third audio data to the outside so that the third audio data is recognized. 4. The voice control device according to 3.
(Appendix 7)
The voice control device according to appendix 6, wherein the second section is variably determined by the wakeup keyword.
(Appendix 8)
The speaker feature vector extraction unit includes:
A first frame feature vector is extracted for each frame of the first audio data, the extracted first frame feature vector is normalized and averaged, and the first speaker feature vector representing the first audio data is represented. Extract
Extracting a second frame feature vector for each frame of the second audio data; normalizing and averaging the extracted second frame feature vectors; and representing the second speaker feature vector representing the second audio data The voice control device according to supplementary note 1, wherein:
(Appendix 9)
The keyword detection unit calculates, for each frame of the audio stream data, a first probability that is a human voice and a second probability that is a background sound, and the first probability is a predetermined probability based on the second probability. A frame that exceeds the reference value is determined as an audio frame,
The speaker feature vector extraction unit includes:
In each frame in the first audio data, a first frame feature vector is extracted for each frame determined to be an audio frame, the extracted first frame feature vector is normalized and averaged, and the first audio data Extracting the first speaker feature vector representative of
A second frame feature vector is extracted for each frame determined to be an audio frame in the frames in the second audio data, and the extracted second frame feature vector is normalized and averaged, and the second audio data The speech control apparatus according to appendix 1, wherein the second speaker feature vector representing the above is extracted.
(Appendix 10)
The speech control apparatus according to appendix 1, wherein the speaker feature vector extraction unit is woken up in response to detection of the candidate keyword by the keyword detection unit.
(Appendix 11)
Receiving an audio signal corresponding to ambient sound and generating audio stream data;
Detecting a candidate keyword corresponding to a predetermined keyword from the audio stream data, and determining a start point and an end point of a first section corresponding to the first audio data in which the candidate keyword is detected in the audio stream data; ,
Extracting a first speaker feature vector related to the first audio data;
Extracting a second speaker feature vector related to the second audio data corresponding to the second section whose end point is the start point of the first section in the audio stream data;
Based on the similarity between the first speaker feature vector and the second speaker feature vector, it is determined whether or not the keyword is included in the first audio data, and whether or not to wake up is determined. And a method of operating the voice control device.
(Appendix 12)
Determining whether to wake up,
Comparing the similarity between the first speaker feature vector and the second speaker feature vector with a predetermined reference value;
When the similarity is equal to or lower than the predetermined reference value, determining that the keyword is included in the first audio data and causing the wakeup to occur;
The audio according to claim 11, further comprising: when the similarity exceeds the predetermined reference value, determining that the keyword is not included in the first audio data and causing the first audio data not to wake up. The operation method of the control device.
(Appendix 13)
When the detected candidate keyword is the candidate keyword corresponding to a single command keyword,
Receiving, in the audio stream data, third audio data corresponding to a third section starting from an end point of the first section;
Extracting a third speaker feature vector of the third audio data;
The similarity between the first speaker feature vector and the second speaker feature vector is less than or equal to a predetermined reference value, and the similarity between the first speaker feature vector and the third speaker feature vector is: The operation method of the voice control device according to appendix 11, further comprising a step of determining that the single audio keyword is included in the first audio data when the value is equal to or less than a predetermined reference value. .
(Appendix 14)
14. The voice control device according to claim 13, further comprising performing a function corresponding to the single command keyword in response to a determination that the single command keyword is included in the first audio data. How it works.
(Appendix 15)
If the detected keyword is the candidate keyword corresponding to a wakeup keyword,
In response to the determination that the wakeup keyword is included in the first audio data, the audio stream data receives third audio data corresponding to a third section starting from the end point of the first section. And the stage of
The operation method of the voice control apparatus according to claim 11, further comprising: recognizing the third audio data or transmitting the third audio data to the outside so that the third audio data is recognized. .
(Appendix 16)
Extracting the first speaker feature vector and the second speaker feature vector;
Extracting a first frame feature vector for each frame of the first audio data;
Normalizing and averaging the extracted first frame feature vectors to extract the first speaker feature vectors representative of the first audio data;
Extracting a second frame feature vector for each frame of the second audio data;
The speech according to claim 11, further comprising: normalizing and averaging the extracted second frame feature vector, and extracting the second speaker feature vector representing the second audio data. The operation method of the control device.
(Appendix 17)
For each frame of the audio stream data, a first probability that is human speech and a second probability that is background sound are calculated, and the first probability is higher than the second probability by exceeding a predetermined reference value. Further comprising determining the frame as an audio frame;
Extracting the first speaker feature vector and the second speaker feature vector;
Extracting a first frame feature vector for each frame determined to be an audio frame in the frames in the first audio data;
Normalizing and averaging the extracted first frame feature vectors to extract the first speaker feature vectors representative of the first audio data;
Extracting a second frame feature vector for each frame determined to be an audio frame in the frames in the second audio data;
The speech according to claim 11, further comprising: normalizing and averaging the extracted second frame feature vector, and extracting the second speaker feature vector representing the second audio data. The operation method of the control device.
(Appendix 18)
A computer program including an instruction word that causes a processor of a voice control device to execute the operation method according to any one of appendices 11 to 17.
(Appendix 19)
A recording medium on which the computer program according to attachment 18 is recorded.

100 Voice control device (electronic equipment)
DESCRIPTION OF SYMBOLS 110 Memory 120 Processor 121 Audio processing part 122 Keyword detection part 123 Speaker feature vector extraction part 124 Wake-up judgment part 125 Speech recognition part 126 Function part

Claims (4)

An audio processing unit that receives an audio signal corresponding to ambient sound and generates audio stream data;
A keyword detection unit for determining a first section in which a candidate keyword corresponding to a predetermined keyword is detected from the audio stream data;
A first speaker feature vector for identifying a speaker in the first interval, and a second speaker feature vector for identifying a speaker in a second interval adjacent to the first interval; A speaker feature vector extraction unit that extracts data;
A wakeup determination unit that determines whether or not the predetermined keyword is included in the first section based on a similarity between the first speaker feature vector and the second speaker feature vector apparatus. A voice control method executed by a voice control device,
Receiving an audio signal corresponding to the ambient sound and generating audio stream data;
Determining a first section in which a candidate keyword corresponding to a predetermined keyword is detected from the audio stream data;
A first speaker feature vector for identifying a speaker in the first interval, and a second speaker feature vector for identifying a speaker in a second interval adjacent to the first interval; Extracting from the data;
Determining whether or not the predetermined keyword is included in the first section based on a similarity between the first speaker feature vector and the second speaker feature vector.   A computer program for causing a computer to execute the voice control method according to claim 2.   A recording medium for storing the computer program according to claim 3.