JP2020064253A - Learning device, detection device, learning method, learning program, detection method, and detection program - Google Patents

Abstract

To improve detection accuracy between speech sections.SOLUTION: A learning device according to this invention comprises: an acquirement unit for acquiring speech information including object speech to be object of detection; and a learning unit having a model learn a final end of the object speech and passage of time from a start end of the object speech. In addition, the detection device according to this invention comprises: an acquirement unit for acquiring the speech information; and a detection unit for detecting a start end of the object speech from the speech information acquired by the acquirement unit by using a model having learned the final end of the object speech and the passage of time from the start end of the object speech.SELECTED DRAWING: Figure 1

Description

  The present invention relates to a learning device, a detection device, a learning method, a learning program, a detection method, and a detection program.

  In recent years, a technique using automatic speech recognition has been known. As an example of such a technique of automatic speech recognition, a technique is known in which a user's utterance is converted into text data and various information processing is executed using the converted text data. Further, in order to improve recognition accuracy, there is known a technique of voice activity detection that detects a voice interval including a user's utterance from an input acoustic signal.

JP, 2008-139654, A

  A process of detecting a voice section including a predetermined voice using such a voice section detection technique is conceivable. For example, a technique is conceivable in which a predetermined voice is extracted from an acoustic signal by using a model such as DNN (Deep Neural Network) in which it is learned whether or not a frame to be processed is a voice section including a voice.

  However, with such a technique, there is room for improving the detection accuracy of the voice section.

  For example, when an attempt is made to extract a voice segment including a keyword composed of a plurality of words or a keyword including an unvoiced segment in the middle, in the above-described technique, a segment including only a part of the keyword is a voice segment. May be detected as.

  The present application has been made in view of the above, and an object thereof is to improve the detection accuracy of a voice section.

  A learning device according to the present application, an acquisition unit that acquires voice information that includes a target voice that is a detection target, a learning unit that causes a model to learn the end of the target voice, and the period elapsed from the start end of the target voice. It is characterized by having.

  According to the aspect of the embodiment, it is possible to improve the detection accuracy of the voice section.

FIG. 1 is a diagram illustrating an example of processing executed by the information providing apparatus and the terminal device according to the embodiment. FIG. 2 is a diagram illustrating a configuration example of the information providing apparatus according to the embodiment. FIG. 3 is a diagram showing an example of information registered in the learning data database according to the embodiment. FIG. 4 is a diagram illustrating a configuration example of the terminal device according to the embodiment. FIG. 5 is a diagram showing an example of information output by the model according to the embodiment. FIG. 6 is a flowchart showing an example of the flow of a learning process executed by the information providing device according to the embodiment. FIG. 7 is a flowchart showing an example of the flow of detection processing executed by the terminal device according to the embodiment. FIG. 8 is a diagram illustrating an example of the hardware configuration.

  Hereinafter, a learning device, a detection device, a learning method, a learning program, a detection method, and a mode for carrying out the detection program (hereinafter, referred to as “embodiment”) according to the present application will be described in detail with reference to the drawings. . Note that the learning device, the detection device, the learning method, the learning program, the detection method, and the detection program according to the present application are not limited by this embodiment. Further, the respective embodiments can be appropriately combined within the range in which the processing content is not inconsistent. Also, in each of the following embodiments, the same parts are designated by the same reference numerals, and duplicated description will be omitted.

[1. Information providing device and terminal device]
First, an example of a learning process executed by the information providing apparatus 10 which is an example of a learning apparatus and an example detection process executed by the terminal device 100 which is an example of a detecting apparatus will be described with reference to FIG. 1. FIG. 1 is a diagram illustrating an example of processing executed by the information providing apparatus and the terminal device according to the embodiment. In FIG. 1, an example of the learning process in which the information providing apparatus 10 learns a model used when extracting a predetermined target voice that is a detection target from voice information including a user's utterance is described. Further, in FIG. 1, an example of the detection process of detecting the keyword section including the predetermined keyword from the voice information including the utterance of the user by the terminal device 100 is described.

  The information providing device 10 illustrated in FIG. 1 is an information processing device that performs a learning process, and is realized by, for example, a server device or a cloud system. For example, the information providing apparatus 10 uses the learning data provided from the data server DS to perform learning of a model used when extracting a keyword section including a predetermined keyword from voice data.

  The data server DS is an information processing device that manages various data, and is realized by, for example, a server device or a cloud system. For example, the data server DS manages the learning data used by the information providing device 10 in the learning process. The details of the learning data managed by the data server DS will be described later.

  The terminal device 100 is an input / output device that includes an acquisition device such as a microphone that acquires ambient sound, and an input / output device that includes an output device such as a speaker that can output an arbitrary sound. Is the device called. For example, the terminal device 100 is a device that can output music and provide information by voice using an output device. In addition, the terminal device 100 has a reception function of receiving a sound input, and has an output function of outputting a sound corresponding to the content of the acquired sound when the sound uttered by the user is acquired.

  For example, when the user utters a voice indicating a song title of a predetermined song, the terminal device 100 identifies the song title indicated by the voice by various voice analysis techniques, and outputs the data of the song indicated by the identified song title, It is acquired from a predetermined external server OS (for example, FIG. 2) via the network N (for example, see FIG. 2). Then, the audio device reproduces the acquired music piece.

  The terminal device 100 has, for example, a function of identifying the content of the voice uttered by the user U by various voice analysis techniques and outputting a response according to the identified content. For example, when the terminal device 100 acquires the voice of the user U such as “What is the weather today?”, The terminal device 100 acquires various weather information such as weather and temperature from the external server OS, and reads the acquired weather information. Then, the weather information is provided to the user U. In addition to the above-described processing, the terminal device 100 also performs various kinds of processing such as ordering products sold in the online shopping mall, controlling various home electric appliances such as air conditioners and lighting devices, and reading out emails and schedules. It is a smart speaker that can realize.

  The terminal device 100 may perform voice analysis by cooperating with the external server OS. For example, the terminal device 100 acquires ambient sound using a microphone and the like, and when the acquired sound satisfies a predetermined condition, transmits the acquired sound to the external server OS. In such a case, the external server OS identifies the content of the acquired voice by various voice analysis techniques and transmits the identification result to the terminal device 100. After that, the terminal device 100 may execute various processes corresponding to the specific result. That is, the terminal device 100 may be a stand-alone smart speaker or a smart speaker that cooperates with an external server such as a cloud.

  Here, the terminal device 100 has a plurality of acquisition devices (for example, microphones) attached to different positions, and executes the above-described various processes using the voice received through each acquisition device. May be. Further, the terminal device 100 may be any device such as a smart device or a recording device, as long as it has a plurality of acquisition devices attached to different positions. The terminal device 100 is connected to a plurality of acquisition devices installed at physically separated positions via wireless communication such as a wireless LAN (Local Area Network) or Bluetooth (registered trademark), and each acquisition device acquires. It may be a device that collects the generated sound.

[1-1. About keyword detection]
Here, when operating a smart speaker or the like, the user utters a predetermined keyword and then utters a process indicating a process to be executed (hereinafter, referred to as a “process utterance”). In such a case, the terminal device 100 determines whether or not the acquired voice includes a predetermined keyword. Then, when it is determined that the predetermined keyword is included, the terminal device 100 utters the user by voice analysis from the section of the voice data including the processing utterance uttered by the user following the keyword. Identify the content.

  Further, such a keyword may be used not only as a start voice of a process but also in a process of clarifying a subsequent process utterance. For example, a keyword section including a keyword is extracted from voice data, a feature is extracted from a voice included in the extracted keyword section, and a user's utterance is emphasized in subsequent voices based on the extracted feature. Therefore, it is possible to reduce the influence of noise such as music and television sound. In addition, the keyword section is extracted from each of the plurality of voice data acquired using the plurality of microphones, the direction in which the user is located is estimated based on the time difference between the extracted keyword sections, and the estimated direction is determined. A beamforming technology that reduces the influence of noise by enhancing the sound from the can be considered. For this reason, when the keyword segment can be detected appropriately, it is possible not only to appropriately determine the presence or absence of the activation voice, but also to improve the recognition accuracy of the processed utterance.

  Here, a model that operates as various classifiers such as SVM (Support Vector Machine) and DNN (Deep Neural Network) is made to learn the characteristics of the keyword, and the keyword is detected from the collected voice using the learned model. Such a mode is conceivable. However, if the model is simply trained to have the characteristics of the voice of the keyword, it is not clear from which point in the keyword the model is determined to be the keyword. It is difficult to estimate whether it is an interval.

[1-2. About learning processing]
Therefore, the information providing device 10 executes the following learning process. First, the information providing apparatus 10 acquires voice information including a target voice to be detected. For example, the information providing apparatus 10 acquires voice information including a voice such as a keyword for causing a predetermined terminal device 100 to perform a predetermined operation as a target voice. Then, the information providing apparatus 10 causes the model to learn at least the end of the target voice and the period elapsed from the start of the target voice. To give a more specific example, the information providing apparatus 10 causes the model to learn the features from the beginning to the end of the target voice, that is, the features of the target voice, and from the beginning of the voice to each section of the target voice. Train the model between and. For example, the information providing apparatus 10 divides the audio information into a plurality of sections, and determines, for each section, whether or not the end of the target sound is included and a period from the start end of the target sound to the section to be processed. Train the model.

  In other words, the information providing apparatus 10 learns a model that outputs the fact that a keyword is detected near the end of the keyword. For example, the information providing apparatus 10 divides the voice of the voice data into a plurality of frames, and inputs the voice information included in each frame to the model in chronological order. Then, the information providing apparatus 10 outputs the information indicating that the end of the keyword is detected when the information of the voice included in the frame including the end of the keyword or the frame near the end is input to the model. Learn. Here, in order to calculate the posterior probability that the newly input voice is the end of the keyword in consideration of the feature amount of the voice input in the past, the information providing apparatus 10 uses an RNN (Recurrent Neural Network) or LSTM. (Long short-term memory) learning model with recurrent neural network configuration.

  In addition to such learning, the information providing apparatus 10 adds a task of classifying the input frame into a class according to the length from the start end of the keyword. That is, the information providing apparatus 10 models how long each frame is from the beginning of the keyword, that is, how much the voice included in each frame is observed when the beginning of the keyword is reached. Let them learn.

  For example, the information providing apparatus 10 allocates a different class for every 10 frames from the start end of the keyword when the keyword is uttered in about 100 frames when one frame is 20 milliseconds. Then, the information providing apparatus 10 learns whether or not each frame includes the end of the keyword, and makes the model learn to which class the input frame is assigned. That is, the information providing apparatus 10 causes the model to perform multitask learning.

  By the learning process described above, the information providing apparatus 10 can realize the learning of the model that can appropriately detect the end of the keyword and can estimate the period from the detected end of the keyword to the detected end. For example, the information providing apparatus 10 realizes a model for detecting the end of the keyword based on the characteristics of the entire keyword (for example, the start end to the end of the keyword) by the above-described learning process. That is, the information providing apparatus 10 learns the model so as to detect the section including the end of the keyword based on the appearance order of the features of each section of the keyword. As a result, the information providing apparatus 10 can improve the detection accuracy of the keyword end.

  For example, the information providing apparatus 10 does not use only the voice included in the frame near the end of the keyword as the learning data, but inputs the voice of each frame of the entire keyword into the model in chronological order, so that each frame of the keyword is input. The model is made to learn the features of the voice in and the feature of the appearance order of each voice. When such learning is performed, the model detects a keyword when a feature in each frame from the beginning to the end and the time-series appearance sequence of the feature are similar to the keyword are input. Will be determined. As a result, the information providing apparatus 10 can appropriately detect a keyword including a plurality of words or silent sections.

  For example, if the keyword is “Hey_Yahoo” including the word “Hey” and the word “Yahoo”, the information providing apparatus 10 causes the information providing device 10 to have a series of audio features including a plurality of words “Hey_Yahoo”. The model is trained as a keyword voice. More specifically, the information providing apparatus 10 inputs each frame of the voice "Hey_Yahoo" into the model in the order of appearance, and detects the keyword when the last frame, that is, the end frame is input. The model is trained to output a message. For example, the information providing apparatus 10 inputs each frame of the sound "Hey_Yahoo" into the model in the order of appearance, outputs "0" each time the sound of each frame other than the end frame is input, and outputs the end The model is trained so that "1" is output when a frame is input.

  When such learning is performed, the model does not output the fact that the keyword is detected (that is, “1”) only by inputting the word “Yahoo”, and the voice of “Hey_Yahoo” is output. When each frame is input in the order of appearance, the fact that the keyword has been detected is output. In addition, in such a model, when a voice similar to a part of a keyword such as “Oi_Yahoo” or “Hey_Yasuko” is input, or a voice such as “Yahoo_Hey” and the sound appearance order are different. Does not output that the keyword has been detected, and detects the end of the keyword only when the frames of the voice similar to the entire keyword are input in the same order as the keyword.

  On the other hand, if the model is made to learn only the features of the speech contained in the end frame of the keyword, the model may erroneously determine that the keyword is detected simply by inputting the speech such as "Yahoo" or "Fu". There is. Therefore, the information providing apparatus 10 learns a model that can appropriately detect the end of the keyword including a plurality of words or silent sections by learning the model so as to detect the end of the keyword from the characteristics of the entire keyword. can do.

  The information providing apparatus 10 also causes the model that detects the end of the keyword to learn the characteristics of the period from the start of the keyword to the detected end. When each frame of the actually measured speech data is input in chronological order to the model on which such learning is performed (hereinafter, referred to as “learning model”), the learning model is the input frame. Whether or not the end of the keyword is included in (or whether or not it is near the end of the keyword), and the information indicating the class of the input frame, that is, the frame input from the start of the keyword Outputs period information that indicates how much time has passed.

  Here, it is estimated that the start point of the keyword is included in the frame that is traced back from the frame in which the learning model is determined to be the end for the period corresponding to the class to which the frame belongs, or in the vicinity of the frame. As a result, the information providing apparatus 10 can realize learning of a learning model that can accurately extract a keyword section.

  The learning model learned by the above-described learning process estimates the end of the keyword based on the characteristics of each frame of the keyword input in chronological order. Therefore, the learning model can appropriately estimate the end of the keyword even when the keyword includes a plurality of words or a silent section.

  In the above description, the information providing apparatus 10 causes the model to learn the characteristics of the entire keyword and the period elapsed from the start of the keyword, but the embodiment is not limited to this. For example, the information providing apparatus 10 may at least let the model learn the characteristics near the end of the keyword and the period elapsed from the start of the keyword. When such learning is performed, the model outputs the fact that the end of the keyword has been detected when a voice similar to the end of the keyword is input, and also the period from the start of the keyword to the end of detection. The information shown will be output. Also from such an output, it is possible to detect a section including a keyword or a voice part of which is similar to the keyword by tracing back from the frame detected as the end for the period detected by the model. Such a section may be detected and whether or not the keyword is actually included may be determined using another model or the like.

[1-3. About detection processing]
On the other hand, the terminal device 100 uses the learning model learned by the information providing device 10 to detect the keyword section from the utterance of the user. For example, the terminal device 100 uses a microphone or the like to acquire voice information including the utterance of the user. Then, the terminal device 100 uses the model in which the end of the target voice to be detected and the period elapsed from the start of the target voice are learned, that is, the learning model learned by the information providing device 10, The starting end of the target voice is detected from the acquired voice information.

  For example, the terminal device 100 divides the voice information acquired using a microphone or the like into a plurality of frames, and inputs each frame into the learning model in chronological order. When a frame is input to the learning model that has been learned by the above-described learning processing, the learning model uses information indicating whether or not the input frame includes a terminal (for example, whether or not the terminal includes a terminal). Information indicating whether or not the voice included in the input frame is the voice from the start end of the keyword That is, the information indicating the class according to the elapsed time from the start end is output. For example, the learning model outputs the posterior probability that the input frame belongs to each class (that is, the probability of belonging to each class). That is, the learning model classifies whether or not the frame includes an end (hereinafter, may be referred to as “end class classification”), and classifies according to the elapsed time from the start ( Hereinafter, it may be referred to as "transition class classification") at the same time.

  Using such a learning model, the terminal device 100 identifies whether or not the input frame includes the end, and how long the voice included in the input frame is the voice after the start end. Specify. For example, when the terminal device 100 specifies that a certain frame includes the end of the keyword based on the result of the end class classification by the learning model, the terminal device 100 specifies the result of the transitive class classification of the frame. Then, the terminal device 100 estimates that the frame starting from the period corresponding to the specified class includes the start point of the keyword, and includes the end point of the keyword from the frame including the start point of the keyword. Extract up to the frame as a keyword section. As a result of such processing, the terminal device 100 can accurately detect the keyword section.

[1-4. Example of processing]
Subsequently, an example of a learning process executed by the information providing device 10 and an example of a detection process executed by the terminal device 100 will be described with reference to FIG. For example, the information providing apparatus 10 acquires learning data used for model learning from the data server DS (step S1). Then, the information providing apparatus 10 causes the model to learn the end time of the keyword and the elapsed time from the start end to each section (step S2).

  For example, the information providing apparatus 10 includes, as learning data, voice data including a speech voice of a keyword, an end label indicating whether or not each section of the voice data includes the end of the keyword, and a class to which each section belongs. Learning data LD1 including a class label is acquired. In addition, when the keyword includes a plurality of words or when a silent section is included, the speech data of a plurality of words or the speech data including the speech including the silent section as the target speech is acquired as the learning data. Will be done.

  For example, in the example shown in FIG. 1, the learning data LD1 has voice data including a keyword having a start end S1 and an end end E1. Further, in the learning data LD1, the voice data is divided into sections “1” to “23”. Further, the learning data LD1 is provided with an end label indicating whether or not the end E1 of the keyword is included in each section. For example, the value "0" indicating that the end E1 is not included in each section "1" to "23" of the learning data LD1 or the value "1" indicating that the end E1 is included is Has been granted.

  Further, the learning data LD1 is provided with a class label indicating a class according to the period elapsed from the starting end S1 for each section. For example, in the example shown in FIG. 1, the start end S1 is included in the section “3”. In such a case, the class label “0” is given to the sections “1” and “2” of the learning data LD1, and the class labels “1” to ““ in order to the sections “3” to “21”. 19 ”is given.

  Here, in the learning data LD1, a class label “0” is given to a section after the section including the end. For example, since the section “21” of the learning data includes the end of the keyword, the end label “1” is added to the sections “22” and “23” after the section “21”. Is assigned the class label “0”.

  When the keywords are spoken in about 20 sections on average, the maximum class level value may be set to 20. Further, in the example shown in FIG. 1, the class label “0” is given to the section after the section including the end of the keyword, but the embodiment is not limited to this. For example, a continuous series of class labels may be given to a section after the section including the end, and a section exceeding the maximum class label value has the same class label as the previous section. May be given. For example, the class labels “20” and “21” may be given to the sections “22” and “23”, or the same class label “20” may be given.

  The learning data LD1 shown in FIG. 1 is divided into 23 sections, but the embodiment is not limited to this. The learning data LD1 shown in FIG. 1 is schematically shown to facilitate understanding of the invention, and is actually divided into more sections. As a specific example, when the frame for processing the audio data is 20 milliseconds and the audio data included in the learning data LD1 is 3 seconds, the audio data is divided into 150 frames. Will be done.

  Each section of the learning data LD1 shown in FIG. 1 may correspond to one frame or may correspond to a plurality of frames. Further, the end label and the class label may be given to each section in arbitrary units. For example, the end label may be given to each frame and the class label may be given to each of a plurality of frames. Further, as the class label, a different value may be given to each frame corresponding to the keyword section.

  First, the information providing apparatus 10 prepares a model M having an LSTM structure. Then, when the information providing apparatus 10 inputs each frame of the audio data included in the learning data LD1 to the model in chronological order, the information providing apparatus 10 outputs the terminal label and the class label given to the input frame, The model M is learned. For such learning, for example, an arbitrary learning method that realizes learning of LSTM such as back propagation or stochastic gradient descent can be adopted.

  For example, the information providing apparatus 10 learns the model M so that the model M outputs the end label “0” and the class label “1” when the frame included in the section “3” is input to the model M. I do. Similarly, the information providing apparatus 10 inputs the frames into the model M in time series order, and learns the model M so as to output the end label and the class label corresponding to each frame. Note that the information providing apparatus 10 may input some learning data in a random order, such as a frame with a terminal label of “0”, for performing appropriate learning.

  As described above, the information providing apparatus 10 receives, when the voice included in the predetermined section is input to the model, the end information indicating whether the end of the target voice is included in the predetermined section, and the target voice. The model M is learned so as to output period information indicating a period from the start end to the predetermined section. Further, the information providing apparatus 10 divides the voice information into a plurality of sections, and when a voice included in the predetermined section is input, outputs the classification result according to the period from the start end of the target voice to the predetermined section. So, learn the model.

  The information providing device 10 learns the model M using not only the learning data LD1 but also a plurality of learning data. Here, the information providing apparatus 10 may use learning data including keywords spoken by various users in order to improve the processing accuracy of the model M. Further, the information providing apparatus 10 may perform learning of the model M by using learning data including voice generated from a television, speech of another user, and voice data added with various noises such as white noise. .

  Then, the information providing device 10 provides the learned model M, which has been learned, to the terminal device 100 (step S3). In such a case, the terminal device 100 receives the utterance of the user (step S4). For example, the terminal device 100 acquires a keyword uttered by the user in order and a voice of a process utterance using a microphone. Then, the terminal device 100 estimates the end of the keyword from the acquired voice using the learning model M, and estimates the start of the keyword section based on the elapsed time to the end of the keyword estimated by the learning model M. (Step S5).

  For example, the terminal device 100 divides the voice (hereinafter, referred to as “speech voice”) acquired from the user into a plurality of sections, and inputs the voices of each section to the learning model M in chronological order. Then, the terminal device 100 acquires, for each section, the terminal label and the class label output by the learning model M. Then, when the learning model M1 outputs the end label "1" when the voice of the section "19" is input, the terminal device 100 estimates that the end of the keyword section is the section "19". Further, when the learning model M1 outputs the class label “15” when the voice of the section “19” is input, the terminal device 100 outputs a section 15 classes before the section “19”, that is, It is estimated that the start point of the keyword is included in the section “4”. Then, the terminal device 100 estimates that the period from section “4” to section “19” is the keyword section.

  Then, the terminal device 100 performs a predetermined process using the voice included in the estimated keyword section (step S6). For example, the terminal device 100 may analyze the voice included in the keyword section to determine whether or not the keyword is uttered, perform beamforming, etc., and emphasize subsequent processing utterances. Good. In addition, the terminal device 100 may simply analyze the process utterance that follows the keyword section and execute the process corresponding to the analysis result. Then, the terminal device 100 provides the processing result to the user (step S7).

  As described above, the terminal device 100 determines whether or not the end of the target voice is included in each section included in the learning information that is the voice information that is the learning target, and from the start end of the target voice to the target section. A period including the starting end of the target voice is detected from the uttered voice using a model in which the period and are learned. For example, the terminal device 100 sequentially inputs the voices included in each section of the uttered voice to the learning model M having the configuration of the recursive neural network from the beginning, and outputs the end information and the period information output by the learning model M. Based on this, a section including the start end of the target voice is detected.

  That is, the terminal device 100, when the voice included in the predetermined section is input, the terminal information indicating whether or not the end of the target voice is included in the predetermined section, and the predetermined information from the start end of the target voice. A section including the start end of the target speech is detected from the uttered speech by using the learning model M in which the learning is performed so as to output the period information indicating the period up to the section. Further, the terminal device 100 divides the uttered voice into a plurality of sections, and the learning model M outputs end information indicating that the end of the target voice is included in the divided sections when the voice is input. The section is specified, and the section including the start end of the target voice is detected based on the period information output from the learning model M for the specified section.

  As a result of such processing, the terminal device 100 can appropriately estimate the keyword section in addition to the keyword detection by using one learning model M. In addition, the terminal device 100 estimates the end of the keyword by using the learning model M configured by the LSTM, and estimates the start of the keyword by tracing back from the estimated end of the keyword. Here, in a recursive neural network such as LSTM, it is possible to determine whether or not the newly input data satisfies a predetermined condition, by taking into consideration the characteristics of the data input so far. Therefore, the terminal device 100 can detect the keyword section after waiting for the utterance of the entire keyword, and thus can detect the keyword section with high accuracy.

  In addition, when performing multitask learning with the task of detecting the end of the keyword and the task of estimating the elapsed time from the start of the keyword to each section, in order to realize each task among the features of the voice. The model learns the features from multiple angles. As a result of such processing, in the learning model M, the keyword end can be detected more accurately than in the model in which only the task of detecting the keyword end is learned.

[1-5. About the model]
In the above description, the information providing apparatus 10 uses the model having the LSTM structure as the learning model M. However, the embodiment is not limited to this. For example, the information providing apparatus 10 may be various neural networks derived from LSTM such as BidirectionalLSTM, or may be various RNNs. Further, if the information providing apparatus 10 simultaneously learns whether or not the end of the keyword is included in the input voice section and the period from the start end of the keyword to the input voice section, the SVM ( A model having an arbitrary configuration such as Support Vector Machine), DNN (Deep Neural Network), CNN (Convolutional Neural Network), or the like may be adopted.

  Moreover, the information providing apparatus 10 may perform learning using a plurality of models. For example, the information providing apparatus 10 learns the first model so as to detect the end of the keyword, and causes the second model to learn the elapsed period from the start end of the keyword to each section. Then, the terminal device 100 inputs each section of the uttered speech individually to the first model and the second model, and from the section determined to be the end of the first model, the second model is The section traced back by the output elapsed period may be set as the section including the start point of the keyword.

[1-6. About section]
In the example described above, the information providing apparatus 10 divides the learning data into a plurality of sections, and causes the model to learn the value of the end label and the value of the class label for each section. However, the embodiment is not limited to this. For example, the information providing apparatus 10 may divide the learning data into frames of a predetermined length, learn the value of the end label for each frame, and learn the value of the class label for each section including a plurality of frames. That is, the information providing apparatus 10 may perform learning on a frame-by-frame basis regarding the end of the keyword and may perform learning on a plurality of frames regarding the elapsed period. In addition, as to how many classes the input frame is classified into, any mode can be adopted.

[1-7. About learning processing]
In the above-mentioned example, the model is trained so as to output a binary value such as "1" or "0" at the end of the keyword, and an integer value such as "1" to "20" for the elapsed period (that is, class). An example in which the model is trained to output is described.

  Here, in reality, the information providing apparatus 10 learns the model so as to output the probability that the keyword end is included in the input frame. In such a case, when a certain frame is input to the learning model M and the accuracy output from the learning model M exceeds a predetermined threshold, the terminal device 100 determines that the frame includes the end of the keyword. It may be estimated.

  The information providing apparatus 10 also learns the model so that the input frame outputs the degrees of certainty that each frame belongs to each class. In such a case, the terminal device 100 may determine, when a certain frame is input to the learning model M, a class whose accuracy output by the learning model M exceeds a predetermined threshold as a class to which the input frame belongs. . In other words, the terminal device 100 identifies the elapsed period in which the accuracy output by the learning model M1 exceeds a predetermined threshold value from among the elapsed periods, and the input frame is after the identified elapsed period from the beginning of the keyword. It may be estimated to be a frame.

  When the learning model is learned so that the input frame outputs the accuracy for each class of the terminal class classification and the elapsed class classification, there may occur a plurality of classes exceeding the predetermined threshold. Therefore, the information providing device 10 may specify the maximum value of the accuracy of each class and adopt the class with the highest accuracy. That is, the information providing apparatus 10 may determine the class by taking argmax for the accuracy of each class. The information providing apparatus 10 may also learn a model including an output layer that performs such argmax processing. Further, the information providing apparatus 10 may perform learning model learning so as to classify the frame into a class having the highest accuracy among the classes whose accuracy exceeds a predetermined threshold.

  The information providing apparatus 10 may solve the elapsed period by a regression problem. For example, the information providing apparatus 10 may perform the model learning so as to output the numerical value itself indicating the period estimated to have passed from the start end, instead of classifying the elapsed time. For example, the information providing apparatus 10 may cause the model to learn the characteristics of the learning data including the elapsed time from the beginning of the keyword to each frame, instead of the class label.

[1-7. About application target]
In the example described above, the information providing apparatus 10 has learned the model for detecting the keyword that is the activation voice. However, the embodiment is not limited to this. The information providing apparatus 10 may perform learning of a model for detecting any voice as long as the voice is a detection target. That is, the information providing apparatus 10 detects a section including a sound to be detected, such as a predetermined mechanical sound, environmental sound, or noise, from various audio data, and therefore, the end of the sound to be detected and the detection are performed. Any sound may be used as the detection target as long as the model learns the period elapsed from the start of the target sound.

[1-8. Execution subject]
In the example described above, the learning process is performed by the information providing device 10, and the detection process is performed by the terminal device 100. However, the embodiment is not limited to this. For example, the learning process and the detection process may be executed by the information providing device 10. In such a case, the information providing apparatus 10 receives the uttered voice acquired by the terminal device 100, and uses the learning model M to detect the keyword section from the received uttered voice. The learning process and the detection process described above may be realized by the terminal device 100.

[2. Example of functional configuration]
Hereinafter, an example of a functional configuration of the information providing apparatus 10 that implements the learning process described above and an example of a functional configuration of the terminal device 100 that implements the detection process described above will be described.

[2-1. Regarding an example of the functional configuration of the information providing device]
First, an example of the functional configuration of the information providing apparatus 10 will be described with reference to FIG. FIG. 2 is a diagram illustrating a configuration example of the information providing apparatus according to the embodiment. As shown in FIG. 2, the information providing device 10 includes a communication unit 20, a storage unit 30, and a control unit 40.

  The communication unit 20 is realized by, for example, a NIC (Network Interface Card) or the like. Then, the communication unit 20 is connected to the network N by wire or wirelessly, and transmits / receives information to / from the terminal device 100, the data server DS, and the external server OS, for example.

  The storage unit 30 is realized by, for example, a semiconductor memory device such as a RAM (Random Access Memory) or a flash memory (Flash Memory), or a storage device such as a hard disk or an optical disk. The storage unit 30 also stores a learning data database 31 and a model database 32.

  Learning data is registered in the learning data database 31. For example, FIG. 3 is a diagram showing an example of information registered in the learning data database according to the embodiment. As shown in FIG. 3, information having items such as “learning data ID (Identifier)”, “section”, “voice data”, “termination tag”, and “class label” is registered in the learning data database 31. It In the example shown in FIG. 3, the audio data, the end tag, and the class label are stored for each “section”, but in reality, the audio data, the end tag, and the class label are stored for each frame. It may be stored.

  Here, "learning data ID" is an identifier of learning data. Further, the “section” is information for identifying each section obtained by dividing the audio data that is the learning data, and is, for example, a series of numbers given to the section. Further, the “voice data” is voice data included in the section indicated by the associated “section”, that is, an acoustic signal. The “end tag” is information indicating whether the associated “section” includes the end of the keyword. Further, the “class label” is section information indicating how long the voice included in the associated “section” is the voice after the beginning of the keyword, and the associated “section” belongs to the “class label”. This is information indicating a class.

  For example, in the example shown in FIG. 3, the learning data ID “LD1”, the section “1”, the voice data “SD1”, the end tag “0”, and the class label “0” are associated and registered in the learning data database 31. Has been done. For such information, voice data “SD1” is registered as voice data included in the section “1” of the learning data indicated by the learning data ID “LD1”, and the value of the end tag in the section “1”. Is "0", indicating that the value of the class label is "0".

  Note that in the example shown in FIG. 3, a conceptual value such as “SD1” is described, but in reality, the learning data database 31 includes information indicating the sound volume and frequency distribution of the sound of each frame as sound data. It will be registered. Further, in the learning data database 31, instead of the “section”, a frame number or the like may be registered. In addition to the information shown in FIG. 3, arbitrary information may be registered in the learning data database 31.

  Returning to FIG. 2, the description will be continued. Learning models are registered in the model database 32. That is, in the model database 32, the data of the learning model M in which the end of the target voice to be detected and the period elapsed from the start of the target voice are learned are registered. For example, in the model database 32, as data of the learning model M1, information on nodes forming a multi-tiered layer each including one or a plurality of nodes, information indicating a connection relationship between each node, A connection coefficient, which is a weight for transmitting information, is registered.

  Here, the learning model M1 has an input layer to which an acoustic signal that is learning data is input. Further, the learning model M1 is termination information indicating whether or not the input sound signal includes the end of the target sound, and the sound signal when the input sound signal has elapsed from the start end of the target sound. The output layer outputs period information indicating whether or not, that is, information indicating a class to which the input acoustic signal is classified.

  Further, the learning model M1 has a value calculated based on the first element belonging to any layer from the input layer to the output layer and other than the output layer, and the first element and the weight of the first element. And a second element which is included in the input layer, and performs an operation based on the first element and the weight of the first element with respect to the information input to the input layer, with each element belonging to each layer other than the output layer as the first element. This causes the computer to function so that the information corresponding to the information input to the input layer is output from the output layer.

  Such a learning model M1 causes a computer to output termination information and period information from the output layer when voice data is input to the input layer during learning and measurement, for example. Then, at the time of learning, the information providing device 10 sets the learning model M1 so that the termination information and the period information output by the learning model M1 indicate the termination information and the period information corresponding to the input voice data. Correct the connection coefficient.

  Here, when the learning model M1 is realized by the SVM or the regression model, the learning model M1 can be regarded as a simple perceptron having an input layer and an output layer. When the learning model M1 is regarded as a simple perceptron, the first element can be regarded as a node included in the input layer, and the second element can be regarded as a node included in the output layer. When the learning model M1 is realized by a neural network having one or a plurality of intermediate layers such as DNN, the first element included in each model is regarded as any node included in the input layer or the intermediate layer. The second element corresponds to the node whose value is transmitted from the node corresponding to the first element, that is, the node in the next stage, and the weight of the first element corresponds to the first element. It is a weight, that is, a connection coefficient, which is considered for the value transmitted from the node to the node corresponding to the second element.

  Here, the information providing apparatus 10 uses the learning data registered in the learning data database 31 to generate the learning model M1 for executing the above-described detection process. That is, the learning data registered in the learning data database 31 is the input layer to which the acoustic signal is input, the output layer, and any layer from the input layer to the output layer that belongs to a layer other than the output layer. Each element that includes one element and a second element whose value is calculated based on the first element and the weight of the first element and that belongs to each layer other than the output layer with respect to the acoustic signal input to the input layer As a first element, the first element and the weighting of the first element, and the calculation based on the weight reflecting the characteristics of the target voice and the characteristics of the period from the start end of the target voice to each section of the target voice Is data for operating the computer so that the terminal information and the period information are output from the output layer by performing.

  The control unit 40 is a controller, for example, a processor such as a CPU (Central Processing Unit) or an MPU (Micro Processing Unit) allows various programs stored in a storage device inside the information providing apparatus 10 to be a RAM or the like. Is implemented as a work area. The control unit 40 is a controller, and may be realized by an integrated circuit such as an ASIC (Application Specific Integrated Circuit) or an FPGA (Field Programmable Gate Array).

  As shown in FIG. 2, the control unit 40 includes a data acquisition unit 41, a learning unit 42, and a providing unit 43. The data acquisition unit 41 acquires voice information including a target voice to be detected. For example, the data acquisition unit 41 uses, as learning data from the data server DS, voice data divided into a plurality of sections, and a termination tag indicating whether or not the speech included in each section includes the termination of the target speech. , Period information indicating a period from the start end of the target voice to the voice included in each section, that is, information associated with the class data is acquired. Then, the data acquisition unit 41 registers the acquired learning data in the learning data database 31.

  The data acquisition unit 41 may acquire voice for causing the terminal device 100 to perform a predetermined operation, that is, voice information including a keyword that is a start voice as a target voice. In addition, the data acquisition unit 41 may acquire voice information that includes, as a target voice, a voice that utters a plurality of words or a voice that includes a silent section. As described above, it is possible to arbitrarily set what kind of sound is the target sound, but the data acquisition unit 41 determines that the set target sound is appropriately detected, in order to properly detect the set target sound. It is desirable to acquire voices having similar characteristics as learning data.

  The learning unit 42 causes the model to learn the end of the target voice and the period elapsed from the start of the target voice. For example, the learning unit 42 learns a model for detecting the end of a keyword based on the characteristics of the entire target voice. As a more specific example, the learning unit 42 performs model learning so as to detect a section including the end of the keyword based on the appearance order of the features of each section of the keyword.

  For example, the learning unit 42 divides the voice information into a plurality of sections, and causes the model to learn whether or not the end of the target voice is included in each section and the period from the start end of the target voice to the section. . To give a more specific example, the learning unit 42, when the voice included in the predetermined section is input to the model, end information indicating whether or not the end of the target voice is included in the predetermined section, The model is learned so as to output period information indicating a period from the start end of the target voice to a predetermined section. That is, the learning unit 42 divides the voice information into a plurality of sections, and when a voice included in the predetermined section is input, outputs the classification result according to the period from the start end of the target voice to the predetermined section. , Model learning should be performed.

  In addition, when a normal DNN or the like is used, a phenomenon in which the end of the target voice is detected based on the features only around the end of the features of the target voice can be considered. When such detection is performed, there is a possibility that a voice having characteristics similar to the end of the target voice may be detected as the end of the target voice. Therefore, in order to detect the end of the target voice based on the characteristics of the entire target voice, the learning unit 42 applies the end of the target voice and the period elapsed from the start of the target voice to the model having the recursive neural network configuration. You can learn and.

  For example, the learning unit 42 generates a model having an LSTM structure and reads one piece of learning data to be processed from the learning data database 31. Subsequently, the learning unit 42 executes the following processing in time-series order (that is, in ascending order of section number) for each section of the read learning data. First, the learning unit 42 inputs the voice data of the processing target section into the model. For example, the learning unit 42 may input the frequency and amplitude of the voice indicated by the voice data, or may input the features of the voice into the model. Then, the learning unit 42 corrects the connection coefficient of the model so that the output of the model to which the voice is input indicates the end tag and the class label of the section to be processed.

  Hereinafter, an example in which the end tag “1” and the class label “18” are registered in association with the voice data “SD10” will be described. For example, the learning unit 42 inputs the voice data “SD10” into the model. In such a case, the learning unit 42, from among the nodes of the output layer included in the model, outputs a value indicating the accuracy of a predetermined threshold value or more from the node for outputting the termination information (that is, corresponds to the termination tag “1”). Connected to the model so that the value corresponding to the class label “18” among the nodes in the output layer of the model is output as a value indicating the accuracy of a predetermined threshold value or more. Correct the coefficient. The learning unit 42 also performs the same processing on other learning data. Then, the learning unit 42 registers the model in the model database 32 as the learning model M.

  The providing unit 43 provides the learning model to the terminal device 100. For example, the providing unit 43 reads the learning model M from the model database 32 based on a request from the terminal device 100, and transmits the read learning model M to the terminal device 100.

[2-2. Example of functional configuration of terminal device]
Next, an example of the functional configuration of the terminal device 100 will be described with reference to FIG. FIG. 4 is a diagram illustrating a configuration example of the terminal device according to the embodiment. As shown in FIG. 4, the terminal device 100 includes a communication unit 120, a storage unit 130, a control unit 140, a microphone MC, and a speaker SP.

  The communication unit 120 is realized by, for example, a NIC or the like. Then, the communication unit 120 is connected to the network N by wire or wirelessly, and transmits / receives information to / from the information providing device 10, the data server DS, and the external server OS, for example.

  The storage unit 130 is realized by, for example, a semiconductor memory device such as a RAM or a flash memory, or a storage device such as a hard disk or an optical disk. The storage unit 130 also stores the learning model M distributed from the information providing apparatus 10.

  The microphone MC is a microphone that receives a voice uttered by the user, that is, a voice that is uttered from the surroundings of the terminal device 100, that is, an input device. The speaker SP is a speaker for outputting various sounds, that is, an output device. The terminal device 100 may include a plurality of microphones MC and a plurality of speakers SP.

  The control unit 140 is a controller, and is realized by a processor such as a CPU or MPU executing various programs stored in a storage device inside the terminal device 100 using a RAM or the like as a work area. The control unit 140 is a controller and may be realized by an integrated circuit such as ASIC or FPGA.

  The control unit 140 also includes a voice acquisition unit 141, a detection unit 142, and a processing unit 143. The voice acquisition unit 141 acquires voice information. For example, the voice acquisition unit 141 acquires the voice uttered by the user as voice information via the microphone MC.

  The detection unit 142 uses the model in which the end of the target voice to be detected and the period elapsed from the start end of the target voice are learned to determine the start end of the target voice from the voice information acquired by the voice acquisition unit 141. To detect. For example, the detection unit 142 reads the learning model M registered in the storage unit 130. Then, the detection unit 42 sequentially inputs the audio information acquired by the audio acquisition unit 141 to the learning model M for each segment along the acquired time series. Then, the detection unit 42 detects the end and the start end of the target voice based on the output of the learning model M, and specifies the range from the detected start end to the end as the keyword section.

  For example, the detection unit 142 divides the voice information acquired by the voice acquisition unit 141 into a plurality of sections, and when the voice included in the section is input among the divided sections, the end of the target voice is included. The section in which the learning model M outputs the end information indicating that the learning model M is output is specified. Subsequently, the detection unit 142 detects a section including the start end of the target voice, based on the period information output by the learning model M1 for the specified section. Then, the detection unit 142 notifies the processing unit 143 of the detected start end to end as a keyword section.

  That is, the detection unit 142 learns whether or not the end of the target voice is included for each section included in the learning information that is the voice information that is the learning target, and the period from the start end of the target voice to the section. Using the model thus generated, a section including the start end of the target voice is detected. In addition, the detection unit 142, when the voice included in the predetermined section is input, end information indicating whether or not the end of the target voice is included in the predetermined section, and a predetermined section from the start end of the target voice. The period including the start end of the target voice is detected by using the model learned to output the period information indicating the period up to. Further, the detection unit 142 sequentially inputs the voices included in each section of the voice information to the model having the configuration of the recursive neural network from the beginning, and based on the end information and the period information output from the model, the target Detects a section including the beginning of voice.

  For example, FIG. 5 is a diagram showing an example of information output by the model according to the embodiment. In the example shown in FIG. 5, the voice including the utterance of the keyword is divided into 72 sections, and an example of information output by the learning model M when the voices in each section are input in chronological order is shown. Further, in the example shown in FIG. 5, the learning model M has 17 nodes to which numbers “0” to “16” are given in the output layer, and a value of 0 or more and 1 or less for each node. That is, it is assumed to be configured to output the accuracy. Further, in the example shown in FIG. 5, a section in which the value output by each node is less than the predetermined first threshold value is shown in white, and a section in which the value output from each node is not less than the first threshold value and less than the second threshold value is shown by upward hatching, The section that is equal to or greater than the second threshold value is indicated by hatching with a downward slope. In the example illustrated in FIG. 5, when the accuracy is equal to or higher than the second threshold value, it is determined that the node outputs “1”.

  For example, in the example shown in FIG. 5, the node 0 outputs a value close to “1” when the input voice is not the end of the keyword, and “0” when the input voice is the end of the keyword. A node that has been trained to output a value close to. Also, the node 1 outputs a value close to “0” when the input voice is not the end of the keyword, and outputs a value close to “1” when the input voice is the end of the keyword. This is the node on which learning was performed. Also, the node 2 learns to output a value close to “0” when the input speech is in the keyword section and to output a value close to “1” when the input speech is not in the keyword section. The node that was made.

  Further, in the example shown in FIG. 5, the nodes 4 to 16 are nodes corresponding to different classes, and have been learned so as to output the period information. For example, the node 4 outputs “1” if the input voice is a voice within 3 sections from the beginning of the keyword, and outputs “0” in other cases. Is. In addition, the node 5 outputs "1" when the input voice is a voice which is included in the range within 6 sections after three or more sections have passed from the beginning of the keyword, and "1" in other cases. This is a node that has been learned so as to output "0". In addition, the node 6 outputs "1" when the input voice is a voice which is included in the range within 9 sections after 6 sections or more have passed from the beginning of the keyword, and "1" otherwise. This is a node that has been learned so as to output "0". Similarly, the other nodes are also nodes that have been learned so as to output "1" when the voices are voices in different sections from the start end of the keyword. The learning model M may have nodes corresponding to more classes than the nodes shown in FIG.

  When the acquired voices are input to the learning model M in chronological order, the nodes 0 to 16 output the values shown in FIG. For example, in the example shown in FIG. 5, the node 1 outputs a value close to “1” as a result of detecting the end of the keyword section in the section “49”. Therefore, the detection unit 142 estimates that the end of the keyword is detected in the section “49” in which the node 1 outputs the value exceeding the second threshold.

  Subsequently, the detection unit 142 refers to the outputs of the nodes 4 to 16 and identifies the class to which the voice included in the section “49” belongs. In the example illustrated in FIG. 5, the node 13 starts outputting a value in the section “49”. Therefore, the detection unit 142 classifies the voice included in the section “49” into a class corresponding to the node 13. Here, there are 10 classes from the class corresponding to the node 4 to the class corresponding to the node 13, and three classes are associated with each class. Therefore, the detection unit 142 estimates that 30 sections exist from the start point of the keyword to the section “49”, and the start point of the keyword is added to the section “19” obtained by subtracting 30 from the section “49”. Is assumed to be included. As a result, the detection unit 142 can detect that the section “19” to the section “49” are keyword sections.

  Returning to FIG. 4, the description will be continued. The processing unit 143 executes various processes according to the sound included in the section detected by the detection unit 142. For example, the processing unit 143 performs voice analysis in the detected section and executes various types of processing according to the analysis result. Then, the processing unit 143 outputs the sound indicating the execution result of the processing from the speaker SP.

[3. Regarding the flow of processing executed by the information providing device and the terminal device]
Next, an example of the flow of processing executed by the information providing device 10 and the terminal device 100 will be described with reference to FIGS. 6 and 7. FIG. 6 is a flowchart showing an example of the flow of a learning process executed by the information providing device according to the embodiment. Further, FIG. 7 is a flowchart showing an example of the flow of detection processing executed by the terminal device according to the embodiment.

  First, an example of the flow of learning processing will be described with reference to FIG. First, the information providing apparatus 10 acquires voice information including the target voice as learning data (step S101), determines whether or not the end of the target voice is included in each section of the voice information, and the progress from the start end of the target voice. The model is made to learn time and (step S102). Then, the information providing device 10 provides the learning model to the terminal device 100 (step S103) and ends the process.

  Next, an example of the flow of detection processing will be described using FIG. 7. First, the terminal device 100 determines whether or not the uttered voice is received (step S201), and when not received (step S201: No), executes step S201. Further, when the terminal device 100 receives the uttered voice (step S201: Yes), the uttered voice is input to the learning model and the end of the target voice is estimated (step S202). Then, the terminal device 100 estimates the starting end based on the elapsed period estimated by the learning model (step S203). Then, the terminal device 100 extracts the keyword section, executes a process according to the voice included in the extracted keyword section (step S204), and ends the process.

[4. Modification example)
In the above, an example of the learning process and the detection process by the information providing device 10 has been described. However, the embodiment is not limited to this. Hereinafter, variations of the learning process and the detection process executed by the information providing device 10 and the terminal device 100 will be described.

[4-1. About the section corresponding to the class]
In the above description using FIG. 5, one class is associated with three sections. However, the embodiment is not limited to this. For example, the information providing apparatus 10 may perform learning such that one class corresponds to one section, or may perform learning such that one class corresponds to ten sections. Here, one section may correspond to one frame or may correspond to a plurality of frames. Further, the information providing device 10 may set an upper limit value for the number of classes.

[4-2. Device configuration〕
The databases 31 and 32 registered in the storage unit 30 may be held in an external storage server. Further, the information providing device 10 and the terminal device 100 may realize the above-described learning process and detection process in cooperation with each other, or one of them may execute them independently.

[4-3. Other]
Further, of the processes described in the above embodiment, all or part of the processes described as being automatically performed may be manually performed, and conversely, the processes described as being manually performed. All or part of the above can be automatically performed by a known method. In addition, the processing procedures, specific names, information including various data and parameters shown in the above-mentioned documents and drawings can be arbitrarily changed unless otherwise specified. For example, the various kinds of information shown in each drawing are not limited to the illustrated information.

  Further, each constituent element of each illustrated device is a functional conceptual one, and does not necessarily have to be physically configured as illustrated. That is, the specific form of distribution / integration of each device is not limited to that shown in the figure, and all or part of the device may be functionally or physically distributed / arranged in arbitrary units according to various loads and usage conditions. It can be integrated and configured.

  Further, the respective embodiments described above can be appropriately combined within a range in which the processing content is not inconsistent.

[4-4. program〕
Further, the information providing device 10 according to the above-described embodiment is realized by, for example, a computer 1000 having a configuration shown in FIG. FIG. 8 is a diagram illustrating an example of the hardware configuration. The computer 1000 is connected to an output device 1010 and an input device 1020, and an arithmetic device 1030, a primary storage device 1040, a secondary storage device 1050, an output IF (Interface) 1060, an input IF 1070, and a network IF 1080 are connected by a bus 1090. Have.

  The arithmetic unit 1030 operates based on a program stored in the primary storage device 1040 or the secondary storage device 1050, a program read from the input device 1020, or the like, and executes various processes. The primary storage device 1040 is a memory device such as a RAM that temporarily stores data used by the arithmetic device 1030 for various calculations. The secondary storage device 1050 is a storage device in which data used by the arithmetic device 1030 for various calculations and various databases are registered, such as a ROM (Read Only Memory), a HDD (Hard Disk Drive), and a flash memory. It is realized by.

  The output IF 1060 is an interface for transmitting information to be output to an output device 1010 that outputs various types of information such as a monitor and a printer. For example, a USB (Universal Serial Bus) or a DVI (Digital Visual Interface), It is realized by a connector of a standard such as HDMI (registered trademark) (High Definition Multimedia Interface). The input IF 1070 is an interface for receiving information from various input devices 1020 such as a mouse, a keyboard, and a scanner, and is realized by, for example, USB.

  The input device 1020 is, for example, an optical recording medium such as a CD (Compact Disc), a DVD (Digital Versatile Disc), a PD (Phase change rewritable Disk), a magneto-optical recording medium such as an MO (Magneto-Optical disk), or a tape. It may be a device that reads information from a medium, a magnetic recording medium, a semiconductor memory, or the like. The input device 1020 may also be an external storage medium such as a USB memory.

  The network IF 1080 receives data from another device via the network N and sends the data to the arithmetic device 1030, and also transmits the data generated by the arithmetic device 1030 via the network N to another device.

  The arithmetic device 1030 controls the output device 1010 and the input device 1020 via the output IF 1060 and the input IF 1070. For example, the arithmetic device 1030 loads a program from the input device 1020 or the secondary storage device 1050 onto the primary storage device 1040, and executes the loaded program.

  For example, when the computer 1000 functions as the information providing device 10, the arithmetic device 1030 of the computer 1000 executes the program or data (for example, the learning model M1) loaded on the primary storage device 1040, so that the control unit 40. Realize the function of. The computing device 1030 of the computer 1000 reads these programs or data (for example, the learning model M1) from the primary storage device 1040 and executes them. As another example, these programs are loaded from other devices via the network N. You may get it.

[5. effect〕
As described above, the information providing apparatus 10 acquires the voice information including the target voice to be detected, and makes the model learn the end of the target voice and the period elapsed from the start of the target voice. Therefore, the information providing apparatus 10 can improve the detection accuracy of the section including the target voice as a result of implementing learning of the model that can appropriately detect the section including the target voice from the input voice information. .

  Further, the information providing apparatus 10 divides the voice information into a plurality of sections, and learns for each section whether or not the end of the target voice is included and the period from the start end of the target voice to the section as a model. Let In addition, the information providing apparatus 10, when the voice included in the predetermined section is input to the model, determines the end information indicating whether the end of the target voice is included in the predetermined section and the start end of the target voice. The model is trained so as to output period information indicating the period up to the predetermined section. Further, the information providing device 10 divides the voice information into a plurality of sections, and when a voice included in the predetermined section is input, outputs a classification result according to a period from the start end of the target voice to the predetermined section. To learn the model.

  In this way, the information providing apparatus 10 detects the end of the target voice and learns a model that enables the detection of the start end of the target voice by tracing back the period information from the detected end. As a result, the information providing apparatus 10 can detect the section including the target voice by using the characteristics of the entire target voice, and as a result, can improve the detection accuracy of the section including the target voice.

  The information providing apparatus 10 also causes the model having the configuration of the recurrent neural network to learn the end of the target voice and the period elapsed from the start of the target voice. The information providing apparatus 10 also acquires voice information including a voice for causing a predetermined terminal device to perform a predetermined operation as a target voice. In addition, the information providing apparatus 10 acquires voice information that includes a voice in which a plurality of words are uttered or a voice including a silent section as a target voice.

  The information providing apparatus 10 also learns a model so as to detect the end of the target voice based on the characteristics of the entire target voice. For example, the information providing apparatus 10 performs model learning so as to detect a section including the end of the target voice based on the appearance order of the features of each section of the target voice. As a result of the above-described processing, the information providing device 10 can improve the detection accuracy of the section including the target voice.

  In addition, the terminal device 100 acquires voice information. Then, the terminal device 100 uses the model in which the end of the target voice to be detected and the period elapsed from the start of the target voice are learned, from the voice information acquired by the acquisition unit, the start end of the target voice. To detect. Therefore, the terminal device 100 can improve the detection accuracy of the section including the target voice.

  In addition, the terminal device 100 determines whether or not the end of the target voice is included in each section included in the learning information that is the voice information that is the learning target, and the period from the start end of the target voice to the section. Using the learned model, a section including the start end of the target voice is detected from the voice information. In addition, the terminal device 100, when the voice included in the predetermined section is input, the terminal information indicating whether or not the end of the target voice is included in the predetermined section, and the predetermined start point of the target voice. A section including the start end of the target voice is detected from the voice information by using the model learned so as to output the period information indicating the period up to the period.

  Further, the terminal device 100 divides the acquired voice information into a plurality of sections, and of the divided sections, termination information indicating that the termination of the target speech is included when the speech included in the section is input. The section output by the model is specified, and the section including the start end of the target voice is detected based on the period information output by the model for the specified section. Further, the terminal device 100 sequentially inputs the voices included in each section of the voice information from the beginning to the model having the configuration of the recurrent neural network, and based on the end information and the period information output by the model, A section including the start end of the target voice is detected. As a result of such processing, the terminal device 100 detects the section in which the target voice is included, based on the characteristics of the entire target voice, so the detection accuracy can be improved.

  As described above, some of the embodiments of the present application have been described in detail based on the drawings, but these are examples, and various modifications based on the knowledge of those skilled in the art, including the modes described in the section of the disclosure of the invention, It is possible to implement the present invention in other forms with improvements.

  Further, the above-mentioned "section (module, unit)" can be read as "means" or "circuit". For example, the detection unit can be replaced with a detection unit or a detection circuit.

10 information providing device 20 communication unit 30 storage unit 31 learning data database 32 model database 40 control unit 41 learning unit 42 acquisition unit 43 detection unit 44 response generation unit 45 providing unit 100 user terminal

Claims (18)

An acquisition unit that acquires audio information that includes the target audio that is the detection target,
A learning device, comprising: a learning unit that causes a model to learn the end of the target voice and the period elapsed from the start of the target voice. The learning unit divides the voice information into a plurality of sections, and for each section, learns whether the end of the target voice is included and a period from the start end of the target voice to the section as a model. The learning apparatus according to claim 1, wherein the learning apparatus is configured to: The learning unit, when a voice included in a predetermined section is input to the model, end information indicating whether or not the end of the target voice is included in the predetermined section, and from a start end of the target voice. The learning device according to claim 2, wherein the model learning is performed so as to output period information indicating a period up to the predetermined section. The learning unit divides the voice information into a plurality of sections, and when a voice included in a predetermined section is input, outputs a classification result according to a period from the start end of the target voice to the predetermined section. The learning device according to any one of claims 1 to 3, wherein the learning of the model is performed. The learning unit causes a model having a configuration of a recursive neural network to learn a terminal end of the target voice and a period elapsed from a start end of the target voice. The learning device according to one. The learning according to any one of claims 1 to 5, wherein the acquisition unit acquires voice information including a voice for causing a predetermined terminal device to perform a predetermined operation as the target voice. apparatus. The said acquisition part acquires the audio | voice information which includes the audio | voice which pronounced several words, or the audio | voice containing a silent area as the said target audio | voice. Learning device. The learning unit performs learning of the model based on a feature of the entire target voice so as to detect an end of the target voice. 8. The learning unit according to claim 1, wherein Learning device. The learning unit performs learning of the model so as to detect a section including a terminal end of the target voice based on an appearance order of features included in each section of the target voice. Learning device. An acquisition unit that acquires voice information,
A detection unit that detects the start end of the target voice from the voice information acquired by the acquisition unit using a model in which the end of the target voice to be detected and the period elapsed from the start end of the target voice are learned. And a detection device. The detection unit determines whether or not the end of the target voice is included, and the period from the start end of the target voice to the section, for each section included in the learning information that is the voice information that is the learning target. The detection device according to claim 10, wherein the learned model is used to detect a section including a start end of the target voice from the voice information acquired by the acquisition unit. The detection unit, when a voice included in a predetermined section is input, end information indicating whether or not the end of the target voice is included in the predetermined section, and the predetermined end from the start end of the target voice. A period including a start end of the target voice is detected from the voice information acquired by the acquisition unit by using a model learned to output period information indicating a period up to the period. The detection device according to claim 11. The detection unit divides the voice information acquired by the acquisition unit into a plurality of sections, and the end of the target voice is included when a voice included in the section is input among the divided sections. 13. A section in which the model outputs end information indicating the section is specified, and a section in which the start end of the target voice is included is detected based on the period information output by the model for the specified section. The detection device according to 1. The detection unit inputs voices included in each section of the voice information acquired by the acquisition unit in order from the beginning with respect to a model having a configuration of a recursive neural network, and outputs the end information and the end information output by the model. The detection device according to claim 11 or 12, wherein a section including a start end of the target voice is detected based on the period information. A learning method executed by the learning device,
An acquisition step of acquiring voice information including a target voice to be detected,
A learning method, comprising: a learning step of causing a model to learn the end of the target voice and the period elapsed from the start of the target voice. An acquisition procedure for acquiring audio information including target audio to be detected,
A learning program for causing a computer to execute a learning procedure for causing a model to learn the end of the target voice and the period elapsed from the start of the target voice. A detection method performed by a detection device, comprising:
An acquisition process for acquiring audio information,
A detection step of detecting the start end of the target voice from the voice information acquired by the acquisition unit using a model in which the end of the target voice to be detected and the period elapsed from the start end of the target voice are learned. A detection method comprising: and. Acquisition procedure to acquire voice information,
A detection procedure for detecting the start end of the target voice from the voice information acquired by the acquisition unit using a model in which the end of the target voice to be detected and the period elapsed from the start end of the target voice are learned. And a detection program including.

Images