JP2021139920A - Voice processing device and voice processing method - Google Patents

Abstract

To provide a voice processing device comprising a voice agent function which improves the accuracy of a response to a user.SOLUTION: In a voice processing device, when voice uttered by a user is inputted to a sound collection unit 232, a voice recognition unit 234 recognizes the voice, and a semantic analysis unit 238 analyses meaning of a character string inputted from the voice recognition unit 234. Then, the semantic analysis unit 238, when understanding that a request of the user is a response of information, extracts a keyword from the character string inputted from the voice recognition unit 234. Further, a sound source direction estimation unit 236 estimates a sound source direction of the voice, and a response control unit 250 extracts response information from a storage unit 240 based on the keyword and the sound source direction of the voice. Then, the response control unit 250 generates response voice by using the extracted response information, and a voice output unit 260 outputs the response voice.SELECTED DRAWING: Figure 2

Description

The present disclosure relates to a voice processing device and a voice processing method.

In recent years, a voice processing device having a voice agent function has become widespread. The voice agent function is a function that analyzes the meaning of the voice spoken by the user and executes processing according to the meaning obtained by the analysis. For example, a voice processing device having a voice agent function can answer a question from a user, add an appointment, set a timer, and the like.

Regarding the voice agent function, Patent Document 1 discloses a method in which a user registers an abbreviation in a voice processing device in advance and calls a complicated word using the abbreviation. Further, Patent Document 2 discloses a technique for understanding the meaning of a user's utterance by using context information when the user utters a voice.

Japanese Unexamined Patent Publication No. 2016-114395 Japanese Unexamined Patent Publication No. 2015-122104

However, regarding the method described in Patent Document 1, even if the same word as the word registered in advance is uttered by the user, the meaning contained in the word may differ depending on the situation. Further, in the technique described in Patent Document 2, the context information is not registered in advance, and only the context information when the user speaks is used as the context information, so that there is room for improvement in accuracy.

Therefore, the present disclosure proposes a new and improved voice processing device and a voice processing method capable of improving the accuracy of the response to the user.

According to the present disclosure, the storage unit that stores the input information obtained based on the recognition of the voice input to the sound input unit is referred to, and the situation associated with the input information is recognized. A voice processing device including a response control unit that controls a response using the input information is provided.

Further, according to the present disclosure, the processor recognizes the situation associated with the input information by referring to the storage unit that stores the input information obtained based on the recognition of the voice input to the sound input unit. Based on the above, a voice processing method including controlling a response using the input information is provided.

As described above, according to the present disclosure, it is possible to improve the accuracy of the response to the user. It should be noted that the above effects are not necessarily limited, and together with or in place of the above effects, any of the effects shown herein, or any other effect that can be grasped from this specification. May be played.

It is explanatory drawing which shows the outline of the voice processing apparatus 20 by embodiment of this disclosure. It is explanatory drawing which shows the structure of the voice processing apparatus 21 by 1st Embodiment. It is explanatory drawing which shows the usage form of the voice processing apparatus 21. It is explanatory drawing which shows the specific example of the stored information. It is explanatory drawing which shows the usage form of the voice processing apparatus 21. It is a flowchart which shows the operation of the voice processing apparatus 21 by 1st Embodiment. It is explanatory drawing which shows the application example of 1st Embodiment. It is explanatory drawing which shows the structure of the voice processing apparatus 22 by 2nd Embodiment. It is explanatory drawing which shows the specific example of the stored information. It is explanatory drawing which shows the specific example of the usage form of a recognition model. It is a flowchart which shows the schematic operation of the voice processing apparatus 22 by 2nd Embodiment. It is a flowchart which shows the generation method of a recognition model. It is a flowchart which shows the response control using a recognition model. It is a flowchart which shows the application example of 2nd Embodiment. It is explanatory drawing which showed the hardware configuration of the voice processing apparatus 20.

Preferred embodiments of the present disclosure will be described in detail below with reference to the accompanying drawings. In the present specification and the drawings, components having substantially the same functional configuration are designated by the same reference numerals, so that duplicate description will be omitted.

Further, in the present specification and the drawings, a plurality of components having substantially the same functional configuration may be distinguished by adding different alphabets after the same reference numerals. However, when it is not necessary to particularly distinguish each of the plurality of components having substantially the same functional configuration, only the same reference numerals are given to each of the plurality of components.

In addition, the present disclosure will be described according to the order of items shown below.
0. Outline of voice processing device 1. First Embodiment 1-1. Configuration of voice processing device according to the first embodiment 1-2. Operation of voice processing device according to the first embodiment 1-3. Action effect 1-4. Application example of the first embodiment 2. Second Embodiment 2-1. Configuration of voice processing device according to the second embodiment 2-2. Operation of voice processing device according to the second embodiment 2-3. Action effect 2-4. Application example of the second embodiment 3. Hardware configuration 4. Conclusion

<0. Overview of voice processing equipment>
First, the outline of the voice processing apparatus according to the embodiment of the present disclosure will be described with reference to FIG.

FIG. 1 is an explanatory diagram showing an outline of the voice processing device 20 according to the embodiment of the present disclosure. As shown in FIG. 1, the voice processing device 20 is arranged in a house as an example. The voice processing device 20 has a voice agent function that analyzes the meaning of the voice spoken by the user of the voice processing device 20 and executes processing according to the meaning obtained by the analysis.

For example, the voice processing device 20 stores the keyword and the response information in association with each other, has the meaning that the user's voice requests a response of the information, and is based on the fact that the stored keyword is recognized from the voice. A response to the user is executed using the response information stored in association with the keyword. In the example shown in FIG. 1, the voice processing device 20 stores the keyword "pasta of company A" and the response information "boil time is 5 minutes" in association with each other, and the user asks "What is the pasta of company A?" The voice processing device 20 responds to the user with a voice saying "I remember that the pasta of company A has a boiling time of 5 minutes" using the response information "boil time is 5 minutes". ..

Although FIG. 1 shows a stationary device as the voice processing device 20, the voice processing device 20 is not limited to the stationary device. For example, the voice processing device 20 may be a portable information processing device such as a smartphone, a mobile phone, a PHS (Personal Handyphone System), a portable music playback device, a portable video processing device, or a portable game device. , It may be an autonomous mobile robot.

Here, there may be keywords having ambiguity. Therefore, it is assumed that different response information is associated with the same keyword. For example, the keyword "pasta of company A" may be associated with the response information indicating the recipe described above, or may be associated with the response information indicating the storage location. For this reason, it has been desired to devise a method for extracting response information according to the user's intention from the keywords included in the voice.

The inventor of the present invention has come to create the voice processing device 20 according to the embodiment of the present disclosure with the above circumstances as the first point of view. The voice processing device 20 according to the embodiment of the present disclosure can improve the accuracy of the response to the voice input by extracting the response information according to the user's intention from the keywords included in the voice. Hereinafter, some embodiments of the present disclosure will be described in detail in sequence.

<1. First Embodiment>
The voice processing device 20 according to the first embodiment stores the keyword, the sound source direction, and the response information in association with each other, and is based on the fact that the sound including the keyword arrives from the sound source direction corresponding to the stored sound source direction. Execute the response using the response information. The keyword and the response information are examples of the input information obtained based on the recognition of the input voice, and the sound source direction is an example of the situation information indicating the situation regarding the sound or the generation of the voice. According to the voice processing device 20 according to the first embodiment, even when different response information is associated with the same keyword, the response information according to the user's intention is provided based on the sound source direction of the voice including the keyword. It is possible to extract.

In the following, in order to distinguish the voice processing device 20 according to each embodiment, the voice processing device according to the first embodiment is referred to as a voice processing device 21, and the voice processing device according to the second embodiment is referred to as a voice processing device 22. It may be called. The voice processing device 21 and the voice processing device 22 according to each embodiment may be simply collectively referred to as the voice processing device 20.

<< 1-1. Configuration of voice processing device according to the first embodiment >>
FIG. 2 is an explanatory diagram showing the configuration of the voice processing device 21 according to the first embodiment. As shown in FIG. 2, the voice processing device 21 according to the first embodiment includes a sound collecting unit 232, a voice recognition unit 234, a sound source direction estimation unit 236, a semantic analysis unit 238, a storage unit 240, a response control unit 250, and a response control unit 250. It has an audio output unit 260.

(Sound collection section)
The sound collecting unit 232 has a function of a sound input unit that acquires an electrical sound signal from aerial vibration including environmental sound and voice. The sound collecting unit 232 outputs the acquired sound signal to the voice recognition unit 234 and the sound source direction estimation unit 236.

(Voice recognition unit)
The voice recognition unit 234 detects a voice signal from the sound signal input from the sound collection unit 232, recognizes the voice signal, and obtains a character string representing the voice spoken by the user.

(Sound source direction estimation unit)
The sound source direction estimation unit 236 is an example of the situational awareness unit, and estimates the sound source direction of the sound that has reached the sound collecting unit 232. When the sound collecting unit 232 is composed of a plurality of sound collecting elements, the sound source direction estimating unit 236 estimates the sound source direction based on the phase difference of the sound signals obtained by each sound collecting element. When the sound signal is a voice signal, the direction of the user as seen from the voice processing device 21 is estimated as the sound source direction.

(Semantic analysis department, memory department)
The semantic analysis unit 238 analyzes the meaning of the character string input from the voice recognition unit 234. The method of semantic analysis may be any of a method of creating a speech corpus and then using machine learning to realize semantic analysis, a method of realizing semantic analysis by rules, or a combination thereof. In addition, morphological analysis, which is a part of semantic analysis processing, has a mechanism for assigning attributes on a word-by-word basis, and holds a dictionary inside. The semantic analysis unit 238 uses a mechanism for assigning this attribute and a dictionary to determine what kind of word the word included in the utterance is, for example, whether it is a person's name, a place name, or a general noun. It is possible to grant.

Then, the semantic analysis unit 238 performs different processing according to the meaning obtained by the analysis. For example, when the meaning obtained by the analysis has a meaning requesting the storage of information, the semantic analysis unit 238 extracts a keyword as the first part from the character string input from the voice recognition unit 234, and the second The response information corresponding to the keyword is extracted as the part of. Then, the semantic analysis unit 238 stores the extracted keyword, the response information, and the sound source direction input from the sound source direction estimation unit 236 in association with each other in the storage unit 240. Hereinafter, a specific example of storing such information will be described with reference to FIGS. 3 and 4.

FIG. 3 is an explanatory diagram showing a usage pattern of the voice processing device 21. In the example shown in FIG. 3, at the position P1 included in the kitchen area A1, the user utters a voice saying "Remember that the pasta of company A is boiled for 5 minutes." In this case, the semantic analysis unit 238 uses "pasta of company A" as a keyword from the character string "remember that the boiling time of pasta of company A is 5 minutes" input from the voice recognition unit 234. Extract and extract "boil time is 5 minutes" as response information. On the other hand, the sound source direction estimation unit 236 estimates as the sound source direction 30 °, which is an angle formed by the direction of the position P1 as seen from the voice processing device 21 and the reference direction d of the voice processing device 21. Therefore, as shown in FIG. 4, the semantic analysis unit 238 stores the keyword “pasta of company A”, the sound source direction “30 °”, and the response information “boil time is 5 minutes” in the storage unit 240 in association with each other. ..

FIG. 3 further shows an example in which the user utters a voice saying "Remember that the pasta of company A was placed on the top shelf of the storage" at the position P2 included in the storage peripheral area A2. Shown. In this case, the semantic analysis unit 238 uses the character string "Remember that the pasta of company A was placed on the top shelf of the storage" input from the voice recognition unit 234 as a keyword "Company A". "Pasta" is extracted, and "placed on the top shelf of storage" is extracted as response information.

On the other hand, the sound source direction estimation unit 236 estimates the angle formed by the direction of the position P2 as seen from the voice processing device 21 and the reference direction d of the voice processing device 21 as the sound source direction. Therefore, as shown in FIG. 4, the semantic analysis unit 238 associates the keyword "pasta of company A", the sound source direction "-20 °", and the response information "placed on the top shelf of the storage". It is stored in the storage unit 240.

(Response control unit)
When the meaning obtained by the analysis of the semantic analysis unit 238 has the meaning of requesting the response of the information, the response control unit 250 refers to the storage unit 240 and the arrival of the sound from the sound source direction associated with the response information. , And based on the recognition that the voice contains a keyword associated with the response information, the response is controlled using the response information. Here, the response control unit 250 indicates that the sound source direction in which the difference from the sound source direction associated with the response information is less than a predetermined reference is recognized, that the sound from the sound source direction associated with the response information is recognized. It may be treated as an arrival. The predetermined reference may be, for example, a value between 10 ° and 20 °.

(Audio output section)
The voice output unit 260 outputs a response voice to the user according to the control from the response control unit 250. However, the voice output unit 260 is only an example of an output unit that outputs a response, and the voice processing device 21 may have a display unit that outputs the response by display. Here, a specific example of the response voice output by the voice output unit 260 will be described with reference to FIG.

FIG. 5 is an explanatory diagram showing a usage pattern of the voice processing device 21. In the example shown in FIG. 5, the user is uttering a voice "What is the pasta of company A?" At the position P1 included in the kitchen area A1. The sound source direction of the sound is "30 °". Therefore, the response control unit 250 refers to the storage unit 240 and extracts the response information of "boil time is 5 minutes" associated with the keyword "pasta of company A" and the sound source direction "30 °". Then, the response control unit 250 uses the extracted response information “boil time is 5 minutes” and stores the “boil time of pasta of company A as 5 minutes” in the audio output unit 260 as shown in FIG. The response voice "Masu." Is output.

On the other hand, when the user utters the voice "What is the pasta of company A?" At the position P2 included in the storage peripheral area A2, the response control unit 250 refers to the storage unit 240 and the keyword "pasta of company A". And the response information "placed on the top shelf of the storage" associated with the sound source direction "-20 °" is extracted. Then, the response control unit 250 uses the extracted response information “placed on the top shelf of the storage” to inform the audio output unit 260 that “the pasta of company A is the top of the storage” as shown in FIG. I remember that I put it on the upper shelf. "

<< 1-2. Operation of voice processing device according to the first embodiment >>
The configuration of the voice processing device 21 according to the first embodiment has been described above. Subsequently, with reference to FIG. 6, the operation of the voice processing device 21 according to the first embodiment is arranged.

FIG. 6 is a flowchart showing the operation of the voice processing device 21 according to the first embodiment. First, when the voice spoken by the user is input to the sound collecting unit 232 (S304), the voice recognition unit 234 recognizes the voice, and the semantic analysis unit 238 determines the meaning of the character string input from the voice recognition unit 234. Analyze (S308).

Then, when the semantic analysis unit 238 understands that the user's request is a response to information (S312 / Yes), the semantic analysis unit 238 extracts a keyword from the character string input from the voice recognition unit 234 (S316). Further, the sound source direction estimation unit 236 estimates the sound source direction of the voice (S340), and the response control unit 250 extracts the response information from the storage unit 240 based on the keyword and the sound source direction of the voice (S344). Then, the response control unit 250 generates a response voice using the extracted response information, and the voice output unit 260 outputs the response voice (S348).

On the other hand, when the semantic analysis unit 238 understands that the user's request is a memory of information (S312 / No, S352 / Yes), the semantic analysis unit 238 extracts the keyword and the response information from the character string input from the voice recognition unit 234 (S356). ). Further, the sound source direction estimation unit 236 estimates the sound source direction of the sound (S360), and the semantic analysis unit 238 stores the keyword, the response information, and the sound source direction input from the sound source direction estimation unit 236 in the storage unit 240 in association with each other. (S364). After that, for example, the audio output unit 260 outputs a registration completion notification indicating that the storage of information is completed (S368).

<< 1-3. Action effect >>
According to the first embodiment described above, various effects can be obtained. For example, according to the first embodiment, it is possible to output different response voices based on the recognition of the situation (in the above example, the estimation of the sound source direction) for the voices containing the same keyword. Therefore, even for ambiguous voices that can have multiple meanings such as "What is the pasta of company A?", The meaning of inquiring about the boiling time, the meaning of inquiring about the location, etc. It is possible to output.

<< 1-4. Application example >>
As an application example of the first embodiment described above, the response control unit 250 controls whether or not to output a response voice to the voice spoken by the user, depending on whether or not the user is the target user. You may. For example, when the information is stored based on the voice spoken by a certain user, the storage unit 240 also stores the target information indicating the user as the target user. The target information may be the identification information of the target user, or may be the feature amount of the voice of the target user. Then, the response control unit 250 may output the response information when the user who utters the voice requesting the response of the information is the target user and other conditions (sound source direction, etc.) are satisfied.

The target user and the user who requested the storage of information may be different. For example, when storing information, the voice processing device 21 may instruct the user who requested the storage of information to input a user to be set as the target user, and set the input user as the target user. For example, as shown in FIG. 7, all users can be set as the target user, or a specific user (user A in the example of FIG. 7) can be set as the target user.

By setting a plurality of users such as all users as the target users, it is not necessary for each user to individually speak for storing information, so that it is possible to reduce the time and effort of the users. For highly versatile information, it is useful to set a plurality of users such as all users as target users in this way. On the other hand, for information specific to a certain user, it is useful to set a specific user as a target user.

<2. Second embodiment>
Next, the voice processing device 22 according to the second embodiment of the present disclosure will be described. The voice processing device 22 according to the second embodiment can output an appropriate response voice to the user without an explicit request from the user. Hereinafter, the configuration and operation of the voice processing device 22 according to the second embodiment will be sequentially described in detail.

<< 2-1. Configuration of voice processing device according to the second embodiment >>
FIG. 8 is an explanatory diagram showing the configuration of the voice processing device 22 according to the second embodiment. As shown in FIG. 8, the voice processing device 22 according to the second embodiment includes a sound collecting unit 232, a voice recognition unit 234, a sound source direction estimation unit 236, a semantic analysis unit 238, a storage unit 242, a learning unit 246, and a response. It has a control unit 252 and an audio output unit 260. Since the configurations of the sound collecting unit 232, the voice recognition unit 234, the sound source direction estimation unit 236, the semantic analysis unit 238, and the voice output unit 260 are as described in the first embodiment, detailed description thereof will be omitted here. ..

Similar to the first embodiment, the storage unit 242 stores the keywords extracted by the semantic analysis unit 238, the response information, and the sound source direction input from the sound source direction estimation unit 236 in association with each other. For example, when the user utters a voice "remember to brush your teeth when you take a bath" at the dressing room, the storage unit 242 uses the keyword "take a bath" as shown in FIG. , The sound source direction "50 °" (the direction of the dressing room as seen from the voice processing device 22), and the response information "brushing teeth" are stored in association with each other. As a result, based on the arrival of the voice from the direction of "50 °" and the recognition that the voice includes the keyword "taken a bath", the "bath" is the same as in the first embodiment. It is possible to output a response voice saying, "I remember brushing my teeth when I got out."

Further, the learning unit 246 generates a recognition model for recognizing the sound generated in the situation indicated by a certain keyword by machine learning, and the storage unit 242 generates the above recognition model which is the learning result of the machine learning by the learning unit 246. Remember. According to the recognition model, for example, when a sound generated when the bath door is closed is input, the keyword "taken out of the bath" can be output. As another example, when the sound of the running shoes hitting the floor of the entrance is input when putting on the running shoes, the keyword "going to run" can be output. A specific example of how to use such a recognition model will be described with reference to FIG.

FIG. 10 is an explanatory diagram showing a specific example of the usage pattern of the recognition model. As shown in FIG. 10, when a sound (bang) of closing the bathroom door is generated at the dressing room P3, the response control unit 252 uses the keyword "taken a bath" corresponding to the sound as a recognition model. Identify based on. Then, the response control unit 252 extracts the response information "brushing teeth" associated with the keyword "out of the bath" based on the information (see FIG. 9) stored in the storage unit 242, and "brushes the teeth". I remember brushing my teeth when I got out of the bath. "

Further, as shown in FIG. 10, when a sound (tonton) of the running shoes hitting the floor of the entrance is generated at the entrance P4, the response control unit 252 uses the keyword "going to run" corresponding to the sound as a recognition model. Identify based on. Then, the response control unit 252 extracts the response information "confirm the weather" associated with the keyword "going to run" based on the information (see FIG. 9) stored in the storage unit 242, and "checks the weather". When I go running, I remember when I check the weather. ”I have the audio output unit 260 output a response voice.

Here, the response control unit 252 obtains the weather information via the network, and as a result of executing the process or action of "confirming the weather", the response such as "the weather after this is rainy". The audio may be output to the audio output unit 260. According to such a configuration, the labor of the user is reduced, so that the convenience of the user can be improved.

<< 2-2. Operation of voice processing device according to the second embodiment >>
Subsequently, with reference to FIGS. 11 to 13, the operation of the voice processing device 22 according to the second embodiment is arranged.

FIG. 11 is a flowchart showing a schematic operation of the voice processing device 22 according to the second embodiment. As shown in FIG. 11, the learning unit 246 generates a recognition model for recognizing the sound generated in the situation indicated by the keyword by machine learning (S410).

Then, the semantic analysis unit 238 analyzes the meaning of the voice spoken by the user, and when the user's request is information storage, the storage unit 242 estimates the sound source direction based on the keyword and response information extracted from the voice. The sound source direction of the sound estimated by the unit 236 is associated and stored (S430). The process of S430 corresponds to the process of S352 to S368 described with reference to FIG.

After that, the response control unit 252 performs sound recognition using the recognition model generated by the learning unit 246, and controls the output of the response voice using the response information associated with the keyword specified based on the sound recognition. (S450).

Hereinafter, the process of S410 will be described more specifically with reference to FIG. 12, and the process of S450 will be described more specifically with reference to FIG.

FIG. 12 is a flowchart showing a method of generating a recognition model. First, when the user utters a certain keyword, the voice processing device 22 instructs the user to generate a sound generated in the situation indicated by the keyword a certain number of times (S411). For example, when the keyword is "taken a bath", the user repeatedly opens and closes the bathroom door a certain number of times. Then, the learning unit 246 saves the generated sound as regular example data for machine learning (S412).

Subsequently, the voice processing device 22 instructs the user not to generate the sound generated in the situation indicated by the keyword for a certain period of time (S413). Then, the learning unit 246 saves the ambient sound as negative example data for machine learning (S414).

After that, the learning unit 246 performs machine learning using the saved positive example data and negative example data, and generates a recognition model that recognizes the situation indicated by the keyword (S415). Then, the storage unit 242 stores the recognition model generated by the learning unit 246 (S416).

FIG. 13 is a flowchart showing response control using the recognition model. First, when a sound is input to the sound collecting unit 232 (S451), the response control unit 252 determines whether or not the input sound is a sound recognized by using the recognition model stored in the storage unit 242. Is determined (S452). When the input sound is a sound recognized by using the recognition model (S452 / Yes), the response control unit 252 specifies the keyword corresponding to the sound based on the recognition model (S453). Further, the sound source direction estimation unit 236 estimates the sound source direction of the sound (S454).

Then, the response control unit 252 is associated with the keyword specified based on the recognition model, and is associated with the sound source direction in which the difference from the sound source direction estimated by the sound source direction estimation unit 236 is less than a predetermined reference. The response information is extracted (S455). Further, the response control unit 252 generates a response voice using the extracted response information, and causes the voice output unit 260 to output the response voice (S456).

<< 2-3. Action effect >>
According to the second embodiment described above, various effects can be obtained. For example, according to the second embodiment, it is appropriate for the user based on the recognition of the situation (in the above example, the recognition of the sound and the estimation of the sound source direction) without the user explicitly speaking the voice. It is possible to output a response voice.

Also, the sounds produced in a given situation may differ depending on which user the situation is associated with. For example, even if it is the sound of opening and closing the bathroom door, the sound that the father makes when opening and closing the bathroom door and the sound that the child makes when opening and closing the bathroom door Can be different. Therefore, when a recognition model is generated by a user generating a sound a certain number of times in S411 described with reference to FIG. 12, the sound generated by another user may not be recognized by the recognition model. obtain. Therefore, according to the second embodiment, it is possible to limit the output target of the response voice based on the sound input to a specific user.

<< 2-4. Application example >>
As an application example of the second embodiment described above, a method of improving the accuracy of the recognition model in the use of the voice processing device 22 will be described.

FIG. 14 is a flowchart showing an application example of the second embodiment, and is a flowchart of the process applied to S430 shown in FIG. Since the processes of S304 to S316 and S340 to S368 in FIG. 14 are as described with reference to FIG. 6, detailed description here will be omitted.

When the user's voice request is a response to information (S312 / Yes) and a keyword is extracted from the voice (S316), the learning unit 246 stores the recognition model corresponding to the keyword in the storage unit 242. Whether or not it is determined (S320 / Yes). Then, the learning unit 246 extracts a sound that matches or is similar to the sound recognized by the recognition model from the sound input to the sound collecting unit 232 (S324).

Further, the learning unit 246 adds the sound extracted in S324 as regular example data (S328), performs machine learning again using the regular example data group including the added regular example data, and regenerates the recognition model. (S332).

According to the application example, the correct example data for machine learning is added even if the user does not speak a voice for machine learning, so that the accuracy of the recognition model can be improved without the user having to feel the trouble. Is possible.

In the above, the example of recognizing the sound as the recognition of the situation corresponding to the keyword has been described, but the situation corresponding to the keyword can also be recognized by other methods. For example, when the voice processing device 22 includes an imaging unit, the learning unit 246 may generate a recognition model that recognizes a situation corresponding to a keyword by machine learning using the captured image obtained by the imaging unit. Here, "going to run" can correspond to the keyword, and the situation corresponding to the keyword includes the clothes of the user when "going to running". The response control unit 252 identifies the keyword "going to run" when the user's clothes at the time of "going to run" are recognized based on the recognition model, and the response associated with the keyword "going to run". The response output may be controlled using the information "Check the weather."

<3. Hardware configuration>
Each embodiment of the present disclosure has been described above. Information processing such as voice recognition and response control described above is realized by the cooperation between the software and the hardware of the voice processing device 20 described below.

FIG. 15 is an explanatory diagram showing a hardware configuration of the voice processing device 20. As shown in FIG. 15, the audio processing device 20 includes a CPU (Central Processing Unit) 201, a ROM (Read Only Memory) 202, a RAM (Random Access Memory) 203, an input device 208, and an output device 210. , A storage device 211, a drive 212, an image pickup device 213, and a communication device 215.

The CPU 201 functions as an arithmetic processing device and a control device, and controls the overall operation in the voice processing device 20 according to various programs. Further, the CPU 201 may be a microprocessor. The ROM 202 stores programs, calculation parameters, and the like used by the CPU 201. The RAM 203 temporarily stores a program used in the execution of the CPU 201, parameters that change appropriately in the execution, and the like. These are connected to each other by a host bus composed of a CPU bus or the like. By the collaboration between the CPU 201, ROM 202 and RAM 203 and the software, functions such as a voice recognition unit 234, a sound source direction estimation unit 236, a semantic analysis unit 238, a learning unit 246, and a response control unit 252 can be realized.

The input device 208 includes input means for the user to input information such as a mouse, keyboard, touch panel, buttons, microphone, switch, and lever, and an input control circuit that generates an input signal based on the input by the user and outputs the input signal to the CPU 201. It is composed of such as. By operating the input device 208, the user of the voice processing device 20 can input various data to the voice processing device 20 and instruct the processing operation.

The output device 210 includes, for example, a liquid crystal display (LCD) device, an OLED (Organic Light Emitting Diode) device, and a display device such as a lamp. Further, the output device 210 includes an audio output device such as a speaker and headphones. For example, the display device displays an captured image, a generated image, or the like. On the other hand, the voice output device converts voice data and the like into voice and outputs the data.

The storage device 211 is a data storage device configured as an example of the storage unit of the voice processing device 20 according to the present embodiment. The storage device 211 may include a storage medium, a recording device that records data on the storage medium, a reading device that reads data from the storage medium, a deleting device that deletes the data recorded on the storage medium, and the like. The storage device 211 stores programs and various data executed by the CPU 201.

The drive 212 is a reader / writer for a storage medium, and is built in or externally attached to the voice processing device 20. The drive 212 reads the information recorded in the removable storage medium 24 such as the mounted magnetic disk, optical disk, magneto-optical disk, or semiconductor memory, and outputs the information to the RAM 203. The drive 212 can also write information to the removable storage medium 24.

The image pickup apparatus 213 includes an imaging optical system such as a photographing lens and a zoom lens that collects light, and a signal conversion element such as a CCD (Charge Coupled Device) or a CMOS (Complementary Metal Oxide Sensor). The imaging optical system collects the light emitted from the subject to form a subject image in the signal conversion unit, and the signal conversion element converts the formed subject image into an electrical image signal.

The communication device 215 is, for example, a communication interface composed of a communication device or the like for connecting to the network 12. Further, the communication device 215 may be a wireless LAN (Local Area Network) compatible communication device, an LTE (Long Term Evolution) compatible communication device, or a wire communication device that performs wired communication.

The network 12 is a wired or wireless transmission path for information transmitted from a device connected to the network 12. For example, the network 12 may include a public line network such as the Internet, a telephone line network, a satellite communication network, various LANs (Local Area Network) including Ethernet (registered trademark), and a WAN (Wide Area Network). Further, the network 12 may include a dedicated line network such as IP-VPN (Internet Protocol-Virtual Private Network).

<4. Supplement>
Although the preferred embodiments of the present disclosure have been described in detail with reference to the accompanying drawings, the technical scope of the present disclosure is not limited to such examples. It is clear that a person having ordinary knowledge in the technical field of the present disclosure can come up with various modifications or modifications within the scope of the technical ideas described in the claims. Of course, it is understood that the above also belongs to the technical scope of the present disclosure.

For example, each step in the processing of the voice processing device 20 of the present specification does not necessarily have to be processed in chronological order in the order described as a flowchart. For example, each step in the processing of the voice processing device 20 may be processed in an order different from the order described in the flowchart, or may be processed in parallel.

Further, some of the functions of the voice processing device 20 described above may be implemented in a cloud server connected to the voice processing device 20 via the network 12. For example, the cloud server may have a function corresponding to a voice recognition unit 234, a sound source direction estimation unit 236, a semantic analysis unit 238, a storage unit 240, and a response control unit 250, that is, a function as a voice processing device. In this case, the voice processing device 20 may transmit the voice signal to the cloud server, and the cloud server may store the information or transmit the response voice to the voice processing device 20.

Further, it is possible to create a computer program for causing the hardware such as the CPU, ROM, and RAM built in the voice processing device 20 to exhibit the same functions as each configuration of the voice processing device 20 described above. A storage medium for storing the computer program is also provided.

In addition, the effects described herein are merely explanatory or exemplary and are not limited. That is, the techniques according to the present disclosure may exhibit other effects apparent to those skilled in the art from the description herein, in addition to or in place of the above effects.

The following configurations also belong to the technical scope of the present disclosure.
(1)
With reference to the storage unit that stores the input information obtained based on the recognition of the voice input to the sound input unit, and based on the recognition of the situation associated with the input information, the input information is used. Response control unit that controls the response,
A voice processing device.
(2)
The input information includes a first part and a second part.
The response control unit is based on the fact that the first portion included in the input information stored in the storage unit is obtained by voice recognition and the situation associated with the input information is recognized. The voice processing device according to (1) above, wherein the response is controlled by using the second portion included in the input information.
(3)
In the storage unit, the sound source direction of the voice when the voice is input to the sound input unit is stored as status information in association with the input information.
The response control unit is associated with the sound source direction in the storage unit based on the recognition of the sound source direction in which the difference from the sound source direction stored as the situation information in the storage unit is less than a predetermined reference. The voice processing device according to (1) or (2), wherein the response is controlled by using the stored input information.
(4)
Target information indicating a target user is stored in the storage unit in association with the input information.
The response control unit receives the input information associated with the target information based on the recognition of the situation associated with the input information with respect to the target user indicated by the target information stored in the storage unit. The voice processing apparatus according to any one of (1) to (3) above, wherein the response is controlled by using the voice processing apparatus.
(5)
The input information includes a first part and a second part indicating the situation.
The storage unit stores a recognition model obtained by machine learning using the data obtained in the situation indicated by the first part.
The response control unit uses the recognition model stored in the storage unit to recognize the situation indicated by the first portion as a situation related to the first portion. The voice processing apparatus according to (1) above, wherein the response is controlled by using the portion 2.
(6)
When a voice indicating a situation recognized by the recognition model is input, the voice processing device additionally performs machine learning using the data obtained when the voice is input, and performs machine learning again to obtain a recognition model. The voice processing device according to (5) above, further comprising a learning unit to be regenerated.
(7)
The plurality of input information stored in the storage unit includes input information in which the first portion has ambiguity or the result of meaning recognition of the first portion is unknown. The voice processing device described in.
(8)
The voice processing device according to any one of (1) to (7) above, wherein the response control unit controls the output unit to output at least a part of the input information as the control of the response.
(9)
The response control unit controls the output unit to output the execution result of the process or action indicated by at least a part of the input information as the control of the response, any one of the above (1) to (7). The audio processing device described in.
(10)
The voice processing device is
A voice recognition unit that recognizes the voice input to the sound input unit,
A situational awareness unit that recognizes the situation and
The voice processing apparatus according to any one of (1) to (9) above.
(11)
The input information is based on the fact that the processor recognizes the situation associated with the input information by referring to the storage unit that stores the input information obtained based on the recognition of the voice input to the sound input unit. To control the response using
Speech processing methods, including.

20, 21, 22 Voice processing device 232 Sound collecting unit 234 Sound recognition unit 236 Sound source direction estimation unit 238 Semantic analysis unit 240 Storage unit 242 Storage unit 246 Learning unit 250 Response control unit 252 Response control unit 260 Voice output unit

Claims (11)

With reference to the storage unit that stores the input information obtained based on the recognition of the voice input to the sound input unit, and based on the recognition of the situation associated with the input information, the input information is used. Response control unit that controls the response,
A voice processing device. The input information includes a first part and a second part.
The response control unit is based on the fact that the first portion included in the input information stored in the storage unit is obtained by voice recognition and the situation associated with the input information is recognized. The voice processing apparatus according to claim 1, wherein the response is controlled by using the second portion included in the input information. In the storage unit, the sound source direction of the voice when the voice is input to the sound input unit is stored as status information in association with the input information.
The response control unit is associated with the sound source direction in the storage unit based on the recognition of the sound source direction in which the difference from the sound source direction stored as the situation information in the storage unit is less than a predetermined reference. The voice processing device according to claim 1, wherein the response is controlled by using the stored input information. Target information indicating a target user is stored in the storage unit in association with the input information.
The response control unit receives the input information associated with the target information based on the recognition of the situation associated with the input information with respect to the target user indicated by the target information stored in the storage unit. The voice processing apparatus according to claim 1, wherein the response is controlled by using the voice processing apparatus. The input information includes a first part and a second part indicating the situation.
The storage unit stores a recognition model obtained by machine learning using the data obtained in the situation indicated by the first part.
The response control unit uses the recognition model stored in the storage unit to recognize the situation indicated by the first portion as a situation related to the first portion. The voice processing apparatus according to claim 1, wherein the response is controlled by using the portion 2. When a voice indicating a situation recognized by the recognition model is input, the voice processing device additionally performs machine learning using the data obtained when the voice is input, and performs machine learning again to obtain a recognition model. The voice processing device according to claim 5, further comprising a learning unit to be regenerated. The plurality of input information stored in the storage unit includes input information in which the first portion has ambiguity or the result of meaning recognition of the first portion is unknown, according to claim 2. The voice processing device described. The voice processing device according to claim 1, wherein the response control unit controls the output unit to output at least a part of the input information as the control of the response. The voice processing device according to claim 1, wherein the response control unit controls the output unit to output the execution result of the process or action indicated by at least a part of the input information as the control of the response. The voice processing device is
A voice recognition unit that recognizes the voice input to the sound input unit,
A situational awareness unit that recognizes the situation and
The voice processing apparatus according to claim 1. The input information is based on the fact that the processor recognizes the situation associated with the input information by referring to the storage unit that stores the input information obtained based on the recognition of the voice input to the sound input unit. To control the response using
Speech processing methods, including.

Images