JP2022054667A - Voice dialogue device, voice dialogue system, and voice dialogue method - Google Patents

Abstract

To provide a voice dialogue device capable of improving the convenience of a user using the voice dialogue device, a voice dialogue system, and a voice dialogue method.SOLUTION: In a voice dialogue system 100, a control unit of a voice dialogue device includes: a detection unit that detects an utterance of a user; a determination unit that determines the possibility of continuation of the dialogue with the user when having detected the utterance of the user; a response processing unit that responds to the utterance of the user when the determination unit determines that the dialogue is being continued. The determination unit uses a domain of the dialogue with the user, which was done just before the detected user's utterance for determining the possibility of the continuation of the dialogue.SELECTED DRAWING: Figure 1

Description

The present invention relates to a voice dialogue device, a voice dialogue system, and a voice dialogue method.

Conventionally, there is known a voice dialogue system that performs a voice dialogue with a user by outputting a response corresponding to a word input by the user (see, for example, Patent Document 1).

In a configuration in which voice recognition and intention understanding are always performed for the user's utterance, the processing load of the system tends to be large. For this reason, many of the current voice dialogue systems are configured such that the dialogue is started when the user presses a button, or the dialogue is started when a predetermined word called a wake word is detected.

Japanese Unexamined Patent Publication No. 2018-109663

In a configuration in which a dialogue is started by pressing a button or uttering a wake word, the user may find it difficult to use because the button must be pressed or the wake word must be spoken for each requirement.

In view of the above points, it is an object of the present invention to provide a technique capable of improving the convenience of a user who uses a voice dialogue device.

In order to achieve the above object, the voice dialogue device of the present invention includes a detection unit that detects the utterance of the user and a determination unit that determines the possibility of continuing the dialogue with the user when the utterance of the user is detected. A response processing unit that responds to the user's utterance when it is determined by the determination unit that the dialogue is continuing is provided, and the determination unit detects the possibility of continuation of the dialogue. It is configured to use the domain of the dialogue with the user performed immediately before the user's utterance (first configuration).

In the voice dialogue device of the first configuration, the determination unit further utilizes the response state of the own device in the dialogue with the user to determine the possibility of continuation of the dialogue (second configuration). It's okay.

In the voice dialogue device of the first or second configuration, the determination unit further utilizes the characteristics of the voice extracted from the voice of the user in determining the possibility of continuation of the dialogue (third configuration). ) May be.

In the voice dialogue device having any of the first to third configurations, the determination unit further uses the information obtained from the captured image of the user to determine the possibility of continuation of the dialogue (fourth). Configuration).

In the voice dialogue device having any of the first to fourth configurations, the determination unit detects the utterance of the user within a certain period of time after the end of the dialogue with the user immediately before the end. , The configuration (fifth configuration) for determining the possibility of continuation of the dialogue may be used.

In the voice dialogue device having any of the first to fifth configurations, the determination unit compares a numerical value indicating the possibility of continuation of the dialogue with a threshold value to determine whether or not the dialogue with the user is continuing. The threshold value may be changed according to the elapsed time from the end of the dialogue with the user performed immediately before (sixth configuration).

In the voice dialogue device having any of the first to sixth configurations, the determination unit compares a numerical value indicating the possibility of continuation of the dialogue with a threshold value to determine whether or not the dialogue with the user is continuing. The determination unit changes the threshold value when the user's utterance is detected at a specific timing, so that it is easy to determine that the dialogue is continuing (seventh configuration). May be.

In the voice dialogue device having any of the first to seventh configurations, the determination unit detects the utterance of the user, and the dialogue is performed when the user does not perform a predetermined dialogue start operation. It may be a configuration (eighth configuration) for determining the possibility of continuation.

In order to achieve the above object, the voice dialogue system of the present invention converts the voice of the user into a voice signal and outputs it to the voice dialogue device and the voice dialogue device having any one of the first to eighth configurations. The configuration (9th configuration) includes a microphone that performs sound, and a speaker that converts a voice signal output from the voice dialogue device into voice and emits sound to the user.

The voice dialogue system having the ninth configuration may be configured to further include a camera that captures the user and outputs the information of the captured image to the voice dialogue device (tenth configuration).

In order to achieve the above object, the voice dialogue method of the present invention is a voice dialogue method in a voice dialogue device, in which a detection step of detecting a user's utterance and a detection step of detecting the user's utterance with the user are performed. A determination step for determining the possibility of continuation of the dialogue and a response processing step for responding to the utterance of the user when it is determined by the determination step that the dialogue is continuing are provided, and the possibility of continuation of the dialogue is provided. In the determination, the domain of the dialogue with the user performed immediately before the utterance of the detected user is used (11th configuration).

According to the present invention, it is possible to improve the convenience of the user who uses the voice dialogue device.

Block diagram showing the configuration of a voice dialogue system A block diagram showing the functions of the control unit of the voice dialogue device. Schematic diagram for explaining the outline of the operation of the voice dialogue device A flowchart showing an operation example in the utterance temporary reception state of the voice dialogue device. A flowchart showing a modified example of the operation of the voice dialogue device in the utterance temporary reception state. A flowchart showing a first detailed example of the threshold value adjustment process shown in FIG. A flowchart showing a second detailed example of the threshold value adjustment process shown in FIG.

Hereinafter, exemplary embodiments of the present invention will be described in detail with reference to the drawings.

<1. Voice Dialogue System>
FIG. 1 is a block diagram showing a configuration of a voice dialogue system 100 according to an embodiment of the present invention. The voice dialogue system 100 of the present embodiment is applied to a vehicle as an example. Vehicles may broadly include vehicles with wheels, such as automobiles and trains. The voice dialogue system 100 of the present invention may be applied to a moving body such as a ship or an aircraft on which a person other than a vehicle is placed, or a moving body other than a moving body such as a house or a facility.

As shown in FIG. 1, the voice dialogue system 100 includes a voice dialogue device 1, a microphone 2, and a speaker 3. Further, the voice dialogue system 100 further includes a camera 4 and a server device 5. The voice dialogue system 100 does not have to include at least one of the camera 4 and the server device 5.

The voice dialogue device 1 is a vehicle dialogue device arranged at an appropriate position in the vehicle. The voice dialogue device 1 is a device that appropriately responds to a user's utterance. In the present embodiment, the user is a occupant of a vehicle such as a driver. The response includes a voice response to the user's utterance. Further, the response may include an answer using a means other than voice, such as displaying the answer to the user's utterance on the screen. In the case of a configuration in which the answer to the user's utterance is displayed on the screen, the voice dialogue system 100 includes a monitor. The term "dialogue" as used herein includes not only a voice interaction between a user and a device but also a case where the device responds to a voice question or instruction from the user by means other than voice. It's okay.

As a general rule, the voice dialogue device 1 of the present embodiment understands the intention of the user's utterance when detecting the wake word uttered by the user, and responds according to the user's utterance. The voice dialogue device 1 is based on one question and one answer, and in principle, it is necessary to utter a wake word for each requirement. However, when the voice dialogue device 1 asks the user back, the voice dialogue device 1 responds without detecting the wake word. Further, the voice dialogue device 1 responds under specific conditions without detecting the way word. The details of this point will be described later.

The wake word is a predetermined word that triggers a dialogue (interaction) with the user. A user who wants to start a dialogue with the voice dialogue device 1 issues a wake word as a signal to start the dialogue. As a result, the voice dialogue device 1 understands the intention of the utterance uttered by the user after the wake word and responds. The wake word may be, for example, "hello, my computer", "hey, vehicle" or the like.

Further, in the present embodiment, the dialogue between the user and the voice dialogue device 1 is started by the detection of the wake word, which is an example. Instead of detecting the wake word, for example, the dialogue may be started by detecting that the button is pressed. Further, the detection of the wake word and the use of the button may be used together.

The microphone 2 collects the voice generated by the user. The microphone 2 is placed in place on the vehicle. The microphone 2 is connected to the voice dialogue device 1 by wire or wirelessly. The microphone 2 converts the user's voice into a voice signal and outputs it to the voice dialogue device 1. The microphone 2 may be included in the voice dialogue device 1.

The speaker 3 is connected to the voice dialogue device 1 by wire or wirelessly. The speaker 3 converts the voice signal output from the voice dialogue device 1 into voice and emits the sound to the user. The speaker 3 is also arranged at an appropriate position in the vehicle like the microphone 2. The speaker 3 may be included in the voice dialogue device 1.

The camera 4 is connected to the voice dialogue device 1 by wire or wirelessly. The camera 4 photographs the user and outputs the information of the captured image to the voice dialogue device 1. The camera 4 is arranged at an appropriate position in the vehicle so that the entire user sitting in the seat of the vehicle can be photographed, for example. Further, for example, the camera 4 is arranged at an appropriate position in the vehicle so that the face of the user sitting in the seat of the vehicle can be photographed. The camera 4 may be included in the voice dialogue device 1.

The server device 5 is a computer device connected to a network such as the Internet. The server device 5 of the present embodiment includes artificial intelligence (AI). The server device 5 can receive various information from any other computer device connected to the network. The voice dialogue device 1 can exchange information with the server device 5 via a network.

In the voice dialogue system 100 of the present embodiment, as will be described in detail later, the voice dialogue device 1 responds to the user's utterance without issuing a wake word under specific conditions, which is convenient for the user. Can be improved. Further, in the present embodiment, since the voice dialogue device 1 can make a judgment related to the dialogue with the user in consideration of the image information of the user acquired from the camera 4, the user can more interact with the human. It is possible to have a dialogue with the voice dialogue device 1 with a feeling close to that of.

<2. Voice dialogue device>
Next, the voice dialogue device 1 will be described in detail. As shown in FIG. 1, the voice dialogue device 1 includes a detection unit 11, an image processing unit 12, a control unit 13, a storage unit 14, and a communication unit 15.

The detection unit 11 inputs an audio signal from the microphone 2. The detection unit 11 detects the user's utterance. In the present embodiment, the detection unit 11 not only detects the user's utterance but also recognizes the user's utterance by voice. Hereinafter, the detection unit 11 will be referred to as a voice recognition unit 11.

The voice recognition unit 11 is composed of a semiconductor integrated circuit. The voice recognition unit 11 is composed of, for example, an AI chip. The voice recognition unit 11 detects the user's utterance based on the input voice signal. The voice recognition unit 11 converts the detected user's spoken voice into text data (character string data), and extracts the characteristics of the voice. Audio features may include, for example, volume, pitch, intonation, and the like. The voice recognition unit 11 is connected to the control unit 13. The voice recognition unit 11 outputs the text data obtained by the conversion and the information related to the voice including the data indicating the characteristics of the voice to the control unit 13.

The image processing unit 12 is input with the data of the image taken by the camera 4. The image processing unit 12 is composed of a semiconductor integrated circuit. The image processing unit 12 is composed of, for example, an AI chip. The image processing unit 12 extracts features related to the user's behavior (behavior) from the input image data. Features related to the behavior of the user may include, for example, a change in the posture of the user, a change in the orientation of the face (specifically, rotation of the face and vertical movement of the face) and the like. The image processing unit 12 is connected to the control unit 13. The image processing unit 12 outputs captured image information including data showing features related to the behavior of the user to the control unit 13.

The control unit 13 is a controller that comprehensively controls the entire voice dialogue device 1. The control unit 13 may be, for example, a computer including a CPU (Central Processing Unit). Various functions realized by the control unit 13 are realized by the computer executing arithmetic processing according to a program stored in the storage unit 14.

The storage unit 14 is composed of, for example, a semiconductor memory element such as a RAM (Random Access Memory) or a flash memory, a hard disk, or a storage device using a portable recording medium such as an optical disk. The storage unit 14 stores a program as firmware and various data.

The communication unit 15 is connected to the control unit 13. The communication unit 15 is connected to the server device 5 via a network using wireless communication, and performs bidirectional communication with the server device 5. That is, the control unit 13 can exchange information with the server device 5 by using the communication unit 15.

FIG. 2 is a block diagram showing a function of the control unit 13 included in the voice dialogue device 1 according to the embodiment of the present invention. The control unit 13 includes a wake word detection unit 131, a response processing unit 132, and a determination unit 133 as functions that the computer constituting the control unit 13 exerts by performing arithmetic processing according to a program. In other words, the voice dialogue device 1 includes a wake word detection unit 131, a response processing unit 132, and a determination unit 133. When the button is used instead of the wake word, the wake word detection unit 131 is unnecessary.

Even if at least one of the wake word detection unit 131, the response processing unit 132, and the judgment unit 133 is configured by hardware such as an ASIC (Application Specific Integrated Circuit) or an FPGA (Field Programmable Gate Array). good. Further, the wake word detection unit 131, the response processing unit 132, and the determination unit 133 are conceptual components. The functions executed by one component may be distributed to a plurality of components, or the functions of the plurality of components may be integrated into one component. Further, at least a part of the functions of the above-mentioned voice recognition unit 11 and at least a part of the functions of the image processing unit 12 may be included in the functions of the control unit 13.

The wake word detection unit 131 detects that the user has issued a wake word when the user's utterance text data includes the wake word. Upon detecting the wake word, as will be described later, the dialogue function of the voice dialogue device 1 is activated, and the voice dialogue device 1 responds to the user's question or command.

When the response processing unit 132 detects that the user has uttered a wake word, the response processing unit 132 performs response processing in response to the utterance following the user's wake word. The response process includes, for example, a process of decoding a user's uttered voice, a solution process of resolving a user's request understood by decoding the voice, and a transmission process of transmitting the result of the solution process to the user. The decoding process includes, for example, an analysis process of the user's utterance intention. The resolution process includes, for example, a search process using the Internet. The transmission process includes, for example, a voice response and a display response. As described above, in the present specification, not only the voice response but also the display response is an element constituting the dialogue.

All of the various processes included in the response process may be performed by the response process unit 132, but in the present embodiment, a part of the process is performed by the server device 5. When the wake word is detected, the response processing unit 132 transmits the voice signal of the user's utterance to the server device 5 via the communication unit 15. The server device 5 generates result data for responding to the user's request by performing detailed voice recognition processing, natural language processing, and the like on the received voice signal, and transmits the result data to the voice dialogue device 1. The response processing unit 132 makes a response to the user based on the received result data.

The determination unit 133 determines the possibility of continuing the dialogue with the user when the user's utterance is detected. The determination unit 133 uses the domain of the dialogue with the user performed immediately before the utterance of the user detected by the detection unit 11 in determining the possibility of continuing the dialogue. In detail, the dialogue with the user means the dialogue between the user and the own device 1. The domain of dialogue is determined in consideration of the intention of the user and the utterance of the own device 1.

The domain of the dialogue is, in detail, the topic of the dialogue. For example, when the user asks the voice dialogue device 1 about "today's weather", the domain of the dialogue is "weather". Further, for example, when the user asks a question about "news with the highest access ranking today", the domain of the dialogue is "news". Depending on the domain of the dialogue, the possibility of continuing the dialogue with the user tends to differ. For example, if the domain of the dialogue is "weather", the possibility of continuation of the dialogue tends to be high, and if the domain of the dialogue is "news", the possibility of continuation of the dialogue tends to be low. Therefore, by using the domain of the dialogue, it becomes possible to appropriately judge the possibility of continuing the dialogue.

In the present embodiment, in detail, the determination unit 133 determines the possibility of continuation of the dialogue when the user's utterance is detected and the predetermined dialogue start operation by the user is not performed. With such a configuration, it is not necessary to uselessly determine the possibility of continuation of dialogue, and the processing load on the voice dialogue device 1 can be reduced. The predetermined dialogue start operation is, in the present embodiment, the utterance of a wake word by the user. The predetermined dialogue start operation may be, for example, pressing a button instead of uttering a wake word.

In the present embodiment, more specifically, the determination unit 133 does not determine the possibility of continuing the user dialogue even when the own device 1 is asking the user a question. As a result, it is not necessary to uselessly determine the possibility of continuing the dialogue, and the processing load on the voice dialogue device 1 can be further reduced.

In the present embodiment, the response processing unit 132 responds to the user's utterance when it is determined by the determination unit 133 that the dialogue (dialogue between the user and the own device 1) is continuing. According to this, it is not always necessary to issue a wake word each time an utterance is made, and the convenience of the user can be improved. Further, the user can interact with the voice dialogue device 1 with a feeling close to that of a dialogue with a human.

FIG. 3 is a schematic diagram for explaining an outline of the operation of the voice dialogue device 1 according to the embodiment of the present invention. As shown in FIG. 3, when the voice dialogue device 1 detects the wake word, it is in the utterance acceptance state of accepting the utterance of the user 6. The utterance acceptance state is a state in which the own device 1 decides to respond to the utterance of the user 6. The voice dialogue device 1 performs response processing to the utterance of the user 6 following the wake word (utterance A in FIG. 3). The response process includes a process of decoding the user's utterance and a process of resolving the decoded user's request (search process in the example of FIG. 3) performed by the server device 5. The voice dialogue device 1 makes a voice response or a display response to the search result. The voice dialogue device 1 recognizes that the dialogue with the user 6 is completed when the response processing is completed. When the voice dialogue device 1 asks the user back, the voice dialogue device 1 recognizes that the dialogue with the user 6 is continuing, and is in the utterance acceptance state.

In the present embodiment, as a characteristic configuration, even after it is recognized that the dialogue with the user 6 is completed, the utterance provisional reception state is set in which the utterance of the user 6 is tentatively accepted only for a certain period of time. The utterance provisional reception state is a state in which the user does not always respond to all the utterances of the user, but responds to the user only when a specific condition is satisfied. In the utterance provisional reception state, the above-mentioned determination unit 133 determines the possibility of continuing the dialogue, and determines whether or not to respond to the user's utterance based on the result of the determination.

In the example shown in FIG. 3, the user 6 is performing the utterance B during the period of the utterance provisional reception state. For this purpose, the determination unit 133 determines the domain of the immediately preceding dialogue (dialogue related to the utterance A) by using the detection of the utterance B as a trigger. The determination unit 133 determines the possibility of continuing the dialogue with the user by using the domain of the determined dialogue. Then, when the determination unit 133 determines that the dialogue is continuing, the response processing unit 132 responds to the user 6. If the determination unit 133 determines that the dialogue is not continuing, the response processing unit 132 does not respond to the user 6. That is, the processing itself of the response processing unit 132 is not performed.

In the present embodiment, the determination unit 133 determines the possibility of continuing the dialogue when the utterance of the user 6 is detected within a certain time after the end of the dialogue with the user 6 performed immediately before. ing. According to such a configuration, it is not always necessary to perform the processing required to respond to the user 6, and the processing load on the voice dialogue device 1 can be reduced. Further, it is possible to reduce the possibility of a malfunction such as responding to the user even though the user 6 has not requested it.

In the present embodiment, as a preferred embodiment, the determination unit 133 further utilizes the characteristics of the voice extracted from the voice of the user 6 in determining the possibility of continuing the dialogue. Audio features may include, for example, volume, pitch, intonation, and the like. For example, when the utterance of the user 6 is an utterance toward the device 1, the intonation tends to be reduced. For example, the determination unit 133 determines that there is a high possibility that the dialogue will continue if the inflection of the user's utterance (speech B in the example of FIG. 3) detected in the utterance provisional reception state is small. With such a configuration, it is possible to judge the possibility of continuing the dialogue by adding not only the domain of the dialogue but also other indicators, so that the utterance of the user 6 is directed to the own device 1. It becomes possible to judge whether or not it is more appropriate.

Further, in the present embodiment, as a preferred embodiment, the determination unit 133 further uses the information obtained from the captured image of the user 6 for determining the possibility of continuing the dialogue. Specifically, the judgment unit 133 determines the possibility of continuing the dialogue by using the characteristics related to the behavior (behavior) of the user who has made an utterance after the utterance provisional reception state is reached. The characteristics related to the behavior of the user may include, for example, a change in the posture of the user, a change in the orientation of the face, and the like. For example, the user 6 tends to glance at the microphone 2 (or device 1) when speaking to the device 1. Therefore, if it is determined that the user faces the device 1 or the microphone 2 from changes in the posture or face orientation of the user, it can be determined that there is a high possibility that the dialogue will continue. With such a configuration, it is possible to judge the possibility of continuing the dialogue by adding not only the domain of the dialogue but also other indicators, so that the utterance of the user 6 is directed to the own device 1. It becomes possible to judge whether or not it is more appropriate.

In the utterance provisional reception state, the voice dialogue device 1 of the present embodiment can continue the dialogue by using a plurality of information such as the domain of the immediately preceding dialogue, the information related to the voice characteristics of the user 6, and the image information of the user 6. The sex is judged, and based on this, the user 6 may or may not respond. Hereinafter, a detailed example of the operation of the voice dialogue device 1 in the temporary reception state will be described with reference to FIG. Note that FIG. 4 is a flowchart showing an operation example of the voice dialogue device 1 in the utterance provisional reception state.

In step S1, the determination unit 133 determines whether or not it is within a certain period of time after the dialogue with the user 6 is completed. The fixed time is, for example, 60 seconds. If the determination unit 133 is out of a certain period of time (No in step S1), the determination unit 133 ends the process. On the other hand, if the determination unit 133 is within a certain period of time (Yes in step S1), the determination unit 133 proceeds to the next step S2.

In step S2, the determination unit 133 determines whether or not the utterance of the user 6 has been detected. The voice recognition unit 11 detects the utterance of the user 6. When the utterance of the user 6 is detected (Yes in step S2), the determination unit 133 proceeds to the next step S3. On the other hand, when the utterance of the user 6 is not detected (No in step S2), the determination unit 133 returns the process to step S1.

In step S3, the determination unit 133 determines whether or not the wake word is included at the beginning of the utterance of the detected user 6. The wake word is detected by the wake word detection unit 131 in which the spoken text data is input from the voice recognition unit 11. If a wake word is included (Yes in step S3), the determination unit 133 proceeds to step S7. On the other hand, if the wake word is not included (No in step S3), the determination unit 133 proceeds to the next step S4.

In step S4, the determination unit 133 calculates a determination value indicating the possibility of continuation of the dialogue. The determination value becomes a high value when the possibility of continuation of dialogue is high, and becomes a low value when the possibility of continuation of dialogue is low. The determination value may be, for example, a number expressed as a percentage. In this case, the configuration may be such that when the possibility of continuation of dialogue is high, it approaches 100%, and when the possibility of continuation of dialogue is low, it approaches 0%. The determination unit 133 obtains a determination value using a trained model (neural network) that has undergone machine learning by, for example, a method such as deep learning. A determination value is obtained by inputting a predetermined feature amount to the trained model. The trained model includes at least one feature amount (inflection, etc.) indicating the characteristics of the utterance voice of the user 6 who is receiving the utterance provisional reception recognized by the voice recognition unit 11, and the behavior before and after the utterance of the user 6 who is receiving the utterance provisional reception. At least one feature amount (such as the rotation speed of the face) indicating the feature of is input. The feature amount indicating the feature of the user's behavior is obtained from the image processing unit 12. When the determination unit 133 obtains the determination value, the determination unit 133 proceeds to the next step S5.

In step S5, the determination unit 133 acquires the coefficient with the aim of making the previously determined determination value a more reliable value. When determining the coefficient, the determination unit 133 determines the domain of the dialogue between the own device 1 and the user 6 performed immediately before the utterance of the user 6 who is temporarily accepting the utterance. The domain of the dialogue may be obtained by the processing of the determination unit 133 itself, or may be obtained by the server device 5 that decodes the intention of the user 6's utterance. The server device 5 may be configured to classify the domain of dialogue each time the user 6's utterance is analyzed, and transmit the classified domain to the voice dialogue device 1. The voice dialogue device 1 may be configured to store the domain of the dialogue with the user 6 sent from the server device 5 in the storage unit 14.

The determination unit 133 obtains a coefficient related to the possibility of continuation of the dialogue from the domain of the determined dialogue. The coefficient may be, for example, a numerical value predetermined for each domain of dialogue, and may be stored in the storage unit 14 in advance. The coefficient is, for example, a value greater than zero and less than or equal to 1.0, and as a rule of thumb, a large coefficient value is assigned to the domain of the dialogue in which the dialogue is likely to continue, and the dialogue is continued. Small coefficient values are assigned to domains of dialogue that are unlikely. For example, in FIG. 3, when the utterance A is "What is the weather today?", Even after the voice dialogue device 1 responds, the user 6 makes a utterance such as "Then, what is tomorrow?" And continues the dialogue. Tend. For this reason, if the domain of the dialogue is the weather, the coefficient will be a large value. For example, if the domain of dialogue is weather, the factor may be 1.0. Although the case where the coefficient does not exceed 1.0 is illustrated here, the coefficient may be a number exceeding 1.0. When the determination unit 133 obtains the coefficient, the process proceeds to the next step S6.

In step S6, the determination unit 133 calculates the correction determination value by multiplying the determination value obtained in step S4 by the coefficient obtained in step S5. Thereby, the possibility of continuation of the dialogue can be evaluated by using the information of the spoken voice of the user 6, the image information of the user 6, and the domain of the immediately preceding dialogue. When the determination unit 133 calculates the correction determination value, the process proceeds to the next step S7.

In step S7, the determination unit 133 compares the correction determination value obtained in step S6 with the threshold value prepared in advance. The determination unit 133 compares the numerical value indicating the possibility of continuing the dialogue with the threshold value and determines whether or not the dialogue with the user 6 is continuing. When the correction determination value is equal to or greater than the threshold value (Yes in step S7), the determination unit 133 determines that there is a high possibility that the dialogue will continue and that the dialogue with the user 6 is continuing. In this case, the process proceeds to step S8. On the other hand, when the correction determination value is smaller than the threshold value (No in step S7), the determination unit 133 determines that the possibility of continuation of the dialogue is low and that the utterance of the user 6 is not directed to the own device 1. In this case, the process proceeds to step S9.

In step S8, the response processing unit 132 performs response processing to the utterance of the user 6 detected during the utterance provisional reception. In step S9, the response processing unit 132 decides to leave the utterance of the user 6 detected during the utterance provisional reception. Once the decision in step S9 is made, the response processing by the response processing unit 132 cannot proceed. That is, the voice dialogue device 1 does not respond to the user's utterance.

According to the configuration of the present embodiment, the user 6 can continue the dialogue without issuing a wake word within a certain period of time after the dialogue with the voice dialogue device 1 is completed, so that the convenience of the user is enhanced. .. Further, the voice dialogue device 1 uses a plurality of information such as voice information, image information, and the content of the dialogue to determine whether or not the user's utterance is directed to the own device 1 and respond. In order to do so, it is possible to realize a dialogue with a feeling close to that of a dialogue with humans.

When obtaining the determination value or acquiring the coefficient, the determination unit 133 may consider the response state of the own device 1 in the dialogue with the user immediately before. That is, the determination unit 133 may further utilize the response state of the own device 1 in the dialogue with the user 6 that was performed immediately before to determine the possibility of continuing the dialogue. The voice dialogue device 1 may succeed in responding to the utterance of the user 6 or may fail. When the response of the user 6 fails, it includes the case where the user 6's intention of utterance cannot be understood, or the case where the user 6's intention of utterance can be understood but the search result cannot be obtained. It was. ”And so on.

If the response fails in the immediately preceding dialogue, the user is more likely to speak to the voice dialogue device 1. For this reason, if the own device 1 fails to respond in the immediately preceding dialogue, the processing may be performed so that the determination value and the coefficient value become large. With such a configuration, it is possible to further increase the index for judging the possibility of continuing the dialogue, and to more appropriately judge whether or not the utterance of the user 6 is directed to the own device 1. Become.

In addition, within a certain period of time after the end of the dialogue (temporary utterance reception state), there is a possibility that the dialogue will continue even if the wake word is not issued. The voice dialogue device 1 or the voice dialogue system 100 may be configured to notify the user 6 of such a state. The means of notification may be, for example, image display, light emission, or the like. With such a configuration, it is possible to prevent the user 6 from unnecessarily saying a wake word.

<3. Modification example>
FIG. 5 is a flowchart showing a modified example of the operation of the voice dialogue device 1 in the utterance temporary reception state. The flowchart shown in FIG. 5 is different from the flowchart of FIG. 4 in that the process of step S10 is added between steps S6 and S7 of the flowchart shown in FIG. In step S10, a process of adjusting the threshold value is performed. That is, in the modified example, the threshold value to be compared with the determination value (correctly, the correction determination value in this example) is not a constant value but is appropriately changed. Below, two examples (first modification and second modification) of the configuration in which the threshold value is appropriately changed are shown.

(3-1. First modification)
In the first modification, the threshold value is changed according to the elapsed time from the end of the immediately preceding dialogue with the user. The time point at which the dialogue with the user ends coincides with the time point at which the above-mentioned utterance provisional reception state starts.

FIG. 6 is a flowchart showing a first detailed example of the threshold value adjustment process shown in FIG. In step S101, the determination unit 133 determines whether or not it is within the first hour from the end of the dialogue with the user. The first time is shorter than the fixed time in step S1 shown in FIG. If the determination unit 133 is within the first hour (Yes in step S101), the determination unit 133 proceeds to the next step S102. On the other hand, when the determination unit 133 is outside the first time (No in step S101), the determination unit 133 proceeds to the process in step S103.

In step S102, the determination unit 133 maintains the threshold value as the initial value. When the determination unit 133 completes the process of step S102, the determination unit 133 performs the process of step S7 shown in FIG. That is, the threshold value maintained at the initial value is compared with the correction determination value, and it is determined whether or not the dialogue is continued.

In step S103, the determination unit 133 determines whether or not it is within the second time from the end of the dialogue with the user. The second time is shorter than the fixed time of step S1 shown in FIG. 5 and longer than the first time. If the determination unit 133 is within the second time (Yes in step S103), the determination unit 133 proceeds to the next step S104. On the other hand, if the determination unit 133 is outside the first time (No in step S103), the determination unit 133 proceeds to the process in step S105.

In step S104, the determination unit 133 changes the threshold value from the initial value to the first threshold value. The first threshold value is a value larger than the initial value. When the determination unit 133 completes the process of step S104, the determination unit 133 performs the process of step S7 shown in FIG. That is, the threshold value (first threshold value) larger than the initial value is compared with the correction determination value, and it is determined whether or not the dialogue is continued. Since the first threshold value is larger than the initial value, it becomes difficult to determine that the dialogue is continuing by changing to the first threshold value.

In step S105, the determination unit 133 changes the threshold value from the initial value to the second threshold value. The second threshold value is a value larger than the first threshold value. When the determination unit 133 completes the process of step S105, the determination unit 133 performs the process of step S7 shown in FIG. That is, the initial value and the threshold value larger than the first threshold value (second threshold value) are compared with the correction determination value, and it is determined whether or not the dialogue is continued. The change to the second threshold makes it more difficult to determine that the dialogue is continuing.

According to the configuration of this modification, there is a possibility that it is determined that the dialogue continues as the elapsed time from the end of the dialogue with the user 6 increases due to the stepwise change of the threshold value. It becomes low. The longer the elapsed time since the end of the dialogue with the user 6, the less likely it is that the dialogue will continue. Therefore, according to the configuration of this modification, it is possible to reduce the possibility of a malfunction such as responding to the user 6 even though the user 6 has not requested it.

In this modification, the threshold value is prepared in three stages, but this is only an example. The number of variable thresholds may be changed as appropriate.

(3-2. Second modification)
In the second modification, the determination unit 133 changes the threshold value when the utterance of the user 6 is detected at a specific timing, so that it is easy to determine that the dialogue is continuing. The specific timing is a timing at which the user 6 has a high tendency to continue the dialogue, and is a timing determined by an empirical rule or the like. The specific timing corresponds to, for example, after the own device 1 fails to respond to the user 6, or after a dialogue belonging to a domain in which the dialogue is likely to continue.

FIG. 7 is a flowchart showing a second detailed example of the threshold value adjustment process shown in FIG. In step S106, the determination unit 133 determines whether or not the detection of the user's utterance is at a specific timing. When the determination unit 133 corresponds to a specific timing (Yes in step S106), the determination unit 133 proceeds to the next step S107. On the other hand, if the determination unit 133 does not correspond to the specific timing (No in step S106), the determination unit 133 proceeds to the process in step S108.

In step S107, the determination unit 133 changes the threshold value from the initial value to the third threshold value. The third threshold value is smaller than the initial value. When the determination unit 133 completes the process of step S107, the determination unit 133 performs the process of step S7 shown in FIG. That is, the threshold value (third threshold value) smaller than the initial value is compared with the correction determination value, and it is determined whether or not the dialogue is continued. Since the third threshold value is smaller than the initial value, it is easy to determine that the dialogue is continuing by changing to the third threshold value.

In step S108, the determination unit 133 maintains the threshold value as the initial value. When the determination unit 133 completes the process of step S108, the determination unit 133 performs the process of step S7 shown in FIG. That is, the threshold value maintained at the initial value is compared with the correction determination value, and it is determined whether or not the dialogue is continued.

According to the configuration of this modification, when the dialogue with the user is likely to continue, it is easy for the voice dialogue device 1 to determine that the dialogue is continuing, and it is appropriate for the user 6 to speak. Can be accommodated.

<4. Points to note>
The various technical features disclosed herein can be modified in addition to the above embodiments without departing from the gist of the technical creation. That is, it should be considered that the embodiments are exemplary in all respects and are not restrictive, and the technical scope of the invention is not the description of the embodiments but the claims. It is shown and should be understood to include all modifications that fall within the meaning and scope of the claims. In addition, a plurality of embodiments and modifications shown in the present specification may be appropriately combined and implemented to the extent possible.

In the above, the voice dialogue device 1 is configured to determine the possibility of continuation of the dialogue by using a plurality of information such as the domain of the immediately preceding dialogue, the information of the spoken voice of the user 6, and the image information of the user 6. Is just an example. For example, the voice dialogue device may be configured to determine the possibility of continuation of dialogue by using only the domain of the immediately preceding dialogue. For example, a numerical value indicating the possibility of continuing the dialogue may be obtained from the domain of the immediately preceding dialogue. Further, for example, the voice dialogue device may be configured to determine the possibility of continuation of the dialogue by using only the domain of the immediately preceding dialogue and the information of the spoken voice of the user. Further, for example, the voice dialogue device may be configured to determine the possibility of continuation of the dialogue by using only the domain of the immediately preceding dialogue and the image information of the user.

Further, instead of correcting the determination value depending on the domain of the dialogue, the threshold value to be compared with the determination value may be changed depending on the domain of the dialogue. This configuration is also included in the configuration for determining the possibility of continuing the dialogue using the domain of the dialogue.

1 ... Voice dialogue device 2 ... Microphone 3 ... Speaker 4 ... Camera 6 ... User 11 ... Detection unit 100 ... Voice dialogue system 132 ... Response processing unit 133 ...・ Judgment department

Claims (11)

A detector that detects the user's utterance,
When the utterance of the user is detected, a judgment unit for determining the possibility of continuing the dialogue with the user, and a judgment unit.
A response processing unit that responds to the user's utterance when it is determined by the determination unit that the dialogue is continuing.
Equipped with
The determination unit is a voice dialogue device that uses the domain of the dialogue with the user performed immediately before the detected utterance of the user to determine the possibility of continuation of the dialogue. The voice dialogue device according to claim 1, wherein the determination unit further utilizes the response state of the own device in the dialogue with the user in determining the possibility of continuation of the dialogue. The voice dialogue device according to claim 1 or 2, wherein the determination unit further utilizes the characteristics of the voice extracted from the voice of the user in determining the possibility of continuation of the dialogue. The voice dialogue device according to any one of claims 1 to 3, wherein the determination unit further utilizes information obtained from the captured image of the user in determining the possibility of continuation of the dialogue. 3. The voice dialogue device according to any one of the following items. The determination unit compares a numerical value indicating the possibility of continuation of the dialogue with a threshold value to determine whether or not the dialogue with the user is continuing.
The voice dialogue device according to any one of claims 1 to 5, wherein the threshold value is changed according to the elapsed time from the end of the dialogue with the user performed immediately before. The determination unit compares a numerical value indicating the possibility of continuation of the dialogue with a threshold value to determine whether or not the dialogue with the user is continuing.
The determination unit changes the threshold value when the user's utterance is detected at a specific timing, so that it is easy to determine that the dialogue is continuing, any one of claims 1 to 6. The voice dialogue device described in. Any one of claims 1 to 7, wherein the determination unit detects the utterance of the user and determines the possibility of continuing the dialogue when the user has not performed a predetermined dialogue start operation. The voice dialogue device described in the section. The voice dialogue device according to any one of claims 1 to 8.
A microphone that converts the user's voice into a voice signal and outputs it to the voice dialogue device.
A speaker that converts a voice signal output from the voice dialogue device into voice and emits sound to the user.
A voice dialogue system. The voice dialogue system according to claim 9, further comprising a camera that shoots the user and outputs information of the shot image to the voice dialogue device. It is a voice dialogue method in a voice dialogue device.
A detection process that detects the user's utterance,
A determination step for determining the possibility of continuing a dialogue with the user when the user's utterance is detected, and
A response processing step that responds to the user's utterance when it is determined by the determination step that the dialogue is continuing.
Equipped with
A voice dialogue method in which the domain of the dialogue with the user performed immediately before the detected speech of the user is used to determine the possibility of continuation of the dialogue.

Images