JP2021099378A - Controller - Google Patents

Abstract

To provide a technique for achieving appropriate operation based on instruction voice even in a situation where a plurality of apparatuses exist as candidates of an instruction object and in a situation where only one apparatus exists as the candidate of the instruction object.SOLUTION: A controller comprises: a first estimation part for estimating an object apparatus of an instruction object from a plurality of candidate apparatuses on the basis of a prescribed first condition when the plurality of candidate apparatuses are set in a case where a voice reception part which can receive voice receives the instruction voice for allowing the apparatus to execute specific operation; a determination part for determining one candidate apparatus as the object apparatus regardless of the first condition when only one candidate apparatus is set in a case where the voice reception part receives the instruction voice; and a first operation relation processing execution part for executing first operation relation processing for allowing the object apparatus to execute specific operation based on the instruction voice.SELECTED DRAWING: Figure 1

Description

The technique disclosed herein relates to a control device for causing a device to perform an operation based on an instruction voice.

In Patent Document 1, when a voice instruction is given by a user, the wearable terminal converts the voice instruction into an electric signal and transmits the converted signal to the network. The network generates a control signal for controlling the home electric appliance according to the signal received from the wearable terminal, and transmits the generated control signal to the home electric appliance. The home electric appliance executes a predetermined operation in response to the control signal.

Japanese Unexamined Patent Publication No. 2018-120627

In Patent Document 1, only one device exists as a candidate for a referent. However, Patent Document 1 does not assume a situation in which a plurality of devices exist as candidates for referents.

In the present specification, it is possible to execute an appropriate operation based on the referent voice in both the situation where a plurality of devices exist as candidates for the referent and the situation where only one device exists as the candidate for the referent. Providing the technology to make it.

The control device disclosed in the present specification is predetermined when a plurality of candidate devices are set when a voice receiving unit capable of receiving voice receives an instruction voice for causing a device to execute a specific operation. The first estimation unit that estimates the target device to be instructed from the plurality of candidate devices based on the first condition of the above, and one candidate device when the voice reception unit receives the instruction voice. When only is set, regardless of the first condition, the determination unit that determines the one candidate device as the target device and the specific operation based on the instruction voice are executed on the target device. A first operation-related processing execution unit for executing the first operation-related processing for causing the operation may be provided.

The configuration of the communication system is shown. In the first embodiment, the case A1 in which the instruction voice for the television is uttered is shown. In the first embodiment, the case A2 in which the instruction voice for the air conditioner is uttered is shown. The continuation of FIG. 3 is shown. In the second embodiment, a case where an instruction voice for the air conditioner is uttered is shown. Cases B and C1 which are continuation of FIG. 5 are shown. A case C2, which is a continuation of FIG. 5, is shown. In the third embodiment, a case where an instruction voice for the air conditioner is uttered is shown.

(First Example)
(Configuration of communication system 2; FIG. 1)
As shown in FIG. 1, the communication system 2 includes a management server 10, a control server 50, a service providing server 90 (hereinafter, referred to as “SP server 90”), and each of the communication systems 2 installed in the house 4. The device 100 to 500 is provided. The management server 10, the control server 50, and the SP server 90 are connected to the Internet 8 and are installed outside the house 4.

In the house 4, three air conditioners 100, 200, 300, one TV 310, a smart speaker 400, and a router 500 are installed. The router 500 relays the communication between the device in the house 4 (for example, the air conditioner 100) and the server on the Internet 8 (for example, 10). The router 500 is connected to the Internet 8 and forms a LAN 5 in the house 4. LAN 5 is a wireless LAN. In the modified example, the LAN 5 may be a wired LAN.

The device name "ac1" is given to the air conditioner 100. The device name is, for example, the model number of the device. Similarly, the air conditioners 200 and 300 are given the device names "ac2" and "ac3". The air conditioner (for example, 100) can communicate with the server (for example, 10) via the router 500 and the Internet 8 by being connected to the LAN 5. The air conditioners 100, 200, and 300 are installed in the first room, the second room, and the third room in the house 4, respectively. Room names R1, R2, and R3 are given to the first room, the second room, and the third room, respectively.

The device name "tv1" is given to the television 310. By connecting to the LAN 5, the television 310 can communicate with a server (for example, 10) via the router 500 and the Internet 8. The television 310 is installed in the third room R3 in the house 4.

The ID "ss1" is assigned to the smart speaker 400. Here, the ID is, for example, a serial number assigned by the manufacturer of the smart speaker 400. The smart speaker 400 is a speaker that can accept operations by voice recognition. The smart speaker 400 is, for example, Google Home (registered trademark), Amazon Echo (registered trademark), or the like. The smart speaker 400 includes a microphone 400a capable of receiving voice and a speaker 400b capable of producing voice. By connecting to the LAN 5, the smart speaker 400 can communicate with the server (for example, 10) via the router 500 and the Internet 8. The smart speaker 400 is installed in the first room R1 in the house 4. The router 500 is also installed in the first room R1 in the house 4.

(Configuration of management server 10)
The management server 10 is a server for managing communication between the smart speaker 400 and the control server 50. The management server 10 includes a network interface 12 and a control unit 30. Each part 12 and 30 is connected to a bus line (reference numeral omitted). The management server 10 is installed, for example, by the manufacturer of the equipment (for example, the air conditioner 100) in the house 4. In the following, the interface will be referred to as "I / F".

The network I / F 12 is an I / F for connecting to the Internet 8 and executing communication via the Internet 8. The control unit 30 includes a CPU 32 and a memory 34. The CPU 32 executes various processes according to the program 40 stored in the memory 34.

(Configuration of control server 50)
The control server 50 is a server for remotely controlling a device (for example, an air conditioner 100) in the house 4. The control server 50 includes a network I / F 52 and a control unit 60. Each part 52, 60 is connected to a bus line (reference numeral omitted). The control server 50 is installed, for example, by the manufacturer of the equipment (for example, the air conditioner 100) in the house 4.

The network I / F 52 is the same as the network I / F 12 of the management server 10. The control unit 60 includes a CPU 62 and a memory 64. The CPU 62 executes various processes according to the program 70 stored in the memory 64. The memory 64 further stores the account information AI and the device information.

The account information AI is information for authenticating the user of the house 4. The device information is information about the device (for example, the air conditioner 100) in the house 4. The device information is stored in the memory 64 in association with the account information AI. The device information is set in the control server 50 (that is, stored in association with the account information AI) by, for example, performing an operation for setting the device information by the user. The device information includes a device name, location information, current status information, and past operation information. The position information is information indicating the position where the corresponding device is installed (for example, the room name R1 indicating the first room).

The current status information includes operating information indicating the current operating status of the corresponding device. The operation information indicates, for example, "stop" when the device is in a stopped state (for example, a sleeve state), and "operation" when the device is in an operating state (for example, a state in which cooling is being executed). Shown. Further, when the corresponding device is an air conditioner, the operation information further includes ambient information indicating the current state around the device. Ambient information includes, for example, the ambient temperature of the device (ie, room temperature). The ambient information is not limited to room temperature, and may include, for example, the humidity around the device, the outdoor temperature, and the like. The current state information is periodically received from, for example, a device in the house 4 (for example, the air conditioner 100) and stored in the memory 64.

The past operation information indicates the past operation of the corresponding device. For example, when the device is an air conditioner, the past operation information indicates an operation (for example, cooling) that was executed immediately before the air conditioner stopped, and a set value of the operation (for example, a set temperature for cooling). Further, for example, when the device is a television, the past operation information indicates a channel number (for example, "1ch") specified by the user immediately before the television is stopped.

(Configuration of SP server 90)
The SP server 90 is a server for providing a service using the smart speaker 400. The SP server 90 is installed, for example, by the manufacturer of the smart speaker 400. The SP server 90 stores the account information AI and the ID “ss1” of the smart speaker 400 in association with each other. The ID "ss1" is set in the SP server 90 (that is, stored in association with the account information AI) when, for example, an operation for starting the use of the smart speaker 400 is performed by the user.

(TV case A1; Fig. 2)
A case A1 in which an instruction voice for a television is uttered will be described with reference to FIG. The communication between the devices is executed via the Internet 8, the Internet 8 and the LAN 5. In the following, from the viewpoint of ease of understanding, the description of "via the Internet 8" and "via LAN 5" may be omitted when explaining the communication. Further, from the viewpoint of ease of understanding, the processing executed by the CPU of the device (for example, the CPU 32 of the management server 10) according to the program (for example, the program 40) will not be described mainly by the CPU, but will be described mainly by the device. In some cases.

First, the user in the first room R1 utters the instruction voice "Turn on the TV". At T10, the smart speaker 400 acquires voice data indicating the instruction voice from the microphone 400a that has received the instruction voice. As a result, the smart speaker 400 detects the instruction voice "turn on the TV".

At T12, the smart speaker 400 executes voice recognition for the detected instruction voice "turn on the TV". For example, a program for executing voice recognition is installed in the smart speaker 400, and the smart speaker 400 uses the program to perform voice recognition for voice data indicated by the detected instruction voice. .. As a result, the smart speaker 400 acquires result information indicating the result of voice recognition for the detected instruction voice. In this case, the result information includes a "television" indicating the type of the device to be instructed and a "start" indicating the content of the instruction. In another example, the smart speaker 400 transmits voice data indicating the instruction voice to a server (for example, SP server 90) on the Internet, and the server executes voice recognition for the voice data, resulting in information. May be transmitted to the smart speaker 400.

In T14, the smart speaker 400 includes the ID “ss1” of the smart speaker 400, the position information indicating the room name “R1” of the room in which the smart speaker 400 is installed, and the result information acquired in T12. The recognition information is transmitted to the management server 10. The position information indicating the room name "R1" is input to the smart speaker 400 by the user.

When the management server 10 receives the recognition information from the smart speaker 400 at T14, the management server 10 transmits the ID “ss1” included in the received recognition information to the SP server 90 at T20. As a result, the management server 10 receives the account information AI stored in association with the ID "ss1" from the SP server 90 at T22.

At T24, the management server 10 transmits a device information request for requesting device information to the control server 50. The device information request includes the account information AI received at T22.

When the control server 50 receives the device information request from the management server 10 in T24, the control server 50 acquires the device information associated with the account information AI in the device information request from the memory 64 in T30. In this case, the four device information is stored in association with the account information AI. The four device information includes three device information corresponding to the three air conditioners 100 to 300 and one device information corresponding to one TV 310.

In T32, the control server 50 transmits the four device information acquired in T30 to the management server 10.

When the management server 10 receives the information of four devices from the control server 50 in T32, the management server 10 executes a determination in T40 using the result information in the recognition information received in T14. Specifically, the management server 10 determines whether or not two or more device names corresponding to the same type as the type included in the result information exist in the four device information. In this case, the management server 10 determines that only one device name "tv1" corresponding to the same type "television" as the type included in the result information exists in the four device information. That is, the management server 10 determines that only one television 310 of the same type as the type "television" included in the result information is set. In this case, the management server 10 determines the television 310 having the device name "tv1" as the referent without executing the estimation using the position information in the recognition information received in T14, and proceeds to T44. In case A2 of FIG. 3, which will be described later, it is determined that two or more device names corresponding to the same type as the type included in the result information exist in the four device information. Details will be described later, but in this case, the management server 10 executes estimation using the position information in the recognition information received at T14.

In the T44, the management server 10 determines the past operation information "1ch" in the device information including the device name "tv1" of the television 310 to be instructed as the target operation information indicating the operation instructing the instructed target. As a result, the management server 10 estimates the current instruction content by using the operation instructed in the past by the user even if the instruction voice does not include a specific instruction content (for example, a channel number). Can be done.

In T60, the management server 10 transmits an operation command to the control server 50. The operation command is a command instructing the television 310 to transmit the television start request to the television 310 indicated by the device name "tv1". The television start request is a command for requesting the television 310 to perform an operation indicated by the target operation information "1ch" (that is, an operation of outputting a program being broadcast on the channel number "1ch"). The operation command includes the target operation information "1ch".

When the control server 50 receives an operation command from the management server 10 at T60, the control server 50 transmits a television start request to the television 310 at T62. The TV start request includes the channel number "1ch".

When the television 310 receives the television start request from the control server 50 in the T62, the television 310 starts outputting the program being broadcast on the channel number "1ch" in the television start request in the T64. As a result, the user in the third room R3 can watch the program being broadcast on the channel number "1ch".

At T66, the television 310 transmits a start notification to the control server 50 indicating that the operation according to the television start request has started. As a result, the start notification is transmitted to the management server 10 via the control server 50.

When the management server 10 receives the start notification from the control server 50 in the T68, the smart speaker 400 outputs the voice data indicating the notification voice "1ch was output on the TV tv1" in the T70 to notify the user of the result of the instruction voice. Send to.

When the smart speaker 400 receives the voice data indicating the notification voice "1ch was output by the television tv1" from the management server 10 at the T70, the smart speaker 400 supplies the voice data to the speaker 400b at the T72. As a result, the speaker 400b utters the notification voice "1ch was output by the television tv1" according to the voice data. The user who has heard the notification voice can know that the program having the channel number "1ch" has been output on the television 310 according to the instruction voice of T10.

(Air conditioner case A2; Fig. 3, Fig. 4)
A case A2 in which an instruction voice for the air conditioner is uttered will be described with reference to FIGS. 3 and 4.

The T110 is the same as the T10 in FIG. 2 except that the instruction voice "turn on the air conditioner" is detected. T112 and T114 are the same as T12 and T14, except that the result information includes an "air conditioner" indicating the type of the device to be referred. T120 to T132 are the same as T20 to T32.

In T140, the management server 10 determines that the three device names "ac1" to "ac3" corresponding to the same type as the type "air conditioner" included in the result information exist in the four device information. That is, the management server 10 determines that three air conditioners 100 to 300 of the same type as the type "air conditioner" included in the result information are set.

In T142, the management server 10 corresponds to the same position information "R1" as the position information "R1" included in the recognition information of T114 from the three device names "ac1" to "ac3" determined in T140. Select the attached device name "ac1". As described above, the position information "R1" included in the recognition information indicates a room in which the smart speaker 400 is installed (that is, a room in which the user who originates the instruction voice is located). That is, the management server 10 instructs the air conditioner 100 having the device name "ac1" installed in the same room as the room where the user who originates the instruction voice is located, from the three air conditioners 100 to 300 in the house 4. Estimate as a target. T144 is the same as T44 in FIG. 2 except that the past operation information “cooling 26 ° C.” in the device information including the device name “ac1” is used.

Subsequently, in T160, the management server 10 transmits an operation command to the control server 50. The operation command is a command instructing to send a cooling start request to the air conditioner 100 indicated by the device name "ac1". The cooling start request is a command for requesting the air conditioner 100 to perform an operation indicated by the target operation information "cooling 26 ° C." (that is, an operation of starting cooling at a set temperature of 26 ° C.). The operation command includes the target operation information "cooling 26 ° C.".

When the control server 50 receives an operation command from the management server 10 at T160, the control server 50 transmits a cooling start request to the air conditioner 100 at T62. The cooling start request includes the set temperature "26 ° C.".

When the air conditioner 100 receives the cooling start request from the control server 50 at T162, the air conditioner 100 starts cooling at the installation temperature "26 ° C." in the cooling start request at T164. T166 and T168 are the same as T66 and T68 in FIG. The T170 and T172 are the same as the T70 and T72, except that the notification voice is "The cooling of the air conditioner ac1 has started". The user who hears the notification voice can know that the air conditioner 100 has started cooling according to the instruction voice of the T110.

When the user utters the instruction voice "Turn on the air conditioner" with the intention of starting the cooling of the air conditioner 100 in the first room R1, the process of T172 ends this case. However, the instruction voice "turn on the air conditioner" does not include a word (for example, a device name) indicating which of the three air conditioners 100 to 300 in the house 4 is. Therefore, the user may utter the instruction voice "turn on the air conditioner" with the intention of starting the cooling of the air conditioner different from the air conditioner 100 of the first room R1. In this case, the user utters a misunderstanding voice "different" indicating that the intention is different from the user's intention in response to the notification voice "Cooling of the air conditioner c1 has started" of T172.

At T174, the smart speaker 400 detects the misunderstood voice "different". At T176, the smart speaker 400 executes voice recognition for the detected misunderstood voice. In this case, the smart speaker 400 acquires result information including a "misunderstanding" indicating that an air conditioner different from the user's intention was erroneously estimated.

In T176, the smart speaker 400 transmits the recognition information including the ID “ss1” of the smart speaker 400 and the result information acquired in T174 to the management server 10.

At T180, the management server 10 sends a stop command to the control server 50. The stop command is a command instructing the air conditioner 100 to send a stop request. The stop request is a command for requesting the air conditioner 100 to stop the cooling.

When the control server 50 receives the stop command from the management server 10 at T180, the control server 50 transmits a stop request to the air conditioner 100 at T182.

When the air conditioner 100 receives a stop request from the control server 50 at T182, the air conditioner 100 stops cooling at T184. In the subsequent T186, the air conditioner 100 transmits a stop notification indicating that the cooling has stopped to the control server 50. As a result, the stop notification is transmitted to the management server 10 via the control server 50.

When the management server 10 receives the stop notification from the control server 50 at T186, the management server 10 proceeds to T190.

In the T190, the management server 10 has two device names "ac2" and "ac3" excluding the device name "ac1" selected in T142 of FIG. 3 from the three device names "ac1" to "ac3". To identify. Then, the management server 10 has a device name associated with the current status information including the highest room temperature and the operation information indicating "stop" from the two specified device names "ac2" and "ac3". The air conditioner having the above is estimated as the next referent. In this case, the current status information corresponding to the device name "ac2" includes the room temperature "27 ° C" and "stop", and the current status information corresponding to the device name "ac3" includes the room temperature "30 ° C" and "stop". Including. Therefore, the management server 10 estimates the air conditioner 300 having the device name "ac3" associated with the current status information including the highest room temperature "30 ° C." and "stop" as the next referent.

T192 is the same as T144 in FIG. 3 except that the past operation information “cooling 26 ° C.” in the device information including the device name “ac3” is used. T200 and T202 are the same as T160 and T162 except that the referent is the air conditioner 300. T204 and T206 are the same as T164 and T166 except that the operating main body is the air conditioner 300. T208 is similar to T168. T210 and T212 are the same as T170 and T172 except that the notification voice is "The cooling of the air conditioner ac3 has started".

(Effect of this example)
As described above, in the control server 50, three air conditioners 100 to 300 are set as candidate devices that can be remotely controlled for the device type "air conditioner". For example, when the user intends to remotely control the air conditioner 100, an instruction voice (for example, "turn on the air conditioner ac1") including a word for identifying the air conditioner 100 (for example, the device name "ac1") is uttered. A comparative example is assumed. In this comparative example, although it is clear that the referent is the air conditioner 100, the user needs to know the words for identifying the air conditioner 100. Further, the instruction voice uttered by the user becomes relatively long. Therefore, the user may find it annoying to utter the instruction voice. On the other hand, in case A2 of FIGS. 3 and 4, when the smart speaker 400 receives the instruction voice "turn on the air conditioner" (T110), the management server 10 has three air conditioners of the same type 100 to 300. The air conditioner 100 installed in the same room as the room in which the smart speaker 400 is installed (that is, the room in which the user who originates the instruction voice is located) is estimated as the instruction target. Therefore, the user does not need to include a word for identifying the air conditioner 100 in the instruction voice. It is possible to suppress the user from feeling annoyed to utter the instruction voice. User convenience is improved.

Further, as described above, in the control server 50, only one TV 310 is set as a candidate device for the device type "TV". In this case, it is not necessary to estimate the device to be referred. In case A1 of FIG. 2, when the smart speaker 400 receives the instruction voice "turn on the TV" (T10), the management server 10 has one TV 310 of the same type as the type "TV" included in the instruction voice. It is determined that only is set (T40). Then, the management server 10 transmits an operation command for the television 310 to be instructed to the control server 50 without executing a process of estimating the referent (for example, T142 in FIG. 4). In this case, since the process of estimating the referent is not executed, it is possible to suppress the execution of unnecessary processes.

As shown in case A2 of FIGS. 3 and 4, when a plurality of candidate devices (for example, three air conditioners) are set, the control server 50 estimates the device to be instructed and gives an instruction. Have the estimated device perform voice-based actions. On the other hand, as shown in Case A1 of FIG. 2, when only one candidate device (for example, one TV) is set, the control server 50 does not estimate the device to be instructed. The one candidate device is determined as the device to be instructed, and the determined device is made to execute the operation based on the instruction voice. That is, according to the configuration of the present embodiment, the operation based on the referent is performed in either the situation where a plurality of devices exist as candidates for the referent or the situation where only one device exists as a candidate for the referent. It can be executed by an appropriate device.

Further, according to the above configuration, when the management server 10 receives the misunderstood voice "different" indicating that the smart speaker 400 is different from the user's intention (T174 in FIG. 4), the air conditioner 100 erroneously estimated. A stop command for is transmitted to the control server 50 (T180). Even if the referent is erroneously estimated by the management server 10, the user can stop the operation of the erroneously estimated referent simply by uttering a misunderstood voice. Further, the user only needs to utter a misunderstanding voice, and does not need to utter a relatively long voice such as an instruction voice "stop the air conditioner ac1". User convenience is improved.

Further, according to the above configuration, when the management server 10 receives the misunderstood voice "different" indicating that the smart speaker 400 is different from the user's intention (T174 in FIG. 4), the air conditioner 100 presumed earlier. The air conditioner 300 is estimated as the next referent from the two air conditioners 200 and 300 excluding the above (T190). Then, the management server 10 transmits an operation command for the estimated air conditioner 300 to the control server 50 (T200). Even if the referent is erroneously estimated by the management server 10, the user simply utters a misunderstood voice and causes the air conditioner 300, which is highly likely to be intended by the user, to perform an operation based on the referent voice. Can be done. The user only needs to utter a misunderstanding voice, and does not need to utter a relatively long voice such as the instruction voice "Turn on the air conditioner ac3". User convenience is improved.

Further, according to the above configuration, when the management server 10 uses the location information and the estimation is incorrect, the management server 10 estimates the next referent by using the current status information different from the location information (FIG. 4). T190). By executing the next estimation under conditions different from the previous estimation, it is possible to prevent the estimation from being erroneous as in the previous estimation.

Further, for example, when the estimation of the air conditioner 100 installed in the room where the user who originates the instruction voice is present is incorrect, a comparative example in which the next instruction target is randomly estimated from the remaining candidates is assumed. In this comparative example, the estimation of the next referent is also likely to be incorrect. On the other hand, when the estimation of the air conditioner 100 installed in the room where the user who originates the instruction voice is located is incorrect, the management server 10 associates it with the current status information including the highest room temperature "30 ° C". The air conditioner 300 having the device name "ac3" is estimated as the next referent (T190). If the instructions are presumed to be the start of cooling, it is likely that the user intends to start cooling the air conditioner installed in the room with the highest room temperature. By using the information on the room temperature of the room in which the air conditioner is installed, the accuracy of estimating the next referent can be improved as compared with the above comparative example. If the instruction content is presumed to be the start of heating, it is highly likely that the user intends to start heating the air conditioner installed in the room having the lowest room temperature. In this case, the management server 10 may estimate an air conditioner having a device name associated with the current status information including the lowest room temperature as the next referent. In the modified example, the above comparative example may be adopted.

(Correspondence)
The management server 10 is an example of a “control device”. The microphone 400a and the speaker 400b of the smart speaker 400 are examples of the "audio reception unit" and the "audio output unit", respectively. Three air conditioners 100 to 300 and one television 310 are examples of "a plurality of candidate devices" and "one candidate device", respectively. The start of the output of the television in case A1 of FIG. 2 and the start of cooling of the air conditioner in case A2 of FIG. 3 are examples of "specific operation". Here, the "specific operation" refers to the content of the operation of the device instructed by the user by voice. "Turn on the TV" in case A1 and "Turn on the air conditioner" in case A2 are examples of "instruction voice". T60 in case A1 and T160 in case A2 are examples of "first operation-related processing". The television start request in case A1 and the cooling start request in case A2 are examples of the "execution request". The voice of T72 in case A1 and the voice of T172 in case A2 are examples of "completion voice". The voice "different" is an example of "misunderstanding voice". The stop request of T182 in FIG. 4 is an example of a “stop request”. The condition using the position information included in the recognition information in T142 in FIG. 3 is an example of the “first condition”. The air conditioner 100 and the air conditioner 300 are examples of a "first device (specific device)" and a "second device", respectively. Two air conditioners 200 and 300 are examples of "one or more candidate devices". T200 in FIG. 4 is an example of "second operation-related processing". The position information "R1" in the recognition information is an example of "position information". The surrounding information and operation information in the current status information are examples of "information about the surrounding state of the device" and "information about the device's own state".

(Second Example)
In the first embodiment, the operation based on the instruction voice is immediately executed. On the other hand, the second embodiment is configured to be able to execute confirmation to the user before executing the operation based on the instruction voice. The configuration of the communication system 2 of the present embodiment is the same as the configuration of the communication system 2 of the first embodiment.

(Air conditioner case; Fig. 5, Fig. 6, Fig. 7)
A case where an instruction voice for the air conditioner is uttered in this embodiment will be described with reference to FIGS. 5 to 7. In this case, the past operation information corresponding to the device name "ac1" indicates "air clearing" or "cooling 26 ° C.". Here, the past operation information "air purifier" indicates that the operation executed immediately before the air conditioner is stopped is air purification.

T310 to T342 are the same as T110 to T142 in FIG. In T344, when the past operation information in the device information including the device name "ac1" indicates "air clear", the management server 10 determines the past operation information "air clear" as the target operation information, and determines the device name "ac1". When the past operation information in the device information including "" indicates "cooling 26 ° C.", the past operation information "cooling 26 ° C." is determined as the target operation information.

In T346, the management server 10 determines whether or not the target operation information determined in T344 indicates a predetermined operation. Here, the predetermined operation is, for example, an operation of starting air cleaning. When the target operation information indicates a predetermined operation (YES in T346), the management server 10 proceeds to case B of FIG. 6, and the target operation information indicates an operation other than the predetermined operation (for example, an operation of starting cooling). In case (NO at T346), proceed to case C1 or case C2 of FIG.

(Air purification case B: Fig. 6)
Case B is the same as T160 in FIG. 4, except that the operation command is a command instructing the air conditioner 100 to send the air cleaning start request. The air cleaning start request is a command for requesting the air conditioner 100 to perform an operation (that is, an operation for starting air cleaning) indicated by the target operation information "air cleaning". The operation command includes the target operation information "Sorasei".

The T160 is the same as the T162 of FIG. 4, except that the air cleaning start request is transmitted to the air conditioner 100. At T346, the air conditioner 100 starts air cleaning. T366 and T368 are the same as T166 and T168. T370 and T372 are the same as T170 and T172, except that the notification voice is "The air conditioner ac1 has started air cleaning".

(Cooling case C1; Fig. 6)
Regarding case C1, in T450, the management server 10 transmits voice data indicating the confirmation voice "Do you want to start cooling the air conditioner ac1?" To the smart speaker 400 without transmitting the operation command to the control server 50. The confirmation voice is a voice for confirming to the user whether the air conditioner estimated as the referent is the same as the user's intention. The confirmation voice includes the device name “ac1” of the air conditioner 100 estimated as the referent in T342 of FIG.

When the smart speaker 400 receives the voice data indicating the confirmation voice from the management server 10 in the T450, the smart speaker 400 utters the confirmation voice "Do you want to start cooling the air conditioner ac1?" In the T452. The user can hear the device name "ac1" included in the confirmation voice and know that the air conditioner 100 has been estimated as the referent. In this case, the user intends to start cooling the air conditioner 100 installed in the first room R1 in which he / she is located. In this case, the user utters the correct answer voice "OK" indicating that the confirmation voice of T452 is the same as the intention of the user.

In T454, the smart speaker 400 detects the correct answer voice "OK". In T456, the smart speaker 400 executes voice recognition for the detected correct answer voice. In this case, the smart speaker 400 acquires result information including a "correct answer" indicating that the same air conditioner as the user's intention was correctly estimated.

At T458, the smart speaker 400 transmits the recognition information including the ID “ss1” of the smart speaker 400 and the result information acquired by T454 to the management server 10.

When the management server 10 receives the recognition information from the smart speaker 400 at the T458, the management server 10 proceeds to the T460. T460 is the same as T160 in FIG. T462 and T464 are the same as T162 and T164. That is, in the air conditioner 100 estimated as the referent, cooling at the installation temperature "26 ° C." is started.

(Cooling case C2; Fig. 7)
Regarding case C2, T550 and T552 are the same as T450 and T452 in FIG. In this case, the user intends to start cooling an air conditioner different from the air conditioner 100 installed in the first room R1 in which he / she is located. In this case, the user utters a misunderstanding voice "different" indicating that the confirmation voice of T552 is different from the user's intention.

T554 and T558 are the same as T176 and T178 in FIG. When the management server 10 receives the recognition information including the result information "misunderstanding" from the smart speaker 400 at T178, the management server 10 estimates the next referent at T560 and T562. T560, T562
Is the same as T190 and T192 in FIG. That is, the management server 10 estimates the air conditioner 300 having the device name "ac3" as the next referent.

T570 and T572 are the same as T550 and T552, except that the confirmation voice is "Do you want to start cooling the air conditioner ac3?". In this case, the user intends to start cooling the air conditioner 300 installed in the third room R3 where he / she is not present. In this case, the user utters the correct answer voice "OK" indicating that the confirmation voice of T552 is the same as the intention of the user.

T576 and T578 are the same as T456 and T458 in FIG. T590 and T592 are the same as T460 in FIG. 6 except that the referent is the air conditioner 300. The T594 is the same as the T464 except that the operating body is the air conditioner 300.

Although not shown, it is assumed that, for example, only one air conditioner 100 is set as a candidate device. In this case, the management server 10 determines the one air conditioner 100 as the referent without estimating the referent device (that is, without executing the process of T342 in FIG. 5). Then, the management server 10 transmits the voice data indicating the confirmation voice similar to the T450 of FIG. 6 to the smart speaker 400.

(Effect of this example)
As shown in case C1 of FIG. 6, when a plurality of candidate devices (for example, three air conditioners) are set, the control server 50 estimates the device to be instructed and estimates it as the referent. The smart speaker 400 is made to utter a confirmation voice confirming to the user whether the air conditioner is the same as the user's intention. On the other hand, when only one candidate device (for example, one air conditioner) is set, the control server 50 sets the one candidate device as the referent device without estimating the referent device. Is determined, and the confirmation voice is uttered to the smart speaker 400. That is, in the configuration of this embodiment, the smart speaker emits an appropriate confirmation voice in either a situation where a plurality of devices exist as candidates for the referent or a situation where only one device exists as a candidate for the referent. It can be run by 400. Further, also in this embodiment, since the estimation of the device to be instructed is executed, the user does not need to utter the instruction voice including the words for identifying the device intended by the user. User convenience is improved. Further, also in this embodiment, when only one candidate device is set, the process of estimating the referent is not executed, so that it is possible to suppress the execution of unnecessary processes.

Further, according to the above configuration, when the smart speaker 400 receives the correct answer voice "Iyo" for the confirmation voice (T454 in FIG. 6), the management server 10 issues an operation command for the air conditioner 100 estimated as the referent. It is transmitted to the control server 50 (T460). On the other hand, when the smart speaker 400 receives the misunderstanding voice "different" for the confirmation voice (T554 in FIG. 7), the management server 10 does not transmit the operation command for the air conditioner 100 estimated as the referent to the control server 50. That is, in the second embodiment, the user's intention can be confirmed by the confirmation voice before the operation command is transmitted to the control server 50. As a result, as compared with the first embodiment in which the operation command is transmitted to the control server 50 without executing the utterance of the confirmation voice, the process for stopping the operation of the referent that is erroneously estimated (for example, FIG. 4). It is possible to suppress the execution of T180). In the first embodiment, the confirmation voice is not uttered. Therefore, the first embodiment has an advantage that the operation based on the instruction voice can be promptly executed by the estimated device.

Further, according to Case B, when the target operation information indicates "air clearing" which is a predetermined operation (YES in T346 of FIG. 5), the management server 10 does not utter a confirmation voice to the smart speaker 400. The operation command is transmitted to the control server 50 (T360 in FIG. 6). The power consumption of the air conditioner in "Sorasei" in Case B is lower than that in "Cooling" in Case C1 in which the confirmation voice is uttered. That is, "Sorasei" has less economic impact than "cooling". Also, "cooling" is more likely to cause discomfort to the user if executed incorrectly than "air conditioning". That is, "Sorasei" has little influence on the user (that is, the human body). In this way, the user may find it annoying to confirm by the confirmation voice for the operation having little economic or human influence. By executing the determination of whether or not the target motion information is a predetermined motion in T346, it is possible to omit the utterance of the confirmation voice in the situation where the motion with little economic or human influence should be executed. .. It is possible to suppress the user from feeling annoyed. User convenience is improved.

(Correspondence)
The predetermined operation of T346 in FIG. 5 is an example of the “predetermined operation”. T450 of FIG. 6 or T550 of FIG. 7 is an example of "first operation-related processing". T570 in FIG. 7 is an example of "second operation-related processing". The confirmation voice of T452 and the correct answer voice "Iyo" of T454 in FIG. 6 are examples of the "confirmation voice" and the "correct answer voice", respectively.

(Third Example)
The third embodiment is the same as the first embodiment except that the conditions for estimating the device to be instructed are different.

(Air conditioner case; Fig. 8)
A case where an instruction voice for the air conditioner is uttered in this embodiment will be described with reference to FIGS. 5 to 7.

T610 and T612 are the same as T110 and T112 in FIG. T614 is similar to T114, except that the authentication information does not include location information. T620 to T640 are the same as T120 to T140. In T642, the management server 10 is associated with the current status information including the highest room temperature and the operation information indicating "stop" from the three device names "ac1" to "ac3" determined in T640. Estimate the air conditioner with the device name as the referent. In this case, the current status information corresponding to the device name "ac1" includes the highest room temperature "31 ° C" and "stop". Therefore, the management server 10 estimates the air conditioner 100 having the device name "ac1" associated with the current status information including the highest room temperature "31 ° C." and "stop" as the referent.

Subsequently, the same processing as in T160 to T172 of FIG. 4 is executed. As a result, cooling is started in the air conditioner 100, and then the notification voice "cooling of the air conditioner ac1 has started" is uttered in the smart speaker 400.

Although not shown, the processing in the situation where only one candidate device (for example, one television) is set is the same as in Case A (see FIG. 2) of the first embodiment.

Also in this case, as in the first embodiment, the instruction is given in either the situation where a plurality of candidate devices (for example, three air conditioners) exist or the situation where only one device exists as a candidate for the referent. It is possible to have an appropriate device perform a target-based operation. The condition using the current state information in T642 of FIG. 8 is an example of the “first condition”.

Modifications of each of the above embodiments are described below.

In each of the above embodiments, the communication system 2 includes a management server 10 and a control server 50. Instead, the communication system 2 may include only one server. In this case, the one server may execute both the process executed by the management server 10 and the process executed by the control server 50. In this modification, the one server is an example of a "control device".

In each of the above embodiments, the communication system 2 includes a smart speaker 400. Instead, the communication system 2 may include a terminal device (for example, a desktop PC, a smartphone) including a microphone and a speaker. In this modification, the microphone of the terminal device and the speaker of the terminal device are examples of the “voice reception unit” and the “voice output unit”, respectively.

In each of the above embodiments, the communication system 2 includes a management server 10 and a control server 50. Instead of this, the communication system 2 may not include the management server 10 and the control server 50, but may include a smart speaker 400 and a plurality of devices in the house 4. In this case, the smart speaker 400 stores a plurality of device information corresponding to the plurality of devices, and receives and stores the current status information in the device information from the devices via the LAN 5 in the house 4. May be good. Then, when the smart speaker 400 receives the instruction voice, the smart speaker 400 estimates the device to be instructed from a plurality of devices corresponding to the plurality of device names in the plurality of device information based on predetermined conditions. May be good. In this modification, the smart speaker 400 is an example of a "control device". Further, in another example, the estimation of the device to be instructed may be executed by a terminal device such as a smartphone. In this modification, the terminal device is an example of a "control device".

The device information does not have to include past operation information. In this case, the instruction voice may include words indicating specific instruction contents (for example, cooling at 26 ° C.). Generally speaking, the "instruction voice" may or may not include a word indicating "specific action".

In the first embodiment, the processes of T70 and T72 of FIG. 2 and T170 and T172 of FIG. 4 may not be executed. In this modification, the "completion voice" and the "output control unit" can be omitted.

In the first embodiment, the processes of T180 to T188 of FIG. 4 may not be executed. In this modification, the "stop processing execution unit" can be omitted.

In the first embodiment, the processing after T190 of FIG. 4 may not be executed. In the second embodiment, the processing after T560 of FIG. 7 may not be executed. In these modified examples, the "second estimation unit" and the "second operation-related processing execution unit" can be omitted.

In the first embodiment, the management server 10 uses the location information to execute the previous estimation (T142 in FIG. 3), and uses the current status information to execute the next estimation (T190 in FIG. 4). Instead, the management server 10 may execute both the previous estimation and the next estimation by using the current status information. Generally speaking, the "second device" may be estimated based on the same conditions as the first condition.

The "location information" is not limited to the room name, and may be, for example, user information indicating the position of the user who utters the instruction voice. For example, the user information uses information indicating the longitude and latitude measured by a GPS sensor included in a device carried by the user, a motion sensor for detecting the user (for example, a sensor using infrared rays, a Wi-Fi radio wave). The output signal from the sensor), the information indicating the distance from the user to the reference point (for example, the output signal of the infrared distance sensor), the information obtained by analyzing the image of the camera that captures the user, and the like may be used.

In the first embodiment, the management server 10 estimates the air conditioner 100 installed in the same room as the room where the user who utters the instruction voice is present as the referent (T142 in FIG. 3). Instead, the management server 10 may estimate the air conditioner installed in the room closest to the room in which the user is located as the referent.

The "equipment" is not limited to an air conditioner and a television, and may be, for example, a washing machine, a rice cooker, a microwave oven, a water heater, a heater (for example, a fan heater), a refrigerator, a terminal device, or the like.

The "predetermined operation" is not limited to air purification. For example, when the referent is a water heater, the "predetermined operation" may be a reheating operation, and the operation other than the "predetermined operation" may be a hot water filling operation. Since the hot water filling operation is an operation of storing hot water in the bathtub, if it is started by mistake, the hot water in the bathtub is wasted, which has a great economic impact. On the contrary. Since the additional cooking operation does not increase the amount of hot water in the bathtub, it has a smaller economic impact than the hot water filling operation. Further, for example, when the instruction target is an air conditioner, the "predetermined operation" is cooling or heating in which the temperature difference between the set temperature and the room temperature is small, and the operation other than the "predetermined operation" is the set temperature. It may be cooling or heating having a large temperature difference from room temperature. The greater the temperature difference, the greater the impact on the human body. Generally speaking, the "predetermined motion" is an motion that has less economic or physical influence as compared with an motion other than the predetermined motion.

Further, the "predetermined operation" may be an operation in which the influence of the start of the operation is minor if the operation is canceled immediately after the start of the operation. For example, when the referent is an air conditioner, the operation of changing the set temperature does not cause a substantial change in the indoor temperature if the set temperature is returned to the original set temperature immediately after the set temperature is changed. Corresponds to the "predetermined operation" in. Further, for example, when the referent is an air purifier, the operation of starting the air purifier merely purifies the air and does not cause an irreparable adverse effect on the user. Corresponds to "predetermined operation". On the other hand, when the referent is a washing machine, in a situation where clothes to be washed are put in the washing machine, the operation of starting washing is performed by adding water even if the washing is stopped immediately after the start. Since the clothes get wet, it does not correspond to the "predetermined operation" of this modification. That is, when the target operation information indicates "start of washing", it is desirable that a confirmation voice (for example, "Do you want to start washing?") Is uttered.

As a modification of the case of FIG. 2, it is assumed that only one device information corresponding to the television 310 is stored in association with the account information AI. In this situation, the instruction voice of T10 may be "turn on" instead of "turn on the TV". Then, the management server 10 may determine the television 310 corresponding to one device information received from the control server 50 as the referent in the T40. In the above situation, the process of determining the referent is "when only one candidate device is set, the one candidate device is determined as the target device regardless of the first condition". This is an example.

In the third embodiment, the management server 10 estimates the device to be instructed based on the conditions using both the ambient information (for example, room temperature) and the operation information (T642 in FIG. 8). Instead, the management server 10 may estimate the referent device based on the condition using only the operation information, or estimate the referent device based on the condition using only the surrounding information. You may. Generally speaking, in this modification, at least one of "information about the state of the device itself" and "information about the state around the device" may be used.

Although some embodiments of the present invention have been described, these embodiments are presented as examples and are not intended to limit the scope of the invention. These embodiments can be implemented in various other forms, and various omissions, replacements, and changes can be made without departing from the gist of the invention. These embodiments and modifications thereof are included in the scope and gist of the invention, as well as in the scope of the invention described in the claims and the equivalent scope thereof.

2: Communication system, 4: Home, 5: LAN, 8: Internet, 10: Management server, 12: Network I / F, 30: Control unit, 32: CPU, 34: Memory, 40: Program, 50: Control server , 52: Network I / F, 60: Control unit, 62: CPU, 64: Memory, 70: Program, 90: SP server, 100-300: Air conditioner, 310: TV, 400: Smart speaker, 400a: Microphone, 400b : Speaker, 500: Router, AI: Account information

Claims (10)

When a voice receiving unit capable of receiving voice receives an instruction voice for causing a device to execute a specific operation, when a plurality of candidate devices are set, the plurality of voice reception units are based on a predetermined first condition. The first estimation unit that estimates the target device to be instructed from among the candidate devices, and
When the voice receiving unit receives the instruction voice and only one candidate device is set, the determination unit that determines the one candidate device as the target device regardless of the first condition. When,
A first operation-related processing execution unit that executes a first operation-related processing for causing the target device to execute the specific operation based on the instruction voice, and a first operation-related processing execution unit.
A control device. The first operation-related process includes a process for transmitting an execution request requesting execution of the specific operation to the target device.
The control device further
The control device according to claim 1, further comprising an output control unit that causes the voice output unit to output a completed voice indicating that the target device has executed the specific operation. The control device further
When the voice receiving unit receives a misunderstood voice indicating that a specific device among the plurality of candidate devices is erroneously estimated as the target device after the voice output unit outputs the completed voice. The control device according to claim 2, further comprising a stop processing execution unit that executes a process for transmitting a stop request requesting the stop of the specific operation to the specific device. The first operation-related processing is
A process for causing the voice output unit to output a confirmation voice for confirming to the user that the target device executes the specific operation.
When the voice receiving unit receives the correct answer voice indicating that a specific device among the plurality of candidate devices is correctly estimated as the target device after the voice output unit outputs the confirmation voice. A process for transmitting an execution request requesting execution of the specific operation to the specific device, and
Including
When the voice receiving unit receives a misunderstood voice indicating that the specific device is erroneously estimated as the target device after the voice output unit outputs the confirmation voice, the execution request is the specific device. The control device according to claim 1, which is not transmitted to the device. The first operation-related process is a process for transmitting the execution request to the target device without outputting the confirmation voice to the voice output unit when the specific operation is a predetermined operation. The control device according to claim 4, which includes. The control device further
When the voice receiving unit receives a misunderstood voice indicating that the first device among the plurality of candidate devices is erroneously estimated as the target device after the first operation-related processing is executed. In addition, a second estimation unit that estimates a second device as a referent from one or more candidate devices excluding the first device from the plurality of candidate devices.
A second operation-related processing execution unit that executes a second operation-related processing for causing the second device to execute the specific operation based on the instruction voice, and a second operation-related processing execution unit.
The control device according to any one of claims 1 to 5. The control according to claim 6, wherein the second estimation unit estimates the second device from the one or more candidate devices based on a predetermined second condition different from the first condition. apparatus. The control device according to any one of claims 1 to 6, wherein the first condition includes a condition for using position information regarding the position of the vocalization source of the instruction voice. The control device according to claim 8, wherein the first estimation unit estimates a device installed at the same place as the place indicated by the position information as the target device from the plurality of candidate devices. .. The first condition according to any one of claims 1 to 9, wherein the first condition includes a condition for using at least one of information about the state of the device itself and information about the state around the device. Control device.

Images