JP6945734B2 - Audio output device, device control system, audio output method, and program - Google Patents

Description

The present invention relates to a voice output device, a device control system, a voice output method, and a program.

Currently, various techniques for controlling equipment according to an operation by a user are known. Such operations include voice operations in addition to screen operations. When the equipment is controlled according to the voice operation, for example, the equipment is controlled by using the equipment control device that controls the equipment based on the voice emitted by the user. The convenience of the user is often improved by using the equipment control device that controls the equipment based on the voice.

By the way, in order to bring the equipment into a desired control state, it is very troublesome for the user to repeatedly execute operations corresponding to various control contents. Therefore, in order to reduce such annoyance, there is known a method of storing information indicating a desired control state for each user. For example, Patent Document 1 includes a portable device that transmits user identification information when a user controls a device using the portable device, and stores personal information acquired by using the device in accordance with the user identification information. The personal information storage system to be used is described.

Japanese Unexamined Patent Publication No. 2008-234371

However, since the technique described in Patent Document 1 is not a technique using a device control device that controls equipment based on voice, it is difficult to immediately apply it to a technique using such a device control device. be. For this reason, there is a demand for a technique for easily bringing equipment into a desired control state by using an equipment control device that controls equipment based on voice.

The present invention has been made in view of the above problems, and is an audio output device and an equipment control system that easily puts an equipment into a desired control state by using an equipment control device that controls an equipment based on voice. , Audio output methods, and programs.

In order to achieve the above object, the audio output device according to the present invention is
A voice output device for putting the equipment into a desired control state by using a device control device that controls the equipment based on voice.
An operation detection means for detecting the first operation for controlling the first equipment for the voice output device, and
When the first operation is detected, an audio output means for outputting a first audio representing the content of the first operation, and an audio output means.
A voice detection means for detecting a user voice that is a voice emitted by a user and represents the first word,
If the first operation within a predetermined time and said user voice is examined knowledge, history information generating means for generating history information the content of the first operation and the first word associated And with
After the history information is generated, the voice output means detects the user voice representing the first word associated with the content of the first operation by the history information , the first voice. Is output.

In the present invention, the operation of the equipment by the user and the action performed by the user have a predetermined relationship, and when the action is detected, a voice representing the content of the operation is output. Therefore, according to the present invention, it is possible to easily bring the equipment into a desired control state by using the equipment control device that controls the equipment based on voice.

Configuration diagram of the device control system according to the first embodiment of the present invention. Configuration diagram of the audio output device according to the first embodiment of the present invention. Configuration diagram of the device control device according to the first embodiment of the present invention. Functional configuration diagram of the audio output device according to the first embodiment of the present invention Functional configuration diagram of the device control device according to the first embodiment of the present invention The figure which shows the history information which concerns on Embodiment 1 of this invention. A flowchart showing an audio output process executed by the audio output device according to the first embodiment of the present invention. Functional configuration diagram of the audio output device according to the second embodiment of the present invention The figure which shows the history information which concerns on Embodiment 2 of this invention. A flowchart showing an audio output process executed by the audio output device according to the second embodiment of the present invention. Functional configuration diagram of the audio output device according to the third embodiment of the present invention Configuration diagram of the device control system according to the fourth embodiment of the present invention. The figure which shows the history information which concerns on Embodiment 4 of this invention. The figure which shows the history information which concerns on Embodiment 5 of this invention. A flowchart showing a setting process executed by the audio output device according to the sixth embodiment of the present invention.

(Embodiment 1)
First, the configuration of the device control system 1000 according to the first embodiment of the present invention will be described with reference to FIG. The device control system 1000 is a system including a voice output device 100 and a device control device 200, in which the voice output device 100 and the device control device 200 cooperate to control equipment. In the present embodiment, the equipment is assumed to be an air conditioner 300. The voice output device 100 receives an operation on the equipment from the user 10 and outputs a voice representing this operation. The device control device 200 detects the voice output by the voice output device 100, generates a control command corresponding to the voice, and transmits this control command to the air conditioner 300. The device control device 200 and the air conditioner 300 are connected to each other via a communication network 600. The communication network 600 is, for example, a wireless LAN (Local Area Network) constructed in the home.

The device control device 200 is generally used to detect a voice spoken by the user 10 and transmit a control command corresponding to the voice to the air conditioner 300. However, in the present embodiment, not the voice spoken by the user 10 but the voice uttered by the voice output device 100 in response to the operation received from the user 10 is detected, and the control command corresponding to this voice is transmitted to the air conditioner 300. Send. In this way, when the voice output device 100 relays between the user 10 and the device control device 200, various effects can be expected. For example, according to such a configuration, the user 10 can control the air conditioner 300 not by voice operation but by screen operation. Further, according to such a configuration, for example, it is possible to automatically execute not only the control corresponding to the operation performed by the user 10 but also the control corresponding to other operations related to this operation.

The voice output device 100 receives from the user 10 an operation instructing control of the air conditioner 300. The voice output device 100 outputs voice corresponding to the content of the operation received from the user 10. The voice output by the voice output device 100 is detected by the device control device 200. The voice output device 100 can automatically output the voice for bringing the air conditioner 300 into the control state desired by the user 10 based on the history of operations received from the user 10. For this reason, the voice output device 100 can output voice other than the voice corresponding to the content of the received operation, or can output the voice when the operation is not accepted. The audio output device 100 is, for example, a smartphone, a tablet terminal, or a personal computer.

Hereinafter, the configuration of the audio output device 100 will be described with reference to FIG. As shown in FIG. 2, the audio output device 100 includes a processor 11, a flash memory 12, a touch screen 13, a microphone 14, a speaker 15, and a communication interface 16. The processor 11 controls the overall operation of the audio output device 100. The processor 11 is, for example, a CPU (Central Processing Unit) having a built-in ROM (Read Only Memory), RAM (Random Access Memory), RTC (Real Time Clock), and the like. The CPU operates according to, for example, a basic program stored in the ROM, and uses the RAM as a work area.

The flash memory 12 is a non-volatile memory that stores various types of information. The flash memory 12 stores, for example, a program executed by the processor 11. The touch screen 13 detects an operation performed by the user and supplies a signal indicating the detection result to the processor 11. Further, the touch screen 13 displays information under the control of the processor 11.

The microphone 14 is a device that converts sound into an electric signal. For example, the microphone 14 converts the voice emitted by the user 10 into an electric signal. The speaker 15 is a device that converts a supplied electric signal into physical vibration to generate sound. For example, the speaker 15 outputs audio for transmitting various messages to the user 10. The communication interface 16 is a communication interface for connecting the voice output device 100 to a telephone network (not shown) or the Internet (not shown).

The device control device 200 detects the voice output by the voice output device 100 and generates a control command corresponding to the detected voice. The device control device 200 transmits the generated control command to the air conditioner 300 via the communication network 600. The device control device 200 has a function of converting voice into words and a function of converting words into control commands. The device control device 200 is, for example, a smart speaker.

Hereinafter, the configuration of the device control device 200 will be described with reference to FIG. As shown in FIG. 3, the device control device 200 includes a processor 21, a flash memory 22, a touch screen 23, a microphone 24, a speaker 25, and a communication interface 26. The processor 21 controls the overall operation of the device control device 200. The processor 21 is, for example, a CPU having a built-in ROM, RAM, RTC, and the like. The CPU operates according to, for example, a basic program stored in the ROM, and uses the RAM as a work area.

The flash memory 22 is a non-volatile memory that stores various types of information. The flash memory 22 stores, for example, a program executed by the processor 21. The touch screen 23 detects an operation performed by the user and supplies a signal indicating the detection result to the processor 21. Further, the touch screen 23 displays information under the control of the processor 21.

The microphone 24 is a device that converts sound into an electric signal. For example, the microphone 24 converts the sound output by the sound output device 100 into an electric signal. The speaker 25 is a device that converts a supplied electric signal into physical vibration and emits a sound. For example, the speaker 25 outputs audio for transmitting various messages to the user 10. The communication interface 26 is a communication interface for connecting the device control device 200 to the communication network 600.

The air conditioner 300 is an equipment device to be controlled by the device control system 1000. The air conditioner 300 is, for example, a device that harmonizes the air in the space in the house. The air conditioner 300 includes, for example, a heating function, a cooling function, a dehumidifying function, and a blowing function. The air conditioner 300 is, for example, a remote controller (not shown) for operating an indoor unit (not shown) installed inside the house, an outdoor unit (not shown) installed outside the house, and the indoor unit and the outdoor unit. (Not shown). The air conditioner 300 has a function of connecting to the communication network 600. The air conditioner 300 is controlled according to a control command received from the device control device 200 via the communication network 600.

Next, the function of the audio output device 100 will be described with reference to FIG. As shown in FIG. 4, the voice output device 100 functionally includes a control unit 101, a voice output unit 103, a voice information storage unit 104, an action detection unit 105, a history information generation unit 106, and a history. It includes an information storage unit 107. The action detection unit 105 includes an operation detection unit 102. The operation detection means corresponds to, for example, the operation detection unit 102. The audio output means corresponds to, for example, the audio output unit 103. The behavior detection means corresponds to, for example, the behavior detection unit 105. The history information generation means corresponds to, for example, the history information generation unit 106.

The control unit 101 controls the overall operation of the audio output device 100. For example, the control unit 101 causes the audio output unit 103 to output audio based on the detection result of the operation detection unit 102. Further, for example, the control unit 101 causes the history information generation unit 106 to generate history information based on the detection result by the operation detection unit 102 and the detection result by the action detection unit 105. Further, for example, the control unit 101 outputs voice from the voice output unit 103 based on the detection result of at least one of the detection result by the operation detection unit 102 and the detection result by the action detection unit 105 and the history information. Output. The function of the control unit 101 is realized, for example, by the processor 11 executing the program stored in the flash memory 12.

The operation detection unit 102 detects the operation of the user 10 on the air conditioner 300. This operation is an operation for controlling the air conditioner 300, and is an operation for instructing the control content for the air conditioner 300. This operation is, for example, an operation of instructing the air conditioner 300 to turn on the power, an operation of switching the air conditioning mode to cooling, or an operation of switching the set temperature to 22 ° C. This operation is a screen operation or a voice operation for the voice output device 100. For example, the operation detection unit 102 accepts screen operations on the operation screen for accepting operations on the air conditioner 300. Alternatively, the operation detection unit 102 detects a voice representing the control content for the air conditioner 300. The operation detection unit 102 can be said to be an operation reception unit that accepts operations by the user 10. The function of the operation detection unit 102 is realized by, for example, the function of the touch screen 13 or the function of the microphone 14.

When the operation detection unit 102 detects the operation, the voice output unit 103 outputs a voice representing the content of the operation. For example, it is assumed that the content of the operation is specified by the control unit 101, and the voice information representing the content of the control is acquired by the control unit 101. In this case, the voice output unit 103 generates an electric signal based on the voice information supplied from the control unit 101, and generates a voice corresponding to the electric signal. The function of the audio output unit 103 is realized, for example, by the cooperation of the processor 11 and the speaker 15.

The voice information storage unit 104 stores voice information. The voice information is, for example, information indicating voice to be output for each operation content, that is, for each control content. For example, the voice information is associated with the content of the operation of air conditioner 300: power: on and the information corresponding to the electric signal for outputting the voice "Please turn on the power of the air conditioner 300." Information. The function of the voice information storage unit 104 is realized by, for example, the function of the flash memory 12.

The action detection unit 105 detects the action taken by the user 10. This action is, for example, an operation of the air conditioner 300 by the user 10 or a utterance of words by the user 10. In the present embodiment, the operation on the air conditioner 300 is substantially an operation on the voice output device 100. The function of the action detection unit 105 is realized by, for example, the function of the touch screen 13 or the function of the microphone 14.

In the history information generation unit 106, when the operation detected by the operation detection unit 102 and the action detected by the action detection unit 105 have a predetermined relationship, the content of the operation and the content of the action are combined. Generate the associated history information. The predetermined relationship is, for example, a relationship in which the difference between the detected times is within the threshold value, or a relationship in which the detected times are both times in the setting mode. The function of the history information generation unit 106 is realized, for example, by the processor 11 executing the program stored in the flash memory 12.

The history information storage unit 107 stores the history information generated by the history information generation unit 106. The function of the history information storage unit 107 is realized by, for example, the function of the flash memory 12.

When the action is detected by the action detection unit 105, the voice output unit 103 outputs the voice representing the content of the operation associated with the content of the action based on the history information. That is, when there is history information in which the content of the detected action and the content of the operation are associated with each other, the voice output unit 103 outputs a voice representing the content of this operation.

Here, when the operation and the action are detected within a predetermined time, the history information generation unit 106 generates history information in which the content of the operation and the content of the action are associated with each other. This predetermined time is, for example, a time of about several minutes. Then, the voice output unit 103 outputs the voice when the action is detected after the history information is generated. As described above, when the user 10 has a track record of continuously executing the operation and the action at a relatively short interval, the user 10 is likely to execute the operation together with the action. Therefore, when there is such a track record, the voice output unit 103 considers that the user 10 is likely to perform the above operation when the above action is detected, and outputs the voice for realizing the control by the above operation. Output automatically.

Specifically, in the present embodiment, the operation detection unit 102 detects the first operation of the user 10 on the air conditioner 300 and the second operation of the user 10 on the air conditioner 300. Here, the voice output unit 103 outputs the first voice representing the content of the first operation when the first operation is detected, and the second voice representing the content of the second operation when the second operation is detected. Output audio. Further, the action detection unit 105 includes an operation detection unit 102, and detects a second operation as an action performed by the user 10.

Further, when the first operation and the second operation are detected within the predetermined time, the history information generation unit 106 provides history information in which the contents of the first operation and the contents of the second operation are associated with each other. To generate. Then, when the second operation is detected after the history information is generated, the voice output unit 103 outputs the first voice and the second voice. As described above, when the first operation and the second operation are continuously detected, there is a high possibility that the first operation and the second operation are continuously executed. Therefore, in the case where the voice output unit 103 has such a track record, when the second operation is detected, the voice output unit 103 considers that the first operation is likely to be performed, and the second voice representing the content of the second operation. Not only that, the first voice representing the content of the first operation is also output.

In the present embodiment, if the history information generation unit 106 detects the first operation before the predetermined time elapses after the second operation is detected, the contents of the first operation and the second operation Generate history information associated with the content of the operation. That is, in the present embodiment, when the first operation is detected after the second operation is detected and the second operation is newly detected, the second voice representing the content of the second operation is used. In addition, a first voice representing the content of the first operation is also output. On the other hand, when the first operation is detected after the second operation is detected and the first operation is newly detected, the first voice representing the content of the first operation is output and the second operation is output. The second voice indicating the content of the operation is not output.

As described above, in the present embodiment, the voice representing the content of the operation that is considered to be more likely to be executed after the certain operation is detected is automatically output, while the voice indicating the content of the operation is further executed after the certain operation is detected. The voice indicating the content of the operation that is unlikely to be performed is not automatically output. For example, assume that the second operation is an operation of turning on the power to the air conditioner 300 and the first operation is an operation of setting the air conditioning mode to the air conditioner 300. In this case, it is highly likely that the first operation will be performed after the second operation has been performed, but it is unlikely that the second operation will be performed after the first operation has been performed. Therefore, only when the operation detected earlier is newly detected, a voice indicating the content of the operation detected later is additionally output.

Here, when the number of times the operation and the action are detected within the latest predetermined time within the latest predetermined period, the history information generation unit 106 reaches a predetermined threshold value. It is preferable to generate history information in which the content of the operation and the content of the action are associated with each other. The most recent predetermined period is, for example, the most recent month. The predetermined time is, for example, a few minutes. The predetermined threshold is, for example, 5 times. In this way, if the operation and the action are continuously detected five times in the last month, for example, it is highly likely that the operation is performed together with the action. Therefore, in such a case, when the above action is detected, it is preferable to output a voice representing the content of the above operation. According to such a configuration, it is possible to suppress inappropriate output of audio.

Next, the function of the device control device 200 will be described with reference to FIG. As shown in FIG. 5, the device control device 200 functionally includes a control unit 201, a voice detection unit 202, a voice output unit 203, a voice information storage unit 204, a device control unit 205, and command information. A storage unit 206 is provided. The voice detection means included in the device control device 200 corresponds to, for example, the voice detection unit 202. The device control means corresponds to, for example, the device control unit 205.

The control unit 201 controls the overall operation of the device control device 200. For example, the control unit 201 identifies the content of control for the air conditioner 300 from the voice detected by the voice detection unit 202, and causes the device control unit 205 to transmit a control command indicating the content of the specified control. The function of the control unit 201 is realized, for example, by the processor 21 executing the program stored in the flash memory 22.

The voice detection unit 202 detects the voice output by the voice output unit 103. Therefore, it is desirable that the voice detection unit 202 is arranged near the voice output unit 103. For example, the voice detection unit 202 is arranged in an area within a few meters from the voice output unit 103. The function of the voice detection unit 202 is realized by, for example, the function of the microphone 24.

The audio output unit 203 outputs various audios according to the control by the control unit 201. For example, the audio output unit 203 outputs an audio representing an announcement to the user 10. The function of the audio output unit 203 is realized, for example, by the cooperation of the processor 21 and the speaker 25.

The voice information storage unit 204 stores voice information. The voice information is, for example, information used to specify the content of control from the voice detected by the voice detection unit 202. For example, the voice information is information indicating information representing an electric signal corresponding to the voice representing the content of control for each content of control. The function of the voice information storage unit 204 is realized by, for example, the function of the flash memory 22.

The device control unit 205 controls the air conditioner 300 based on the content of the operation represented by the voice detected by the voice detection unit 202. For example, the device control unit 205 transmits a control command corresponding to the detected voice to the air conditioner 300 via the communication network 600 according to the control by the control unit 201. The function of the device control unit 205 is realized, for example, by the cooperation of the processor 21 and the communication interface 26.

The command information storage unit 206 stores command information. The command information is, for example, information in which the control content corresponding to the operation content and the control command are associated with each other. The function of the command information storage unit 206 is realized by, for example, the function of the flash memory 22.

Next, the history information will be described with reference to FIG. The history information shown in FIG. 6 is information indicating all combinations of a plurality of operations performed consecutively in the past. The detection start time is the time when the detection of the corresponding record combination is started. Operation A is the first operation among a plurality of operations performed consecutively in the past. Operation A corresponds to, for example, the second operation. Operation B is the second operation among the plurality of operations performed consecutively in the past. Operation B corresponds to, for example, the first operation. Operation C is the third operation among the plurality of operations performed consecutively in the past. Operation C corresponds to, for example, the first operation. In the present embodiment, the second operation is one, and the first operation is one or more.

The top record of the history information shown in FIG. 6 is the operation of turning on the power of the air conditioner 300 and the operation of setting the air conditioning mode of the air conditioner 300 to cooling at 12:00 on May 18, 2018. The operation of setting the set temperature of the air conditioner 300 to 28 ° C. is a record showing the results of continuous execution. When the operation of turning on the power of the air conditioner 300 is detected in such a case, the operation of setting the air conditioning mode of the air conditioner 300 to cooling and the operation of setting the set temperature of the air conditioner 300 to 28 ° C. are executed. It is likely to be done. Therefore, when the operation of turning on the power of the air conditioner 300 is detected, in addition to the voice instructing to turn on the power of the air conditioner 300, the voice instructing to set the air conditioning mode of the air conditioner 300 to cooling and the voice instructing to turn on the air conditioning mode. A voice instructing that the set temperature of the air conditioner 300 is set to 28 ° C. is output. In the example shown in FIG. 6, the content of the operation A corresponding to the second operation is the same for the top record and the bottom record. In such a case, it is preferable that the top record with the newest detection start time is adopted.

Note that FIG. 6 shows an example in which all combinations of detected operations are included in the history information, but the history information is not limited to this example. For example, only the combination of operations detected in the most recent predetermined period may be included in the history information. Further, the history information may include only a combination of operations detected more than a predetermined number of times in the latest predetermined period. Further, the record having the oldest detection start time among the competing combinations may be excluded from the history information. The combinations in a competing relationship are, for example, combinations in which the second operation is the same and at least one first operation is different.

Next, the audio output process executed by the audio output device 100 will be described with reference to the flowchart of FIG. 7. The audio output process is executed, for example, in response to the power of the audio output device 100 being turned on.

First, the processor 11 determines whether or not the operation has been detected (step S101). When the processor 11 determines that the operation has not been detected (step S101: NO), the processor 11 returns the process to step S101. On the other hand, when the processor 11 determines that the operation has been detected (step S101: YES), the processor 11 stores the detection start time (step S102). When the processor 11 completes the process of step S102, the processor 11 determines whether or not there is an interlocking setting (step S103). Specifically, the processor 11 determines whether or not the history information includes a record whose second operation is the operation detected in step S101.

When the processor 11 determines that there is an interlocking setting (step S103: YES), the processor 11 outputs a voice group (step S104). For example, the processor 11 performs a process of selecting one operation content included in the record and a process of outputting a voice representing the selected operation content from the speaker 15, all of the operation content included in the record. Repeat until you select it. On the other hand, when the processor 11 determines that there is no interlocking setting (step S103: NO), the processor 11 outputs a single sound (step S105). For example, the processor 11 causes the speaker 15 to output a voice representing the content of the operation detected in step S101.

When the process of step S104 or the process of step S105 is completed, the processor 11 determines whether or not the operation is detected (step S106). When the processor 11 determines that the operation has been detected (step S106: YES), the processor 11 determines whether or not there is an interlocking setting (step S107). Specifically, the processor 11 determines whether or not the history information includes a record whose second operation is the operation detected in step S106. When the processor 11 determines that there is an interlocking setting (step S107: YES), the processor 11 outputs a voice group (step S108). On the other hand, when the processor 11 determines that there is no interlocking setting (step S107: NO), the processor 11 outputs a single sound (step S109).

When the processor 11 completes the process of step S108 or the process of step S109, or determines that the operation has not been detected (step S106: NO), whether or not the first time has elapsed from the detection start time. (Step S110). The first time is the predetermined time described above, for example, a few minutes. When the processor 11 determines that the first time has not elapsed from the detection start time (step S110: NO), the processor 11 returns the process to step S106.

On the other hand, when the processor 11 determines that the first time has elapsed from the detection start time (step S110: YES), the processor 11 determines whether or not a plurality of operations have been detected within the first period (step S111). The first period is a period from the detection start time to the elapse of the first time. When the processor 11 determines that a plurality of operations have been detected within the first period (step S111: YES), the processor 11 generates history information (step S112). For example, the processor 11 updates the history information so as to include a record in which the operation detected in step S101 is the second operation and the operation detected in step S106 is the first operation. When it is determined that the processor 11 has not detected a plurality of operations within the first period (step S111: NO), or when the process of step S112 is completed, the process returns to step S101.

The audio output method according to the present embodiment is realized by the audio output device 100 according to the present embodiment executing the audio output process shown in FIG. 7. In this voice output method, first, an operation on the equipment by the user 10 is detected, and when this operation is detected, a voice representing the content of this operation is output. Further, in this voice output method, the action taken by the user 10 is detected. Then, in this voice output method, when this action is detected and this operation and this action have a predetermined relationship, the above sound is output.

As described above, in the present embodiment, the operation of the equipment by the user 10 and the action performed by the user 10 have a predetermined relationship, and when the action is detected, a voice representing the content of the operation is heard. Is output. Therefore, according to the present embodiment, the equipment control device 200 that controls the equipment based on the voice can be used to easily bring the equipment into a desired control state. For example, it can be expected that the control of equipment and devices that reflects the tastes of the user 10 can be realized by a single operation on the voice output device 100.

Further, in the present embodiment, when a plurality of operations have been executed within a predetermined time, and when any of these plurality of operations is executed, the contents of the other operations are displayed. The voice to be represented is automatically output. Therefore, according to the present embodiment, the equipment can be brought into a desired control state with a small number of operations.

Further, in the present embodiment, when a plurality of operations have been executed within a predetermined time, when the first operation among these a plurality of operations is executed, the contents of other operations are represented. Audio is automatically output. Therefore, according to the present embodiment, the equipment can be appropriately put into a desired control state with a small number of operations.

(Embodiment 2)
In the first embodiment, an example in which keywords are not associated with operations and actions that are continuously executed has been described. In this embodiment, an example of associating a keyword with a continuously executed operation and action will be described. Hereinafter, a part different from the first embodiment will be basically described.

First, the function of the audio output device 120 will be described with reference to FIG. As shown in FIG. 8, the voice output device 120 functionally includes a control unit 101, an operation detection unit 102, a voice output unit 103, a voice information storage unit 104, an action detection unit 105, and history information. A generation unit 106 and a history information storage unit 107 are provided. The action detection unit 105 includes a voice detection unit 108. The voice detection means included in the voice output device 120 corresponds to, for example, the voice detection unit 108.

The operation detection unit 102 detects the first operation of the equipment by the user 10. When the first operation is detected, the voice output unit 103 outputs the first voice representing the content of the first operation. The action detection unit 105 includes a voice detection unit 108 that detects a third voice representing a word uttered by the user 10, and detects the utterance of the word by the user 10 as an action performed by the user 10. The function of the voice detection unit 108 is realized by, for example, the function of the microphone 14.

When the first operation and the third voice are detected within a predetermined time, the history information generation unit 106 generates history information in which the content of the first operation and the above words are associated with each other. The audio output unit 103 outputs the first audio when the third audio is detected after the history information is generated.

The word represented by the third voice is a word that is likely to be uttered with the execution of the first operation, and is treated as a keyword. Then, when this keyword has a track record of being issued at the same time as the execution of the first operation, when this keyword is issued, a voice representing the first operation is automatically output. The number of first operations associated with this keyword based on the history information may be any number as long as it is one or more. Further, the history information may be information in which the content of the first operation is associated with the keyword issued immediately before or immediately after the first operation, or is issued immediately before the content of the first operation and the first operation. The information may be associated with a keyword, or the information may be associated with the content of the first operation and the keyword issued immediately after the first operation.

Next, the history information in the present embodiment will be described with reference to FIG. In the present embodiment, the history information is information in which the keyword, the content of the operation A, the content of the operation B, and the content of the operation C are associated with each other. The operation A, the operation B, and the operation C are operations performed together with the utterance of the keyword, and are the first operations. The top record shown by the history information is the operation of turning on the power of the air conditioner 300, the operation of setting the air conditioning mode of the air conditioner 300 to cooling, and the operation of the air conditioner 300, along with the utterance of the keyword "air conditioner". It shows that the operation of setting the set temperature to 28 ° C. has been performed. In the case where there is such a track record, when the keyword "air conditioner" is spoken, a voice instructing to turn on the power of the air conditioner 300 and an instruction to set the air conditioning mode of the air conditioner 300 to cooling are instructed. The voice and the voice instructing to set the set temperature of the air conditioner 300 to 28 ° C. are automatically output. Note that FIG. 9 shows an example in which all combinations of the detected keyword and the first operation are included in the history information, but the history information is not limited to this example. For example, only the combination of the keyword and the first operation detected in the latest predetermined period may be included in the history information. Further, only the combination of the keyword and the first operation detected more than a predetermined number of times in the latest predetermined period may be included in the history information. Further, the record having the oldest detection start time among the competing combinations may be excluded from the history information. Competitive combinations are, for example, combinations in which the keywords are the same and at least one first operation is different.

Next, the audio output process executed by the audio output device 120 will be described with reference to the flowchart of FIG. The audio output process is executed, for example, in response to the power of the audio output device 120 being turned on. Here, an example in which the content of a series of operations performed after the utterance of a keyword is associated with the keyword will be described.

First, the processor 11 determines whether or not the word has been detected (step S201). For example, the processor 11 determines whether or not a voice representing a word that can be a keyword is detected by the microphone 14. When the processor 11 determines that the word has not been detected (step S201: NO), the processor 11 returns the process to step S201. On the other hand, when the processor 11 determines that the word has been detected (step S201: YES), the processor 11 stores the detection start time (step S202).

When the processor 11 completes the process of step S202, the processor 11 determines whether or not there is an interlocking setting (step S203). Specifically, the processor 11 determines whether or not the history information includes a record using the word detected in step S201 as a keyword. When the processor 11 determines that there is an interlocking setting (step S203: YES), the processor 11 outputs a single voice or voice group (step S204). If the record contains the content of a single operation, a single voice is output. On the other hand, when the record contains the contents of a plurality of operations, a voice group is output.

When the processor 11 completes the process of step S204 or determines that there is no interlocking setting (step S203: NO), the processor 11 determines whether or not an operation has been detected (step S205). When the processor 11 determines that the operation has been detected (step S205: YES), the processor 11 outputs a single voice (step S206). When the processor 11 completes the process of step S206 or determines that the operation has not been detected (step S205: NO), the processor 11 determines whether or not the first time has elapsed from the detection start time (step S205). S207).

When the processor 11 determines that the first time has not elapsed from the detection start time (step S207: NO), the processor 11 returns the process to step S205. On the other hand, when the processor 11 determines that the first time has elapsed from the detection start time (step S207: YES), the processor 11 determines whether or not the operation is detected within the first period (step S208). When the processor 11 determines that the operation has been detected within the first period (step S208: YES), the processor 11 generates history information (step S209). The processor 11 updates the history information so as to include a record in which the keyword which is the detected word and the content of the operation detected within the first period are associated with each other. When the processor 11 completes the process of step S209, or determines that the operation has not been detected within the first period (step S208: NO), the processor 11 returns the process to step S201.

In the present embodiment, when at least one operation is detected together with the utterance of the keyword, when the utterance of the keyword is detected, a voice representing the content of the at least one operation is automatically output. Therefore, according to the present embodiment, the equipment can be put into a desired control state by speaking the keyword. Further, in the present embodiment, the keyword associated with the series of operations for bringing the equipment into the desired control state can be freely selected by the user, which enhances the convenience of the user.

(Embodiment 3)
In the first embodiment, an example in which the action detected by the action detection unit 105 is an operation detected by the operation detection unit 102 has been described. Further, in the second embodiment, an example in which the action detected by the action detection unit 105 is a voice utterance detected by the voice detection unit 108 has been described. In the present embodiment, an example will be described in which the action detected by the action detection unit 105 is both an operation detected by the operation detection unit 102 and a voice utterance detected by the voice detection unit 108. Hereinafter, the parts different from those of the first and second embodiments will be basically described.

First, the function of the audio output device 130 will be described with reference to FIG. As shown in FIG. 11, the voice output device 130 functionally includes a control unit 101, a voice output unit 103, a voice information storage unit 104, an action detection unit 105, a history information generation unit 106, and a history. It includes an information storage unit 107. The action detection unit 105 includes an operation detection unit 102 and a voice detection unit 108.

The action detection unit 105 includes a voice detection unit 108 that detects a third voice representing a word uttered by the user 10, and detects the utterance of the word by the user 10 as an action performed by the user 10. When the first operation, the second operation, and the third voice are detected within a predetermined time, the history information generation unit 106 associates the contents of the first operation, the contents of the second operation, and the above words. Generate the history information. The voice output unit 103 outputs the first voice and the second voice when the third voice is detected after the history information is generated.

That is, in the present embodiment, when the first operation, the second operation, and the third voice are continuously detected, and at least one of the second operation and the third voice is detected, the first operation is performed. The first voice and the second voice are output. For example, as in the second embodiment, it is assumed that the history information shown in FIG. 9 is generated. In the present embodiment, for example, the power of the air conditioner 300 is turned on regardless of whether the keyword "air conditioner" is pronounced or the operation of turning on the power of the air conditioner 300 is performed. Is output, a voice instructing that the air conditioning mode of the air conditioner 300 is to be cooled, and a voice instructing that the set temperature of the air conditioner 300 is set to 28 ° C. are output.

In the present embodiment, when a plurality of operations have been detected together with the utterance of the keyword, when the utterance of the keyword is detected, or when the first operation among the plurality of operations is detected, these Voices representing the contents of multiple operations are automatically output. Therefore, according to the present embodiment, the equipment can be put into a desired control state by uttering a keyword or the first operation in a series of operations.

(Embodiment 4)
In the first to third embodiments, an example in which the number of equipment to be controlled is only one has been described. In the present embodiment, an example in which the number of equipment to be controlled is a plurality of equipment will be described. In the present embodiment, as shown in FIG. 12, an example in which the equipment to be controlled is the air conditioner 300, the bathroom heater 310, and the water heater 320 will be described.

The operation detection unit 102 detects the first operation of the plurality of equipment by the user 10 for the first equipment and the second operation of the plurality of equipment by the user 10 for the second equipment. In the present embodiment, it is assumed that any one of the air conditioner 300, the bathroom heater 310, and the water heater 320 is the second equipment, and the remaining two equipments are the first equipment. .. However, any of the three equipments may be the second equipment.

When the first operation is detected, the voice output unit 103 outputs the first voice representing the content of the first operation, and when the second operation is detected, outputs the second voice representing the content of the second operation. do. The action detection unit 105 includes an operation detection unit 102, and detects a second operation as an action performed by the user 10. When the first operation and the second operation are detected within a predetermined time, the history information generation unit 106 generates history information in which the contents of the first operation and the contents of the second operation are associated with each other. .. Then, when the second operation is detected after the history information is generated, the voice output unit 103 outputs the first voice and the second voice.

Next, the history information in the present embodiment will be described with reference to FIG. In the present embodiment, the history information is information in which the detection start time, the content of the operation A, the content of the operation B, and the content of the operation C are associated with each other. Operation A, operation B, and operation C are a series of operations that are continuously detected within a predetermined time. This series of operations may include a plurality of operations for one equipment. The operation of any one of the operation A, the operation B, and the operation C is the second operation. The remaining two operations are the first operation.

The top record indicated by the history information is the operation of turning on the power of the air conditioner 300 between the detection start time of 12:00 on May 18, 2018 and the elapse of a predetermined time. It shows that the operation of turning on the power of the bathroom heater 310 and the operation of turning on the power of the water heater 320 have been continuously executed. In the case where there is such a track record, one of the operation of turning on the power of the air conditioner 300, the operation of turning on the power of the bathroom heater 310, and the operation of turning on the power of the water heater 320 can be performed. When detected, a voice instructing the power of the air conditioner 300 to be turned on, a voice instructing the bathroom heater 310 to be turned on, and a voice instructing the water heater 320 to be turned on. Is automatically output.

In the present embodiment, when a series of operations have been performed on a plurality of equipments and devices, and any one of the series of operations is detected, the content of the series of operations is represented. Audio is automatically output. Therefore, according to the present embodiment, a plurality of equipment can be brought into a desired control state by one of a series of operations.

(Embodiment 5)
In the fourth embodiment, an example in which a keyword is not associated with a series of operations for a plurality of equipments and devices has been described. In this embodiment, an example in which a keyword is associated with a series of operations for a plurality of equipments and devices will be described. Hereinafter, basically, the parts different from the fourth embodiment will be described.

The action detection unit 105 includes a voice detection unit 108 that detects a third voice representing a word uttered by the user 10, and detects the utterance of the above word by the user 10 as an action performed by the user 10. When the first operation, the second operation, and the third voice are detected within a predetermined time, the history information generation unit 106 associates the contents of the first operation, the contents of the second operation, and the above words. Generate the history information.

The voice output unit 103 outputs the first voice and the second voice when the third voice is detected after the history information is generated. As described above, in the present embodiment, when a series of operations for a plurality of equipments and a utterance of a keyword corresponding to the third voice have been continuously detected, one of the series of operations is performed. Alternatively, when the utterance of the keyword is detected, a series of voices representing a series of operations are output.

Next, the history information in the present embodiment will be described with reference to FIG. In the present embodiment, the history information is information in which the keyword, the content of the operation A, the content of the operation B, and the content of the operation C are associated with each other. Operation A, operation B, and operation C are a series of operations that are continuously detected within a predetermined time. The keyword is a word detected within the predetermined time together with the above series of operations. The operation of any one of the operation A, the operation B, and the operation C is the second operation. The remaining two operations are the first operation.

The top record shown by the history information is the pronunciation of the keyword "I'm back", the operation to turn on the air conditioner 300, the operation to turn on the bathroom heater 310, and the water heater 320. It shows that the operation of turning on the power and the operation of turning on the power have been executed continuously. When there is such a track record, the keyword "I'm back now" or the operation to turn on the power of the air conditioner 300, the operation to turn on the power of the bathroom heater 310, and the power of the water heater 320 are turned on. When any of the on operations is detected, a voice instructing to turn on the power of the air conditioner 300, a voice instructing to turn on the power of the bathroom heater 310, and a water heater A voice instructing to turn on the power of the 320 is automatically output.

In the present embodiment, when a series of operations have been performed on a plurality of equipments and devices together with the utterance of the keyword, and when the utterance of the keyword or any one of the series of operations is detected, these The voice indicating the contents of the series of operations of is automatically output. Therefore, according to the present embodiment, it is possible to bring a plurality of equipments into a desired control state by speaking a keyword or operating one of a series of operations.

(Embodiment 6)
In the first to fifth embodiments, an example in which history information is automatically generated has been described. In this embodiment, an example in which history information is manually generated will be described. In the present embodiment, the touch screen 13 or the microphone 14 receives an instruction to shift to the setting mode. The transition instruction receiving means corresponds to, for example, a touch screen 13 or a microphone 14.

When the transition instruction is received by the touch screen 13 or the microphone 14 and the operation and the action are detected while the setting mode is set based on the transition instruction, the history information generation unit 106, the content and the action of the operation. Generates history information associated with the contents of. The voice output unit 103 outputs the voice when the action is detected after the history information is generated.

For example, it is assumed that the user 10 gives a transition instruction to the audio output device 100. In the voice output device 100, the processor 11 and the speaker 15 cooperate with each other to utter a voice, and the voice uttered by the user 10 is detected by the microphone 14. In this case, for example, the voice output device 100 utters "What do you want to work with?". When the user 10 utters "water heater, power supply, on", the voice output device 100 utters "what is linked?". When the user 10 utters "bathroom heating, power, on", the voice output device 100 utters "what is linked?". When the user 10 utters "end", the voice output device 100 utters "what is the keyword?". When the user 10 utters "I'm back now", the voice output device 100 utters "The setting is completed".

Hereinafter, the setting process executed by the audio output device 100 will be described with reference to FIG.

First, the processor 11 determines whether or not there is an instruction to shift to the setting mode (step S301). When the processor 11 determines that there is no instruction to shift to the setting mode (step S301: NO), the processor 11 returns the process to step S301. On the other hand, when the processor 11 determines that there is an instruction to shift to the setting mode (step S301: YES), the processor 11 utters a message prompting the operation (step S302).

When the processor 11 completes the process of step S302, it determines whether or not there is a control designation operation (step S303). The control designation operation is, for example, an operation of designating control by voice. When the processor 11 determines that there is a control designation operation (step S303: YES), the processor 11 stores the designated control (step S304). When the processor 11 determines that there is no control designation operation (step S303: NO), or when the process of step S304 is completed, the processor 11 determines whether or not there is a setting end operation (step S305). When the processor 11 determines that there is no setting end operation (step S305: NO), the processor 11 returns the process to step S302. On the other hand, when the processor 11 determines that there is a setting end operation (step S305: YES), the processor 11 utters a message prompting the utterance of the keyword (step S306).

When the processor 11 completes the process of step S306, it determines whether or not there is an utterance of the keyword (step S307). When the processor 11 determines that there is an utterance of the keyword (step S307: YES), the processor 11 generates history information with the keyword (step S308). On the other hand, when the processor 11 determines that there is no utterance of the keyword (step S307: NO), the processor 11 generates history information without the keyword (step S309). When the processor 11 completes the process of step S308 or the process of step S309, the processor 11 returns the process to step S301.

In this embodiment, a series of operations desired by the user is manually set. Therefore, according to the present embodiment, it is possible to prevent the user from making an unintended setting.

(Modification example)
Although the embodiments of the present invention have been described above, various modifications and applications are possible in carrying out the present invention.

In the present invention, which part of the configuration, function, and operation described in the above embodiment is adopted is arbitrary. Further, in the present invention, in addition to the above-described configurations, functions, and operations, further configurations, functions, and operations may be adopted. Further, the configurations, functions, and operations described in the above embodiments can be freely combined.

For example, in the second embodiment, an example in which a series of operations on the equipment is performed after the utterance of the keyword has been described. A keyword may be spoken after a series of operations on the equipment.

The operation described as a screen operation may be a voice operation, or the operation described as a voice operation may be a screen operation.

By applying an operation program that defines the operation of the audio output device 100 according to the present invention to an existing personal computer or information terminal device, it is possible to make the personal computer or the like function as the audio output device 100 according to the present invention. be. The distribution method of such a program is arbitrary, and is stored and distributed in a computer-readable recording medium such as a CD-ROM (Compact Disk Read-Only Memory), a DVD (Digital Versatile Disk), or a memory card. It may be distributed via a communication network such as the Internet.

The present invention allows for various embodiments and modifications without departing from the broad spirit and scope of the present invention. Moreover, the above-described embodiment is for explaining the present invention, and does not limit the scope of the present invention. That is, the scope of the present invention is indicated not by the embodiment but by the claims. Then, various modifications made within the scope of the claims and within the equivalent meaning of the invention are considered to be within the scope of the present invention.

The present invention is applicable to an equipment control system including an equipment control device that controls equipment equipment based on voice.

10 users, 11,21 processors, 12,22 flash memory, 13,23 touch screens, 14,24 microphones, 15,25 speakers, 16,26 communication interfaces, 100,120,130 voice output devices, 101,201 controls , 102 operation detection unit, 103, 203 voice output unit, 104, 204 voice information storage unit, 105 action detection unit, 106 history information generation unit, 107 history information storage unit, 108, 202 voice detection unit, 200 device control device, 205 Equipment control unit, 206 Command information storage unit, 300 Air conditioner, 310 Bathroom heater, 320 Water heater, 600 Communication network, 1000 Equipment control system

Claims (10)

A voice output device for putting the equipment into a desired control state by using a device control device that controls the equipment based on voice.
An operation detection means for detecting the first operation for controlling the first equipment for the voice output device, and
When the first operation is detected, an audio output means for outputting a first audio representing the content of the first operation, and an audio output means.
A voice detection means for detecting a user voice that is a voice emitted by a user and represents the first word,
If the first operation within a predetermined time and said user voice is examined knowledge, history information generating means for generating history information the content of the first operation and the first word associated And with
After the history information is generated, the voice output means detects the user voice representing the first word associated with the content of the first operation by the history information , the first voice. To output,
Audio output device. The operation detecting means detects a second operation for controlling the first equipment with respect to the voice output device, and detects the second operation.
When the second operation is detected, the voice output means outputs a second voice representing the content of the second operation.
When the first operation, the second operation, and the user voice are detected within the predetermined time, the history information generating means includes the contents of the first operation, the contents of the second operation, and the above. Generate the history information associated with the first word,
After the history information is generated, the voice output means performs the second operation corresponding to the content of the second operation associated with the content of the first operation and the first word by the history information. Is detected, the first voice and the second voice are output.
The audio output device according to claim 1. When the first operation is detected before the predetermined time elapses after the second operation is detected, the history information generating means has the contents of the first operation and the contents of the second operation. Generates the history information in which the first word is associated with the first word.
The audio output device according to claim 2. The operation detecting means detects a second operation for controlling the second equipment with respect to the voice output device, and detects the second operation.
When the second operation is detected, the voice output means outputs a second voice representing the content of the second operation.
When the first operation, the second operation, and the user voice are detected within the predetermined time, the history information generating means includes the contents of the first operation, the contents of the second operation, and the above. Generate the history information associated with the first word,
After the history information is generated, the voice output means detects the user voice representing the first word associated with the content of the first operation and the content of the second operation by the history information. If so, the first voice and the second voice are output.
The audio output device according to claim 1. In the history information generation means, the number of times that the first operation and the user voice representing the first word are detected within the predetermined time within the latest predetermined period is predetermined. When the threshold value is reached, the history information in which the content of the first operation and the first word are associated with each other is generated.
The audio output device according to any one of claims 1 to 4. A voice output device for putting the equipment into a desired control state by using a device control device that controls the equipment based on voice.
An operation detection means for detecting the first operation for controlling the first equipment for the voice output device, and
When the first operation is detected, an audio output means for outputting a first audio representing the content of the first operation, and an audio output means.
A voice detection means for detecting a user voice that is a voice emitted by a user and represents the first word,
A transition instruction receiving means that accepts a transition instruction to the setting mode, and
The shift instruction is accepted by the prior SL migration instruction receiving means, while the setting mode based on the shift instruction is set, when the first operation and said user voice is detected, the first operation comprising of a history information generation means for generating the history information and the first words associated with the content, and
After the history information is generated, the voice output means detects the user voice representing the first word associated with the content of the first operation by the history information, the first voice. To output,
Voice output device. A voice output device for putting the equipment into a desired control state by using a device control device that controls the equipment based on voice.
For the voice output device, the first operation and for controlling the equipment, for the audio output device, an operation detection means for detecting a second operation for controlling the equipment,
When the first operation is detected, a first voice representing the content of the first operation is output, and when the second operation is detected, a second voice representing the content of the second operation is output . Output means and
When the number of times the first operation and the second operation are detected within the latest predetermined period and within a predetermined time reaches a predetermined threshold value, the content of the first operation and A history information generation means for generating history information associated with the content of the second operation is provided.
When the second operation corresponding to the content of the second operation associated with the content of the first operation is detected by the history information after the history information is generated, the voice output means said. Outputs the first voice and the second voice,
Audio output device. A device control apparatus which controls the equipment based on the voice, by using the device control apparatus, the equipment comprising a device control system including an audio output device to the desired control state,
The audio output device is
An operation detecting means for detecting an operation for controlling the equipment of the audio output device, and an operation detecting means for detecting the operation.
When the operation is detected, an audio output means for outputting a first audio representing the content of the operation, and
A voice detection means for detecting a user voice that is a voice emitted by a user and represents the first word,
If said operation with the user voice within a predetermined time is examined knowledge, and a history information generating means for generating history information the content of the operation and the first word associated ,
The device control device is
A voice detecting means for detecting the first voice output by the voice output means, and a voice detecting means.
A device control means for controlling the equipment based on the content of the operation represented by the first voice detected by the voice detection means is provided.
After the history information is generated, the voice output means outputs the first voice when the user voice representing the first word associated with the content of the operation is detected by the history information. do,
Equipment control system. It is a voice output method executed by a voice output device for putting the equipment into a desired control state by using a device control device that controls equipment based on voice.
The operation for controlling the equipment with respect to the audio output device is detected, and the operation is detected.
When the operation is detected, the first sound representing the content of the operation is output.
Detects the user voice that is the voice uttered by the user and represents the first word,
Wherein if the operation and said user voice is examined knowledge, generates history information the content of the operation and the first word associated within a predetermined time,
After the history information is generated , when the user voice representing the first word associated with the content of the operation is detected by the history information, the first voice is output.
Audio output method. A computer provided with an audio output device for putting the equipment into a desired control state by using an equipment control device that controls equipment based on voice.
An operation detecting means for detecting an operation for controlling the equipment of the audio output device,
When the operation is detected by the operation detecting means, the voice output means for outputting the first voice representing the content of the operation,
A voice detecting means for detecting a user voice that is a voice emitted by a user and represents the first word.
If said operation with the user voice within a predetermined time is examined knowledge, history information generating means for generating history information the content of the operation and the first word is associated, to function as a Program for
After the history information is generated, the voice output means outputs the first voice when the user voice representing the first word associated with the content of the operation is detected by the history information. do,
program.

Images