JP2016114744A - Electronic device control system, terminal device and server - Google Patents

Abstract

PROBLEM TO BE SOLVED: To enable control of an electronic device with a small amount of utterance.SOLUTION: When character data included in voice data outputted from a microphone 22 matches a predetermined keyword, a microcomputer 21 of a terminal device 2 transmits voice data outputted from the microphone 22 after the predetermined keyword to a home server 3, receives a control command transmitted by the home server 3 and transmits a remote control code corresponding to the received control code to an electronic device through an infrared LED 24. A CPU 31 of the home server 3 receives voice data transmitted by the terminal device 2, transmits the received voice data to a cloud server 4, receives a voice command corresponding to the transmitted voice data from the cloud server 4, determines a control command on the bases of the received voice command and transmits the determined control command to the terminal device 2.SELECTED DRAWING: Figure 2

Description

  The present invention relates to an electronic device control system for controlling electronic devices such as a television and lighting, a terminal device that constitutes the electronic device control system, and a server.

  2. Description of the Related Art In recent years, a technique for controlling electronic devices such as a television and lighting using a voice spoken by a user has been developed (see, for example, Patent Document 1). FIG. 6 is a diagram showing a system configuration of a conventional electronic device control system. The electronic device control system 11 includes a terminal device 12 (12a to 12c), a home server 13, a cloud server 14, and a router 15. For example, the terminal device 12a, the home server 13, and the router 15 are installed in the living room on the first floor of the home. The terminal device 12b is installed in the study on the second floor of the home. The terminal device 12c is installed in a child room on the second floor of the home. The cloud server 14 is an external server installed outside the home.

  The terminal devices 12a to 12c have the same configuration. The terminal device 12 includes a microcomputer 121, a microphone 122, a speaker 123, an infrared LED 124, and the like. The microcomputer 121 controls each part constituting the terminal device 12. The microcomputer 121 has a wireless LAN network processor, and performs communication with the router 15 according to the wireless LAN. The microphone 122 collects external sound and outputs it as sound data. The audio data output from the microphone 122 is input to the microcomputer 121 via a DSP (Digital Signal Processor) (not shown). The microphone 122 is a digital microphone, and A / D converts the collected sound data and outputs it to the DSP. The speaker 123 outputs sound based on the sound data. The speaker 123 outputs audio data amplified by a power amplifier (not shown) after the D / A converter (not shown) D / A converts the audio data (digital data). The infrared LED 124 transmits a remote control code for controlling the television 16 and the illumination 17 to the television 16 and the illumination 17.

  The home server 13 includes a CPU (Central Processing Unit) 131, a RAM (Random Access Memory) 132, a storage unit 133, and a network interface (hereinafter referred to as “NW I / F”) 134. The CPU 131 controls each unit constituting the home server 13. The RAM 132 functions as a work memory for the CPU 131. The storage unit 133 stores various programs and data. The NW I / F 134 is for communicating with an external device (for example, the router 15) according to a wired LAN or a wireless LAN.

  The router 15 has an access point function and communicates with the terminal device 12 according to the wireless LAN. The router 15 communicates with the home server 13 according to a wired LAN. The router 15 connects the home server 13 and the cloud server 14 to the Internet.

  The home server 13 described above stores a device control information database (hereinafter referred to as “device control information DB”) in the storage unit 133 in order to control electronic devices (the television 16 and the illumination 17). FIG. 7 is a diagram illustrating the device control information DB. As shown to Fig.7 (a), ID of terminal device 12a-12c and the position of terminal device 12a-12c are matched and memorize | stored in apparatus control information DB. For example, it is stored that the terminal device 12a whose ID is “XXX1” is in the living room on the first floor of the home. As shown in FIG. 7B, the device control information DB stores the IDs of the terminal devices 12a to 12c and the profiles of the electronic devices in association with each other. Electronic device profiles include “Accessory”, “Service”, and “Characteristics”. “Accessory” indicates an electronic device (lighting, television) to be controlled. “Service” indicates a function to be controlled (Light, tv_control). “Characteristics” indicates an execution content (Power State, Volume Control, etc.).

  Further, as shown in FIG. 7C, in order to identify the electronic device to be controlled, the Accessory and the voice command are stored in association with each other in the device control information DB. Also, as shown in FIG. 7D, in order to specify the function to be controlled, the device control information DB stores a service and a voice command in association with each other. Further, as shown in FIG. 7 (e), in order to specify the execution contents, the device control information DB stores Characteristics and voice commands in association with each other.

  For example, if the voice command is “Turn on TV”, the Accessory (the electronic device to be controlled) corresponding to the voice command “TV” is “TV”. Further, Characteristics (execution content) corresponding to the voice command “Take” is “Power State”. Therefore, the CPU 131 can determine to turn on the television based on the voice command and the device control information DB. Then, the CPU 131 transmits a control command for turning on the television to the terminal device 12.

  FIG. 8 is a sequence diagram showing the operation of the conventional electronic device control system. Hereinafter, a series of operations for controlling the television 16 will be described with reference to FIG. First, the user speaks a predetermined keyword in order to set the data transmission mode in which the terminal device 12 transmits audio data to the home server 13. For example, the user speaks "Hello Onkyo" (shown in FIG. 8 (1)). The microphone 122 of the terminal device 12 collects the voice uttered by the user and outputs it to the microcomputer 121 as voice data. The microcomputer 121 has a voice recognition function, recognizes character data included in the voice data, and determines whether or not the recognized character data matches a predetermined keyword. When the microcomputer 121 determines that the recognized character data matches the predetermined keyword, the microcomputer 121 shifts to the data transmission mode ((2) in FIG. 8).

  Next, for example, when the user wants to turn on the TV, the user speaks “Turn on TV” ((3) in FIG. 8). The microphone 122 of the terminal device 12 collects the voice uttered by the user and outputs it to the microcomputer 121 as voice data. The microcomputer 121 transmits audio data ("Turn on TV") to the home server 14 ((4) in FIG. 8). The CPU 131 of the home server 13 receives the audio data transmitted from the terminal device 12 through the NW I / F 134. Next, the CPU 131 transmits voice data to the cloud server 14 by the NW I / F 134 and makes a voice recognition inquiry ((5) in FIG. 8).

  The cloud server 14 receives the audio data transmitted from the home server 13. The cloud server 14 has a voice recognition function and recognizes character data included in the voice data. Next, the cloud server 14 transmits a voice command (“Turn on TV”) corresponding to the recognized character data to the home server 13 and responds to the result of the voice recognition ((6) in FIG. 8). The CPU 131 of the home server 13 receives the voice command transmitted from the cloud server 14 via the NW I / F 134. The CPU 131 analyzes the speech recognition result ((7) in FIG. 8). Specifically, the CPU 131 determines a control command corresponding to the voice command based on the device control information DB and the voice command. The CPU 131 transmits the control command determined by the NW I / F 134 (control command for turning on the power of the television) ((8) in FIG. 8).

  At the same time, the CPU 131 transmits a request for voice synthesis to the cloud server 14 by the NW I / F 134 ((9) in FIG. 8). For example, if the voice command is “Turn on TV”, the CPU 131 causes the cloud server 14 to perform speech synthesis of “Turn on TV” corresponding to this. The cloud server 14 performs speech synthesis, transmits speech synthesis data to the home server 13, and responds with the result of speech synthesis ((10) in FIG. 8). The CPU 131 of the home server 13 receives voice synthesis data through the NW I / F 134. Next, the CPU 131 transmits the speech synthesis data received by the NW I / F 134 to the terminal device 12 ((11) in FIG. 8).

  On the other hand, the microcomputer 121 of the terminal device 12 receives the control command transmitted by the home server 13. Next, the microcomputer 121 transmits a remote control code (remote control code for turning on the TV) corresponding to the received control command by the infrared LED 124 ((12) in FIG. 8). The television 16 is turned on by a remote control code that turns on the television transmitted by the terminal device 12 ((13) in FIG. 8). Further, the microcomputer 121 receives the speech synthesis data transmitted from the home server 13. The microcomputer 121 causes the speaker 123 to output sound (“TV set”) based on the received voice synthesis data.

JP2013-088535A

  In the prior art, the user needs to speak a predetermined keyword in order to set the terminal device 12 to the data transmission mode. For this reason, for example, after having turned on the TV, if you want to increase the volume of the TV, "Hello Onkyo", "put TV", "Hello Onkyo", have to recite a predetermined keyword as "volume up to the TV." I must. In addition, the user needs to repeat the electronic device (for example, a television) to be controlled. Further, since it is necessary to perform voice recognition of keywords one by one on the terminal device 12 side, the terminal device 12 may not transition to the data transmission mode when the processing capability of the terminal device 12 is low and the keyword recognition rate is low. .

  An object of the present invention is to enable control of an electronic device with a small amount of speech.

  An electronic device control system according to a first aspect of the present invention has a voice section detection function and a voice recognition function, communicates with an external server that transmits voice commands corresponding to voice data, and a terminal device that communicates with the server An electronic device control system comprising: a microphone that collects external sound and outputs it as sound data; a transmitter that transmits a code for controlling the electronic device; and the microphone When the character data included in the voice data to be output matches a predetermined keyword, the voice data output by the microphone after the predetermined keyword is transmitted to the server, and the control command transmitted by the server is received. A first control unit that transmits a code corresponding to the received control command to the electronic device by the transmission unit; The voice data transmitted by the terminal device is received, the received voice data is transmitted to the external server, a voice command corresponding to the transmitted voice data is received from the external server, and a control command based on the received voice command And a second control unit for transmitting the determined control command to the terminal device.

  In the present invention, the first control unit of the terminal device transmits, to the server, voice data output from the microphone after the predetermined keyword when the character data included in the voice data output from the microphone matches the predetermined keyword. Send. The second control unit of the server transmits the voice data transmitted from the terminal device to the external server, and receives a voice command corresponding to the transmitted voice data from the external server. Further, the second control unit determines a control command based on the received voice command, and transmits the determined control command to the terminal device. And the 1st control part of a terminal unit receives the control command which a server transmits, and transmits the code corresponding to the received control command to an electronic device by a transmission part. Therefore, the user only has to speak a predetermined keyword once, and does not need to repeat the predetermined keyword, so that the electronic device can be controlled with a small amount of utterance. In addition, it is not necessary to perform voice recognition of keywords one by one on the terminal device side.

  An electronic device control system according to a second aspect is the electronic device control system according to the first aspect, wherein the server further includes a storage unit, and the second control unit is an electronic device to be controlled corresponding to the received voice command. A device, a function to be controlled, and an execution content are stored in the storage unit, and a control command is determined based on the electronic device to be controlled, the function to be controlled, and the execution content stored in the storage unit. And

  In the present invention, the second control unit of the server sets the electronic device to be controlled (for example, a television) corresponding to the received voice command, the function to be controlled (for example, television control), and the execution content (for example, power on). Store in the storage unit. Then, the second control unit performs control based on the electronic device to be controlled (for example, a television), the function to be controlled (for example, television control), and the execution content (for example, power on) stored in the storage unit. A command (for example, a control command for turning on the television) is determined. For example, the second control unit stores the electronic device to be controlled (for example, a television), the function to be controlled (for example, television control), and the execution content (for example, power on) in the storage unit, and executes When a voice command corresponding to the content (for example, power off) is received, a control command (for example, a control command for turning off the power of the television) can be determined. For this reason, the user utters a voice corresponding to an electronic device to be controlled (for example, a television), a function to be controlled (for example, television control), and an execution content (power on), and the execution content (for example, a power source) When it is desired to perform control that differs only by (off), it is sufficient to speak only the voice corresponding to the execution content. Thus, the user can control the electronic device with a small amount of speech.

  An electronic device control system according to a third aspect is the electronic device control system according to the second aspect, wherein the second control unit is configured to control the electronic device to be controlled when the received voice command corresponds to the electronic device to be controlled. When the control target function and execution contents stored in the storage unit, which have a lower priority than the device, are erased and the received voice command corresponds to the control target function, the control target function is lower than the control target function. The execution content stored in the storage unit with the lower priority is erased.

  In the present invention, when the received voice command corresponds to the electronic device to be controlled (for example, a television), the second control unit of the server has a function to be controlled (for example, a television control) stored in the storage unit. ), The execution contents (for example, power-on) are deleted. In addition, when the received voice command corresponds to a function to be controlled (for example, television control), the second control unit deletes the execution content (for example, power on) stored in the storage unit. For this reason, the user can utter the voice corresponding to the electronic device to be controlled and the function to be controlled, and change them.

  An electronic device control system according to a fourth aspect is the electronic device control system according to the third aspect, wherein the second control unit stores the electronic device to be controlled, the function to be controlled, and the execution content in the storage unit. If there is no initial state and the received voice command corresponds to the execution content, the execution content is stored in the storage unit, and then the received voice command is sent to the electronic device to be controlled or the function to be controlled. In the case where it corresponds, the electronic device to be controlled or the function to be controlled corresponding to the received voice command is stored in the storage unit without erasing the execution content stored in the storage unit. To do.

  In the present invention, the second control unit of the server is in an initial state in which the electronic device to be controlled, the function to be controlled, and the execution content are not stored in the storage unit, and the received voice command executes the execution content (for example, power supply On), the execution contents are stored in the storage unit. After that, when the received voice command corresponds to the electronic device to be controlled or the function to be controlled, it is stored in the storage unit. Without deleting the execution contents, the control target electronic device (for example, television) or the control target function (for example, television control) corresponding to the received voice command is stored in the storage unit. Then, the second control unit performs control based on the electronic device to be controlled (for example, a television), the function to be controlled (for example, television control), and the execution content (for example, power on) stored in the storage unit. A command (for example, a control command for turning on the television) is determined. Therefore, in the initial state, the user can control the electronic device by speaking without considering the speech order.

  The electronic device control system according to a fifth aspect of the present invention is the electronic device control system according to the third aspect of the present invention, wherein the second control unit is one of the electronic device whose control is the control target, the function of the control target, and the execution content. When it corresponds to two or more, the electronic device to be controlled, the function to be controlled, and the execution content corresponding to the received voice command are stored in the storage unit regardless of the priority order.

  In the present invention, when the received voice command corresponds to any two or more of the electronic device to be controlled, the function to be controlled, and the execution content, the second control unit of the server receives the command regardless of the priority order. The electronic device to be controlled corresponding to the voice command, the function to be controlled, and the execution content are stored in the storage unit. And a 2nd control part determines a control command based on the electronic device of the control object memorize | stored in the memory | storage part, the function of a control object, and the execution content. For this reason, the user can control the electronic device by continuously speaking the sound corresponding to the electronic device to be controlled, the function to be controlled, and the execution content. In addition, the user can control the electronic device by speaking without considering the utterance order by continuously speaking the electronic device to be controlled, the function to be controlled, and the voice corresponding to the execution content. it can. In addition, if the electronic device to be controlled, the function to be controlled, and the voice corresponding to the execution content are continuously spoken, it is not necessary to speak in the order of the electronic device to be controlled, the function to be controlled, and the execution content. For this reason, in Japanese, for example, “Turn on the TV” is generally spoken in the order of the electronic devices to be controlled and the execution contents, such as “Turn on TV”. In addition, it is possible to cope with electronic devices to be controlled and utterances in the order of execution contents.

  A terminal device of a sixth invention is a terminal device in the electronic device control system of any one of the first to fifth inventions.

  A server of a seventh invention is a server in the electronic device control system of any of the first to fifth inventions.

  According to the present invention, it is possible to control an electronic device with a small amount of speech.

It is a figure which shows the system configuration | structure of the electronic device control system which concerns on embodiment of this invention. It is a figure which shows a command information database. It is a sequence diagram which shows operation | movement of an electronic device control system. It is a sequence diagram which shows operation | movement of an electronic device control system. It is a flowchart which shows the processing operation of a home server when the voice command which the cloud server transmitted is received. It is a figure which shows the system configuration | structure of the conventional electronic device control system. It is a figure which shows an apparatus control information database. It is a sequence diagram which shows operation | movement of the conventional electronic device control system.

  Hereinafter, embodiments of the present invention will be described. FIG. 1 is a block diagram illustrating a configuration of an electronic device control system according to the present embodiment. The electronic device control system 1 includes a terminal device 2 (2a to 2c), a home server 3, a cloud server 4, and a router 5. For example, the terminal device 2a, the home server 3, and the router 5 are installed in a living room on the first floor of the home. The terminal device 2b is installed in the study on the second floor of the home. The terminal device 2c is installed in a child room on the second floor of the home. The cloud server 4 is an external server installed outside the home. The terminal device 2 communicates with the home server 3. The home server 3 communicates with the cloud server 4.

  The terminal device 2 includes a microcomputer 21, a microphone 22, a speaker 23, an infrared LED 24, and the like. The microcomputer 21 (first control unit) controls each unit constituting the terminal device 2. The microcomputer 21 has a wireless LAN network processor, and performs communication with the router 5 in accordance with the wireless LAN. The microphone 22 collects external sound and outputs it as sound data. The audio data output from the microphone 22 is input to the microcomputer 21 via a DSP (Digital Signal Processor) (not shown). The microphone 22 is a digital microphone, and A / D converts the collected voice data and outputs it to the DSP. The speaker 23 outputs sound based on the sound data. The speaker 23 outputs audio data amplified by a power amplifier (not shown) after D / A conversion of the audio data (digital data) by a D / A converter (not shown). The infrared LED 24 (transmission unit) transmits a remote control code for controlling the television 6 and the illumination 7 to the television 6 (electronic device) and the illumination (electronic device) 7.

  The home server 3 (server) includes a CPU (Central Processing Unit) 31, a RAM (Random Access Memory) 32, a storage unit 33, and a network interface (hereinafter referred to as “NW I / F”) 34. The CPU 31 (second control unit) controls each unit constituting the home server 3. The RAM 32 functions as a work memory for the CPU 31. The storage unit 33 stores various programs and data. The storage unit 33 stores a device control information database (hereinafter referred to as “device control information DB”). The device control information DB is the same as the conventional device control information DB shown in FIG. The NW I / F 34 is for communicating with an external device (for example, the router 5) according to a wired LAN or a wireless LAN.

  The router 5 has an access point function, and communicates with the terminal device 2 according to the wireless LAN. The router 5 communicates with the home server 3 according to a wired LAN. The router 5 connects the home server 3 and the cloud server 4 to the Internet.

  The cloud server 4 (external server) has a voice activity detection (hereinafter referred to as “VAD”) function. The cloud server 4 has a voice recognition function. Since the cloud server 4 has a VAD function, the cloud server 4 can detect a silent section and appropriately perform speech recognition. For example, if a user utters "Hello Onkyo", it detects a silent section, recognizes the character data as "Hello", "Onkyo". On the other hand, if there is no VAD function, if the user utters "Hello Onkyo", for example, it would be recognized as "the crowded," "Harukazu", "on" and "Kiyoｰ". In this specification, a blank part in a sentence may indicate a silent section.

  In the electronic device control system 1, the terminal device 2 transmits collected sound data (for example, “Turn on TV”) to the home server 3. The home server 3 receives the audio data and transmits the received audio data to the cloud server 4. The cloud server 4 receives audio data. The cloud server 4 recognizes the received voice data and determines a corresponding voice command (for example, “TV” and “Turn”). The cloud server 4 transmits the determined voice command to the home server 3. The home server 3 receives the voice command transmitted from the cloud server 4. The home server 3 transmits a control command corresponding to the received voice command (for example, “control command to turn on the TV”) to the terminal device 2. The terminal device 2 receives the control command transmitted from the home server 3. The terminal device 2 transmits a remote control code (for example, “a remote control code for turning on the TV”) corresponding to the received control command via the infrared LED 24.

  In order for the CPU 31 of the home server 3 to determine a control command corresponding to the received voice command (for example, “control command to turn on the TV”), the storage unit 33 stores the Accessory corresponding to the received voice command. Information on (electronic device to be controlled), Service (function to be controlled), and Characteristics (execution content) is stored. FIG. 2 is a diagram showing a command information database (hereinafter referred to as “command information DB”) that stores information on Accessory, Service, and Characteristics corresponding to received voice commands. As shown in FIG. 2A, nothing is stored in the command information DB in the initial state (“None”). For example, when the CPU 31 receives the voice command “TV” transmitted from the cloud server 4 by the NW I / F 34, the CPU 31 sets the Accessory “TV” corresponding to the voice command “TV” to the command information DB based on the device control information DB. (FIG. 2B). At this time, since the Service and Characteristics for the television are not fixed, the CPU 31 is in a “No Action” state in which the operation is not determined.

  Subsequently, when the CPU 31 receives the voice command “Tsuke” transmitted from the cloud server 4 via the NW I / F 34, based on the device control information DB, the Characteristics “Power State ON” corresponding to the voice command “Take” is displayed. Is stored (FIG. 2C). At this time, the CPU 31 can determine that the service is “tv_control” from the Characteristics corresponding to the voice command “attach”, and therefore stores the service “tv_control” in the command information DB. The CPU 31 can determine the control command as a “control command to turn on the TV” because the Accessory (electronic device to be controlled), Service (the function to be controlled), and Characteristics (execution content) have been determined. In the state shown in FIG. 2C, when the CPU 31 receives the voice command “OFF” transmitted from the cloud server 4 by the NW I / F 34, the CPU 31 sets the voice command “OFF” based on the device control information DB. The corresponding Characteristics “Power State OFF” is stored (FIG. 2D). In the command information DB, Accessory “TV” and Service “tv_control” are already stored. Therefore, Accessory, Service, and Characteristics are determined, and the CPU 31 assigns a control command corresponding to the received voice command to “TV power supply”. Control command to turn off.

  As described above, the CPU 31 stores Accessory (control target electronic device), Service (control target function), and Characteristics (execution content) corresponding to the received voice command in the command information DB of the storage unit 33. Then, the CPU 31 determines a control command based on Accessory (control target electronic device), Service (control target function), and Characteristics (execution content) stored in the command information DB of the storage unit 33. Here, the Accessory (control target electronic device), Service (control target function), and Characteristics (execution contents) stored in the command information DB of the storage unit 33 correspond to the received voice command. The CPU 31 determines a control command based on the received voice command.

  The command information DB has a priority order. The priority order is Accessory> Service> Characteristics. In the state shown in FIG. 2C, when the CPU 31 receives the voice command “ONRYO” transmitted from the cloud server 4 via the NW I / F 34, the CPU 31 changes the voice command “ONRYO” to the voice command “ONRYO” based on the device control information DB. The corresponding Service “tv_control” is stored (FIG. 2E). Here, the CPU 31 deletes the Characteristics having a lower priority than the Service (changes it to “None”). In the state shown in FIG. 2C, when the CPU 31 receives the voice command “Denki” transmitted from the cloud server 4 via the NW I / F 34, the CPU 31 changes the voice command “Denki” based on the device control information DB. The corresponding Accessory “illumination” is stored (FIG. 2F). Here, the CPU 31 deletes (changes to “None”) the Service and Characteristics that have a lower priority than the Accessory.

  As described above, when the received voice command corresponds to Accessory (controlled electronic device), the CPU 31 has a command information DB in the storage unit 33 having a lower priority than Accessory (controlled electronic device). Deletes the Service (function to be controlled) and Characteristics (execution content) stored in the. In addition, when the received voice command corresponds to Service (control target function), the CPU 31 stores the command in the command information DB of the storage unit 33 having a lower priority than Service (control target function). Delete the current Characteristics (execution content).

  In the initial state shown in FIG. 2A, for example, when the CPU 31 receives the voice command “attach” transmitted from the cloud server 4 by the NW I / F 34, the CPU 31 attaches the voice command “attach” based on the device control information DB. "Characteristics" Power State ON "corresponding to" "is stored in the command information DB (FIG. 2G). At this point in time, the Accessory and Service have not been determined, so the CPU 31 enters a “No Action” state in which no operation has been determined. Subsequently, when the CPU 31 receives the voice command “Denki” transmitted from the cloud server 4 by the NW I / F 34, the command 31 sets the Accessory “lighting” corresponding to the voice command “Denki” to the command information based on the device control information DB. Store in the DB (FIG. 2 (h)). In the initial state, after information having a lower priority is determined, if information having a higher priority is determined, information having a lower priority is not deleted (not changed to “None”). Thus, in the initial state, the CPU 31 determines the “control command to turn on the illumination” when the voice command “Take” and “Denki” transmitted from the cloud server 4 by the NW I / F 34 are received, for example. (See FIG. 2 (h)).

  As described above, the CPU 31 has received the initial state in which the Accessory (control target electronic device), Service (control target function), and Characteristics (execution content) are not stored in the command information DB of the storage unit 33. When the voice command corresponds to Characteristics (execution content), Characteristics (execution content) is stored in the command information DB of the storage unit 33. After that, when the received voice command corresponds to Accessory (electronic device to be controlled) or Service (functional to be controlled), the CPU 31 determines the characteristics (characteristics) stored in the command information DB of the storage unit 33. Without deleting the (execution content), the Accessory (control-target electronic device) or Characteristics (control-target function) corresponding to the received voice command is stored in the command information DB of the storage unit 33.

  As described above, the cloud server 4 has a VAD function. For example, when the user speaks with silence between “TV” and “Turn”, such as “Turn on TV”, the cloud server 4 sends voice commands “Television” and “Turn on” to the home server. 3 to send. For example, when the user speaks continuously like “Turn on TV”, the cloud server 4 transmits a voice command “Turn on TV” to the home server 3. In this case, the CPU 31 stores Accessory “TV” corresponding to the voice command “TV” and Characteristics “Power State ON” corresponding to the voice command “ON” in the command information DB regardless of the priority order. Similarly, for example, when the user continuously speaks like “Turn on TV”, the cloud server 4 transmits a voice command of “Turn on TV” to the home server 3. In this case, the CPU 31 stores the Characteristics “Power State ON” corresponding to the voice command “Turn” and the Accessory “TV” corresponding to the voice command “TV” in the command information DB regardless of the priority order.

  As described above, when the received voice command corresponds to any two or more of Accessory (control target electronic device), Service (control target function), and Characteristics (execution content), the CPU 31 determines the priority order. Regardless of the received voice command, Accessory (control target electronic device), Service (control target function), and Characteristics (execution content) are stored in the command information DB of the storage unit 33.

  3 and 4 are sequence diagrams showing the operation of the electronic device control system. Hereinafter, a series of operations for controlling the television 6 will be described with reference to FIGS. 3 and 4. First, the user speaks a predetermined keyword in order to set the data transmission mode in which the terminal device 2 transmits audio data to the home server 3. For example, the user utters "Hello Onkyo" (in FIG. 3 (1)). The microphone 22 of the terminal device 2 collects the voice spoken by the user and outputs it to the microcomputer 21 as voice data. The microcomputer 21 has a voice recognition function, recognizes character data included in the voice data, and determines whether or not the recognized character data matches a predetermined keyword. When the microcomputer 21 determines that the recognized character data matches the predetermined keyword, the microcomputer 21 shifts to the data transmission mode ((2) in FIG. 3).

  Next, for example, when the user wants to turn on the TV, the user speaks “Turn on TV” ((3) and (4) in FIG. 3). The microphone 22 of the terminal device 2 collects the voice spoken by the user and outputs it to the microcomputer 21 as voice data. The microcomputer 21 transmits audio data (“Turn on TV”) to the home server 3 ((5) and (6) in FIG. 3). The CPU 31 of the home server 3 receives the audio data transmitted by the terminal device 2 through the NW I / F 34. Next, the CPU 31 transmits voice data to the cloud server 4 by the NW I / F 34 and makes a voice recognition inquiry ((7) and (8) in FIG. 3).

  The cloud server 4 receives the audio data transmitted from the home server 3. The cloud server 4 has a voice recognition function and recognizes character data included in the voice data. As described above, since the cloud server 4 has the VAD function, it can detect a silent section and appropriately perform speech recognition. Next, the cloud server 4 transmits a voice command (“TV” and “attach”) corresponding to the recognized character data to the home server 3 and responds to the result of the voice recognition ((9) and (9) in FIG. 10)). The CPU 31 of the home server 3 receives the voice command transmitted from the cloud server 4 through the NW I / F 34. The CPU 31 analyzes the speech recognition result ((11) and (12) in FIG. 3). Specifically, the CPU 31 determines a control command corresponding to the voice command based on the device control information DB and the voice command.

  Here, when receiving the voice command “TV”, the CPU 31 stores the Accessory “TV” corresponding to the voice command “TV” in the command information DB based on the device control information DB (FIG. 2B). reference.). Further, when receiving the voice command “Tsuke”, the CPU 31 stores Characteristics “Power State ON” corresponding to the voice command “Take” based on the device control information DB (see FIG. 2C). ). At this time, since it is possible to determine that the service is “tv_control” from the Characteristics corresponding to the voice command “tick”, the CPU 31 stores the information of the service “tv_control” in the command information DB. The CPU 31 determines that the control command corresponding to the received voice command is the “control command to turn on the TV” because the Accessory (electronic device to be controlled), Service (the function to be controlled), and Characteristics (execution content) have been determined. Can be determined. The CPU 31 transmits the control command determined by the NW I / F 34 (control command for turning on the power of the television) to the terminal device 2 ((13) in FIG. 3).

  The microcomputer 21 of the terminal device 2 receives the control command transmitted from the home server 3. Next, the microcomputer 21 transmits a remote control code (remote control code for turning on the TV) corresponding to the received control command by the infrared LED 24 ((14) in FIG. 3). The television 6 is turned on by a remote control code that turns on the television transmitted from the terminal device 2 ((15) in FIG. 3).

  Next, for example, when the user wants to turn off the TV, the user speaks “OFF” ((16) in FIG. 4). The microphone 22 of the terminal device 2 collects the voice spoken by the user and outputs it to the microcomputer 21 as voice data. The microcomputer 21 transmits the audio data (“OFF”) to the home server 3 ((17) in FIG. 4). The CPU 31 of the home server 3 receives the audio data transmitted by the terminal device 2 through the NW I / F 34. Next, the CPU 31 sends voice data to the cloud server 4 by the NW I / F 34 and makes a voice recognition inquiry ((18) in FIG. 4).

  The cloud server 4 receives the audio data transmitted from the home server 3. The cloud server 4 recognizes the character data included in the voice data, transmits a voice command (“OFF”) corresponding to the recognized character data to the home server 3, and responds to the result of the voice recognition ((FIG. 4 ( 19)). The CPU 31 of the home server 3 receives the voice command transmitted from the cloud server 4 through the NW I / F 34. The CPU 31 analyzes the speech recognition result ((20) in FIG. 4).

  When the CPU 31 receives the voice command “OFF”, the CPU 31 stores Characteristics “Power State OFF” corresponding to the voice command “OFF” based on the device control information DB (see FIG. 2D). In the command information DB, Accessory “TV” and Service “tv_control” are already stored. Therefore, Accessory, Service, and Characteristics are determined, and the CPU 31 assigns a control command corresponding to the received voice command to “TV power supply”. Control command to turn off. The CPU 31 transmits the control command determined by the NW I / F 34 (control command for turning off the power of the television) to the terminal device 2 ((21) in FIG. 4).

  The microcomputer 21 of the terminal device 2 receives the control command transmitted from the home server 3. Next, the microcomputer 21 transmits a remote control code corresponding to the received control command (remote control code for turning off the power of the television) by the infrared LED 24 ((22) in FIG. 4). The television 6 is turned off by a remote control code for turning off the television transmitted from the terminal device 2 ((23) in FIG. 4).

  The microcomputer 21 of the terminal device 2 ends the data transmission mode after a predetermined time has elapsed since the transition to the data transmission mode. Further, when the cloud server 4 detects silence for a predetermined time, the microcomputer 21 may end the data transmission mode.

  Next, the processing operation of the home server 3 when the voice command transmitted by the cloud server 4 is received will be described based on the flowchart shown in FIG. When the CPU 31 of the home server 3 receives the voice command transmitted from the cloud server 4 by the NW I / F 34 (S1), the CPU 31 determines whether the received voice command matches the device control information based on the device control information DB. Judgment is made (S2). CPU31 complete | finishes a process, when it is judged that the received voice command does not correspond with apparatus control information (S2: No).

  When it is determined that the received voice command matches the device control information (S2: Yes), the CPU 31 determines whether the received voice command corresponds to Accessory (S3). If the CPU 31 determines that the received voice command corresponds to Accessory (S3: Yes), the CPU 31 determines whether or not it is in the initial state (see FIG. 2A) based on the command information DB (see FIG. 2A). S4). If the CPU 31 determines that it is not in the initial state (S4: No), it deletes (changes to “None”) the Service and Characteristics that have a lower priority than the Accessory in the command information DB (S5).

  When the CPU 31 determines that the received voice command does not correspond to Accessory (S3: No), the CPU 31 determines whether the received voice command corresponds to Service (S6). When the CPU 31 determines that the received voice command is compatible with Service (S6: No), the CPU 31 determines whether or not it is in the initial state (see FIG. 2A) based on the command information DB (see FIG. 2A). S7). When determining that the current state is not the initial state (S7: No), the CPU 31 deletes the Characteristics having a lower priority than the Service in the command DB information (changes to “None”) (S8).

  When the CPU 31 determines that it is in the initial state (S4, S7: Yes), when it is determined that the received voice command does not correspond to the Service (S6: No), after the processing of S5 and S8, the information command Based on the DB, it is determined whether Accessory, Service, and Characteristics have been determined and a control command can be determined (S9). If the CPU 31 determines that the Accessory, Service, and Characteristics have not been determined and the control command cannot be determined (S9: No), the process ends.

  If the CPU 31 determines that the Accessory, Service, and Characteristics have been determined and the control command can be determined (S9: Yes), the CPU 31 determines the control command based on the command information DB, and determines the control command determined by the NW I / F 34 Is transmitted to the terminal device 2 (S10).

  As described above, in the present embodiment, when the character data included in the audio data output from the microphone 22 matches the predetermined keyword, the microcomputer 21 of the terminal device 2 uses the microphone after the predetermined keyword. The voice data output by the terminal 22 is transmitted to the home server 3. The CPU 31 of the home server 3 transmits the voice data transmitted from the terminal device 2 to the cloud server 4 and receives a voice command corresponding to the transmitted voice data from the cloud server 4. Further, the CPU 31 determines a control command based on the received voice command, and transmits the determined control command to the terminal device 2. The microcomputer 21 of the terminal device 2 receives the control command transmitted from the home server 3 and transmits a remote control code corresponding to the received control command to the television 6 or the illumination 7 by the infrared LED 24. Therefore, the user only has to speak a predetermined keyword once, and does not need to repeat the predetermined keyword, so that the electronic device can be controlled with a small amount of utterance. Further, it is not necessary to recognize the voice of the keyword one by one on the terminal device 2 side.

  Further, in the present embodiment, the CPU 31 of the home server 3 has an Accessory corresponding to the received voice command (an electronic device to be controlled (for example, a television)), a Service (a function to be controlled (for example, tv_control)), a Characteristics ( The execution content (for example, Power State ON) is stored in the command information DB of the storage unit 33. Then, the CPU 31 stores an Accessory (an electronic device to be controlled (for example, a television)), a Service (a function to be controlled (for example, tv_control)), a Characteristics (for example, an execution content (for example, an execution content (for example, an execution content)). , Power State ON)), a control command (for example, a control command for turning on the power of the television) is determined. For example, the CPU 31 includes an Accessory (electronic device to be controlled (for example, a television)), a Service (a function to be controlled (for example, tv_control)), and a Characteristics (execution content (for example, Power State ON)) in the command of the storage unit 33. When a voice command stored in the information DB and corresponding to Characteristics (execution content (for example, Power State OFF) is received, a control command (for example, a control command for turning off the power of the television) is determined. For this reason, the user can use Accessory (controlled electronic device (eg, television)), Service (controlled function (eg, tv_control)), Characteristics ( When speech corresponding to the line content (for example, Power State ON) is uttered, and when it is desired to perform control that differs only by Characteristics (for example, Power State OFF), only the sound corresponding to Characteristics (for execution content) As a result, the user can control the electronic device with a small amount of speech.

  In the present embodiment, the CPU 31 of the home server 3 stores the received voice command in the command information DB of the storage unit 33 when the received voice command corresponds to Accessory (an electronic device to be controlled (for example, a television)). Service (function to be controlled (for example, tv_control), Characteristics (execution content (for example, Power State ON) is deleted. Further, the CPU 31 receives the received voice command for Service (function to be controlled (for example, tv_control))). In this case, the Characteristics (execution content (for example, Power State ON)) stored in the command information DB of the storage unit 33 is deleted. For this reason, the user can access the Accessory (control target). Child device), and speech sound corresponding to the Service (function of the control object), it is possible to change them.

  In the present embodiment, the CPU 31 of the home server 3 is an initial stage in which Accessory (control target electronic device), Service (control target function), and Characteristics (execution content) are not stored in the command information DB of the storage unit 33. If the received voice command is in the state and corresponds to Characteristics (execution content (for example, Power State On)), the Characteristics (execution content) is stored in the command information DB of the storage unit 33 and then received. When the voice command corresponds to Accessory (electronic device to be controlled) or Service (control target function), without deleting the Characteristics (execution contents) stored in the command information DB of the storage unit 33, The Accessory (the electronic device to be controlled (for example, a television)) or the Service (the function to be controlled (for example, tv_control)) corresponding to the received voice command is stored in the command information DB of the storage unit 33. Accessory (electronic device to be controlled (for example, television)), Service (function to be controlled (for example, tv_control)), Characteristics (execution content (for example, Power State ON)) stored in the command information DB of the storage unit 33 ) To determine a control command (for example, a control command to turn on the TV), so that, in the initial state, the user speaks without considering the speech order and controls the electronic device. be able to.

  In the present embodiment, the CPU 31 of the home server 3 corresponds to any two or more of the received voice commands of “Accessory (control target electronic device)”, “Service (control target function)”, and “Characteristics (execution content)”. If there is, the Accessory (control target electronic device), Service (control target function), and Characteristics (execution content) corresponding to the received voice command are stored in the command information DB of the storage unit 33 regardless of the priority order. . Then, the CPU 31 determines a control command based on Accessory (control target electronic device), Service (control target function), and Characteristics (execution content) stored in the command information DB of the storage unit 33. For this reason, the user can control the electronic device by continuously speaking the voice corresponding to Accessory (electronic device to be controlled), Service (function to be controlled), and Characteristics (execution content). In addition, the user can continuously speak the voice corresponding to Accessory (control target electronic device), Service (control target function), and Characteristics (execution contents), so that the user can speak without considering the order of the voices. Thus, the electronic device can be controlled. Also, if a voice corresponding to Accessory (control target electronic device), Service (control target function), Characteristics (execution content) is continuously spoken, Accessory (control target electronic device), Service (control target) ) And Characteristics (execution content) in this order. For this reason, for example, in Japanese, it is common to utter in the order of Accessory (electronic device to be controlled) and Characteristics (execution content) like “Turn on TV”, but for example, “Turn in English” As in “on the TV”, it is also possible to deal with utterances in the order of Characteristics (execution contents) and Accessory (electronic device to be controlled).

  As mentioned above, although embodiment of this invention was described, the form which can apply this invention is not restricted to the above-mentioned embodiment, It is possible to add a change suitably in the range which does not deviate from the meaning of this invention. is there.

  The present invention can be suitably employed in an electronic device control system for controlling electronic devices such as a television and lighting, a terminal device constituting the electronic device control system, and a server.

DESCRIPTION OF SYMBOLS 1 Electronic device control system 2 Terminal device 21 Microcomputer (1st control part)
22 Microphone 24 Infrared LED (Transmitter)
3 Home server (server)
31 CPU (second control unit)
33 Storage unit 4 Cloud server (external server)
6 TV (electronic equipment)
7 Lighting (electronic equipment)

Claims (7)

An electronic device control system comprising a server that has a voice section detection function and a voice recognition function, communicates with an external server that transmits voice commands corresponding to voice data, and a terminal device that communicates with the server ,
The terminal device
A microphone that collects external audio and outputs it as audio data;
A transmitter for transmitting a code for controlling the electronic device;
When the character data included in the voice data output from the microphone matches a predetermined keyword, the voice data output from the microphone after the predetermined keyword is transmitted to the server,
Receiving a control command sent by the server;
A first control unit that transmits a code corresponding to the received control command to the electronic device by the transmission unit;
The server
Receiving voice data transmitted by the terminal device;
Send the received audio data to the external server,
Receiving a voice command corresponding to the transmitted voice data from the external server;
Determine the control command based on the received voice command,
An electronic device control system comprising: a second control unit that transmits the determined control command to the terminal device. The server further includes a storage unit,
The second controller is
The electronic device to be controlled corresponding to the received voice command, the function to be controlled, and the execution content are stored in the storage unit,
The electronic device control system according to claim 1, wherein a control command is determined based on an electronic device to be controlled, a function to be controlled, and an execution content stored in the storage unit. The second controller is
When the received voice command corresponds to the electronic device to be controlled, the priority order is lower than the electronic device to be controlled, the function to be controlled and the execution content stored in the storage unit are deleted,
3. When the received voice command corresponds to a function to be controlled, the execution content stored in the storage unit having a lower priority than the function to be controlled is erased. The electronic device control system described. The second controller is
In the initial state in which the electronic device to be controlled, the function to be controlled, and the execution content are not stored in the storage unit, and the received voice command corresponds to the execution content, the execution content is stored in the storage unit Thereafter, when the received voice command corresponds to the electronic device to be controlled or the function to be controlled, the received voice command corresponds without erasing the execution content stored in the storage unit. The electronic device control system according to claim 3, wherein an electronic device to be controlled or a function to be controlled is stored in the storage unit. The second controller is
When the received voice command corresponds to any two or more of the electronic device to be controlled, the function to be controlled, and the execution contents, the electronic device to be controlled corresponding to the received voice command regardless of the priority order, The electronic device control system according to claim 3, wherein functions to be controlled and execution contents are stored in the storage unit.   The terminal device according to claim 1.   The server according to any one of claims 1 to 5.

Images