JP6053097B2 - Device operating system, device operating device, server, device operating method and program - Google Patents

Description

  The present invention relates to a device operating device, a server, a device operating system, a device operating method, and a program.

Many home appliances come with a remote controller. By operating this remote controller, the user can perform a desired operation by remote operation without directly operating the home appliance.
On the other hand, for example, there is a desire to operate other home appliances without interrupting washing or cooking work. However, in the remote operation by the remote controller, it is necessary to interrupt the work in order to operate the remote controller.

In recent years, the accuracy of voice recognition has improved. Therefore, if the device can be controlled using the voice recognition, the device can be remotely operated without interrupting the work.
Therefore, the following is known as a voice recognition system for remotely operating a device. That is, in this speech recognition system, the speech feature amount extracted from the speech speech data is transmitted from the mobile phone to an authentication / recognition server having a speech recognition function via the communication network. The authentication / recognition server recognizes the voice content of the speech voice data based on the received voice feature amount. The authentication / recognition server recognizes the instruction voice content when the recognized voice content is an instruction voice to the controlled device, and transmits a corresponding control signal to the controlled device via the communication network. (For example, see Patent Document 1).

JP 2006-203900 A

  However, in the configuration described in Patent Document 1, a mobile phone is used as a device for a user to input a voice for operation. For this reason, the user needs to perform an operation for transmitting utterance voice data to the mobile phone when the device is remotely operated, and to speak to the mobile phone. Therefore, in patent document 1, the problem that the operation | work which was performed until now is interrupted is not solved.

  SUMMARY OF THE INVENTION Accordingly, an object of the present invention is to enable accurate remote control of a device by voice without interrupting work that has been performed by a user so far.

  The present invention has been made to solve the above-described problems, and an apparatus operating system according to an aspect of the present invention includes an apparatus operating device and a biological information analysis server, and the device operating device is self or other. A biometric information input unit that inputs biometric information for operating any of the devices to be controlled by the device operating device, a biometric information processing unit that generates biometric information data including the contents of the input biometric information, and The biometric information data is transmitted to the biometric information analysis server, and the first communication unit that receives the command transmitted from the biometric information analysis server, and the self control so that the operation in response to the received command is executed. A device control unit that controls a target device, wherein the biological information analysis server receives the biological information data and transmits the command An analysis unit that analyzes the received biological information data, a command determination unit that determines the command content and the operation target device based on the content of the biological information recognized by analyzing the biological information data, and the determined command A command generation unit configured to generate the command according to the content, and to transmit the generated command to the device operation device whose control target is the operation target device by the second communication unit.

  In addition, the device operating device as one aspect of the present invention includes a biometric information input unit that inputs biometric information for operating any of the devices to be controlled by itself or another device operating device; A biometric information processing unit that generates biometric information data including information content, a communication unit that transmits the biometric information data to the biometric information analysis server, and that receives a command transmitted from the biometric information analysis server; A device control unit that controls a device that is controlled by the device so that an operation in response to the command is executed.

  In addition, the server as one aspect of the present invention receives the biological information data generated based on the biological information input by the device operating device from the device operating device, and transmits the command and the received biological information. An analysis unit that analyzes data, a command determination unit that determines a command content and an operation target device based on the content of the biological information recognized by analyzing the biological information data, and a command that corresponds to the determined command content A command generation unit configured to generate the command and cause the communication unit to transmit the generated command to the device operation apparatus whose control target is the operation target device;

  An apparatus operating method according to an aspect of the present invention is an apparatus operating method in an apparatus operating system including an apparatus operating device and a biological information analysis server, and the device operating device is self or another device operating device. A biological information input step for inputting biological information for operating any of the devices to be controlled, a biological information processing step for generating biological information data including the content of the input biological information, and the biological information data A first communication step for transmitting a command transmitted from the biological information analysis server and receiving a command transmitted from the biological information analysis server, and a device to be controlled by itself so that an operation in response to the received command is executed The biological information analysis server receives the biological information data and receives the command. Based on the content of the 2nd communication step to transmit, the analysis step which analyzes the received biometric information data, and the biometric information recognized by analyzing the biometric information data, the command content and the operation target device are determined. Generating a command corresponding to the determined command content, and transmitting the generated command to the device operating device whose control target is the operation target device in the second communication step; A command generation step.

  In addition, a program as one embodiment of the present invention is input to a computer with a biometric information input step for inputting biometric information for operating either a device to be controlled by itself or another device operating device. A biological information processing step for generating biological information data including the content of the biological information, a communication step for transmitting the biological information data to the biological information analysis server, and receiving a command transmitted from the biological information analysis server; And a device control step for controlling a device to be controlled by itself so that an operation in response to the command is executed.

  According to another aspect of the present invention, there is provided a program for receiving, from a device operating device, biological information data generated based on biological information input by the device operating device, and transmitting a command to the computer. An analysis step of analyzing the biometric information data, a command determination step of determining the command content and the operation target device based on the content of the biometric information recognized by analyzing the biometric information data, and the determined command content The command is generated in accordance with the command, and the generated command is transmitted to the device operating device whose control target is the operation target device through the communication step.

  According to the present invention, there is an effect that it is possible to perform remote control of a device accurately by voice without interrupting the work that the user has been doing so far.

It is a figure which shows the structural example of the apparatus operating system in the 1st Embodiment of this invention. It is a figure which shows the structural example of the apparatus operating device in 1st Embodiment. It is a figure which shows the structural example of the audio | voice analysis server in this embodiment, and a gesture analysis server. It is a figure which shows the example of the content of the apparatus management information in 1st Embodiment. It is a figure which shows the example of the content of the command management information in 1st Embodiment. It is a figure which shows the example of a process sequence which the audio | voice information processing part in the apparatus operating device of 1st Embodiment performs. It is a figure which shows the modification of the process sequence which the audio | voice information processing part in the apparatus operating device of 1st Embodiment performs. It is a figure which shows the example of a process sequence which the audio | voice analysis server in 1st Embodiment performs. It is a figure which shows the structural example of the apparatus operation system in 2nd Embodiment. It is a figure for demonstrating the group management of the apparatus in 2nd Embodiment. It is a figure which shows the structural example of the apparatus operation system in 3rd Embodiment. It is a figure which shows the structural example of the apparatus operating device in 4th Embodiment. It is a figure which shows the modification of the apparatus operating device in 4th Embodiment.

[First Embodiment]
FIG. 1 shows a configuration example of a device operation system as a first embodiment of the present invention. The device operation system shown in this figure includes an air conditioner 100-1, a television receiver 100-2, device operation devices 200-1 and 200-2, and a router 300 in a house 1. The device operation system also includes a voice analysis server (biological information analysis server) 400.

The air conditioner 100-1 is a home appliance provided in the house 1, and is one of the operation target devices in the device operation system of the present embodiment. The television receiver 100-2 is also a home electric appliance provided in the house 1, and is one of the devices to be operated in the device operation system of the present embodiment.
Hereinafter, the air conditioner 100-1 and the television receiver 100-2 will be referred to as the device 100 unless otherwise distinguished.

  The device operating device 200-1 is a device for operating the air conditioner 100-1, and is connected to the air conditioner 100-1 via a predetermined interface. The device operating device 200-1 is connected to the voice analysis server 400 via a network such as the Internet from the router 300 so as to be communicable.

The device operating device 200-2 is a device for operating the television receiver 100-2, and is connected to the television receiver 100-2 via a predetermined interface. In addition, the device operating device 200-1 is connected to the voice analysis server 400 so as to be communicable from the router 300 via the network.
Hereinafter, the device operation devices 200-1 and 200-2 are referred to as device operation devices 200 unless otherwise distinguished.

  The router 300 is a relay for enabling the device operating device 200 in the house 1 and the voice analysis server 400 to communicate via a network.

  The voice analysis server 400 converts voice information data received from the device operation device 200 via the router 300 into a command code and remotely connects the operation target device 100 to a device connected to the operation target device 100. It transmits with respect to the operating device 200.

With reference to the same FIG. 1, the operation example for the remote operation with respect to the apparatus 100 in the information processing system of 1st Embodiment is demonstrated.
Here, it is assumed that the user 2 of the house 1 wants to change the temperature of the air conditioner 100-1 to 28 degrees. Therefore, the user 2 speaks, for example, “28 degrees”.
The device operating devices 200-1 and 200-2 include microphones that collect ambient sounds. Speech speech (biological information) uttered from the user 2 as described above is collected by a microphone provided on the side of the device operating devices 200-1 and 200-2 located near the user 2. Alternatively, when there is no great difference in the distance between the device operation devices 200-1 and 200-2 and the user 2, the uttered voice is collected by the microphones of both the device operation devices 200-1 and 200-2. .

  The device operating device 200 that picks up the uttered voice generates voice information data (biological information data) by converting the collected voice signal into, for example, a compressed voice signal of a predetermined format and adding metadata. The metadata is, for example, an identifier corresponding to a house or room in which the device operating device 200 is provided. Such information is memorize | stored in the apparatus operating device 200 by user operation etc., for example.

The voice analysis server 400 analyzes the received voice information data to perform voice recognition on the component of the uttered voice in the voice signal included in the voice information data. Thereby, the content of the uttered voice is recognized. Then, using the utterance content recognition result, the voice analysis server 400 determines the operation target device and the command content indicated by the utterance content. In this case, since the utterance content is “28 degrees”, the voice analysis server 400 determines that the operation target device is the air conditioner 100-1, and the command content is to set the set temperature to 28 degrees. Is determined. Thus, the command content is the content of the operation instructed by the utterance.
The voice analysis server 400 generates a command code corresponding to the determined command content, and transmits the command code by designating the device operating device 200 connected to the determined operation target device as a transmission destination. Specifically, the voice analysis server 400 in this case designates a command code for instructing to set the set temperature at 28 degrees as the transmission destination of the device operating device 200-1 connected to the air conditioner 100-1. To send.

The device operating device 200-1 transmits the received command code to the air conditioner 100-1. In response to receiving this command code, the air conditioner 100-1 changes the set temperature to 28 degrees.
In the present embodiment, as described above, the user can remotely operate the device 100 by giving a voice and giving an instruction. In the present embodiment, the device operating device 200 collects a user's uttered voice in a state where the device operating device 200 is fixedly disposed with the corresponding device 100 in the house 1. That is, the user 2 only has to speak the voice for operation in the house 1 and does not have to hold the mobile phone again. Thereby, when operating the air conditioner 100-1, the user 2 does not need to interrupt the work which has been performed so far.

  FIG. 2 shows a configuration example of the device operating device 200 in the first embodiment. The device operating device 200 shown in this figure includes a microphone (biological information input unit) 210, a voice information processing unit (biological information processing unit) 220, a communication unit 230, and a device control unit 240.

  The microphone 210 picks up surrounding sounds. That is, the microphone 210 inputs sound as biometric information for operating either the operation target device that is controlled by itself or another device operation device 200.

  The voice information processing unit 220 generates voice information data (biological information data) including the content of voice (biological information) collected (input) by the microphone 210. The voice information processing unit 220 generates the biological information data based on the biological information input within a predetermined time after inputting a predetermined operation start notice signal.

The voice information processing unit 220 includes an utterance voice component extraction unit 221, a format conversion unit 222, a metadata storage unit 223, and a voice information data generation unit 224.
The sound collected by the microphone 210 includes ambient sounds other than the speech sound. The uttered voice component extraction unit 221 extracts the uttered voice from the voice signal collected by the microphone 210. In this extraction, the utterance voice component extraction unit 221 filters the voice signal using, for example, information on frequency characteristics unique to the utterance voice.
The format conversion unit 222 converts the extracted speech signal of the speech component into speech data of a predetermined format, for example, by performing compression encoding.
The metadata storage unit 223 is, for example, predetermined metadata including a device operation device ID for identifying the device operation device 200, a group ID corresponding to a house or a room where the device operation device 200 is disposed, and the like. Remember.
The audio information data generation unit 224 generates audio information data by adding the metadata read from the metadata storage unit 223 to the audio data converted by the format conversion unit 222.

  In the device operating device 200, the communication unit (first communication unit) 230 transmits the voice information data generated by the voice information data generation unit 224 from the router 300 (not shown in this figure) via the network to the voice analysis server. 400 is transmitted. Further, the communication unit 230 receives the command code transmitted from the voice analysis server 400.

  The device control unit 240 controls the device 100 to be controlled connected to the device control unit 240 according to the command code received by the communication unit 230. For example, the device control unit 240 converts the command code received by the communication unit 230 into a format suitable for the device 100 as necessary, and transmits the converted code to the device 100. The device 100 executes an operation in response to the command code received from the device control unit 240.

  FIG. 3 shows a configuration example of the voice analysis server 400. The voice analysis server 400 shown in this figure includes a duplicate reception handling processing unit 410, an analysis unit 420, a command determination unit 430, a command generation unit 440, device management information 450, command management information 460, and a communication unit 470 (second communication unit). ).

In the present embodiment, there is a possibility that a plurality of device operation devices 200 installed in the same room pick up the same uttered voice and transmit voice information data having the same content. In this case, processing the same voice information data a plurality of times on the voice analysis server 400 side increases the processing load. In addition, if the same command is transmitted based on the voice information data having the same operation instruction content, there is a possibility that the operation of the device 100 may be a hindrance to stable remote operation such that the operation of the device 100 does not conform to the user's intention.
In order to avoid the problems as described above, the duplicate reception handling processing unit 410 should use only one voice information data for remote operation when receiving voice information data of the same utterance voice from a plurality of device operation devices 200. Sort as kimono.
For this reason, the duplicate reception handling processing unit 410 matches the result of comparing the voice information data (biological information data) received this time with the voice information data received by the communication unit 470 within a certain past time. If so, the audio information data received this time is discarded.

  The analysis unit 420 analyzes the voice information data received by the communication unit 470. The analysis unit 420 recognizes the content of the uttered speech and the content of the metadata included in the speech information data as the analysis result.

  The command determination unit 430 determines the command content and the operation target device according to the operation intended by the user 2 based on the content of the uttered voice recognized by the analysis unit 420. At this time, the command determination unit 430 uses the device management information 450 and the command management information 460.

  The command generation unit 440 generates a command code (command) of the command content determined by the command determination unit 430. Then, the command generation unit 440 causes the communication unit 470 to transmit the generated command code, specifying the device operating device 200 connected to the determined operation target device as a transmission destination.

  The communication unit 470 receives the audio information data transmitted from the device operating device 200. Further, the command code generated by the command generation unit 440 is transmitted to the device operating device 200.

  FIG. 4 shows an example of the structure of the device management information 450 stored in the voice analysis server 400. The device management information 450 has a structure in which a device ID that uniquely identifies the device 100 is associated with a model ID, a device operation device ID, a device operation device address, a main group ID, and a sub group ID.

The device ID is an identifier that uniquely identifies the device 100.
The model ID is an identifier that uniquely identifies the model of the device 100 having the associated device ID.
The device operation device ID is an identifier for uniquely identifying the device operation device 200 connected to the device 100 having the associated device ID.
The device operating device address indicates an address (IP address) assigned to the device operating device 200 on the network.

The main group ID is an identifier that uniquely identifies the main group to which the device 100 having the associated device ID belongs. The main group corresponds to each house or family, for example. That is, the device 100 having the device ID associated with the same main group ID is installed in the same house.
The subgroup ID is an identifier that uniquely identifies the subgroup to which the device 100 having the associated device ID belongs. In addition, a subgroup respond | corresponds for every room in a house (or family), for example. That is, the device 100 having the device ID associated with the same subgroup ID is installed in the same room.

  FIG. 5 shows a structural example of the command management information 460. The command management information 460 is information in which a model ID and a command set are associated as shown in FIG. The command code has a different set for each model of the device 100. One command set has a structure in which a command code, command content, and speech voice content are associated with each other.

  The command determination unit 430 determines the command content by searching the command management information 460 for the command content associated with the recognized utterance voice content, for example. Further, the command determination unit 430 recognizes the model ID associated with the determined command code from the command management information 460, and further recognizes the device ID associated with the recognized model ID from the device management information 450. . The command determination unit 430 determines the device ID recognized in this way as a control target device.

Specifically, when it is recognized that the utterance voice content is “28 degrees”, the command determination unit 430 searches the command management information 460 for a command set including the “28 degrees” utterance voice content. . The command determination unit 430 recognizes the command content included in the searched command set. In addition, the command determination unit 430 recognizes the model ID associated with the command set searched in the command management information 460. In this case, the model ID of the model of the air conditioner 100-1 is recognized.
Next, the command determination unit 430 searches the device management information 450 for a record including the recognized model ID and the device operating device ID included in the received audio information data metadata, and determines the device ID in this record. Recognized as an operation target device. Specifically, the command determination unit 430 in this case recognizes the device ID of the air conditioner 100-1.
The command determination unit 430 uses the command content and the device ID recognized as described above as the determination result.

  The command generation unit 440 recognizes a command code corresponding to the command content determined by the command determination unit 430 from the command management information 460, and generates the recognized command code.

The flowchart in FIG. 6 illustrates an example of a processing procedure executed by the voice information processing unit 220 in the device operation apparatus 200.
The uttered voice component extraction unit 221 waits for the uttered voice component to be extracted while executing processing for extracting the uttered voice component from the voice signal input from the microphone 210 (step S101—NO). . When the utterance voice component is extracted (step S101—YES), the format conversion unit 222 converts the extracted voice signal of the utterance voice component into voice data of a predetermined format (step S102).
Next, the audio information data generation unit 224 generates audio information data by adding metadata to the audio data after the conversion in step S102 (step S103), and the generated audio information data is transmitted by the communication unit 230. It is transmitted to the voice analysis server 400 (step S104).

The flowchart of FIG. 7 shows a modification of the processing procedure executed by the voice information processing unit 220.
In this modification, the user 2 inputs operation start notice information for notifying the start of operation to the device operation device 200 at a timing before sounding a sound for remote operation. As a specific example, the user 2 makes a sound by hitting his hand as input of operation start information. The microphone 210 of the device operating device 200 inputs the sound of hitting this hand. The utterance voice component extraction unit 221 extracts not only the utterance voice but also the component of the sound of clapping the hand.

  The speech component extraction unit 221 performs a process of extracting a sound component of the clapping hand from the sound signal input from the microphone 210 and waits for the component of the sound of the clapping hand to be extracted. Yes. That is, the utterance voice component extraction unit 221 is on standby for input of operation start advance notice information (step S201—NO).

  Then, in response to the input of the operation start advance notice information (step S201-YES), the utterance voice component extraction unit 221 starts from the timing when the operation start advance notice information is input until a predetermined time elapses ( Step S202—NO), waiting for the speech component to be extracted (step S203—NO).

When a certain period of time has passed without extracting the speech component (step S202—YES), the speech component extraction unit 221 returns to step S101.
On the other hand, when the utterance voice is extracted before the predetermined time elapses (step S203—YES), the processing from step S204 to S206 is executed. The processing in steps S204 to S206 is the same as that in steps S102 to S104 in FIG.

For example, in the case where the sound of a program is output from the television receiver 100-2 in the house 1 or there is a person other than the user 2, the speech sound component collected by the microphone 210 is It is not necessarily for remote operation.
Therefore, the user 2 in the modification first taps his hand for an operation start notice as an action for remote operation, and then utters a speech voice for operation after a certain time. According to the processing in FIG. 7, the voice information processing unit 220 generates only voice speech uttered after a predetermined time from hitting a hand as an operation start notice as voice information data and transmits it to the voice analysis server 400. Can do. As a result, the possibility that the uttered voice uttered for other than the operation purpose is mistakenly recognized as the voice information data and the process for the remote operation is executed is reduced. In addition, as an operation start notice, for example, a chime or a bell may be sounded in addition to clapping. Alternatively, a specific button on a specific device may be pressed, or a specific word, for example, an “operation start” message may be used as the operation start notification.

  FIG. 8 shows an example of a processing procedure executed by the voice analysis server 400. First, the communication unit 470 receives the voice information data in response to the voice information data being transmitted from the device operating device 200 (step S301).

The duplicate reception handling processing unit 410 compares the audio information data received this time in step S301 with the contents of the audio information data received within a certain past time from the reception timing of the current audio information data (step S302). ). At this time, the duplicate reception handling processing unit 410 may compare the bit arrangement patterns of the audio information data information, for example. The fixed time is set to, for example, about 3 seconds in consideration of an error in processing time until the device operating device 200 inputs sound and generates sound information data.
Then, the duplicate reception handling processing unit 410 determines whether or not there is a match between the current voice information data and the voice information data received within a certain past time based on the comparison result in step S302 ( Step S303).

If it is determined that there is a match (step S303—YES), the duplicate reception handling processing unit 410 discards the audio information data received this time (step S315).
(Step S303).
On the other hand, when it is determined that there is no match (NO in step S303), the analysis unit 420 analyzes the voice information data received this time (step S304). Based on the result of this analysis, the analysis unit 420 determines whether or not the voice data included in the voice information data is an uttered voice (step S305). If it is determined that the voice is not an uttered voice (step S305—NO), the analysis unit 420 discards the voice information data received this time (step S315).
Note that if the command determination unit 430 determines that the voice is an utterance (step S305—YES), the command determination unit 430 recognizes the content of the utterance. Further, the analysis unit 420 recognizes the content of metadata included in the audio information data as a result of the analysis.

When it is determined that the voice is an uttered voice (YES in step S305), the analysis unit 420 recognizes the uttered voice and the content of the metadata included in the voice information data. Therefore, the command determination unit 430 in this case uses the recognition result of the analysis unit 420 to determine the command content and the operation target device as described above (step S306).
However, when the user utters content not related to the operation and the voice information data of this uttered voice is transmitted, the command set corresponding to the uttered voice of the voice information data is selected from the device management information 450. I can't search. In response to such a case, the command determination unit 430 determines whether or not a command set corresponding to the recognized speech content is retrieved from the command management information 460 as a determination result (step S307). .

When the command set is not searched (step S307—NO), the command determination unit 430 discards the voice information data that is the current determination target (step S315).
On the other hand, when the command set is searched (step S307: YES), the command determination unit 430 indicates that the device 100 determined as the operation target device exists in the house 1 where the device operation device 200 that is the voice information data transmission source is installed. It is determined whether or not to perform (step S308).
Therefore, the command determination unit 430 determines that the main group ID included in the metadata of the received audio information data includes the main group ID in the record of the device management information 450 including the device ID of the operation target device determined in step S306. It is determined whether or not it matches the ID. Since the main group ID corresponds to the house, if the main group ID matches, the main group ID exists in the house from which the audio information data is transmitted.
When it is determined that the operation target device does not exist (step S308—NO), the command determination unit 430 discards the currently received voice information data (step S315).

  On the other hand, when the recognized operation target device exists (step S308-YES), the command determination unit 430 determines whether there is only one operation target device determined in step S306 (step S309). ). For example, when the uttered voice is unclear, the command determination unit 430 may determine a plurality of operation target models in step S306.

  When there are a plurality of determined operation target devices (NO in step S309), the command determination unit 430 generates candidate presentation information for narrowing down the operation target devices to one, and the candidate presentation information is converted into voice information. It transmits with respect to the apparatus operating device 200 of a data transmission origin (step S313). The candidate presentation information is information that presents a plurality of operation target devices determined by the command determination unit 430 in step S306 as selection candidates.

The device operating device 200 that has received the candidate presentation information, for example, presents the name of the operation target device that is a candidate for selection and outputs a sound that prompts selection of the device. For example, the device operating device 200 outputs a sound such as “air conditioner and television receiver are candidates. Please say the name of the correct device”.
The user who has heard this utters the name of the selected operation target device, for example. The device operating device 200 generates a selection notification including the uttered voice and transmits it to the voice analysis server 400. In the voice analysis server 400, the selection notification is passed to the command determination unit 430 via the duplicate reception handling processing unit 410 and the analysis unit 420. The command determination unit 430 sets the operation target device indicated by the selection notification as a final determination result (step S314).

In addition, the command determination unit 430 may determine the contents of a plurality of commands in step S306 when the uttered voice is still unclear. Therefore, the command determination unit 430 determines whether there is only one command content determined in step S306 (step S310).
When the determined command content is plural (step S310-NO), the command determination unit 430 generates candidate presentation information for narrowing the command content to one, and transmits this candidate presentation information as voice information data transmission. It transmits with respect to the original apparatus operating device 200 (step S313). Thereafter, the command determination unit 430 sets the operation target device indicated by the selection notification returned from the device operation apparatus 200 in response to the transmission of the candidate presentation information as a final determination result (step S314). When it is determined that there is only one command content determined in step S306, the command determination unit 430 finally determines each command content and operation target device.

  By such processing, the command determination unit 430 finally determines the contents of each command and the operation target device. In response to this, the command generation unit 440 generates a command code by reading the command code associated with the determined command content from the command set in the command management information 460 (step S311). Then, the command generation unit 440 transmits the generated command code to the device operating device 200 connected to the determined operation target device (step S312). At this time, the command generation unit 440 reads a device operating device address (IP address) associated with the device ID of the operation target device in the device management information 450 and designates this IP address as a destination.

[Second Embodiment]
A remote operation example of the second embodiment will be described with reference to FIG.
In the house 1 shown in FIG. 9, two rooms R1 and R2 are shown. In the room R1, the user 2 exists, and the air conditioner 100-1 and the device operating device 200-1 connected to the air conditioner 100-1 are installed. In the room R2, an air conditioner 100-3 and a device operating device 200-3 connected to the air conditioner 100-3 are installed. There is no user in room R2. In the second embodiment, the configurations of the device operating device 200 and the voice analysis server 400 may be the same as those in FIGS. 2 and 3, respectively.

  It is assumed that the user 2 in the room R1 utters voices “This room is hot” and “The room R2 is turned off” for remote control. The device operating device 200 transmits the voice information data of the voice data of the uttered voice to the voice analysis server 400.

  The command determination unit 430 in the voice analysis server 400 that has received the voice information data collates the utterance voice content of “This room is hot” with the utterance voice content of the command management information 460, so that the command content is the air conditioner. It is determined that this is an instruction to lower the set temperature of the current temperature. Further, the command determination unit 430 determines that the operation target device is the air conditioner 100-1 provided in the room R1.

Then, in response to the determination of the command content and the operation target device as described above, the command generation unit 440 in the voice analysis server 400 decreases the set temperature of the air conditioner 100-1 by a predetermined value from the current set value. Command code for Then, the generated command code is transmitted to the device operating device 200-1.
Moreover, the command generation part 440 produces | generates the command code for turning off the power supply of the air conditioner 100-3, and transmits to the apparatus operating device 200-2.
As a result, the set temperature of the air conditioner 100-1 in the room R1 is changed to be a predetermined value lower, and the air conditioner 100-3 in the room R2 stops operating by switching the power supply from on to off.

  As described above, in the second embodiment, the user 2 can remotely control not only the device 100 in the room where the user 2 exists but also the device 100 in the room where the user 2 does not exist.

In order to realize such remote operation, the voice analysis server 400 uses information on the subgroup ID in the device management information 450. The concept of subgroups will be described with reference to FIG.
In FIG. 10, six devices A, B, C, D, E, and F are installed in a house 601. In this house 601, two devices A and B are installed in the room 611-1 and two devices C and D are installed in the room 611-2. The devices E and F are portable devices whose installation location cannot be specified, for example.
The house 602 is provided with six devices G, H, I, J, K, and L. In this house 602, two devices G and H are installed in the room 621-1, and two devices I and J are installed in the room 621-2. The devices K and L are portable devices whose installation locations cannot be specified, for example.

Main group 1 is assigned to house 601, and main group 2 is assigned to house 602. In this way, the main group is a group that is divided corresponding to each house, for example.
In main group 1, subgroup 1 is assigned to room 611-1 and subgroup 2 is assigned to room 611-2. Accordingly, devices A and B are associated with subgroup 1, and devices C and D are associated with subgroup 2.
In the main group 2, the sub group 4 is assigned to the room 621-1, and the sub group 5 is assigned to the room 621-2. Therefore, devices C and D are associated with subgroup 4, and devices I and J are associated with subgroup 5.

  In addition, although a subgroup is a group divided according to each room as described above, for example, a user who is the owner of the portable device E, such as the subgroup 3 in the room 611, for example, You may set a subgroup including the apparatus B of the room 611-1 which a user mainly uses. The subgroup 6 in the room 621 also sets a subgroup including the user who is the owner of the portable device K and the device H in the room 621-1 used mainly by this user. Further, there may be devices that do not belong to any subgroup, such as the device F in the room 611 and the device L in the room 621.

The main group and subgroup set as described above are assigned a main group ID and a subgroup ID that uniquely identify each individual group.
In the example of FIG. 10, the main group IDs assigned to the main group 1 and the main group 2 are represented as “MG1” and “MG2”, respectively. The subgroup IDs assigned to the subgroups 1, 2, and 3 in the main group 1 are represented as “SG11”, “SG12”, and “SG13”, respectively. Further, the subgroup IDs assigned to the subgroups 3, 4, and 5 in the main group 2 are represented as “SG14”, “SG15”, and “SG16”, respectively.
The device IDs of devices A, B, C, D, E, F, G, H, I, J, K, and L are “A”, “B”, “C”, “D”, “E”, “F”, “G”, “H”, “I”, “J”, “K”, “L”.
The content exemplified in the device management information 450 of FIG. 4 reflects the group setting shown in FIG.

  In addition, although illustration is abbreviate | omitted, the audio | voice analysis server 400 of this embodiment memorize | stores the main group management information which matched the name of the house (or family) which the user registered for every set main group ID, for example. Moreover, the subgroup management information which matched the name of the room for every set subgroup ID is memorize | stored. By using these pieces of information, in this embodiment, it is possible to perform a remote operation in response to an utterance voice that is performed by a user specifying a room name.

As described above, the command determination unit 430 compares the utterance voice content of “This room is hot” with the utterance voice content of the command management information 460, so that the command content lowers the set temperature of the air conditioner from the current level. It is determined that it is an instruction.
Further, as described with reference to FIG. 9, in response to the user 2 uttering a voice, the command determination unit 430 extracts a record including the device operating device ID included in the metadata of the voice information data from the device management information 450. To do. From the device ID and the subgroup ID in the extracted record, it is recognized that the operation target device is the air conditioner 100-1 and the installed room is the room R1.

  Further, the command determination unit 430 recognizes that the command content is power off by comparing the utterance content of “room R2 is powered off” with the utterance voice content of the command management information 460. The operation target device is recognized as the air conditioner 100-3 in the room R2. In recognizing the operation target device, the command determination unit 430 needs to recognize that the power-off target device 100 is installed in the room R2. In this case, as described above, a table that manages subgroups and room names in association with each other may be referred to. The command determination unit 430 can recognize that the operation target installed in the room R2 is the air conditioner 100-3 by extracting the device ID associated with the subgroup ID of the room R2.

In response to the determination of the command content and the operation target device as described above, the command generation unit 440 generates a command code for lowering the temperature of the air conditioner 100-1 by a predetermined value from the current time, and the device operation device 200-1 is transmitted.
Moreover, the command generation part 440 produces | generates the command code for turning off the power supply of the air conditioner 100-3, and transmits to the apparatus operating device 200-2.
Thereby, the air conditioner 100-1 in the room R1 operates so as to set the set temperature low, and the air conditioner 100-3 in the room R2 stops the operation by switching the power supply from on to off.

[Third Embodiment]
A remote operation example of the third embodiment will be described with reference to FIG. Note that the configurations of the device operating device 200 and the voice analysis server 400 in the third embodiment may be the same as those in FIGS. 2 and 3, respectively.
In the house 1 shown in FIG. 11, an air conditioner 100-1 and a device operating device 200-1 connected to the air conditioner 100-1 are installed. Further, in the same room where the air conditioner 100-1 is installed, a heater 100-4 and a device operating device 200-4 connected to the heater 100-4 are installed. That is, two kinds of heating devices are installed in the house 1 in the same room.
Here, the heater 100-4 can rapidly heat the room, but has a feature of high power consumption. On the other hand, the air conditioner 100-1 cannot warm the room as rapidly as the heater 100-4, but has a feature of low power consumption.

  Assume that the user 2 utters a voice saying “This room is cold” for remote control. The device operating device 200 transmits the voice information data of the voice data of the uttered voice to the voice analysis server 400.

  The command determination unit 430 in the voice analysis server 400 that has received the voice information data collates the utterance voice content of “This room is cold” with the utterance voice content of the command management information 460, so that the air conditioner that is a heating appliance is used. It is determined that the Shona 100-1 and the heater 100-4 are candidates for the operation target device. At this time, it is assumed that the air conditioner 100-1 and the heater 100-4 are both turned off. Note that the command determination unit 430 has the air conditioner 100-1 and the heater 100-4 turned off based on, for example, the operation history of the device 100 stored in the voice analysis server 400 so far. Can be recognized.

  In this case, it is necessary to warm the room rapidly. Therefore, the command determination unit 430 determines the heater 100-4 as the operation target device among the air conditioner 100-1 and the heater 100-4 determined as the operation target device candidates. It is determined that the heater 100-4 is turned on to designate a predetermined set temperature.

  The command code generated according to the determination result is transmitted from the voice analysis server 400 to the device operating device 200 connected to the heater 100-4, so that the heater 100-4 is turned on and operates in the house 1. To start.

  In addition, in the third embodiment, for example, when the room temperature reaches a set temperature, or when a certain time elapses after the heater 100-4 is turned on, the command determination unit 430 performs the following command content: And the determination result for the operation target device is output. That is, the command determination unit 430 determines that the heater 100-4 should be turned off and the air conditioner 100-1 should be turned on to specify a predetermined set temperature. As described above, the command determination unit 430 according to the third embodiment determines two or more operation target devices and command contents for performing a predetermined operation in cooperation with these operation target devices.

  According to the determination result, the command generation unit 440 generates a command code for turning off the power of the heater 100-4 and transmits the command code to the device operating device 200-4. Further, a command code for turning on the power of the air conditioner 100-1 and a command code for instructing the air conditioner 100-1 to set a predetermined set temperature are generated and transmitted to the device operating device 200-1. .

Thereby, in the house 1, the heater 100-4 is turned off while the room is sufficiently warmed. Instead, the air conditioner 100-1 is turned on to perform heating in a low power consumption state. Will continue.
As described above, in the third embodiment, a plurality of devices 100 can be remotely operated in different time series based on, for example, an utterance voice “This room is cold”.

[Fourth Embodiment]
Subsequently, a fourth embodiment will be described.
FIG. 12 shows a configuration example of a device operation system as the fourth embodiment. In this figure, the same parts as those in FIG.
The device operation system shown in FIG. 12 includes a device operation device 200A and a gesture analysis server (biological information analysis server) 400A instead of the device operation device 200 and the voice analysis server 400 shown in FIG.
The device operating device 200 </ b> A includes an imaging unit 250 instead of the microphone 210. Further, instead of the voice information processing unit 220, a gesture information processing unit 260 is provided.
The gesture information processing unit 260 includes a subject extraction unit 261, a gesture information data generation unit 262, and a metadata storage unit 263.

  The configuration of the gesture analysis server 400A is the same as that shown in FIG. 3, for example. However, as will be described below, the operation of each unit in the gesture analysis server 400A differs appropriately depending on the analysis target being gesture information data.

  The operation of the device operation device 200A and the gesture analysis server 400A shown in FIGS. 12 and 3 will be described together with an example of remote operation performed by the user in the fourth embodiment. In the following description, it is assumed that the installation mode of the device 100 and the device operating device 200 in the house 1 is the same as that in FIG.

In the fourth embodiment, the user 2 performs a predetermined operation (gesture) instead of voice when performing remote operation. That is, the biological information in the fourth embodiment is a body movement.
As a specific example, the gesture of shaking the hand to the right is defined as an operation for instructing channel switching so as to increase the channel number of the television receiver 100-2 by one. On the other hand, the gesture of shaking the hand to the left is defined as an operation for instructing channel switching so that the channel number of the television receiver 100-2 is decreased by one.

  Here, it is assumed that the user 2 wants to perform channel switching so as to increase the channel number by one. In response to this, the user performs a gesture of waving his hand to the right toward any of the device operation devices 200 installed indoors.

In the device operating device 200A, the imaging unit 250 images the user 2 who is making this gesture.
Next, in the gesture information processing unit 260, the subject extraction unit 261 detects an image portion of the subject as a user performing a gesture from the image data captured by the imaging unit 250, and extracts this image portion. Next, the gesture information data generation unit 262 adds the predetermined metadata stored in the metadata storage unit 263 to the extracted image portion data to generate gesture information data, and the generated gesture information data Is transmitted to the gesture analysis server 400A.

  The gesture information data transmitted as described above is received by the communication unit 470 of the gesture analysis server 400A. When the same gesture information data transmitted from different device operation devices 200 is received at this time, the duplicate reception handling processing unit 410 discards the gesture information data received after the second time by the processing according to FIG.

  The analysis unit 420 analyzes the gesture information data. The analysis unit 420 recognizes the content of the gesture and the content of the metadata from the image data of the subject portion by analysis. In this case, the analysis unit 420 recognizes that the gesture content is that the hand is shaken to the right.

The command management information 460 in the fourth embodiment stores gesture content data in place of the speech voice content in the command set.
The command determination unit 430 collates the gesture content recognized by the analysis unit 420 with the gesture content in the command management information 460. Accordingly, the command content corresponding to the gesture content recognized by the analysis unit 420 and the operation target device are recognized. In this case, the command determination unit 430 determines that the operation target device is the television receiver 100-2, and the command content is to perform channel switching so as to increase the channel number by one.

The command generation unit 440 generates a command code corresponding to the determination result and transmits the command code to the device operating device 200-2 connected to the television receiver 100-2.
The device operating device 200-2 transmits the received command code to the television receiver 100-2. In response to receiving this command code, the television receiver 100-2 performs channel switching so as to increase the channel number by one.

  As described above, in the fourth embodiment, the device 100 can be remotely operated by a gesture performed by the user 2.

FIG. 13 shows a configuration example of a device operating device 200B corresponding to a modification of the fourth embodiment. In this figure, the same parts as those in FIG.
In the modified example of the fourth embodiment, the user performs a gesture by holding the gesture detection device 500. The gesture detection device 500 detects a movement according to a gesture performed by the user, and transmits the detection information to the device operation device 200B.

  As shown in FIG. 13, the gesture detection device 500 includes an acceleration sensor 510 and a transmission unit 520. The acceleration sensor 510 detects a movement according to the gesture and outputs an acceleration detection signal. The transmission unit 520 transmits the acceleration detection signal to the device operating device 200B wirelessly.

  The gesture detection device 500 has a shape like a wristwatch, for example. The user performs, for example, a gesture of moving his / her hand while wearing the gesture detection device 500 on the arm. Alternatively, the gesture detection device 500 may be of a type that is shaped like a headset and is worn on the user's head. As described above, the gesture detection device 500 is a type that the user can wear, so that the user can perform remote operation with almost no interruption to the work that has been performed.

  The device operating device 200B includes a receiving unit 270 instead of the imaging unit 250. The receiving unit 270 receives an acceleration detection signal transmitted from the transmitting unit 520 of the gesture detection device 500.

  In the device operating device 200B, the gesture information processing unit 260 includes a gesture information data generation unit 262 and a metadata storage unit 263. In this case, the gesture information data generation unit 262 generates gesture information data by adding the metadata read from the metadata storage unit 263 to the data of the acceleration detection signal received by the reception unit 270. This gesture information data is transmitted from the communication unit 230 to the gesture analysis server 400A.

In the gesture analysis server 400 </ b> A, the analysis unit 420 recognizes a gesture based on, for example, a change pattern according to the passage of time indicated by the acceleration detection signal in the gesture information data by analysis processing.
The command determination unit 430 determines the command content and the operation target device based on the recognized gesture. The command generation unit 440 generates a command code according to the determination result of the command determination unit 430, and transmits the command code to the device operation device 200B connected to the operation target device.
Thus, in the modification of the fourth embodiment, the device 100 can be remotely operated by a user performing a gesture using a gesture detection device including an acceleration sensor.

  For example, although the device 100 and the device operation device 200 have been described as separate devices, the device operation device 200 may be mounted in the device 100 in advance. Further, in this embodiment, an air conditioner and a television receiver are given as examples of the device 100 to be controlled, but any other devices that require operation, such as a refrigerator, a cooking utensil, and a video. Equipment may be targeted. In the description of the above embodiment, the device operating device 200 is connected to correspond to, for example, one device 100. For example, a function as the device operating device 200 is mounted on a mobile phone or a smartphone. Thus, various devices may be configured to be operated as control targets.

  In addition, a program for realizing the functions of the respective units shown in FIGS. 2, 3, 12 and 13 is recorded on a computer-readable recording medium, and the program recorded on the recording medium is read into a computer system. The processing for remote operation may be performed by executing the command. Here, the “computer system” includes an OS and hardware such as peripheral devices.

Further, the “computer system” includes a homepage providing environment (or display environment) if a WWW system is used.
The “computer-readable recording medium” refers to a storage device such as a flexible medium, a magneto-optical disk, a portable medium such as a ROM and a CD-ROM, and a hard disk incorporated in a computer system. Further, the “computer-readable recording medium” refers to a volatile memory (RAM) in a computer system that becomes a server or a client when a program is transmitted via a network such as the Internet or a communication line such as a telephone line. In addition, those holding programs for a certain period of time are also included. The program may be a program for realizing a part of the functions described above, and may be a program capable of realizing the functions described above in combination with a program already recorded in a computer system.

  The embodiment of the present invention has been described in detail with reference to the drawings. However, the specific configuration is not limited to this embodiment, and includes designs and the like that do not depart from the gist of the present invention.

DESCRIPTION OF SYMBOLS 1 House 100 apparatus 200, 200A, 200B apparatus operation apparatus 210 microphone 220 audio | voice information processing part 221 speech audio | voice component extraction part 222 format conversion part 223 metadata memory | storage part 224 audio | voice information data generation part 230 communication part 240 apparatus control part 400 audio | voice analysis Server 400A Gesture analysis server 410 Duplicate reception processing unit 420 Analysis unit 430 Command determination unit 440 Command generation unit 450 Device management information 460 Command management information 470 Communication unit

Claims (14)

A device operating device and a biological information analysis server;
The device operating device is
It is mounted on or connected to a device to be operated, which is a household electrical machine that is installed at home.
A biometric information input unit that inputs speech sound for operating any one of the operation target devices to be controlled by the device itself or another device operation device;
A biological information processing unit that generates biological information data including an audio signal representing the input utterance voice ;
A first communication unit for transmitting the biometric information data to the biometric information analysis server and receiving a command transmitted from the biometric information analysis server;
A device control unit that controls the operation target device so that an operation in response to the received command is performed;
The biological information analysis server is
A second communication unit that receives the biological information data and transmits the command;
An analysis unit a voice signal to voice analysis, recognize contents of the speech contained in the received biometric information data,
A command determination unit that determines the command content and the operation target device based on the content of the utterance voice recognized by voice analysis of the biological information data;
Command generation for generating the command corresponding to the determined command content, and transmitting the generated command to the device operation device whose control target is the determined operation target device by the second communication unit Comprising a part,
A device operation system characterized by that. The analysis unit recognizes the content indicating the temperature as the content of the speech voice,
The command determination unit determines the operation target device having a temperature adjustment function when the content indicating the temperature is recognized.
The apparatus operation system according to claim 1. The device operating device is mounted on or connected to the first operation target device,
The biological information processing unit generates biological information data including information for specifying a place where the household electric machine is installed,
The command determination unit determines a second operation target device different from the first operation target device based on information specifying the location,
The command generation unit causes the second communication unit to transmit the generated command to the device operation device whose control target is the second operation target device. 2. The device operation system according to 2. The biological information analysis server is
If the result of comparing the biometric information data received this time with the biometric information data received by the second communication unit within a predetermined time in the past matches, the biometric information data received this time is discarded Further comprising a duplicate reception handling processing unit,
The device operation system according to any one of claims 1 to 3, wherein The biological information analysis server is
Group management information for managing the operation target device and the device operation device that controls the operation target device separately for each group corresponding to the room in which the operation target device and the device operation device are installed Prepared,
The command determination unit refers to the group management information when it is analyzed that the biometric information data has contents for operating the operation target device by specifying a room, and the operation target in the specified room Determine the command content for operating the device,
The apparatus operating system according to any one of claims 1 to 4, wherein The biological information processing unit
Generating the biometric information data based on the utterance voice input within a predetermined time after inputting a predetermined operation start warning signal,
The apparatus operating system according to any one of claims 1 to 5, wherein The command determination unit
Based on the content of the uttered voice , two or more types of the operation target devices and a command content for performing a predetermined operation in cooperation with the operation target devices are determined.
The device operation system according to any one of claims 1 to 6, wherein The biological information input unit includes an imaging unit to enter the image of the captured user,
The apparatus operation system according to claim 1 , wherein the biological information processing unit generates the biological information data including a captured image . The biological information input unit includes a receiving unit that receives a detection signal of the acceleration sensor for detecting acceleration based on a movement of the user,
The apparatus operation system according to claim 1 , wherein the biological information processing unit generates the biological information data including a detection signal . A biometric information input unit that inputs biometric information for operating any of the operation target devices that are to be controlled by the device itself or another device operation device;
A biological information processing unit that generates biological information data including an audio signal representing the input utterance voice ;
A communication unit that transmits the biological information data to the biological information analysis server and receives a command transmitted from the biological information analysis server;
A device control unit that controls the operation target device so that an operation in response to the received command is executed;
Equipped with a,
Equipment operating device is mounted or connected to the operation target apparatus is a Consumer Electronics installed in the home, characterized in Rukoto. A communication unit that receives the biological information data generated based on the uttered voice input by the device operating device from the device operating device, and transmits a command;
An analysis unit a voice signal to voice analysis, recognize contents of the speech contained in the received biometric information data,
A command determination unit for determining the command content and the operation target device based on the content of the utterance voice recognized by performing voice analysis on the biological information data;
A command generation unit that generates the command according to the determined command content, and causes the communication unit to transmit the generated command to the device operation device whose control target is the determined operation target device;
A server comprising: A device operation method in a device operation system comprising a device operation device and a biological information analysis server mounted or connected to an operation target device that is a home electric machine instrument installed in a home ,
The device operating device is
A biometric information input step for inputting speech sound for operating any one of the operation target devices to be controlled by the device itself or another device operation device;
A biometric information processing step for generating biometric information data including a voice signal representing the input speech voice ;
A first communication step of transmitting the biological information data to the biological information analysis server and receiving a command transmitted from the biological information analysis server;
A device control step for controlling the operation target device so that an operation in response to the received command is executed,
The biological information analysis server is
A second communication step of receiving the biological information data and transmitting the command;
An analysis step of the speech signal and voice analysis, recognizing the contents of the speech contained in the received biometric information data,
A command determination step of determining the command content and the operation target device based on the content of the utterance voice recognized by performing voice analysis on the biological information data;
Command generation for generating the command according to the determined command content, and transmitting the generated command to the device operating device whose control target is the determined operation target device in the second communication step Comprising steps,
A device operating method characterized by the above. To a computer mounted on or connected to an operation target device that is a home electric machine / equipment installed at home ,
A biometric information input step for inputting speech sound for operating any one of the operation target devices to be controlled by the device itself or another device operation device;
A biometric information processing step for generating biometric information data including a voice signal representing the input speech voice ;
A communication step of transmitting the biological information data to the biological information analysis server and receiving a command transmitted from the biological information analysis server;
A device control step for controlling the device to be operated so that an operation in response to the received command is executed;
A program for running On the computer,
A communication step of receiving biological information data generated based on the speech voice input by the device operating device from the device operating device and transmitting a command;
An analysis step of the speech signal and voice analysis, recognizing the contents of the speech contained in the received biometric information data,
A command determination step for determining the command content and the operation target device based on the content of the utterance voice recognized by performing voice analysis on the biological information data;
A command generation step of generating the command according to the determined command content, and transmitting the generated command to the device operating device whose control target is the determined operation target device by the communication step;
A program for running

Images