JP2007061970A - Processing method in home robot and home robot - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To provide a home robot, reducing a user's time and trouble when the robot needs the user's assist when he/she is in a remote place. <P>SOLUTION: An execution request for a task requiring to separate from a user is detected, an interactive mode with the user using a household electric device is selected on detecting the execution request, and according to the selected interactive mode, a dialog with the user including inquiry to the user and answer from the user is performed. <P>COPYRIGHT: (C)2007,JPO&INPIT

Description

  The present invention relates to a home robot that controls home appliances through a home network.

  In recent years, domestic robots, so-called home robots, have been put into practical use, and robots that can be moved around the home for cleaning and answering machines have been realized. However, the home environment is complicated for robots to act autonomously. For example, if a chair becomes an obstacle and the robot cannot pass through the passage, it is necessary to have the person take the chair away, or if you do not know whether the falling object can be judged as garbage, There are situations where it is necessary to get help from a person who needs to be received.

Patent Document 1 discloses a technology for changing whether or not a robot asks for help from a human depending on the importance of the situation.
JP 2004-249393 A

  It is difficult for a robot working in a complex home environment to be completely autonomous, and it is difficult to eliminate the occurrence of scenes that require people to work or make decisions as described above. In this case, it is important how to reduce the labor of the person when the robot asks for help. Patent Document 1 does not disclose a device from such a viewpoint.

  As a method for a robot to ask for help from a person, it is common to output a message by voice. If the request for help involves a human task such as removing the chair, the user must go to the robot to perform the task, which is an unavoidable task. On the other hand, when a robot makes an inquiry requesting whether or not a fallen object is garbage, if the user can send an image to a user at a distant location, the user does not have to go to the location where the robot is located. Should be done. Some home robots have a function of notifying an image taken on a mobile phone possessed by a user who is out of the office as an e-mail when an abnormality in his / her home is detected.

  Even at home, if the user always has a small terminal such as a mobile phone or a dedicated controller, the robot can make an inquiry to the user via the terminal. If there is no work involved, it is possible to reduce the labor of the person when the robot asks for help from the person. However, it is unlikely that the user always has a mobile phone or terminal at home. In addition, the percentage of mobile phones with internet functions in the elderly is low, and operation with a small terminal is often difficult.

  On the other hand, there are many electrical appliances in the home, and some household appliances and their remote control are nearby wherever they are. If you can interact with the robot through these household appliances, you can answer the inquiry from the robot on the spot without having to carry the terminal.

  An object of the present invention is to provide a processing method in a home robot and a home robot for reducing the trouble of the user when the robot needs the help of a user at a remote place.

  In one aspect of the present invention, in a home robot used in a home where at least one home appliance connected to a network is installed, a task execution request that needs to be separated from the user is detected, and the execution request is detected. To select a dialog method with the user using the household electrical appliance, and execute a dialog with the user including an inquiry to the user and a response from the user according to the selected dialog method.

  The selection of the interaction method includes, for example, selection of an output device that makes an inquiry to the user and an input device that receives an answer from the user. You may select household appliances as one or both of an output device and an input device. In this case, it is preferable to preferentially select household appliances that can be used for both the output device and the input device.

  It is preferable that the conversation execution means with the user outputs the inquiry as a message with an explanatory text indicating a response method for the user.

  Furthermore, when performing an answer from the user according to the dialogue method selected at the time of executing the dialogue, a normal operation on the home appliance is ignored.

  It is possible to reduce the time and effort of the user when the robot needs the help of the user who is away from the user by executing the dialogue with the user through the home appliance at the time of requesting the execution of the task that needs to leave the user. it can.

  FIG. 1 is a diagram showing an outline according to an embodiment of the present invention, and shows a layout in a home and an arrangement of home appliances. The robot 11 is a home robot used by the user 10 in the home, and includes wheels and limbs so as to be autonomously movable. The access point (AP) 12 is a base station that connects the home appliances 13 to 15 to the network, and communicates with the home appliances 13 to 15. The home appliances 13 to 15 are provided with a communication terminal called a wireless adapter for communication.

  In the present embodiment, the access point 12 is also network-connected between the home appliances 13 to 15 and the robot 11. As a communication system including the access point 12, for example, a wireless LAN such as IEEE 802.11a / b / g or a close proximity wireless communication system called Bluetooth (registered trademark) is used. The communication method between the robot 11 and the access point 12 is preferably a wireless method because the robot 11 moves, but the communication between the access point 11 and the home appliances 13 to 16 is performed by a wired network or a light. You can also use a line.

  The home appliances 13 to 15 include a television receiver (TV1, TV2) and an air conditioner in this example, and correspond to a home network. That is, the home appliances 13 to 15 are devices called so-called information home appliances or network home appliances, and the robot 11 can perform operations and state confirmation of the home appliances 13 to 15 via the network. The household electrical appliance as used in this embodiment means the household electrical appliance which can be connected to a network. A remote controller (hereinafter referred to as a remote controller) 16 is a dedicated remote controller for the home appliances 13 to 15 or a remote controller common to all. Hereinafter, the home appliance 13 will be described as TV 1, the home appliance 14 as TV 2, and the home appliance 15 as the air conditioner 1.

  The robot 11 normally stands by on the side of the user 10 (side), and executes various tasks in response to instructions from the user 10. For example, when the user 10 is instructed by voice “Turn on TV”, the robot 11 understands the utterance content by voice recognition, operates the TV 1 via the network, and turns on the TV 1. In this way, commands to the robot 11 are often performed using voice.

  When the user 10 instructs the robot 11 to see the state of the kitchen, the robot 11 moves from the living room to the kitchen, and checks whether there is an abnormality in the kitchen using image recognition technology or the like.

  When the robot 11 asks the user 10 for determination when the robot 11 is away from the user 10 in the kitchen, conventionally, the robot 11 displays a message such as “Can this package be moved?” By voice synthesis or the like. Output. In the current voice recognition technology, the distance between the user 10 and the robot 11 needs to be a short distance of about 1 m or less, so the user 10 hears the voice message from the robot 11 and moves to the kitchen to perform voice recognition. After speaking to the robot 11 as far as possible, an answer such as “OK” is uttered. Alternatively, the robot 11 returns to the place where the user 10 is located, approaches the user 10 to a distance where voice recognition is possible, asks a question, obtains an answer from the user 10, returns to the kitchen, and executes the commanded task. to continue.

  On the other hand, in the present embodiment, when the user 10 and the robot 11 are separated from each other, between the user 10 and the robot 11 via home appliances such as the TV 1 and the air conditioner 1 on the user 10 side. By performing a dialogue, the robot 11 does not go to the robot 11 and does not return to the user 10 side, and the robot 11 performs a dialogue that asks the user 10 for any inquiry, that is, a determination. Can do.

  FIG. 2 is a block diagram showing a system configuration of the robot 11. The person extraction unit 102 is a device for finding a person (user 10) around the robot 11, and is realized by, for example, known image processing such as moving object extraction or face recognition.

  The position measuring unit 103 is a device that measures the position (current position) of the robot 11. The robot 11 stores, for example, a home map as shown in FIG. 1 in the map memory 110, and always measures the current position of the robot 11 itself by, for example, measuring an appropriate mark in the home with stereo vision. I know. In the present embodiment, the position measuring unit 103 is also used to assist the main control unit 101 to measure the position of the user 10.

  The task database 104 is a device that stores a task execution method to be executed by the robot 11. The voice recognizing unit 107 recognizes the command utterance of the user 10 from surrounding sounds input by a microphone (not shown). The voice synthesizer 109 generates a voice waveform from text and outputs it from a speaker in order to transmit a message to the user 10. The air conditioner 111, the TV 112, and the lighting 11N are home appliances that can be controlled via a network in the home, and support home network standards such as UPnP and Echonet (registered trademark).

  The home appliance control unit 108 has a function of detecting home appliances in the home, controlling the home appliance via the network, and examining the operation state of the appliance, and corresponds to a controller in the home network standard. In UPnP and Echonet, the controller can not only control the target device, but can also sequentially grasp changes in the operating state of the device as event notifications.

  The input control unit 105 selects one device as an input device from a plurality of devices such as the voice recognition unit 107 and the home appliance control unit 108 in accordance with an instruction from the dialogue method selection unit 100, and inputs the selected input device. After the converted content is converted according to the input device, it is notified to the main control unit 101. The output control unit 106 selects one device as an output device from a plurality of devices such as the voice synthesis unit 109 and the home appliance control unit 108 in accordance with an instruction from the dialogue method selection unit 100.

  When the dialogue method selection unit 100 receives a dialogue method selection instruction from the main control unit 101, the dialogue method selection unit 100 acquires information on household electrical appliances in the home from the household appliance control unit 108, and recognizes speech that is an input / output device of the robot 11. The input / output device to be used is recognized from the available input / output devices including the unit 107 and the speech synthesis unit 109, and the input control unit 105 and the output control unit 105 are instructed to select the device to be used.

  Furthermore, when the robot 11 detects a task execution request that requires the robot 11 to move away from the user 10, the dialog method selection unit 100 selects an interaction method with the user 10 via the home appliance, specifically, an inquiry to the user 10. The output device for performing the selection and the input device for performing the answer from the user 10 are selected.

  The main control unit 101 executes a task stored in the task database 104 in response to a task execution request from the user 10. At that time, the main control unit 101 executes the task execution requested by the robot 11 based on the contents of the task stored in the task database 104 and the position of the user 10 obtained from the outputs of the person extraction unit 102 and the position measurement unit 103. If it is determined that it is necessary to leave the user 10 for the purpose, the dialogue method selection unit 100 is instructed to select the dialogue method.

  Next, the operation flow in the present embodiment will be described with reference to the flowcharts of FIGS. 3 and 4.

  FIG. 3 shows an overall rough process. First, the robot 11 is initialized when the power is turned on (step S301). That is, the OS (operating system) of the robot 11 is started, then the drive system control program, the sensor system control program, and the human interface system control program are started, and finally an application program for providing services to the user 10 is provided. The process proceeds to the next step S302. In step S302, a task instructed by the user 10, that is, an application program is executed.

  The application program executed in step S302 varies depending on how the robot 11 is used. In the present embodiment, a program for executing a task instructed by the user 10 on the user 10 side is executed according to the flowchart shown in FIG.

  During the execution of step S302, if the user 10 gives an instruction to end or an instruction to turn off the power is given, the process proceeds to step S303, where the application program and the control program are sequentially ended, and finally the OS Ends and the power is turned off.

Details of step S302 in FIG. 3 will be described below with reference to FIG.
(Step S401)
The robot 11 stands by on the user 10 side. In step S401, the main control unit 101 determines whether there is a task execution request according to a command from the user 10 or a predetermined schedule. For example, when the robot 11 receives a voice command “go to meet” from the user 10, the voice recognition unit 107 recognizes the command, and the recognition result is notified to the main control unit 101 via the input control unit 105. The In response to this notification, the main control unit 101 determines that execution of the task “greeting” is requested, and the process proceeds to step S402.

(Step S402)
When it is determined in step S401 that a task has been requested, the main control unit 101 extracts the contents of the requested task from the task database 104, and the robot 11 leaves the user 10 side to execute the task. Determine whether it is necessary. As a result of the determination, if the robot 11 needs to leave the user 10, the process proceeds to step S403 to select a dialogue method via the home appliance, and if there is no need to leave, the dialogue method is not switched. That is, the process proceeds to step S405 for task execution without selecting an interactive method via the home appliance.

  In the task database 104, information on various tasks that can be executed by the robot 11 is stored as shown in FIG. Each line in FIG. 5 represents information related to each task, the task name is described in the “task” column, the program of the basic operation program installed in the robot 11 is specified in the “basic operation” column, In the "" column, the target for executing the basic operation is specified, and in the "Location" column, the location for executing the basic operation is specified.

  For example, the task “greeting” on the first line indicates that a program “greet” is executed for a person at the entrance. Therefore, when a task execution request “greeting” is received from the user 10, it is understood that it is necessary to move to the “entrance” from the information stored in the task database 104, and the robot measured by the position measuring unit 103. 11 current positions are compared to determine whether it is necessary to leave the user 10 side. Here, the example in which task information is stored in a table format as shown in FIG. 5 has been described. However, a plan for task execution is generated using a planning technique, and determination is made based on the presence or absence of movement performed in the process of the generated plan. You may do it.

(Step S403)
As described above, if it is determined in step S402 that the robot 11 needs to leave the user 10, the process proceeds to step S403 in order to select an interaction method between the user 10 and the robot 11. In step S403, a dialogue method using home appliances is selected. When the dialogue method selection unit 100 is instructed to select a dialogue method from the main control unit 101, the dialogue method selection unit 100 is likely to be used for dialogue with the robot 11 from the household appliances in the home to the household appliance control unit 108. Request a list of input / output devices.

  The home appliance control unit 108 checks device information of home appliances in the home and lists functions that can be used as input / output devices for dialogue with the robot 11. In home network standards such as Echonet, interface specifications for accessing devices from the network are determined for each type of home appliances such as TVs and air conditioners. FIG. 6 is an example of device information received by the home appliance control unit 108 from a home appliance that conforms to the UPnP standard. According to FIG. 6, the device information is expressed in an XML (extended markup language) format, and provides the home appliance control unit 108 with information necessary for controlling the home appliance. For example, <Device> represents the name of the device, <DeviceType> represents the type of device, <Action> represents a command for operating the device, and <Event> represents an event for notifying a change in the state of the device. The home appliance control unit 108 can list what devices are in the home and what functions are in the home by comparing the information of the stored standard specifications with the device information.

  The home appliance control unit 108 stores commands and events that can be used as input / output devices from the listed home appliance functions, for example, as shown in FIG. The row in FIG. 7 represents one means, where “equipment type” is the type of equipment such as a TV or air conditioner, “means” is a command or event, “means name” is a description of the command or event, and “means type”. Represents a command or an event. “Input / output” indicates whether it is used for input or output, and “input method” indicates how to use the means. “UP / DOWN” indicates that the upper and lower values of the event notification value are used in an alternative, and “SELECT” indicates that the event notification value is used for multiple selection.

  The home appliance control unit 108 is an input / output device that can be used as an input / output method in the current home environment based on the device information shown in FIG. 6 and the information shown in FIG. 7 received from home appliances in the home. (FIG. 8) is passed to the dialogue method selection unit 100. Based on the list of input / output devices received from the home appliance control unit 108, the dialogue method selection unit 100 presents to the user 10 a dialogue method while the robot 11 is away from the user 10 side. I take the. The procedure for selecting the dialogue method is described below.

  First, the home appliance on the user 10 side is selected from the position of the user 10 and the location of the home appliance stored in the map memory 110 from the list received from the home appliance control unit 108. In the example of FIG. 1, the target is narrowed down to the TV 1 and the air conditioner 1 as home appliances on the user 10 side.

  Next, if there is a device having both an input device and an output device in one home appliance, the device is determined to be used as an input device and an output device. If there is no device having both an input device and an output device, the device used for the input device and the device used for the output device are selected separately. If there is no device that can be used as an output device, the speech synthesizer 109 in the robot 11 is set as an output device. If there is no device that can be used as an input device, the voice recognition unit 107 in the robot 11 is set as the input device. If there are multiple home appliances that can be used as input devices, output devices, or input / output devices, determine the priority according to the type of device, or consider whether the device is in use as a home appliance. Select the device.

  Through the above processing, the TV 1 is finally selected as an input / output device as shown in FIG. This is the choice of interaction method.

(Step S404)
The dialogue method selection unit 100 notifies the user 10 of a message to change to the selected dialogue method and confirms the dialogue method. A message sentence to be notified to the user 10 is generated from the information shown in FIG.

(Execution task) + "I will contact you. Since I will be sent to" + (output device name) + ", please answer by the operation of" + (input device name) + "."
As a result, the voice synthesizing unit 109 outputs a message “I will greet you. Since I will contact you later on TV 1, please answer by operating TV 1.” Thereafter, the speech recognition unit 107 confirms an utterance such as “I understand” indicating that the change from the user 10 is accepted.

(Step S405)
After step S404, the task requested in step S401 is executed.

(Step S406)
If a situation occurs in which the robot 11 wants to determine the user 10 during the execution of the task in step S405, the process proceeds to step S407. Otherwise, the execution of the task is continued.

(Step S407)
An inquiry to the user 10 is executed. The main control unit 101 executes, for example, a command “Dialog” when making an inquiry to the user 10. Dialog has three options:
[Option 1] Dialog (Message, MB_OK)
[Option 2] Dialog (Message, MB_YESNO)
[Option 3] Dialog (MessageList, MB_SELECT)
[Option 1] is a case in which only notification is given to the user 10, and only the text specified in the first argument Message is output. Normally, synthesized speech is output from the speech synthesizer 109, but in this embodiment, text is displayed on the screen of the TV 1 as shown in FIG. [Option 2] is a case where the answer of YES / NO is input from the user 10, the text specified in the message of the first argument is output, and the answer from the user 10 is received. Normally, a message is output from the voice synthesizer 109 as a synthesized sound, and a reply “Yes” or “No” of the user 10 is input from the voice recognizer 107. In the present embodiment, for example, text is displayed on the screen of the TV 1 as shown in FIG. 11, and YES / NO is input by volume operation using the remote control 16 of the TV 1. [Option 3] is a case where a plurality of lines of text specified by the first argument are output, and the user 10 selects which line of text to select. In general, an inquiry such as “Which drink should be tea, juice, or coffee?” Is output from the voice synthesis unit 109 as a synthesized sound, and a reply such as “tea” from the user 10 is input from the voice recognition unit 107. In the present embodiment, for example, options are displayed on the screen of the TV 1 as shown in FIG.

  Taking the inquiry of [Option 2] as an example, the following inquiry execution procedure will be described. For example, according to the example of FIG. 11, in response to an inquiry command “Ret = Dialog (“ Can I turn on the entrance lighting? ”, MB_YESNO)”, the main control unit 101 sends “ "Is it OK to turn on?" Is output, and the input control unit 105 is instructed to obtain a YES / NO answer. The output control unit 106 determines that the TV1 should be instructed to execute the outputText command according to the output method of FIG. 9 determined in step S403, and generates a message to be passed as an argument of the outputText. In [Option 2], since the answer method designation is an MB_YESNO alternative query, it is decided to use the volume increase / decrease among the input methods shown in FIG. 9, and a message explaining the answer method is displayed. The inquiry shown in FIG. 11 is output to the TV 1.

  When the user 10 performs a volume operation with the remote controller of the TV 1 in accordance with the inquiry of FIG. 11, the internal state (Volume value) of the TV 1 is changed, and accordingly, the volume event is notified from the TV 1 to the home appliance control unit 108. The home appliance control unit 108 transfers the notified volume event to the input control unit 105. The input control unit 105 is instructed by the dialogue method selection unit 100 to input the volume value change of the TV 1 from the user 10, so the content of the volume value change, that is, the content of the volume event is checked against FIG. 9. If it is a volume increase event, it is converted to “YES”, and if it is a volume decrease event, it is converted to “NO” and notified to the main control unit 101.

  In this way, the main control unit 101 obtains an answer from the user 10 as a return value for the inquiry of [Option 2], and continues to execute the task based on the answer from the user 10. As described above, the user 10 and the robot 11 that are located apart from each other can interact with each other via the home appliance, in this example, the TV 1.

  In addition, the main control unit 101 may perform an instruction to return the operation to the home appliance performed when the user 10 answers to the home appliance, and may automatically return the state of the home appliance. Here, since the user 10 performed an operation to increase the volume of the TV 1 is an action performed as an answer to the robot 11, the user 10 does not intend to increase the volume of the TV 1. It is desirable to automatically restore the change in the state of the TV 1 that occurs due to the answer of the user 10 by instructing the TV 1. Moreover, the event from the household appliance notified to the household appliance control part 108 other than step S407 is disregarded, and it processes as normal household appliance operation by the user 10. FIG.

(Step S408)
It is determined whether or not the task executed in step S405 has been completed, and when it is completed, the process proceeds to step S409.

(Step S409)
When the task is completed, the robot 11 returns to the user 10 side, and the dialogue method selection unit 100 returns the input device and the output device to the standard voice recognition unit 107 and the voice synthesis unit 108, respectively. Notify you that you have returned to the conversation. Thereafter, the process returns to step S401, and the robot 11 again stands by on the user 10 side.

  As described above, according to the present embodiment, when the robot 11 away from the user 10 asks the user 10 for help, the user 10 and the robot 11 interact through a home appliance such as a TV. The trouble that the user 10 goes to the position where the robot 11 is present and the waste that the robot 11 returns to the user 10 can be saved.

  A TV has both functions of an input device and an output device, whereas an air conditioner has only the function of an input device. When the home appliance that can be used for the dialogue between the user 10 and the robot 11 is a device having only an input device function such as an air conditioner, the speech synthesizer 109 in the robot 11 is used as an output device as usual. In this case, the message shown in FIG. 11 is output as a voice from the voice synthesizer 109, and the user 10 may make a reply by an operation of raising or lowering the air volume of the air conditioner.

  Note that the present invention is not limited to the above-described embodiment as it is, and can be embodied by modifying the components without departing from the scope of the invention in the implementation stage. In addition, various inventions can be formed by appropriately combining a plurality of constituent elements disclosed in the embodiment. For example, some components may be deleted from all the components shown in the embodiment.

The figure which shows the example of household appliances arrangement | positioning in the home to which the home robot which concerns on one Embodiment of this invention is applied The block diagram which shows the system configuration | structure of the home robot which concerns on one Embodiment of this invention. A flowchart showing a schematic processing procedure in the embodiment The flowchart which shows the detailed process sequence of the user task execution step in FIG. The figure which shows the example of the content of the task database in FIG. The figure which shows the example of the apparatus information of the household appliances in FIG. The figure which shows the example of the function definition of the household appliances used for an input / output device A diagram showing a list of input / output devices that can be used in the example of FIG. Diagram showing examples of selected input / output devices The figure which shows the example of the inquiry screen to the user The figure which shows the example of the inquiry screen to the user The figure which shows the example of the inquiry screen to the user

Explanation of symbols

10 ... user;
11 ... Home robot;
12 ... access point;
13-15 ... household appliances;
16 ... remote control;
100 ... Dialog method selection unit;
101 ... main control unit;
102 ... person extraction part;
103 ... position measuring unit;
104 ... task database;
105... Input control unit;
106... Output control unit;
107 ... voice recognition unit;
108 ... home appliance control unit;
109 ... speech synthesis unit;
110: Map memory;
111-11N ... Household appliances

Claims (8)

In a processing method in a home robot used in a home where at least one home appliance connected to a network is installed,
Detecting a request to execute a task that needs to leave the user;
Selecting an interaction method with a user using the household electrical appliance by detecting the execution request;
A processing method in a home robot comprising: executing an interaction with a user including an inquiry to the user and an answer from the user according to a selected interaction method.   The processing method in the home robot according to claim 1, wherein the selection of the interaction method includes selection of an output device that makes an inquiry to the user and an input device that receives an answer from the user. In a home robot used in a home where at least one home appliance connected to the network is installed,
Detection means for detecting an execution request of a task that needs to be separated from the user;
A dialogue method selection means for selecting a dialogue method with a user using the home appliance by detecting the execution request;
A home robot comprising dialog execution means for executing a dialog with a user including an inquiry to the user and an answer from the user according to a selected dialog method.   The home robot according to claim 3, wherein the interaction method selection unit selects an interaction method including selection of an output device that makes an inquiry to a user and an input device that receives an answer from the user.   The interaction method selection means selects an interaction method including selection of an output device that makes an inquiry to a user and an input device that receives an answer from the user, and the home appliance is used as at least one of the output device and the input device. The home robot according to claim 3, wherein a device is selected.   The home robot according to claim 5, wherein the interaction method selection unit preferentially selects household electrical appliances that can be used for both the output device and the input device.   The home robot according to claim 3, wherein the conversation execution unit outputs the inquiry as a message to which an explanatory text indicating a response method for the user is added.   The home robot according to claim 3, further comprising means for ignoring an operation on the home appliance when the dialog execution unit performs an answer from a user according to the selected dialog method.

Images