JP2011143511A - User support system - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To appropriately support a user's operation of equipment. <P>SOLUTION: The user support system includes: the equipment operated by the user; a support device executing processing for supporting the operation of the equipment by the user; a user information acquisition means acquiring user information showing the position and attitude of the user; an equipment information acquisition means acquiring equipment information showing the position and attitude of the equipment; and an operating state information acquisition means acquiring operating state information showing the operating state of the equipment. The support device has a movable part moving so that at least one of the position and the attitude of the support device itself changes. In addition, the user support system is provided with a support device control means controlling the movement of the movable part so that support processing for the user can be executed by the support device based on the user information and the equipment information according to the change of the history of the operating state information, and controlling so that the support processing by the support device is executed. <P>COPYRIGHT: (C)2011,JPO&INPIT

Description

  The present invention relates to a user support system, and more particularly to a user support system that supports user operation of a device.

  In recent years, robots that support human behavior in various places such as homes and public facilities have appeared. For this reason, as compared with the support method using a medium such as a manual or a display screen, the support can be realized by a method that does not apply human cognitive load by utilizing the physicality of the robot.

  For example, Patent Document 1 discloses a technique in which a robot presents a user with an operation method of a station ticket vending machine. Specifically, the robot autonomously presents an operation method according to the user's request autonomously in accordance with the contents distributed from the content server. When it becomes difficult to interact with the user autonomously, an operator who can operate the robot from a remote location connects to the robot, and the operator presents an operation method to the user via the robot. Depending on the situation, the robot performs a device operation proxy for the user.

  Further, Patent Document 2 discloses that a robot performs an explanation for a device operated by a user. Specifically, when it is confirmed that no operation is performed on the operation panel operated by the user, the user determines that the user is in trouble, and the robot Explanation operation including conversation is performed to the user. Patent Document 2 also discloses that the robot moves to the position of the user by the self-supporting moving means.

JP 2007-11880 A JP 2008-142876 A

  However, the technique of Patent Document 1 is configured to perform support only when a user requests support, and determines whether the user needs support and acts in accordance with the situation. Absent. For this reason, the problem that it cannot support appropriately with respect to a user arises.

  Further, in the technique of Patent Document 2, “when there is no operation from the user” is cited as the timing of support for the user. However, when the user continues to perform the operation, the support is not performed and the user is appropriately selected. The problem of being unable to support arises. Furthermore, although it is disclosed that the robot moves to the position of the user, it may be annoying to the user because it moves according to the position of every user, and the user still cannot be supported appropriately. The problem arises.

  For this reason, an object of the present invention is to solve the above-mentioned problem that “the user cannot appropriately support such an operation when operating the device”.

In order to achieve such an object, a user support system according to one aspect of the present invention provides:
A device operated by a user, a support device that executes processing for supporting the operation of the device by the user, a user information acquisition unit that acquires user information representing at least one of the position and orientation of the user, and the device Device information acquisition means for acquiring device information representing at least one of the position and orientation of the device, and operation state information acquisition means for acquiring operation state information representing the operation state of the device from the device.
And the said assistance apparatus has a movable part movable so that at least one changes among the position and attitude | position of the said assistance apparatus itself.

  Furthermore, the user support system uses the user information acquired by the user information acquisition unit and the device information acquisition unit according to a change in the history of the operation state information acquired by the operation state information acquisition unit. Support for controlling the operation of the movable part so that the support process for the user by the support device can be executed based on at least one of the acquired device information, and for performing the support process by the support device It is configured to include device control means.

Moreover, the user assistance control apparatus which is the other form of this invention,
A device operated by a user, a support device that executes processing for supporting the operation of the device by the user, a user information acquisition unit that acquires user information representing at least one of the position and orientation of the user, and the device User information system comprising: device information acquisition means for acquiring device information representing at least one of the position and orientation of the device; and operating state information acquisition means for acquiring operation state information representing the operation state of the device from the device Is a device that controls the operation of the support device.

  Then, the user support control device sends the user information acquired by the user information acquisition unit and the device information acquisition unit according to a change in the history of the operation state information acquired by the operation state information acquisition unit. At least one of the position and posture of the support device itself mounted on the support device changes so that the support process for the user by the support device can be executed based on at least one of the device information acquired In addition to controlling the operation of the movable part movable so as to perform the control, control is performed to execute the support process by the support device.

Moreover, the program which is the other form of this invention is:
A device operated by a user, a support device that executes processing for supporting the operation of the device by the user, a user information acquisition unit that acquires user information representing at least one of the position and orientation of the user, and the device User information system comprising: device information acquisition means for acquiring device information representing at least one of the position and orientation of the device; and operating state information acquisition means for acquiring operation state information representing the operation state of the device from the device A user support control device that controls the operation of the support device.
At least of the user information acquired by the user information acquisition unit and the device information acquired by the device information acquisition unit according to a change in the history of the operation state information acquired by the operation state information acquisition unit. The operation of the movable part that is movable so that at least one of the position and posture of the support device itself mounted on the support device changes so that the support process for the user by the support device can be executed based on one side. It is a program for executing a process of controlling to execute a support process by the support apparatus.

Moreover, the user assistance control method which is the other form of this invention is the following.
In a user support system comprising a device operated by a user and a support device that executes processing for supporting the operation of the device by the user,
User information representing at least one of the position and posture of the user, device information representing at least one of the position and posture of the device, and operating state information representing the operating state of the device from the device are acquired. ,
In response to a change in the history of the acquired operating state information, the support device can execute support processing for the user based on at least one of the acquired user information and the acquired device information. Controlling the operation of the movable part movable so that at least one of the position and posture of the support device itself mounted on the support device changes, and controlling to execute the support process by the support device;
The configuration is as follows.

  With the configuration as described above, the present invention can appropriately support the operation of the device by the user, and can improve the convenience of the user.

1 is a configuration diagram of a robot system in Embodiment 1. FIG. It is a block diagram for demonstrating the structure of the robot system in Embodiment 1. FIG. It is a block diagram for demonstrating the minimum structure of the robot system in Embodiment 1. FIG. It is a figure which shows the example of the log | history data format stored in the status log | history collection part in Embodiment 1. FIG. 3 is a flowchart illustrating a method for repeatedly determining the internal state of the television system according to the first embodiment. 6 is a diagram for explaining a method of determining a movement target point of a robot when there is one user in Embodiment 1. FIG. 6 is a diagram for explaining a method of determining a movement target point of a robot when there are a plurality of users in Embodiment 1. FIG. It is a figure for demonstrating the determination method of the movement target point which makes it easy to teach a user how to operate an apparatus in Embodiment 1. 3 is a flowchart illustrating an operation of the television system in the first embodiment. 6 is a flowchart illustrating an operation of a user attitude transmission unit according to the first embodiment. 3 is a flowchart showing the operation of the robot in the first embodiment. It is a block diagram which shows the structure of the user assistance system in Embodiment 2.

<Embodiment 1>
A first embodiment of the present invention will be described with reference to FIGS. 1 to 3 are diagrams for explaining the configuration of the robot system, and FIGS. 4 to 11 are diagrams for explaining the operation thereof.

[Constitution]
In the robot system according to the present embodiment, the robot 3 (support device) presents the operation method of the television system 2 to the user 1 when the user 1 operates the television system 2 (device). It is a user support system which performs processing which supports operation. The operation method is presented to the user by, for example, autonomous operation by the robot 3 or remote operation of the robot 3 by the operator 8, as will be described later.

  Here, the television system 2 may be one using IP communication, one using broadcast waves, or any television. In addition, the device to be operated by the user 1 is not limited to the television system 2 and may be any device such as an electrical product or an ATM (Automated Teller Machine) installed in a bank.

  As shown in FIG. 1, the robot system according to the present embodiment is divided into a user side that operates the device and a support side that supports the user. Specifically, a television system 2, a robot 3, a home server 4, and a user attitude transmitter 5 are installed on the user side. On the support side, a robot operation terminal 6 and a content server 7 are installed.

  The home server 4 installed on the user side is connected to the television system 2, the robot 3, and the user attitude transmitter 5 through a communication line such as a LAN (Local Area Network). The robot 3, the robot operation terminal 6, and the content server 7 are connected via a public line 9 such as the Internet.

  Next, each structure mentioned above is demonstrated with reference to the block diagram of FIG. First, the television system 2 that is a device operated by the user 1 includes a user operation input unit 21, a device posture acquisition unit 22, and a device state transmission unit 23.

  The user operation input unit 21 is an input means for the user 1 to operate the television system 2, and is configured by an input interface such as a remote controller of the television system 2 and switches and buttons installed in the main body. . However, the user operation input unit 21 may be realized by a remote operation means for performing an operation from an outside place via a mobile phone or the Web. In addition, the input means with respect to apparatuses, such as the television system 2, may be equipped with not only one but the several by different methods.

  Moreover, the said apparatus attitude | position acquisition part 22 (apparatus information acquisition means) is a means to acquire the information (apparatus information) regarding the position and attitude | position of the television system 2 (apparatus) in a room | chamber interior. The posture includes the orientation of the display screen of the television system 2 and the like.

  And the apparatus attitude | position acquisition part 22 reads the some RFID tag information which is attached to the television system 2 using the RFID tag information acquisition device, for example, or is arrange | positioned indoors and which contains the positional information. This can be realized by a configuration for estimating the posture. Moreover, the apparatus attitude | position acquisition part 22 may be the structure which estimates an attitude | position by observing the landmark for attitude | position identification arrange | positioned in the television system 2 or room | chamber interior using a camera, for example. In addition, the device posture acquisition unit 22 has a configuration in which an acceleration sensor is attached in the television system 2 and the posture is estimated using a detection value from the acceleration sensor, or a user inputs posture information of the television system 2 in advance. You may implement | achieve in any structures, such as memorize | storing. In addition, the apparatus attitude | position acquisition part 22 may acquire only any one information among the position and attitude | positions of the television system 2, and may acquire both information.

  In addition, the device state transmission unit 23 acquires internal state information (operation state information) representing the internal state of the television system 2, and device information representing the position and posture of the television system 2 from the device posture acquisition unit 22. It is a means for obtaining and transmitting to the home server 4 using a communication line. Here, the internal state information of the television system 2 is information representing the operating state of the television system 2, such as information specifying a screen displayed on the television system 2, information representing settings of the television system 2, and the like. In particular, the information represents the operation content input by the user operation input unit 21. However, the information indicating the internal state of the television system 2 may be other information.

  And the said apparatus state transmission part 23 is the case where the input from the user operation input part 21 is performed, for example, when the screen of the television system 2 changes by a user's operation, when the setting of the television system 2 is changed. When the attitude of the television system 2 changes, the above-mentioned internal state information and device information representing the position and attitude of the television system are transmitted to the home server 4 at such timing. However, the transmission timing may be any timing.

  Next, the user attitude transmitter 5 will be described. The user attitude transmitter 5 includes a user attitude acquisition unit 51 and a user attitude transmission unit 52. The user posture acquisition unit 51 is means for acquiring information (user information) representing the position and posture of the user 1 in the room. Note that the posture includes a direction representing the front of the user 1.

  Then, the user posture acquisition unit 51 reads a plurality of RFID tag information including position information attached to the user 1 itself or placed indoors by using an RFID tag information acquisition device, for example. It is realized by a configuration for estimating the position and posture, a configuration for estimating the posture of a person using a pressure sensor on the floor in the room, and the like. However, the user posture acquisition unit 51 may be realized by another configuration for acquiring user posture information. In addition, the user attitude | position acquisition part 51 may acquire only any one information among the position and attitude | positions of the user 1, and may acquire both information.

  The user attitude transmission unit 52 is means for transmitting user information representing the position and attitude of the user 1 acquired by the user attitude acquisition unit 51 to the home server 4 using a communication line. Here, the transmission of information by the user posture transmission unit 52 may be transmitted when the position or posture of the user has changed, or when a certain time has passed since the last transmission of information. Note that the user posture transmission unit 52 may identify a user based on the user information acquired from the user posture acquisition unit 51 and transmit a user ID or the like for identifying the user to the home server 4.

  Next, the home server 4 will be described. The home server 4 includes a state history recording unit 41. The state history recording unit 41 includes the internal state of the television system 2 transmitted from the device posture acquisition unit 23 and the device information indicating the position and posture, the user position transmitted from the user posture transmission unit 52, It is means for recording user information representing the posture. For example, it is realized by a NAS (Network Attached Storage) composed of a storage device (ROM, RAM, HDD, etc.) and a LAN connection device. In addition, the home server 4 can be realized by a computer connected to a LAN including a central processing unit (CPU) and a storage device.

  Then, in the home server 4, the transmitted information is stored in a spreadsheet format as shown in FIG. The information storage format may be one that can be used in a database, one that can be used as a general text log file, or may be stored in a format suitable for this system. Furthermore, the transmitted information may be aggregated by adding the number of users from the number of user posture information and recorded, and information suitable for this system may be added and recorded.

  Next, the configuration of the robot 3 will be described. First, the robot 3 in this embodiment is a humanoid robot having at least one of a face part, a body part, a foot part, and a hand part. The face portion, the torso portion, the foot portion, the hand portion, and the like are movable portions whose positions and postures are movable, and are configured to change the position and posture of the robot. In particular, in this embodiment, the body part and the foot part are equipped with a body movable means (device body movable means) such as a traveling means for changing the position and posture of the robot body.

  In addition, although not shown, the robot 3 is equipped with an input device such as a touch panel for inputting a question about operation by the user 1 and a microphone for inputting a question by voice, and assists an operation such as answering the question. It is equipped with an output device such as a display that outputs information.

  As shown in FIG. 2, the robot 3 includes a state history acquisition unit 301, an operation determination unit 302, an operator operation request unit 303, a content request unit 304, a robot control unit 305, and a voice recognition unit 306. , A speech unit 307, a movement mechanism unit 308, a display input unit 309, and an instruction mechanism unit 310. These units are realized by incorporating a program into an arithmetic device such as a CPU provided in the robot 3. Hereafter, each said part is explained in full detail.

  The state history acquisition unit 301 is device information representing the internal state, position, and orientation of the television system 2 recorded in the state history recording unit 41 of the home server 4 via a communication line connected to the home server 4. And means for acquiring user information representing the position and orientation of the user. Then, the state history acquisition unit 301 uses the information recorded in the state history recording unit 41 when a certain time has elapsed since the last acquisition of information, or when the user interacts with the robot by voice. You may acquire it at times. In addition, the information to be acquired is recorded in the state history recording unit 41 such as information within a certain time from the current time, user information associated with the user ID of the user who is interacting, and information only on the internal state of the television system 2. It may be limited to a part of the information. In addition, information on timing and contents suitable for this system may be acquired.

  In addition, the operation determination unit 302 uses the internal state and device information of the television system 2 acquired by the state history acquisition unit 301 and the user information, and the robot executed by the robot and the robot necessary for executing the content. 3 is a means for determining the operation of No. 3. The operation determination unit 302 transmits the determined content to the operator operation request unit 303, the content request unit 304, and the robot control unit 305.

  Here, the content is information representing processing details for supporting actions related to user operations, for example, processing details for presenting the operation method of the television system to the user, or processing for operating the television system on behalf of the user. The contents are listed. Further, the operation necessary for executing the content is an operation performed for appropriately executing the content for the user 1. For example, when executing the content presenting the operation method of the television system 2, The robot 3 is moved to a position where the user can easily see the situation, or the robot 3 is moved to a position where the user can easily operate the television system 2 when executing the content for the robot 3 to operate the television system 2 on behalf of the user 1. Moving.

Specifically, the operation determination unit 302 performs the following processes (a), (b), and (c).
(A) When the user 1 is operating the device, it is determined whether or not the device 1 is confused. If the user 1 is confused, the content waiting for the question from the user 1 is determined and necessary for executing the content. As an operation, a movement target point for the robot 3 to enter the field of view of the user 1 is determined so that the user 1 can easily ask the robot 3 how to use.

(B) When the user 1 asks the robot 3 about how to use, the robot 3 determines the content to be used according to the situation of the user 1, and asks the user 1 as an operation necessary for executing the content. Decide the movement target point that makes it easy to teach how to operate.

(C) When the robot 3 is executing content that teaches the user how to operate the device, it is determined whether the position or number of users 1 has changed. As a necessary operation for executing the above, a movement target point that makes it easy to teach the user 1 how to operate the device is determined.

  In the following, detailed processing contents will be described for (a) to (c). First, in the process (a), it is determined whether or not the user 1 is at a loss in device operation. For this reason, the robot 3 acquires the history of the internal state information accumulated within the predetermined time at regular intervals via the state history acquisition unit 301. Then, when either the repetition of the internal state of the television system 2 or a long no-input period is detected from the acquired history, it is determined that the user 1 is at a loss in device operation.

  Here, an example of repetition of the internal state will be described. First, it is assumed that the screen states (operating states) of the television system 2 by the operation of the user 1 are A, B, and C. Now, it is assumed that the screen state is changed to “A, B, C, B, C, A, B, C” by the operation of the user 1. In this history, it can be seen that a series of states in which the screen state changes to “B, C” is repeated three times. Another example will be described. Here, it is assumed that the screen states (operating states) of the television system 2 by the operation of the user 1 are A, B, C, D, and E. It is assumed that the screen state is changed to “A, B, C, D, A, B, C, D, E, A, B, C” by the operation of the user 1. In this history, it can be seen that a series of states in which the screen state changes to “A, B, C” is repeated three times. In this way, when there is a series of repeated operation states, the user 1 can determine that he / she is lost in operation without proceeding to a further screen.

  Specifically, a method for repeatedly determining the internal state of the television system 2 will be described with reference to FIG. The history of the screen state is “R01, R02,..., R0n” in time order.

  First, in the variables a, i, k, m for operation, a = 0, i = 1, k = 2, and m = 1 are set (step S1). Next, it is confirmed whether or not “Rak” exists (step S2). If not, the process proceeds to step S9. If it exists, it is subsequently checked whether Rai and Rak are equal (step S3). If equal, R (a + 1) m = Rai is set (step S4), 1 is added to each of m, i, and k (step S5), and the process returns to step S2. If they are not equal, first, 1 is added to i (step S6), and it is checked whether i = k (step S7). When i = k, k is incremented by 1 to set i = 1 (step S8). Thereafter, the process returns to step S2. If i = k does not hold, the process directly returns to step S2.

  In step S9, it is confirmed whether m = 1. When m = 1, the operation is completed, and an operation sequence in which the value of a + 1 is the number of repetitions and Rax (x is a natural number) is repeated. At this time, if the number of repetitions is 2 or more, it is determined that there is repetition, and if not, it is determined that there is no repetition (step S10). If m ≠ 1, n = m−1, 1 is added to a (step S11), i = 1, k = 2, m = 1 (step S12), and the process returns to step S2.

  Furthermore, a specific determination example by the above-described method for repeatedly determining the internal state of the television system 2 will be described. Here, when the screen state (operation state) of the television system 2 by the operation of the user 1 is A, B, C, the screen state is “A, B, C, B, C, A, B, C”. Suppose that it has changed.

  First, in the variables a, i, k, m for operation, a = 0, i = 1, k = 2, and m = 1 are set (step S1). Then, R01 = A, R02 = B, R03 = C, R04 = B, R05 = C, R06 = A, R07 = B, R08 = C. Next, it is confirmed whether or not “Rak” exists (step S2). Here, since Rak = R02 = B, Rak exists. Accordingly, the process proceeds to step S3.

  Subsequently, it is confirmed whether Rai and Rak are equal (step S3). Here, since Rai = R01 = A and Rai ≠ Rak, the process proceeds to step S6. Then, 1 is added to i (step S6), and it is checked whether i = k (step S7). Here, i = 1 + 1 = 2 and k = 2, so i = k. Accordingly, the process proceeds to step S8. Subsequently, 1 is added to k so that i = 1 (step S8). Thereafter, the process returns to step S2. Here, k = 2 + 1 = 3, i = 1, and the process returns to step S2.

  Subsequently, it is confirmed whether or not “Rak” exists (step S2). Now, since a = 0, i = 1, k = 3, m = 1, and Rak = R03 = C, Rak exists. Accordingly, the process proceeds to step S3.

  Subsequently, it is confirmed whether Rai and Rak are equal (step S3). Here, since Rai = R01 = A and Rai ≠ Rak, the process proceeds to step S6. Then, 1 is added to i (step S6), and it is confirmed whether i = k is satisfied (step S7). Here, i = 1 + 1 = 2, and now k = 3, so i ≠ k. Therefore, the process returns to step S2.

  Subsequently, it is confirmed whether or not “Rak” exists (step S2). Now, since a = 0, i = 2, k = 3, m = 1, and Rak = R03 = C, Rak exists. Accordingly, the process proceeds to step S3.

  Subsequently, it is confirmed whether Rai and Rak are equal (step S3). Here, since Rai = R02 = B and Rai ≠ Rak, the process proceeds to step S6. Then, 1 is added to i (step S6), and it is confirmed whether i = k is satisfied (step S7). Here, i = 2 + 1 = 3, and now k = 3, so i = k. Accordingly, the process proceeds to step S8. 1 is added to k so that i = 1 (step S8). Thereafter, the process returns to step S2. Here, k = 3 + 1 = 4, i = 1, and the process returns to step S2.

  Subsequently, it is confirmed whether or not “Rak” exists (step S2). Now, since a = 0, i = 1, k = 4, m = 1, and Rak = R04 = B, Rak exists. Accordingly, the process proceeds to step S3.

  Subsequently, it is confirmed whether Rai and Rak are equal (step S3). Here, since Rai = R01 = A and Rai ≠ Rak, the process proceeds to step S6. Then, 1 is added to i (step S6), and it is confirmed whether i = k is satisfied (step S7). Here, i = 1 + 1 = 2, and since k = 4 now, i ≠ k. Therefore, the process returns to step S2.

  Subsequently, it is confirmed whether or not “Rak” exists (step S2). Now, since a = 0, i = 2, k = 4, m = 1, and Rak = R04 = B, Rak exists. Accordingly, the process proceeds to step S3.

  Subsequently, it is confirmed whether Rai and Rak are equal (step S3). Here, since Rai = R02 = B and Rai = Rak, the process proceeds to step S4. Then, R (a + 1) m = Rai is set (step S4), 1 is added to each of m, i, and k (step S5), and the process returns to step S2. Here, R11 = B from R (a + 1) m = R11 and Rai = R02 = B. Further, i = 2 + 1 = 3, k = 4 + 1 = 5, m = 1 + 1 = 2, and the process returns to step S2.

  Subsequently, it is confirmed whether or not “Rak” exists (step S2). Now, since a = 0, i = 3, k = 5, m = 2, and Rak = R05 = C, Rak exists. Accordingly, the process proceeds to step S3.

  Subsequently, it is confirmed whether Rai and Rak are equal (step S3). Here, since Rai = R03 = C and Rai = Rak, the process proceeds to step S4. Then, R (a + 1) m = Rai is set (step S4), 1 is added to each of m, i, and k (step S5), and the process returns to step S2. Here, R (a + 1) m = R12 and Rai = R03 = C, so that R12 = C. Further, i = 3 + 1 = 4, k = 5 + 1 = 6, m = 2 + 1 = 3, and the process returns to step S2.

  Subsequently, it is confirmed whether or not “Rak” exists (step S2). Now, since a = 0, i = 4, k = 6, m = 3 and Rak = R06 = A, Rak exists. Accordingly, the process proceeds to step S3.

  Subsequently, it is confirmed whether Rai and Rak are equal (step S3). Here, since Rai = R04 = B and Rai ≠ Rak, the process proceeds to step S6. Then, 1 is added to i (step S6), and it is confirmed whether i = k is satisfied (step S7). Here, i = 4 + 1 = 5, and now k = 6, so i ≠ k. Therefore, the process returns to step S2.

  Subsequently, it is confirmed whether or not “Rak” exists (step S2). Now, since a = 0, i = 5, k = 6, m = 3 and Rak = R06 = A, Rak exists. Accordingly, the process proceeds to step S3.

  Subsequently, it is confirmed whether Rai and Rak are equal (step S3). Here, since Rai = R05 = C and Rai ≠ Rak, the process proceeds to step S6. Then, 1 is added to i (step S6), and it is confirmed whether i = k is satisfied (step S7). Here, i = 5 + 1 = 6, and now k = 6, so i = k. Accordingly, the process proceeds to step S8. Then, 1 is added to k so that i = 1 (step S8). Thereafter, the process returns to step S2. Here, k = 6 + 1 = 7, i = 1, and the process returns to step S2.

  Subsequently, it is confirmed whether or not “Rak” exists (step S2). Now, since a = 0, i = 1, k = 7, m = 3 and Rak = R07 = B, Rak exists. Accordingly, the process proceeds to step S3.

  Subsequently, it is confirmed whether Rai and Rak are equal (step S3). Here, since Rai = R01 = A and Rai ≠ Rak, the process proceeds to step S6. Then, 1 is added to i (step S6), and it is confirmed whether i = k is satisfied (step S7). Here, i = 1 + 1 = 2, and now k = 7, so i ≠ k. Therefore, the process returns to step S2.

  Subsequently, it is confirmed whether or not “Rak” exists (step S2). Now, since a = 0, i = 2, k = 7, m = 3 and Rak = R07 = B, Rak exists. Accordingly, the process proceeds to step S3.

  Subsequently, it is confirmed whether Rai and Rak are equal (step S3). Here, since Rai = R02 = B and Rai = Rak, the process proceeds to step S4. Then, R (a + 1) m = Rai is set (step S4), 1 is added to each of m, i, and k (step S5), and the process returns to step S2. Here, R13 = B from R (a + 1) m = R13 and Rai = R02 = B. Further, i = 2 + 1 = 3, k = 7 + 1 = 8, m = 3 + 1 = 4, and the process returns to step S2.

  Subsequently, it is confirmed whether or not “Rak” exists (step S2). Now, since a = 0, i = 3, k = 8, m = 4 and Rak = R08 = C, Rak exists. Accordingly, the process proceeds to step S3.

  Subsequently, it is confirmed whether Rai and Rak are equal (step S3). Here, since Rai = R03 = C and Rai = Rak, the process proceeds to step S4. Then, R (a + 1) m = Rai is set (step S4), 1 is added to each of m, i, and k (step S5), and the process returns to step S2. Here, R (a + 1) m = R14 and Rai = R03 = C, so that R14 = C. Further, i = 3 + 1 = 4, k = 8 + 1 = 9, m = 4 + 1 = 5, and the process returns to step S2.

  Subsequently, it is confirmed whether or not “Rak” exists (step S2). Now, since a = 0, i = 4, k = 9, m = 5 and Rak = R09, there is no Rak. Accordingly, the process proceeds to step S9.

  In step S9, it is confirmed whether m = 1. Here, since m = 5 ≠ 1, the process proceeds to step S11. Then, n = m−1, 1 is added to a (step S11), i = 1, k = 2, m = 1 (step S12), and the process returns to step S2. Here, n = 5-1 = 4, a = 0 + 1 = 1, i = 1, k = 2, and m = 1, and the process returns to step S2.

  Up to this point, R11 = B, R12 = C, R13 = B, and R14 = C. That is, the first operation sequence “A, B, C, B, C, A, B, C” is reduced to “B, C, B, C” configured by repeated operation sequences; Become. Next, the contracted operation sequence is further contracted. Since the procedure is the same as above, description thereof is omitted. Finally, “B, C, B, C” is reduced to “B, C”.

  Finally, the operation when the contraction to “B, C” is attempted will be described. As described above, R21 = B, R22 = C, m = 1, and a = 2 before reaching step S9. . In step S9, it is confirmed whether m = 1. Here, since m = 1 = 1, the process proceeds to step S10.

  When m = 1, the operation ends, and an operation sequence in which the value of a + 1 is repeated and Rax (x is a natural number) is repeated. At this time, if the number of repetitions is 2 or more, it is determined that there is repetition, and if not, it is determined that there is no repetition (step S10). Here, the repeated operation sequence is R31 = B, R32 = C, that is, “B, C”, and the number of repetitions is a + 1 = 2 + 1 = 3, which is three times. That is, it is obtained that a series of operations “B, C” is repeated three times. Since the number of repetitions is 2 or more, it is determined that there is repetition.

  Next, a method of detecting “a long non-input period” in process (a) will be described. The history of user input is R1, R2,..., Rn in time order, and the time when the history Rn is recorded is T (Rn). The current time is Tn. When the state history is acquired, when the following equation 1 is satisfied, it is determined that a long no-input period is detected, and when the equation 1 is not satisfied, it is determined that no input period is not detected. When a long non-input period is detected, it is determined that the user 1 is lost in operation.

  Note that “α” in the above equation 1 is a parameter for adjusting the unsatisfactory determination of the no-input period, and may be adjusted in advance by the system designer so as to be suitable for the system. The parameters may be automatically changed according to features such as the above and the state of the system.

  After determining that the user 1 is at a loss in the device operation by the process (a) described above, the robot 3 determines the content waiting for the question from the user 1 so that the user can easily ask the robot how to use it. The movement target point at which the robot 3 moves so as to enter the field of view of the user 1 is determined.

  As described above, in order to determine the content, for example, a label is assigned to the content in advance, and it is noted that the content is waiting on the label. When determining the content, the content to be waited for may be determined with reference to the label. In addition to this, a suitable method may be used in determining the content to wait.

  In order to determine a movement target point, first, user information representing the latest position and orientation (orientation) of the user is acquired from the history collected by the state history acquisition unit 301. Based on the user position and orientation information, the visual field position of the user 1 is specified. Here, the field of view of the user 1 will be described with reference to FIG. As shown in FIG. 6, if the user U is facing the direction of the vector V0, the robot U has a fan shape with a constant left and right angle centered on the vector V0, and the robot S of the boundary Sf1 is 3m to 5m away from the user. The point closest to the current position is determined as the movement target point of the robot. The fan-shaped angle may be set at the time of design. For example, it may be 55 degrees left and right, which is a viewing angle at which a person can recognize the presence of an object. If the region boundary Sf1 is not on the robot traveling surface, the region boundary Sf1 may be projected onto the robot traveling surface, and a point on the boundary may be determined as the robot movement target point.

  In addition, from the acquired user information, when there are two or more users U as shown in FIG. 7, the nearest point from the current position of the robot among the boundary Sf <b> 2 of the region in the product of each region (AND operation) is It is determined as the movement target point of the robot. However, if the product of each area (AND operation) is an empty area, the number of users who can take the area product is reduced so that the product of the area of as many users as possible does not become an empty area. Determine the target point. Then, information on the movement target point of the robot determined by the motion determination unit 302 is transmitted to the robot control unit 305.

  Next, the contents of the process (b) will be described. In the process (b), the content that first shows how to use according to the user's situation depends on the content that the user 1 asks the robot 3, the operation history of the user 1, and the internal state of the television system 2. decide. The content is classified into two types: autonomous content in which the robot autonomously presents the operation method to the user, and manual content in which the operation method is presented to the user by the operator operating the robot. The autonomous content teaches the operation method of the television system 2, and is composed of, for example, content that presents an operation method related to the internal state of the plurality of television systems 2.

  Here, the content determination method will be specifically described. First, it is confirmed whether or not the content that the user 1 has asked the robot 3 can answer depending on the content. This process is confirmed by using the contents of the user 1 asking the robot 3 a question. For example, at the time of system design, information regarding a user's assumed question content and one or more contents that can be an answer to the question is held. This can be realized by recognizing the contents of the question input from the user 1 to the robot 3 and examining whether one or more contents associated with the question can be extracted. However, other methods may be used.

  If the content that the user 1 has asked the robot 3 can answer depending on the content, the candidate content is determined as autonomous content, and if it cannot be answered, it is determined as manual content. When manual content is determined, a command to perform an operation request by the operator is transmitted to the operator operation request unit 61.

  When autonomous content is determined, which content is to be executed next is selected. In the selection of content, a user operation history is used. First, an operation sequence repeated by the user is obtained by determining whether the internal state of the television system 2 described in (a) is repeated. Next, all contents that include this operation sequence are selected. If no operation sequence repeated by the user is included in any content, the last operation is deleted from the operation sequence and an attempt is made to select content that includes this operation sequence again. . After selecting the content, it is next determined which part of the content to start execution. The internal state of the television system 2 is used as to which part of the content starts execution. The latest internal state, that is, the current state of the television system 2 is acquired from the history of the internal state of the television system 2, and the process starts from a portion where an operation method related to the state is presented.

  When there are a plurality of contents extracted by the above procedure and a plurality of content start positions, a plurality of contents and content start positions may be presented as candidates on the display input unit 309 so that the user can select them. In addition, a method suitable for this embodiment may be used. When the content and the start position of the content are determined, the information is transmitted to the content request unit 304.

  After the content is determined, a movement target point that makes it easy to teach the user how to operate the device is determined. First, it is confirmed whether or not the content includes an operation for requesting the user to operate the television system 2. If included, the latest position and orientation of the television system 2 and the position and orientation information of the user are acquired from the history collected by the state history acquisition unit 301, and the movement target point of the robot is determined from these information. .

  Here, a method of determining the movement target point will be described with reference to FIG. First, a vector V2 indicating the position and orientation of the television system 2 is projected onto the robot traveling surface to generate a vector V3. Next, a fan-shaped region having a constant left and right angle around the vector V3 is generated. Similarly, a vector V1 is generated by projecting a vector indicating the position and orientation of the user onto the robot travel surface (projection is not required in FIG. 8), and then a fan-shaped region having a constant left and right angle centered on the vector V3 is formed. Generate. In both fan-shaped areas, the boundary Sf3 of the common area is obtained, and the point on the boundary Sf3 closest to the current position of the robot is determined as the movement target point of the robot.

  If there are two or more users U, the closest point from the current position of the robot to the boundary of the common area between the product of the areas for all users (AND operation) and the area by the vector V3 is Determined as a moving target point. However, if the product of the areas for all users (AND operation) is an empty area, reduce the number of users to be the product of the areas so that the product of the areas of as many users as possible will not become empty areas, Determine the movement target point. Information on the determined movement target point of the robot is transmitted to the robot control unit 305.

  Next, the contents of the process (c) will be described. In the process (c), first, in order to determine whether or not the position and the number of users have changed, from the history collected by the state history acquisition unit 301, the latest information on the user's position and the number of persons, the previous information, Check if there is any difference. If there is a difference, it is determined that there is a change in the position and the number of users, and if there is no difference, it is determined that there is no change in the position and the number of users. If there is a change, the robot movement target point is determined next so that it can be easily explained to as many users as possible.

  Specifically, first, it is confirmed whether or not the content of the content being executed includes an operation for requesting the user to operate the television system 2. If not included, the information on the position and orientation of the user is acquired from the history information in the same manner as described in the process (a), and for example, as shown in FIG. Of the boundary Sf2 of the area in the AND operation), a point closest to the current position of the robot is determined as a movement target point of the robot.

  When the operation of requesting the user to operate the television system 2 is included, the position and orientation of the television system 2 and the position of the user are determined from the history information in the same manner as described in the process (b). Get information about and orientation. Then, as shown in FIG. 8, a vector V2 indicating the position and orientation of the television system 2 is projected onto the robot traveling surface to generate a vector V3. Next, a fan-shaped region having a constant left and right angle around the vector V3 is generated. Similarly, a vector V1 is generated by projecting a vector indicating the position and orientation of the user onto the robot traveling surface, and then a fan-shaped region having a constant left and right angle centered on the vector V3 is generated. In both fan-shaped areas, the boundary Sf3 of the common area is obtained, and the point on the boundary Sf3 closest to the current position of the robot is determined as the movement target point of the robot. Information on the determined movement target point of the robot is transmitted to the robot control unit 305.

  Further, when the operator operation requesting unit 303 of the robot 3 receives the command for performing the operation by the operator transmitted from the operation determining unit 302, the operator operation requesting unit 303 sends the operator operation requesting unit 303 to the robot operation terminal 6 that is located away from the robot 3. Is a means for requesting input by. At this time, information regarding the user's operation history may be transmitted to the robot operation terminal 6 at the same time so that the operator can understand what kind of support the user needs.

  The content request unit 304 is a unit that acquires the content recorded in the content recording unit 71 using the content determined by the operation determination unit 302 and the content start position information. The content request unit 304 accesses the content recording unit 71 and searches for content that matches the content determined by the operation determination unit 302 from the content stored therein. If a match can be found, the content start position information determined by the motion determination unit 302 is added to the content, and is acquired by the robot control unit 305 of the robot 3.

  Further, the robot control unit 305 operates the robot 3 based on the information about the movement target point of the robot determined by the operation determination unit 302 and the content for which the start point transmitted from the content recording unit 71 is determined. The control of each part is performed. The movement of the robot is executed in the foreground, while the content operation is executed in the background. That is, they are executed in different threads.

  Specifically, when information on the movement target point of the robot 3 is transmitted, the robot control unit 305 generates a route connecting the current location of the robot 3 and the movement target point, and the movement mechanism unit 308 is based on the route. Is activated. In addition, when content whose start point is determined is transmitted, each unit is controlled according to the content. For example, when it is necessary to utter a voice to the user, the voice recognition unit 306 transmits the voice to be uttered to the utterance unit 307 and when the user needs to acquire the utterance content. When there is a figure or character to be shown to the user, or when an instruction is received from the user, the display input unit 309 causes the display input unit 309 to present or input the user, the television system 2 or the like When it is necessary to point to the object, processing related to the control of the robot such as performing an instruction operation by the instruction mechanism unit 310 is performed.

  As described above, the motion determination unit 302 and the robot control unit 305 equipped in the robot 3 can be operated by the robot 3 with respect to the user 1 according to a change in the history of the internal state of the television system 2. It moves to a position where it can be performed, and also functions as support device control means for controlling to execute support processing, such as executing content that accepts a question. In the above description, the case where the position of the robot 3 is controlled to move according to the change in the history of the internal state of the television system 2 has been exemplified. However, the operation of the face portion and the hand portion of the robot 3 described above is controlled. Then, the posture of the robot 3 may be changed. As a result, the robot 3 has a position and posture where the robot 3 can provide more appropriate operation support to the user 1.

  The voice recognition unit 306 is a unit that receives the user's voice and performs voice recognition processing under the control of the robot control unit 305. For example, a microphone installed in the robot and a program for recognizing voice information input from the microphone may be used.

  The utterance unit 307 is a unit that utters the user under the control of the robot control unit 305. For example, a synthetic utterance program that converts text data into speech data and a speaker that outputs speech may be used. Moreover, you may have the function to output the audio | voice signal transmitted from the operator operation input part 61 as an audio | voice.

  Further, the movement mechanism unit 308 is a unit that moves the robot 3 and moves the face unit, the hand unit, and the like under the control of the robot control unit 305. For example, the moving means may be constituted by a motor-equipped wheel, a caterpillar, a biped walking system, or the like that allows the amount of movement to be observed by odometry.

  The display input unit 309 is a unit that presents characters and drawings information to the user and receives input from the user under the control of the robot control unit 305. For example, you may comprise by the structure by a touchscreen display, the structure by a projector and a controller, etc.

  In addition, the instruction mechanism unit 310 is a unit that is controlled by the robot control unit 305 to indicate an object to be watched by the robot. For example, you may comprise by the laser pointer, the pointing stick, or the manipulator attached to the mechanical mechanism which can change a point position. Further, in the case of a laser pointer, a pointing pattern may be changed by using a matrix light source or a laser pointer having multiple color light sources in order to display a hand shape.

  Next, the robot operation terminal 61 will be described. The robot operation terminal 61 includes an operator operation input unit 61. The operator operation input unit 61 is a means for an operator to operate the robot remotely. For example, when moving the robot, the robot may be moved according to the input contents of the joypad, keyboard, or mouse. Further, when it is desired to speak to the user via the robot 3, voice input may be performed by a microphone. Furthermore, when it is desired to display a figure or character or to perform a pointing operation, the user may operate software that assists the input.

  Next, the content server 7 will be described. The content server 7 includes a content recording unit 71. The content recording unit 71 is a unit for recording content in which operations necessary for the robot 3 to present the operation method of the television system 2 to the user 1 are recorded. Is transmitted to the robot controller 305. The content server 7 can be realized by, for example, a NAS that includes a storage device (ROM, RAM, HDD, etc.) and a LAN connection device. However, in addition to this, a computer connected to a LAN composed of a central processing unit (CPU) and a storage device can also be realized, and a computer suitable for this system may be used.

  The robot system in the present embodiment has the above-described configuration, but FIG. 3 shows a simplified configuration of the robot system. In this figure, the history collection unit M1 shows the portion A1 in FIG. 2, and is configured to acquire user information, device information, and internal state information of the device. Further, the motion determination unit M2 indicates the parts 302 and 305 in FIG. 2, and controls changes in position and posture such as movement of the robot 3 according to changes in the history of the internal state information of the device. It has a configuration. Further, the robot M3 indicates a portion indicated by reference numeral 3 in FIG. In FIG. 2, the motion determination unit M2 is embedded in the robot 3, but it may be configured by a computer installed outside.

[Operation]
Next, the overall operation of the system will be described. First, the operation of the television system is shown in FIG. The television system 2 checks whether or not a user operation has been received from the user operation input unit 21 (step S21). When a user operation is accepted, the operation content is transmitted to the state history recording unit 41 (step S22).

  If the user operation has not been accepted, it is confirmed whether or not the attitude of the television system 2 acquired by the apparatus attitude acquisition unit 22 has changed (step S23). When the attitude of the television system 2 changes, attitude information is transmitted to the state history recording unit 41 (step S24). If the attitude of the television system 2 has not changed, after transmitting the operation content or attitude information to the state history recording unit 41, the process returns to step S21.

  Next, the operation of the user attitude transmission unit 52 is shown in FIG. First, it is confirmed whether or not the user posture acquired by the user posture acquisition unit 51 has changed (step S31). When the user posture changes, the user posture information is transmitted to the state history recording unit 41 by the user posture transmission unit 52 (step S32). When the posture of the user has not changed, or after transmitting the posture information of the user, the process returns to step S31.

  Next, the operation of the robot of the robot 3 is shown in FIG. First, the robot acquires history information from the state history recording unit 41 by the state history acquisition unit 301, and checks whether there is any change in the number or position of users (step S41). If there is no change in the number or position of the users, the process proceeds to step S44. If there is a change in the number or position of the users, it is checked whether or not the content being executed next includes a teaching operation of the operation method (step S42). When the operation content includes the teaching operation of the operation method, a position that is easy to explain to the user is determined as the movement target point (the process (c) of the operation determination unit 302 described above) and transmitted to the robot control unit 305. (Step S43), the process proceeds to Step S44.

  When the operation content teaching operation is not included in the content being executed, the operation determination unit 302 determines whether or not the user 1 is in trouble with the operation. The target point is determined (the process (a) of the operation determination unit 302 described above), transmitted to the robot control unit 305 (step S44), and the process proceeds to step S44.

  In step S45, it is confirmed whether there is a question from the user. When there is a question from the user 1, the operation determination unit 302 determines whether the content is autonomous content or manual content based on the content (step S46). If it is determined that the content is autonomous content, the content requesting unit 304 determines the content that can answer the question and the starting point of the content, acquires the content from the content recording unit 71, and transmits the content to the robot control unit 305 (step S47). When it is determined that the content is manual content, the operator operation request unit 303 requests the operator to operate the robot (step S48). The robot control unit 305 receives the operation details of the robot by the operator.

  Next, based on the information transmitted to the robot control unit 305, the robot is moved, the content is executed in the background, or the operation according to the robot operation by the user is performed (step 49). Thereafter, the process returns to step S41.

  With such a configuration, the robot senses the situation where the user 1 is confused about the device operation and enters the user's field of view, so that the user can easily ask the robot a question, and the user has an opportunity to support the device operation. Can be increased. Further, by using the operation history of the user's television system 2, the content to be taught to the user can be selected pinpointed, and the interaction with the user for identifying the content to be taught can be reduced. Furthermore, in order to be able to move to an appropriate position for presenting the operation method of the television system 2 to the user, there is no time loss due to the movement of the robot when the operation method is presented, and the operation method is presented smoothly. be able to.

<Embodiment 2>
Next, a second embodiment of the present invention will be described with reference to FIG. FIG. 12 is a block diagram showing an outline of the configuration of the user support system in the present embodiment.

As shown in FIG. 1, the user support system which is one form of this invention is
User information that obtains user information representing at least one of the position and posture of the user 500, the device 510 operated by the user 500, the support device 520 that executes processing for supporting the operation of the device 510 by the user 500, An acquisition unit 501, a device information acquisition unit 511 that acquires device information representing at least one of the position and orientation of the device 510, and operating state information that acquires operating state information indicating the operating state of the device from the device 510. Acquisition means 512.
The support device 520 has a movable portion 521 that can move so that at least one of the position and posture of the support device itself changes.

Further, the user support system includes the user information acquired by the user information acquisition unit 501 and the device information acquisition unit according to a change in the history of the operation state information acquired by the operation state information acquisition unit 512. Based on at least one of the device information acquired in 511, the operation of the movable unit 521 is controlled so that the support process for the user by the support apparatus 520 can be executed, and the support process by the support apparatus 520 is performed. Provided with support device control means 530 for controlling to execute,
The structure is taken.

  And the said assistance apparatus control means takes the structure that it operate | moves when the log | history of the said operation state information has repeated the series of operation states which consist of a several operation state. Moreover, the said operation state information takes the structure that it is the information showing the operation state of the said apparatus determined by the said user's operation.

In the user support system,
The movable part of the support device has a device main body moving means for changing at least one of the position and posture of the support device main body,
The support apparatus control means controls the operation of the apparatus main body moving means to change at least one of the position and posture of the support apparatus main body;
The configuration is as follows.

  And the said assistance apparatus control means takes the structure of changing at least one among the position and attitude | position of the said assistance apparatus so that the said assistance apparatus may enter into the said user's visual field based on the said user information.

Furthermore, in the user support system,
The support device control means changes at least one of a position and a posture of the support device so that the support device can execute support processing on the device based on the device information.
The configuration is as follows.

In the user support system,
The support device control means controls the operation of the movable unit based on the user information acquired by the user information acquisition means while the support device is executing support processing.
The configuration is as follows.

  As described above, by configuring the user support system, the user support system first includes information indicating the position and posture of the user, the position and posture of the device operated by the user, and the operating state of the device. To get. Then, in response to a change in the operating state history of the device, for example, when the operating state history is a state in which the same operating state is repeated, the support device can execute the support process for the user. While controlling the movable part of an apparatus, a support process is performed. For example, the support device moves so as to be in a position and posture that can easily assist the user, such as moving so that the support device enters the user's field of view. Thereby, according to the actual operation situation by the user, a support process can be executed automatically and appropriately by a support device for a user who needs support. As a result, user convenience can be improved.

  The present invention is useful for home service robots that support human housework and work, service robots in public facilities such as public halls, government offices, and hospitals, and business support robots in general offices and work sites. When the present invention is applied to these, the robot continues to perform work on behalf of humans, presents things to humans, supports actions such as human movement and dialogue, etc. It is possible to realize a robot system that appropriately supports the above.

(Appendix 1)
A device operated by a user, a support device that executes processing for supporting the operation of the device by the user, user information acquisition means for acquiring user information representing at least one of the position and orientation of the user, and the device A device information acquisition unit that acquires device information representing at least one of the position and orientation of the device, and an operation state information acquisition unit that acquires operation state information representing the operation state of the device from the device, and
The support device has a movable part that is movable so that at least one of the position and posture of the support device itself changes.
At least of the user information acquired by the user information acquisition unit and the device information acquired by the device information acquisition unit according to a change in the history of the operation state information acquired by the operation state information acquisition unit. And controlling the operation of the movable part so that the support process for the user by the support apparatus can be executed based on the one side, and provided with a support apparatus control means for controlling to execute the support process by the support apparatus,
User support system.

(Appendix 2)
The user support system according to attachment 1, wherein
The support device control means operates when the history of the operation state information repeats a series of operation states composed of a plurality of operation states.
User support system.

(Appendix 3)
The user support system according to attachment 2, wherein
The support device control means executes a support process set in advance corresponding to a series of repeated operation states in the history of the operation state information.
User support system.

(Appendix 4)
A user support system according to any one of appendices 1 to 3,
The operating state information is information representing an operating state of the device determined by an operation by the user.
User support system.

(Appendix 5)
The user support system according to any one of appendices 1 to 4,
The movable part of the support device has a device main body moving means for changing at least one of the position and posture of the support device main body,
The support apparatus control means controls the operation of the apparatus main body moving means to change at least one of the position and posture of the support apparatus main body;
User support system.

(Appendix 6)
The user support system according to appendix 5,
The support device is a humanoid robot having at least one of a face part, a body part, a foot part, and a hand part as the movable part,
The support device control means includes the position and posture of the support device body, the position and posture of the face, the position and posture of the torso, the position and posture of the foot, and the position and posture of the hand. Change at least one,
User support system.

(Appendix 7)
The user support system according to appendix 5 or 6,
The support device control means changes at least one of a position and a posture of the support device based on the user information so that the support device enters the visual field of the user.
User support system.

(Appendix 8)
A user support system according to any one of appendices 5 to 7,
The support device control means changes at least one of a position and a posture of the support device so that the support device can execute support processing on the device based on the device information.
User support system.

(Appendix 9)
The user support system according to any one of appendices 5 to 8,
The support device control means controls the operation of the movable part based on the user information acquired by the user information acquisition means while the support device is executing support processing.
User support system.

(Appendix 10)
A device operated by a user, a support device that executes processing for supporting the operation of the device by the user, user information acquisition means for acquiring user information representing at least one of the position and orientation of the user, and the device A user support system comprising: device information acquisition means for acquiring device information representing at least one of the position and orientation of the device; and an operation state information acquisition means for acquiring operation state information representing the operation state of the device from the device. A user support control device for controlling the operation of the support device,
At least of the user information acquired by the user information acquisition unit and the device information acquired by the device information acquisition unit according to a change in the history of the operation state information acquired by the operation state information acquisition unit. The operation of the movable part that is movable so that at least one of the position and posture of the support device itself mounted on the support device changes so that the support processing for the user by the support device can be executed based on one side. And controlling to execute support processing by the support device,
User support control device.

(Appendix 11)
The user support control device according to attachment 10, wherein
When the history of the operating state information repeats a series of operating states consisting of a plurality of operating states, the operation of the movable unit is controlled and control is performed to execute support processing by the support device.
User support control device.

(Appendix 12)
A device operated by a user, a support device that executes processing for supporting the operation of the device by the user, user information acquisition means for acquiring user information representing at least one of the position and orientation of the user, and the device A user support system comprising: device information acquisition means for acquiring device information representing at least one of the position and orientation of the device; and an operation state information acquisition means for acquiring operation state information representing the operation state of the device from the device. A user support control device that controls the operation of the support device.
At least of the user information acquired by the user information acquisition unit and the device information acquired by the device information acquisition unit according to a change in the history of the operation state information acquired by the operation state information acquisition unit. The operation of the movable part that is movable so that at least one of the position and posture of the support device itself mounted on the support device changes so that the support processing for the user by the support device can be executed based on one side. A program for executing control processing to control and execute support processing by the support device.

(Appendix 13)
The program according to attachment 12, wherein
When the history of the operation state information repeats a series of operation states composed of a plurality of operation states, the operation of the movable unit is controlled and the support device is controlled to execute the support process.
program.

(Appendix 14)
In a user support system comprising a device operated by a user and a support device that executes processing for supporting the operation of the device by the user,
User information representing at least one of the position and posture of the user, device information representing at least one of the position and posture of the device, and operating state information representing the operating state of the device from the device are acquired. ,
In accordance with a change in the history of the acquired operation state information, the support device can execute support processing for the user based on at least one of the acquired user information and the acquired device information. Controlling the operation of the movable part movable so that at least one of the position and posture of the support device itself mounted on the support device changes, and controlling to execute the support process by the support device;
User support control method.

(Appendix 15)
The user support control method according to appendix 14,
When the history of the operation state information repeats a series of operation states composed of a plurality of operation states, the operation of the movable unit is controlled and the support device is controlled to execute the support process.
User support control method.

DESCRIPTION OF SYMBOLS 1 User 2 Television system 21 User operation input part 22 Apparatus attitude | position acquisition part 23 Equipment state transmission part 3 Robot 301 State history acquisition part 302 Operation judgment part 303 Operator operation request part 304 Content request part 305 Robot control part 306 Voice recognition part 307 Utterance Unit 308 movement mechanism unit 309 display input unit 310 instruction mechanism unit 4 home server 41 status recording unit 5 user attitude transmitter 5
51 User posture acquisition unit 52 User posture transmission unit 6 Robot operation terminal 61 Operator operation input unit 7 Content server 71 Content recording unit 8 Operator 9 Public line 500 User 501 User information acquisition unit 510 Device 511 Device information acquisition unit 512 Operation state information acquisition Means 520 Support Device 521 Movable Unit 530 Support Device Control Unit

Claims (10)

A device operated by a user, a support device that executes processing for supporting the operation of the device by the user, user information acquisition means for acquiring user information representing at least one of the position and orientation of the user, and the device A device information acquisition unit that acquires device information representing at least one of the position and orientation of the device, and an operation state information acquisition unit that acquires operation state information representing the operation state of the device from the device, and
The support device has a movable part that is movable so that at least one of the position and posture of the support device itself changes.
At least of the user information acquired by the user information acquisition unit and the device information acquired by the device information acquisition unit according to a change in the history of the operation state information acquired by the operation state information acquisition unit. And controlling the operation of the movable part so that the support process for the user by the support apparatus can be executed based on the one side, and provided with a support apparatus control means for controlling to execute the support process by the support apparatus,
User support system. The user support system according to claim 1,
The support device control means operates when the history of the operation state information repeats a series of operation states composed of a plurality of operation states.
User support system. The user support system according to claim 1 or 2,
The operating state information is information representing an operating state of the device determined by an operation by the user.
User support system. The user support system according to any one of claims 1 to 3,
The movable part of the support device has a device main body moving means for changing at least one of the position and posture of the support device main body,
The support apparatus control means controls the operation of the apparatus main body moving means to change at least one of the position and posture of the support apparatus main body;
User support system. The user support system according to claim 4,
The support device control means changes at least one of a position and a posture of the support device based on the user information so that the support device enters the visual field of the user.
User support system. The user support system according to claim 4 or 5,
The support device control means changes at least one of a position and a posture of the support device so that the support device can execute support processing on the device based on the device information.
User support system. The user support system according to any one of claims 4 to 6,
The support device control means controls the operation of the movable part based on the user information acquired by the user information acquisition means while the support device is executing support processing.
User support system. A device operated by a user, a support device that executes processing for supporting the operation of the device by the user, user information acquisition means for acquiring user information representing at least one of the position and orientation of the user, and the device A user support system comprising: device information acquisition means for acquiring device information representing at least one of the position and orientation of the device; and an operation state information acquisition means for acquiring operation state information representing the operation state of the device from the device. A user support control device for controlling the operation of the support device,
At least of the user information acquired by the user information acquisition unit and the device information acquired by the device information acquisition unit according to a change in the history of the operation state information acquired by the operation state information acquisition unit. The operation of the movable part that is movable so that at least one of the position and posture of the support device itself mounted on the support device changes so that the support processing for the user by the support device can be executed based on one side. And controlling to execute support processing by the support device,
User support control device. A device operated by a user, a support device that executes processing for supporting the operation of the device by the user, user information acquisition means for acquiring user information representing at least one of the position and orientation of the user, and the device A user support system comprising: device information acquisition means for acquiring device information representing at least one of the position and orientation of the device; and an operation state information acquisition means for acquiring operation state information representing the operation state of the device from the device. A user support control device that controls the operation of the support device.
At least of the user information acquired by the user information acquisition unit and the device information acquired by the device information acquisition unit according to a change in the history of the operation state information acquired by the operation state information acquisition unit. The operation of the movable part that is movable so that at least one of the position and posture of the support device itself mounted on the support device changes so that the support processing for the user by the support device can be executed based on one side. A program for executing control processing to control and execute support processing by the support device. In a user support system comprising a device operated by a user and a support device that executes processing for supporting the operation of the device by the user,
User information representing at least one of the position and posture of the user, device information representing at least one of the position and posture of the device, and operating state information representing the operating state of the device from the device are acquired. ,
In accordance with a change in the history of the acquired operation state information, the support device can execute support processing for the user based on at least one of the acquired user information and the acquired device information. Controlling the operation of the movable part movable so that at least one of the position and posture of the support device itself mounted on the support device changes, and controlling to execute the support process by the support device;
User support control method.