JP6369392B2 - Remote control system - Google Patents

Description

  The present invention relates to a remote control system, for example, a remote control system in which a real or virtual robot performs a biped walking operation following a predetermined delay in accordance with an operator's biped walking operation.

  Tasks can be executed flexibly by controlling the robot remotely in housework support and extreme environments. If a detailed advance map of the work area can be obtained at that time, various autonomous movement techniques can be used, but such a situation is rare, and a robot using a master-slave control system (that is, a remote control system) is used. It is useful to control the remote.

  For example, the remote control system disclosed in Patent Document 1 detects, by a six-axis force sensor, the acting force received from the foot platform by the operator's foot placed on the left and right foot platform of the foot support mechanism. Based on the information, the robot's legs are master-slave controlled.

Japanese Patent No. 3666714

  The general remote control system is set so that the robot operates using the operation information about one second before the operator as future information. That is, the biped walking operation of the robot is executed so as to trace the biped walking operation with a delay of about 1 second with respect to the current biped walking operation of the operator. Therefore, a time lag occurs between the biped walking motion of the robot and the biped walking motion of the operator, and the operability of the robot is poor.

  The present invention has been made to solve such problems, and realizes a remote control system with excellent operability of the robot.

A remote control system according to an aspect of the present invention is a remote control system in which a real or virtual robot performs a biped walking operation following a biped walking operation after a predetermined delay in accordance with a biped walking operation of an operator. Because
A first detection unit for detecting a position and posture of a foot of a free leg with respect to a foot of a support leg in the operator;
The leg of the free leg with respect to the foot of the support leg in the robot corresponding to the position and posture of the foot of the free leg with respect to the foot of the support leg of the operator detected by the detection information of the first detection unit. An estimation unit for estimating the position and orientation of the plane;
Based on the position and posture of the free leg's foot with respect to the foot of the supporting leg in the robot estimated by the estimating unit, a free leg foot display representing the free leg's foot of the robot is generated, The superimposed information obtained by superimposing the free leg foot display information indicating the leg foot display and the surrounding environment information of the robot is preceded by the time when the foot of the free leg reaches their position and posture in the robot, A display unit to display;
Is provided.

  In the above-described remote control system, the estimation unit calculates a first movable area of the free leg in the robot according to an elapsed time of leaving the free leg of the operator, and the first movable area is within the first movable area. If the free leg foot display is not arranged, it is preferable to correct the free leg foot display so that the free leg foot display is arranged in the first movable region.

  In the above-described remote control system, it is preferable that the display unit displays first movable area information indicating the first movable area.

  In the above-described remote control system, the estimation unit calculates a second movable area of the free leg in the robot according to a preset stride of the robot, and the second movable area includes the second movable area. If the free leg foot display is not arranged, it is preferable to correct the free leg foot display so that the free leg foot display is arranged in the second movable region.

  In the above-described remote control system, it is preferable that the display unit displays second movable area information indicating the second movable area.

  In the above-described remote control system, the estimation unit displays the free leg foot display when the free leg foot display interferes with a self-interference area determined based on a position and a posture of a support leg in the robot. It is preferable to correct the free leg foot display so as not to interfere with the self-interference area.

  In the above-described remote control system, the estimation unit determines a non-landing area based on the surrounding environment information of the robot, and the non-landing area under a condition in which the free leg foot display is set in advance. Preferably, the free leg foot display is corrected so that the free leg foot display does not interfere with the non-landing area under the above condition.

  In the above-described remote control system, it is preferable that the display unit displays non-implantable area information indicating the non-implantable area.

In the above-described remote control system, a second detection unit that detects a load acting on the foot of the right leg and a load acting on the foot of the left leg in the operator, respectively,
The estimating unit estimates a posture of the waist of the robot and a position of a target ZMP based on a ratio of a load acting on the right leg foot and a load acting on the left leg foot;
Preferably, the display unit displays posture information indicating a posture of the waist of the robot and position information indicating a position of the target ZMP.

  As described above, a remote control system with excellent operability of the robot can be realized.

1 is a configuration diagram of a remote control system according to Embodiment 1. FIG. 4 is a flowchart showing a flow of displaying free leg foot display information using the remote control system of the first embodiment. It is a figure which shows the example of a display of the foot display information of the supporting leg in a robot, and the foot display information of the free leg before leaving the bed. It is a figure which shows the example of a display of the foot display information of the support leg in a robot, and the foot display information of the free leg immediately after leaving the bed. It is a figure which shows typically operation | movement of the leg in an operator. It is a figure for demonstrating the coordinate system of the support leg in an operator, and the coordinate system of a free leg. It is a figure which shows the example of a display of the foot display information of the supporting leg in a robot, the foot display information of the free leg just after leaving the floor in a robot, and the foot display information of the free leg just before landing. It is a figure which shows superimposition information. (A) is a figure which shows the example of a display before the free leg in an operator lands. (B) is a figure which shows the example of a display after the free leg in an operator has landed. It is a figure which shows the example of a display which corrected the foot display of the free leg. It is a different figure which shows the example of a display which corrected the foot display of the free leg. It is a figure for demonstrating the distance between the foot centers of the leg on either side of a robot. It is a different figure for demonstrating the distance between the foot centers of the leg on either side of a robot. It is a figure which shows the example of a display which corrected the foot display of the free leg. (A) is a figure which shows the arrangement | positioning relationship between the foot display of a support leg, and the foot display of a free leg when the foot display of a free leg interferes with the foot display of a support leg. (B) is a figure which shows the arrangement | positioning relationship between the foot display of the free leg corrected so that the foot display of the free leg may not interfere with the foot display of the support leg, and the foot display of the support leg. (A) is a figure which shows the arrangement | positioning relationship between the foot display of a support leg, and the foot display of a free leg when the knee of a support leg and the knee of a free leg interfere. (B) is a figure which shows the arrangement | positioning relationship between the foot display of the free leg and the foot display of the support leg corrected so that the knee of a support leg and the knee of a free leg may not interfere. It is a figure which shows typically the environment where a robot walks. It is a figure which shows superimposition information. It is a figure for demonstrating the sampling point provided in the foot display of the free leg. (A) is a figure which shows the example of a display of the state which the foot display of the free leg interfered with the convex part. (B) is a figure which shows the example of a display of the state which corrected the foot display of the free leg so that the foot display of the free leg may not interfere with a convex part. (C) is a figure which shows the foot locus which interpolated. (A) is a figure which shows the example of a display of the state which the foot display of the free leg interfered with the recessed part. (B) is a figure which shows the example of a display of the state which corrected the foot display of the free leg so that the foot display of the free leg may not interfere with a recessed part. (C) is a figure which shows the foot locus which interpolated. It is a figure which shows typically the load which acts on the foot of a left leg in an operator, and the load which acts on the foot of a right leg. It is a figure which shows the example of a display of the attitude | position of the waist of a robot, and the position of target ZMP.

  The best mode for carrying out the present invention will be described below with reference to the accompanying drawings. However, the present invention is not limited to the following embodiment. In addition, for clarity of explanation, the following description and drawings are simplified as appropriate.

<Embodiment 1>
First, the configuration of the remote control system of the present embodiment will be described. FIG. 1 is a configuration diagram of a remote control system according to the present embodiment.

  The remote control system 1 is a so-called master-slave control system, and as shown in FIG. 1, a biped walking robot 3 (hereinafter sometimes simply referred to as a robot), a first detection unit 4, and a second detection system. A detection unit 5, a master calculation unit 6, and a display unit 7 are provided. Here, the unit including the first detection unit 4, the second detection unit 5, the master calculation unit 6, and the display unit 7 corresponds to the master device 2 in the master-slave control, and the robot 3 is a slave in the master-slave control. It corresponds to a device.

  The first detection unit 4 detects the relative position and relative posture between the foot of the support leg of the operator and the foot of the free leg. The first detection unit 4 of the present embodiment includes a plurality of motion capture devices 4a. Each of the plurality of motion capture devices 4 a captures an operator having a plurality of markers 8 attached to the foot, generates image information indicating the image, and outputs the image information to the master calculation unit 6. Here, the “free leg” includes not only the state in which the foot has left the floor, but also the state immediately before the foot has left the floor and the state immediately after the landing.

  The second detection unit 5 detects a load acting on the right leg foot and a load acting on the left leg foot of the operator. The second detection unit 5 of the present embodiment includes a pair of force plates 5a. The pair of force plates 5a is used when the operator places the left leg on the left force plate 5a and the right leg on the right force plate 5a. Thereby, each of a pair of force plate 5a detects the load which acts on an operator's right leg and left leg. Each of the pair of force plates 5 a outputs load information indicating the detected loads on the right leg and the left leg of the operator to the master calculation unit 6.

  The master calculation unit 6 can be configured by, for example, a PC (Personal Computer). Based on the image information (detection information) input from the first detection unit 4, the master calculation unit 6 determines the leg raising height of the operator's free leg and the right leg's foot and left leg of the operator. Foot position indicating the relative position and relative posture of the foot, the calculated leg lift height, and the relative position and relative position of the right and left foot of the operator Posture information is output to the robot 3.

  Based on the load information input from the second detection unit 5, the master calculation unit 6 determines whether the operator has left the floor and has landed. In addition, the master calculation unit 6 outputs the load information input from the second detection unit 5 to the robot 3.

  The master calculation unit 6 supports the supporting legs in the robot 3 so as to trace the movement of the operator's leg based on the calculated relative position and relative posture of the right leg and the left leg. The relative position and relative posture between the foot of the foot and the foot of the free leg are estimated (calculated). That is, the master calculation unit 6 functions as an estimation unit that estimates the relative position and relative posture between the foot of the supporting leg and the foot of the free leg in the robot 3.

The master calculation unit 6 displays the foot when the free leg is landed at the relative position and relative posture between the estimated foot of the supporting leg and the foot of the free leg (that is, the foot of the free leg is projected). Virtual landing foot display). In addition, the master calculation unit 6 superimposes the generated foot display information indicating the foot display of the free leg on the surrounding environment information indicating the surrounding environment of the robot 3 detected by the detecting unit 9 mounted on the robot 3. Then, superimposition information obtained by superimposing the foot display information of the free leg on the surrounding environment information of the robot 3 is output to the display unit 7.
The display unit 7 displays the superimposition information input from the master calculation unit 6.

  The robot 3 includes a detection unit 9 that detects the surrounding environment of the robot 3. The detection unit 9 of the present embodiment includes KINECT (registered trademark), detects the surrounding environment of the robot 3, and includes surrounding environment information (for example, 3D point cloud information) indicating the detected surrounding environment of the robot 3. Peripheral environment image information and grid information) is output to the master calculation unit 6. However, the detection part 9 should just be able to detect surrounding environment, and may be provided with a laser range sensor, a stereo camera, etc.

  Here, the “peripheral environment information” is a model on the assumption that there are no obstacles or unevenness in the surroundings, information on the floor surface that has been modeled in advance, or information on the real surface detected by the detection unit 9. Information on the horizontal plane is included. However, the “peripheral environment information” only needs to include at least information related to the floor surface on which the robot 3 walks, and it does not conform to the actual environment and may be an assumed model.

  Further, the robot 3 executes master-slave control based on the foot position / posture information and the load information input from the master calculation unit 6. More specifically, the slave calculation unit 10 mounted on the robot 3 operates based on the foot position / posture information and the load information input from the master calculation unit 6, and the operator after the bipedal walking motion of the robot 3 is delayed by a predetermined delay. The target ZMP of the robot 3, the target position and target posture of the left leg, the target position and target posture of the right leg, and the target position and target posture of the waist are calculated so as to follow the biped walking motion. Here, the slave calculation part 10 can be comprised by CPU (Central Processing Unit), for example. The slave calculation unit 10 can be included in a microcontroller (microcomputer) that is mounted on the robot 3 and comprehensively controls the operation of the robot 3.

  The slave calculation unit 10 calculates each joint angle of the robot 3 based on the calculated information (target ZMP, target position and target posture of the left leg, target position and target posture of the right leg, and target position and target posture of the waist). Each joint is controlled so that the calculated joint angle is obtained. Specifically, the robot 3 includes a plurality of actuators (not shown) that function as each joint. The slave calculation unit 10 calculates the target angle of each actuator as each joint angle, and controls each actuator so that the angle of each actuator becomes the calculated target angle. That is, the target ZMP, the target position and target posture of the left leg, the target position and target posture of the right leg, and the target angle and the target position of the waist and the target posture are calculated. Thereby, master-slave control is performed so that the biped walking motion of the robot 3 follows the biped walking motion of the operator.

  Next, the flow of displaying the foot display information of the free leg using the remote control system 1 of the present embodiment will be described. FIG. 2 is a flowchart showing a flow of displaying the foot display information of the free leg using the remote control system of the present embodiment. FIG. 3 is a diagram illustrating a display example of the foot display information of the supporting leg and the foot display information of the free leg before leaving the bed in the robot. FIG. 4 shows a display example of the foot display information of the support leg in the robot and the foot display information of the free leg immediately after leaving the bed, and the coordinate system of the support leg and the free leg coordinate system before leaving the bed in the robot will be described. FIG. FIG. 5 is a diagram schematically illustrating the movement of the leg of the operator. FIG. 6 is a diagram for explaining the coordinate system of the support leg and the coordinate system of the free leg for the operator. FIG. 7 shows a display example of the foot display information of the support leg in the robot, the foot display information of the free leg immediately after leaving the robot, and the foot display information of the free leg just before landing. It is a figure for demonstrating a coordinate system, the coordinate system of the free leg before leaving the floor, and the coordinate system of the free leg after leaving the floor. FIG. 8 is a diagram showing superposition information. Fig.9 (a) is a figure which shows the example of a display before a free leg in an operator lands. FIG. 9B is a diagram illustrating a display example after the free leg of the operator has landed. 3, 4, and 7, the surrounding environment information of the robot 3 is omitted so that the foot display information is clear.

  First, as shown in FIG. 3, the display unit 7 displays the surrounding environment information of the robot 3 (not shown), the foot display information of the supporting leg in the robot 3, and the foot display information of the free leg before leaving the floor. (S1). The foot display information of the support leg in the robot 3 is foot display information indicating the foot display generated when the foot of the support leg is landed by the free leg motion. Further, the foot display information of the free leg before leaving the floor in the robot 3 is foot display information indicating the foot display generated when the foot of the free leg has landed in the previous swing leg operation.

Here, as shown in FIG. 4, if the global coordinate system used as a reference in the robot 3 is defined as Σ ₀ , the coordinate system of the support leg is defined as Σ _rS, and the coordinate system of the free leg before leaving the floor is defined as Σ _{rF 0.} , homogeneous transformation matrix ⁰ T _RF0 representing the position and orientation of the foot centers in the free leg of the front lifting for homogeneous transformation matrix ⁰ T _rS and the reference coordinates representing the position and orientation of the foot centers in the support leg relative to the reference coordinates These are generally used in calculations in the robot 3.

  Next, the operator recognizes the robot 3 while viewing the display unit 7 on which the surrounding environment information of the robot 3, the foot display information of the supporting legs in the robot 3, and the foot display information of the free leg before leaving the floor are displayed. In order to perform master-slave control of the legs, the legs are operated as shown in FIG. 5 (S2).

  Next, the master calculation unit 6 determines whether or not the free leg of the operator has left the floor (S3). Specifically, the master calculation unit 6 is input from the pair of force plates 5a that the left or right leg of the operator is a free leg and the other leg is a support leg in the previous walking motion (one step). Is recognized based on the load indicated by the load information. Then, the master calculation unit 6 determines whether or not the foot of the leg that becomes the free leg has left the floor in the current walking motion by the operator based on the load indicated by the load information input from the force plate 5a. To do. That is, for example, as shown in FIG. 5, the master calculation unit 6 receives the load information input from the right force plate 5 a when the foot of the right leg, which is a free leg, leaves the floor in the current walking motion by the operator. If it becomes smaller than the preset first threshold value, it is determined that the foot of the right leg of the operator has left the floor.

  Next, the master calculation unit 6 calculates the position (coordinates) and posture (orientation) of the free leg's foot relative to the foot of the support leg of the operator (S4). Specifically, the master calculation unit 6 calculates the foot position and posture of both legs of the operator from the movement of the marker 8 included in the image indicated by the image information input from each of the plurality of motion capture devices 4a. .

For example, as shown in FIG. 6, _hS the coordinate system of the support leg sigma in the _operator, defining a coordinate system of the free leg and sigma _hF in the operator, the foot of the support leg in the operator flat center (e.g., foot The homogeneous transformation matrix representing the position and posture of the foot center of the free leg with respect to the position of the center of gravity) is expressed as <Equation 1>, and the master calculation unit 6 calculates xh, yh, and θ of <Equation 1>.

Next, the master calculation unit 6 corresponds to the position and posture of the foot of the free leg with respect to the foot of the support leg in the operator, and the position of the foot center of the free leg with respect to the center of the foot of the support leg in the robot 3 and The posture is estimated (calculated) (S5). For example, as shown in FIG. 7, the coordinate system after leaving the free leg of the robot 3 is defined as _ΣrF, and the center of the foot of the supporting leg in the robot 3 is defined by The position (x, y and z coordinates) and posture (rotation angle around the z axis) of the foot center of the free leg in the robot 3 applying the amount of change of the foot center of the free leg with respect to the foot center of the support leg Estimate in the coordinate system.

However, when the length of the leg portion is greatly different between the operator and the robot 3, xh and yh are multiplied by a constant rate (r) that takes into account the difference in length as shown in <Equation 3>. The position and posture of the foot center of the free leg in the robot 3 may be estimated using ^hS T _hF ′ as ^hS T _hF .

  Next, the master calculation unit 6 generates a foot display when the free leg has landed at the estimated position and posture of the foot center of the free leg in the robot 3 (S6). Specifically, the master calculation unit 6 generates a foot display based on the foot shape of the robot 3 set in advance and the calculated position and posture of the foot center of the free leg in the robot 3.

  Next, as shown in FIG. 8, the master calculation unit 6 uses the foot display information indicating the generated foot display to indicate the surrounding environment of the robot 3 acquired by the detection unit 9 mounted on the robot 3. Superimposition information that is superimposed on the environment information and the foot display information is superimposed on the surrounding environment information of the robot 3 is output to the display unit 7 (S7). Incidentally, the foot display information may be superimposed on the surrounding environment information of the robot 3 based on the coordinates of the center of the foot in the foot display and the coordinates of the grid of the surrounding environment.

  Next, the display unit 7 displays the input superimposition information (S8). That is, the foot display information of the free leg of the robot 3 is displayed on the display unit 7 even if the operator's free leg has not yet landed. Then, the master calculation unit 6 generates a foot display of the free leg in the robot 3 at a predetermined time interval, and causes the display unit 7 to display foot display information indicating the foot display. Here, the “predetermined time interval” is preferably a high rate (for example, 30 fps or more) so that the operator can perform master-slave control with good response by looking at the display unit 7.

  Thereafter, the master calculation unit 6 determines whether or not the load indicated by the load information input from the free leg side force plate 5a of the operator is greater than a preset second threshold value, and the free leg of the operator. If the load indicated by the load information input from the side force plate 5a is greater than the second threshold, it is determined that the operator's free leg has landed. Then, the master calculation unit 6 sets the foot display of the free leg when it is determined that the operator's free leg has landed as a confirmed foot display, and position information indicating the position of the foot center in the confirmed foot display. And posture information indicating the posture is output to the slave calculation unit 10. The slave calculation unit 10 executes master-slave control as described above based on the input position information, posture information, and the like. That is, before landing on the posture indicated by the position and posture information indicated by the position information to which the foot of the free leg of the robot 3 is input, the foot indicating the foot display based on the position information and posture information on the display unit 7. Flat display information is displayed. Incidentally, the master-slave control can be executed by, for example, control based on the center-of-gravity trajectory calculation that satisfies the target ZMP trajectory and the inverse kinematic calculation that satisfies the target foot trajectory. .

  The above-described steps S1 to S8 are continued while the operator executes master-slave control of the robot 3.

  As described above, in this embodiment, the foot display information of the free leg in the robot 3 corresponding to the position and posture of the foot of the free leg with respect to the foot of the support leg of the operator is displayed on the display unit 7. Therefore, the operator can intuitively perform master-slave control of the robot 3 while looking at the display unit 7, and the operability of the robot 3 is improved.

  In addition, since the foot display information of the free leg in the robot 3 is displayed so as to be superimposed on the surrounding environment information of the robot 3, the operator can more intuitively perform master-slave control of the robot 3 while looking at the display unit 7. Can do.

  Here, as shown in FIG. 7, when the arrow is extended from the foot center of the foot display information of the free leg before leaving the floor to the foot center of the foot display information of the free leg in the leaving state, as shown in FIG. It becomes easy for the operator to visually recognize the movement of the foot of the free leg.

  In addition, as shown in FIGS. 4, 7, 9A, and 9B, each foot display information is displayed so that the operator can distinguish each foot display information of the free leg. It is preferable to vary the line type and display color. Thereby, confusion of foot display information can be suppressed.

<Embodiment 2>
The robot 3 has a lower maximum operating speed than the operator. Therefore, there are cases where the legs of the robot 3 cannot be moved so as to correspond to the movement of the operator's legs. Therefore, in the present embodiment, the foot display of the free leg generated in the first embodiment is corrected within a range that can be realized by the robot 3.

  10 and 11 are diagrams illustrating display examples in which the foot display of the free leg is corrected. Incidentally, FIG. 11 exemplifies a case where the elapsed time of leaving the free leg of the operator is longer than that in FIG. 10 and 11, the surrounding environment information of the robot 3 is omitted so that the foot display information is clear.

  The master calculation unit 6 of the present embodiment calculates the first movable area of the free leg in the robot 3 according to the elapsed time of leaving the free leg of the operator. Specifically, the master calculation unit 6 determines the relationship between the preset elapsed time and the first movable area centered on the foot center of the free leg before leaving the floor in the robot 3, and the free leg for the operator. The first movable area is calculated on the basis of the elapsed time from when the person leaves the bed. For example, the first movable region becomes wider in proportion to the elapsed time of leaving the free leg of the operator.

  The master calculation unit 6 determines whether or not the free leg's foot display is arranged in the first movable region. Then, the master calculation unit 6 determines that if the free leg's foot display is not arranged in the first movable area, the free leg's foot display is arranged in the first movable area. The position and posture of the center of the foot in the foot display of the free leg is corrected, and foot display information indicating the foot display generated (corrected) based on the corrected position and posture is displayed on the display unit 7. However, even if all of the free leg's foot display is not arranged in the first movable area, for example, the foot of the free leg's foot display is on the boundary of the first movable area. If it is arrange | positioned, it can also be considered that the foot display of a free leg is arrange | positioned in the 1st movable area | region.

Here, the amount of change in the position and posture of the foot center from the coordinate system before leaving the floor of the robot 3 relative to the coordinate system (Σ _rF0 ) of the robot 3 to the coordinate system after leaving the floor of the robot 3 can be expressed by < _Equation 4>. . However, Δx, Δy, and Δθ in <Equation 4> are changes in posture and position. ^RF0 T _rF is a homogeneous transformation matrix that represents the position and posture of the foot center of the free leg after getting out of bed with respect to the foot center of the free leg before getting out of the robot 3. ⁰ T _rF0 is a homogeneous transformation matrix that represents the position and posture of the foot center of the free leg before leaving the floor in the robot 3 with respect to the reference coordinates.

  The master calculation unit 6 modifies Δθ, Δx, Δy to Δθ ′, Δx ′, Δy ′ so that the foot display of the free leg falls within the first movable region with the amount of change according to <Equation 5>. The position and posture of the foot center in the foot display of the free leg are corrected.

  Thereby, the foot display of the free leg can be stored in the first movable region in which the robot 3 can operate.

  Here, as shown in FIGS. 10 and 11, the display unit 7 includes first movable area information R1 indicating the first movable area in addition to the corrected foot display information of the free leg. It is preferable to display. Thereby, the operator can operate intuitively so that the foot of the free leg does not deviate from the first movable region.

  At this time, as shown in FIGS. 10 and 11, any of the foot display information of the supporting leg, the foot display information of the free leg immediately after leaving the floor, and the foot display information of the free leg before correction is selectively displayed. It is preferable to display in the part 7.

Further, the foot display information of the supporting leg, the foot display information of the free leg immediately after leaving the floor, the foot display information of the free leg before correction, and the foot display information of the free leg after correction can be distinguished. Thus, it is preferable to change the line type and display color, respectively. Thereby, confusion of foot display information can be suppressed.

  In the present embodiment, the first movable area information R1 is widened at a constant speed. However, the first movable area information is strictly at a foot movable speed based on the posture of the robot 3. R1 may be widened.

  Further, the first movable region of the present embodiment is centered on the foot center of the free leg before leaving the floor in the robot 3, but the position of the center is not particularly limited. For example, the robot 3 is in an upright state. The foot center of the leg that becomes the free leg may be the center of the first movable region.

<Embodiment 3>
The robot 3 is more restrictive in terms of the range of movement of the joints than humans, and is set in advance with a leg swing width (step length) and an allowable posture value that can be safely walked. Therefore, in the present embodiment, if the position and posture of the foot center in the foot display of the free leg generated in the first or second embodiment deviates from the allowable value, the correction is made so that it falls within the allowable value. To do.

  12 and 13 are diagrams for explaining the distance between the foot centers of the left and right legs of the robot. FIG. 14 is a diagram illustrating a display example in which the foot display of the free leg is corrected. In FIG. 12, only the legs of the robot 3 are shown. Further, in FIG. 14, the surrounding environment information of the robot 3 is omitted so that the foot display information of the free leg is clear.

As shown in FIGS. 12 and 13, the distance between the foot centers of the left and right legs when the robot 3 is in the upright state is w, and the foot center of the leg on the supporting leg side of the robot 3 (the coordinate system is _ΣrS ) Is defined as the free leg foot reference coordinates (coordinate system is Σ _rFC ), the free leg foot reference coordinates are as follows: Can be represented.

However, ⁰ T _rFC is a homogeneous transformation matrix that represents the position and posture of the foot center of the leg that is the free leg when the robot 3 stands upright with respect to the reference coordinates. ⁰ T _rS is a homogeneous transformation matrix that represents the position and posture of the foot center of the leg on the side that becomes the supporting leg when the robot 3 is upright with respect to the reference coordinates. ^rS T _rFC is a homogeneous transformation matrix representing the position and posture of the foot center of the leg on the free leg side with respect to the foot center of the leg on the side of the supporting leg when the robot 3 is in the upright state.

  In <Equation 6>, when the left leg is a free leg, w is attached with a + sign, and when the right leg is a free leg, w is attached with a-sign.

  The master calculation unit 6 determines whether or not the position and posture of the foot center in the foot display of the free leg with respect to the free foot foot reference coordinates deviate from the preset step and posture tolerances of the robot 3. judge. Then, the master calculation unit 6 determines that the position and posture of the foot center in the foot display of the free leg with respect to the free foot foot reference coordinates deviate from the predetermined step length and posture tolerance of the robot 3. The position and posture of the foot center in the foot display are corrected so that the position and posture of the foot center in the foot display of the free leg are within the allowable value, and generated based on the corrected position and posture (corrected) ) Foot display information indicating the displayed foot display is displayed on the display unit 7.

Specifically, the master calculation unit 6 first calculates the position and posture of the foot center in the foot display of the free leg with respect to the free foot foot reference coordinates by the following equation (7). However, ^rFC T _rF is a homogeneous transformation matrix that represents the position and posture of the foot center of the free leg after leaving the floor with respect to the foot center of the leg on the side that becomes the free leg when the robot 3 is in the upright state.

  Then, the master calculation unit 6 modifies Δθ ′, Δx ′, and Δy ′ so that Δθ, Δx, and Δy are within the allowable values of the stride and posture, and the position of the foot center in the foot display of the free leg And the posture is corrected as shown in <Equation 8>.

  As a result, the position and posture of the foot center in the foot display of the free leg can be kept within the preset allowable values of the stride and posture of the robot 3.

  Here, in addition to the corrected foot display information, the display unit 7 includes an area where the foot of the free leg can land at least within the stride of the robot 3 (second movement as shown in FIG. 14). It is preferable to display the second movable area information R2 indicating the possible area). Thereby, the operator can operate intuitively so that the foot of the free leg does not deviate from the second movable region.

  At this time, as shown in FIG. 14, the foot display information of the left and right legs when the robot 3 is in the upright state, the foot display information of the free leg immediately after leaving the robot 3, and the foot display of the free leg before correction Any of the information is preferably selectively displayed on the display unit 7.

  Also, the foot display information of the left and right legs when the robot 3 is upright, the foot display information of the free leg immediately after leaving the robot 3, the foot display information of the free leg before correction, and the play after correction It is preferable to change the line type and display color so that the leg foot display information can be distinguished from each other. Thereby, confusion of foot display information can be suppressed.

  In the second movable region of the present embodiment, the center of the foot on the side that becomes the free leg when the robot 3 is in the upright state is the free leg foot reference coordinate, but the position that becomes the free leg foot reference coordinate. For example, the foot center of the free leg before leaving the floor in the robot 3 may be used as the free leg foot reference coordinates.

<Embodiment 4>
In this embodiment, self-interference is estimated by the foot display of the free leg interfering with the self-interference area, and the foot display of the free leg is displayed so that the foot display of the free leg does not interfere with the self-interference area. Correct it.

  Here, “self-interference” refers to contact with the robot itself such as the free leg and support leg of the robot 3 or the knees of the robot 3. In this embodiment, when the free leg's foot display generated in Embodiments 1 to 3 interferes with the foot display of the supporting leg, the free leg's foot display interferes with the foot display of the supporting leg. The foot display of the free leg is corrected so that it does not. That is, the self-interference area of the present embodiment is an area where the foot display of the support leg exists and corresponds to the interference between the support leg and the free leg.

  FIG. 15A is a diagram showing the relationship between the foot display of the supporting leg and the foot display of the free leg when the foot display of the free leg interferes with the foot display of the supporting leg. FIG. 15B is a diagram illustrating the relationship between the foot display of the free leg and the foot display of the support leg, which is corrected so that the foot display of the free leg does not interfere with the foot display of the support leg.

  As shown in FIG. 15A, when the free leg's foot display interferes with the supporting leg's foot display, the master calculation unit 6 does not interfere with the free leg's foot display with the supporting leg's foot display. As described above, the position and posture of the foot center in the foot display is corrected, and foot display information indicating the foot display of the free leg generated (corrected) based on the corrected position and posture is displayed on the display unit 7. .

  For example, the foot display of the free leg is displayed so that the foot display of the free leg does not interfere with the foot display of the support leg by moving the foot display of the free leg in the y-axis direction of the coordinate system of the support leg. Correct it. However, it is only necessary to correct the foot display of the free leg so as not to interfere with the foot display of the support leg. For example, the foot display of the free leg may be moved in the y-axis direction of the coordinate system of the free leg.

  Thereby, interference with the foot of the supporting leg and the foot of the free leg in the robot 3 can be suppressed. At this time, the foot display information of the free leg before correction may be displayed on the display unit 7 in addition to the foot display information of the free leg after correction.

<Embodiment 5>
Also in the present embodiment, self-interference is estimated by the foot display of the free leg interfering with the self-interference area, and the foot display of the free leg is displayed so that the foot display of the free leg does not interfere with the self-interference area. Correct it.

  Specifically, when performing master-slave control of the robot 3 based on the position and posture of the foot center in the foot display of the free leg generated in the first to fourth embodiments, the knee of the supporting leg and the knee of the free leg If there is interference (that is, self-interference), the foot display of the free leg is corrected so that the knee of the support leg and the knee of the free leg do not interfere.

  FIG. 16A is a diagram illustrating the relationship between the foot display of the support leg and the foot display of the free leg when the knee of the support leg and the knee of the free leg interfere with each other. FIG. 16B is a diagram illustrating the relationship between the foot display of the free leg and the foot display of the support leg that has been corrected so that the knee of the support leg and the knee of the free leg do not interfere with each other.

  When the supporting leg knee and the free leg knee interfere with each other as shown in FIG. 16A, the master calculation unit 6 determines that the supporting leg knee and the free leg knee are in contact with each other as shown in FIG. The position and posture of the foot center in the foot display of the free leg is corrected so as not to interfere, and the foot display information indicating the foot display of the free leg generated (corrected) based on the corrected position and posture is displayed. This is displayed on part 7.

  For example, if the foot display of the free leg interferes with the front area of the foot display of the support leg (the hatched portion in FIG. 16A and FIG. 16B), the master calculation unit 6 After determining that the knee and the knee of the free leg are interfering with each other, the foot display of the free leg is corrected so that the foot display of the free leg does not interfere with the front area of the foot display of the supporting leg. Foot display information indicating the foot display of the free leg is displayed on the display unit 7. That is, the self-interference area of the present embodiment is the front area of the support leg and corresponds to the interference between the knee of the support leg and the knee of the free leg. However, since the robot 3 of the present embodiment is not a robot that bends the knees backward like a bird's leg, the self-interference area is only the front area of the support legs, but the knees backward like the bird's legs. When using a type of robot that bends, the rear region of the support leg may be a self-interference region.

  Specifically, the free leg's foot display is moved in the y-axis direction of the coordinate system of the support leg so that the free leg's foot display does not interfere with the front area of the support leg's foot display. Correct the foot display of. However, the foot display of the free leg may be corrected so that the foot display of the free leg does not interfere with the front area of the foot display of the support leg. May be moved in the y-axis direction.

  Thereby, the interference of the knee of the supporting leg and the knee of the free leg in the robot 3 can be suppressed. At this time, the foot display information of the free leg before correction may be displayed on the display unit 7 in addition to the foot display information of the free leg after correction.

<Embodiment 6>
In the present embodiment, a non-walkable area (non-landing area) of the robot 3 is determined based on the surrounding environment of the robot 3, and the foot display of the free leg generated in the first to fifth embodiments is a walk. If there is an interference with the impossible area under a preset condition, the foot display of the free leg is corrected so that the foot display of the free leg does not interfere with the non-walkable area under the condition.

  First, the avoidance operation of the convex portion that is an unwalkable area will be described. Here, FIG. 17 is a diagram schematically illustrating an environment in which the robot walks. FIG. 18 is a diagram showing superposition information. FIG. 19 is a diagram for explaining sampling points provided in the foot display of the free leg. FIG. 20A is a diagram illustrating a display example in a state where the foot display of the free leg interferes with the convex portion. FIG. 20B is a diagram illustrating a display example in a state where the foot display of the free leg is corrected so that the foot display of the free leg does not interfere with the convex portion. FIG. 20C shows the interpolated foot trajectory. In addition, in FIG. 18 etc., the walkable area | region (landing possible area | region) is shown by (circle), the convex part is shown by ●, and the recessed part is shown by x.

  In the environment as shown in FIG. 17, when the surrounding environment of the robot 3 is detected by the detection unit 9 mounted on the robot 3, for example, the surrounding environment image information and grid information shown in FIG. 18 can be obtained. Here, the surrounding environment image information and the grid information are superimposed, and the grid indicated by each grid information has coordinates in the reference coordinate system. And the master calculation part 6 determines whether it is a convex area and a recessed part which are a walkable area | region or a non-walkable area | region based on the said coordinate (especially z coordinate).

  Then, as shown in FIG. 19, the master calculation unit 6 causes the sampling points provided in the foot display of the free leg to interfere with the grid of convex portions that are non-walkable regions as shown in FIG. If it is, it is determined that walking is impossible.

  Incidentally, in the example of FIG. 19, 5 × 7 sampling points are regularly arranged, but the number and arrangement of the sampling points are not particularly limited, and are arranged at least along the periphery of the free leg's foot display. It only has to be done.

  When the master calculation unit 6 determines that walking is impossible, the foot locus between the free leg's foot display and the free leg's foot display immediately after leaving the robot 3 as shown in FIG. Interpolate.

  Then, as shown in FIG. 20B, the master calculation unit 6 immediately precedes the foot trajectory where the sampling point contacts the convex grid (that is, the foot where the sampling point contacts the convex grid). The foot display of the free leg is corrected so that the foot trajectory on the one floor side of the trajectory becomes the foot display of the free leg (and thus the estimated position of the center of the foot of the free leg in the estimated robot 3 and The posture is corrected), and the foot display information indicating the foot display of the free leg after the correction is displayed on the display unit 7. Thereby, when the robot 3 walks, it can suppress that the foot of the said robot 3 contacts a convex part.

  At this time, the display unit 7 displays the foot display information of the supporting leg in the robot 3 and the foot display of the free leg immediately after leaving the robot 3 in addition to the corrected foot display information and the surrounding environment information of the corrected leg. Either the information or the foot display information of the free leg before correction may be selectively displayed.

  Further, as shown in FIG. 20B, it is preferable to display the grid information indicating the grid touched by the sampling points separately from the grid information indicating other grids. Thereby, the operator can specify the position where the foot locus contacts.

  Next, the avoidance operation of the recessed portion that is an unwalkable area will be described. Here, FIG. 21A is a diagram illustrating a display example in which the foot display of the free leg interferes with the recess. FIG. 21B is a diagram illustrating a display example in a state where the foot display of the free leg is corrected so that the foot display of the free leg does not interfere with the recess. FIG. 21C shows the interpolated foot trajectory.

  As described above, when the grid information shown in FIG. 18 is obtained, the master calculation unit 6 is based on the coordinates of the grid indicated by each grid information, and a convex part that is a walkable area or a non-walkable area and It is judged whether it is a recessed part.

  Then, when the sampling point of the foot display of the free leg interferes with the grid of the concave portion that is a non-walkable area as shown in FIG. It is determined that walking is impossible. Here, the “preset condition” is set when the center of the foot is arranged in the non-walking area or when a preset number of sampling points are arranged in the non-walking area. can do.

  When the master calculation unit 6 determines that walking is impossible, the foot locus between the foot display of the free leg and the foot display of the free leg immediately after leaving the robot 3 as shown in FIG. Interpolate.

  Then, as shown in FIG. 21 (b), the master calculation unit 6 detects the foot just before the foot locus in which the sampling point of the foot display of the free leg interfered with the grid of the recess in a preset condition. The foot display of the free leg is corrected so that the trajectory becomes the foot display of the free leg (and thus the estimated position and posture of the foot center of the free leg in the robot 3 is corrected). Foot display information indicating foot display is displayed on the display unit 7. Thereby, when the robot 3 walks, it is possible to suppress the foot of the robot 3 from landing in the recess.

  At this time, the display unit 7 displays the foot display information of the supporting leg in the robot 3 and the foot display of the free leg immediately after leaving the robot 3 in addition to the corrected foot display information and the surrounding environment information of the corrected leg. Either the information or the foot display information of the free leg before correction may be selectively displayed. In addition, as shown in FIG. 21B, it is preferable to display the sampling points that protrude into the recesses. Thereby, the operator can visually recognize how the foot protrudes into the recess.

<Embodiment 7>
In addition to the display information of the first to sixth embodiments, the posture information indicating the waist posture of the robot 3 (that is, the rotation angle around the z axis) and the position information indicating the position of the target ZMP are displayed on the display unit 7. Is preferred. Here, FIG. 22 is a diagram schematically showing the load acting on the foot of the left leg and the load acting on the foot of the right leg in the operator. FIG. 23 is a diagram illustrating a display example of the posture of the robot waist and the position of the target ZMP.

  The master calculation unit 6 sets the ratio of the load acting on the foot of the left leg and the load acting on the foot of the right leg, and the posture of the waist of the robot 3 and the position of the target ZMP. Based on the relationship and the ratio of the load acting on the left leg foot and the load acting on the right leg foot of the detected operator, the waist posture of the robot 3 and the position of the target ZMP are estimated ( And the position information indicating the estimated posture of the robot 3 and the position of the target ZMP are displayed on the display unit 7.

  Specifically, the master calculation unit 6 determines that the posture of the waist of the robot 3 is the left leg of the robot 3 when the load acting on the foot of the left leg of the operator is equal to the load acting on the foot of the right leg. It is estimated that the target ZMP is arranged at the center of the line connecting the foot centers of the left and right feet, in the middle of the direction of the right leg and the direction of the right leg.

  Then, as shown in FIG. 23, the master calculation unit 6 increases the robot acting as shown in FIG. 23 as the load acting on the foot of the right leg increases with respect to the load acting on the foot of the left leg of the operator. The posture of the waist of the robot 3 and the position of the target ZMP are estimated so that the posture of the waist of 3 approaches the direction in which the right leg faces and the target ZMP moves to the right on the line connecting the foot centers of the left and right legs. To do.

  Further, the master calculation unit 6 indicates that the posture of the waist of the robot 3 is directed toward the left leg as the load acting on the left leg foot increases with respect to the load acting on the right leg foot of the operator. The waist posture of the robot 3 and the position of the target ZMP are estimated so that the position of the target ZMP moves to the left on the line connecting the foot centers of the left and right legs.

  Thereby, the operator can visually recognize the posture of the waist of the robot 3 and the position of the target ZMP in relation to the foot display of the supporting leg and the foot display of the free leg.

  The embodiment of the present invention has been described above. However, the present invention is not limited to the above, and can be changed without departing from the technical idea of the present invention.

  For example, Embodiments 1 to 7 can be implemented in combination.

  For example, in Embodiments 1 to 7, the foot display is generated based on the foot center of the leg of the operator and the foot center of the leg of the robot 3, but the reference position is not particularly limited, and the foot tip is not particularly limited. But you can.

  For example, in Embodiments 1 to 7, the robot 3 is used as a slave, but a character (for example, a virtual robot) that walks on two legs in an image may be a slave. In this case, the surrounding environment information indicating the surrounding environment of the character set in advance may be used as the surrounding environment information of the character.

  For example, in Embodiments 1 to 7, the master calculation unit 6 displays the foot display when the free leg has landed at the estimated relative position and relative position of the foot of the supporting leg and the foot of the free leg. The generated foot display information indicating the foot display of the free leg is superimposed on the surrounding environment information indicating the surrounding environment of the robot 3 detected by the detection unit 9 mounted on the robot 3. This is not the case. The display unit 7 generates a foot display when the free leg is landed at the estimated relative position and relative position of the foot of the supporting leg and the foot of the free leg, and the foot display of the generated free leg May be superimposed on the surrounding environment information indicating the surrounding environment of the robot 3 detected by the detection unit 9 mounted on the robot 3.

  For example, as an example of using the optical motion capture as the motion capture method of the first detection unit, another motion capture method such as a mechanical method or a magnetic method may be used.

  For example, the second detection unit, for example, applies the force applied to the force sensor from each of the right leg and the left leg of the operator to the right leg and the left leg by the operator wearing shoes equipped with the force sensor. You may make it detect as a load which each acts. Then, each of the right leg and left leg force sensors may generate load information indicating the detected force and output the load information to the master calculation unit 6.

  For example, the form of communication between the master calculation unit 6 and the slave calculation unit 10 may be wired or wireless. That is, each of the master calculation unit 6 and the robot 3 includes a communication module corresponding to the employed communication method of wired and wireless, and performs communication via the communication module. Therefore, the description is omitted.

1 Remote control system 2 Master device 3 Biped robot (robot)
4 First detection unit, 4a Motion capture device 5 Second detection unit, 5a Force plate 6 Master calculation unit 7 Display unit 8 Marker 9 Detection unit 10 Slave calculation unit

Claims (9)

A remote control system in which a real or virtual robot performs a biped walking operation following a biped walking operation after a predetermined delay in accordance with the biped walking operation of an operator,
A first detection unit for detecting a position and posture of a foot of a free leg with respect to a foot of a support leg in the operator;
The leg of the free leg with respect to the foot of the support leg in the robot corresponding to the position and posture of the foot of the free leg with respect to the foot of the support leg of the operator detected by the detection information of the first detection unit. An estimation unit for estimating the position and orientation of the plane;
Based on the position and posture of the free leg's foot with respect to the foot of the supporting leg in the robot estimated by the estimating unit, a free leg foot display representing the free leg's foot of the robot is generated, The superimposed information obtained by superimposing the free leg foot display information indicating the leg foot display and the surrounding environment information of the robot is preceded by the time when the foot of the free leg reaches their position and posture in the robot, A display unit to display;
A remote control system comprising:   The estimation unit calculates a first movable area of the free leg in the robot according to an elapsed time of leaving the free leg of the operator, and the free leg foot display is displayed in the first movable area. 2. The remote control system according to claim 1, wherein, if not arranged, the free leg foot display is modified so that the free leg foot display is arranged in the first movable region.   The remote control system according to claim 2, wherein the display unit displays first movable area information indicating the first movable area.   The estimation unit calculates a second movable area of the free leg in the robot according to a preset stride of the robot, and the free leg foot display is arranged in the second movable area. The remote leg display according to any one of claims 1 to 3, wherein if not, the free leg foot display is corrected so that the free leg foot display is arranged in the second movable region. Control system.   The remote control system according to claim 4, wherein the display unit displays second movable area information indicating the second movable area.   When the free leg foot display interferes with a self-interference area determined based on the position and posture of a support leg in the robot, the estimation unit interferes with the self-interference area. The remote control system according to any one of claims 1 to 5, wherein the free leg foot display is corrected so as not to occur.   The estimation unit determines a non-landing area based on the surrounding environment information of the robot, and when the free leg foot display interferes with the non-landing area under a preset condition, The remote control system according to any one of claims 1 to 6, wherein the free leg foot display is corrected so that the free leg foot display does not interfere with the non-landing area under the condition.   The remote control system according to claim 7, wherein the display unit displays non-implantable area information indicating the non-implantable area. A second detection unit for detecting a load acting on the right leg foot and a load acting on the left leg foot of the operator, respectively;
The estimating unit estimates a posture of the waist of the robot and a position of a target ZMP based on a ratio of a load acting on the right leg foot and a load acting on the left leg foot;
The remote control system according to claim 1, wherein the display unit displays posture information indicating a posture of the waist of the robot and position information indicating a position of a target ZMP.

Images