WO2024019107A1

WO2024019107A1 - Multiple-robot control method, multiple-robot control device, and multiple-robot control system

Info

Publication number: WO2024019107A1
Application number: PCT/JP2023/026552
Authority: WO
Inventors: 弘樹西條; 慶直袖山; 栄良笠井; 亘小久保
Original assignee: ソニーグループ株式会社
Priority date: 2022-07-22
Filing date: 2023-07-20
Publication date: 2024-01-25

Abstract

[Problem] To provide a multiple-robot control method that enables a single operator to control multiple robots. [Solution] A multiple-robot control method according to the present disclosure comprises: a step for receiving an operational command for multiple robots from an operator; a step for, by using an environment model and multiple robot models corresponding to the respective robots, causing the multiple robot models to operate faster than in real time within the environment model on the basis of the operational command, and acquiring predictive observation values obtained as a result; a step for displaying the predictive observation values toward the operator; a step for accumulating task information that contains operational commands and predictive observation values; and a step for transmitting the accumulated task information to the multiple robots.

Description

Multiple robot control method, multiple robot control device, and multiple robot control system

The present disclosure relates to a method for controlling multiple robots, a control device for multiple robots, and a control system for multiple robots.

In recent years, robots have been increasingly utilized in service fields such as housework, nursing care, and assistance. In such fields, multiple robots are often operated simultaneously within one work environment. Furthermore, in situations where it is difficult for a robot to autonomously perform all tasks, a robot that is remotely controlled by an operator is often utilized (see, for example, Patent Document 1). However, a control system in which one robot occupies one operator is not cost effective.

Japanese Patent Application Publication No. 9-254065

The present disclosure is intended to solve the above problems, and provides a method for controlling multiple robots, a control device for multiple robots, and a control device for multiple robots, in which multiple robots can be controlled by one operator. The purpose is to provide a system.

In order to solve the above problems, a method for controlling multiple robots according to the present disclosure includes the steps of receiving operation commands from an operator to multiple robots, and controlling multiple robot models and environment models corresponding to the multiple robots. using the robot model to operate multiple robot models in an environment model faster than in real time based on movement instructions, obtaining predicted observed values as a result, and directing the predicted observed values to an operator. The method includes displaying, accumulating task information including motion instructions and predicted observed values, and transmitting the accumulated task information to a plurality of robots.

Further, a control device for multiple robots according to the present disclosure includes an input unit that receives operation commands from an operator to the multiple robots, and multiple robot models and environment models corresponding to the multiple robots, and includes a plurality of robot models and an environment model corresponding to the multiple robots, and A simulation unit that operates multiple robot models in an environment model faster than real time and outputs the resulting predicted observed values, a display unit that displays the predicted observed values to the operator, and a movement command. and a storage unit that stores task information including predicted observed values, and a transmission unit that transmits the task information stored in the storage unit to a plurality of robots.

Further, a control system for a plurality of robots according to the present disclosure includes a plurality of robots and a control device for the plurality of robots, and the control device includes an input unit that receives operation commands from an operator to the plurality of robots; Contains multiple robot models and environment models corresponding to multiple robots, based on motion instructions, simulates the motion of multiple robot models within the environment model faster than real time, and the resulting predicted observed values a simulation unit that outputs predicted observed values; a display unit that displays predicted observed values to the operator; a first storage unit that accumulates task information including operation commands and predicted observed values; a transmitter that transmits task information to a plurality of robots, and each of the plurality of robots includes a receiver that receives task information from a control device, and a second receiver that accumulates task information received by the receiver. The storage unit includes a storage unit, and a control unit that operates the device in a real environment according to operation instructions included in the task information stored in the second storage unit.

1 is a diagram showing the configuration of a control system for multiple robots according to Embodiment 1. FIG. 1 is a diagram showing detailed configurations of a control device and a robot according to Embodiment 1. FIG. FIG. 3 is a diagram showing a logical configuration of a simulator and a storage unit according to the first embodiment. FIG. 3 is a diagram showing a logical configuration of a robot and a storage unit according to the first embodiment. 3 is a flowchart illustrating the operation of the control system for a plurality of robots according to the first embodiment. It is a time chart for transmitting and executing task information from a control device to a robot. It is a figure which shows the definition example of environmental complexity. FIG. 2 is a diagram showing a configuration in which a simulator and a robot correspond to each other on a one-to-one basis. FIG. 2 is a diagram showing a configuration in which a simulator and a robot correspond to each other in a 1:N ratio. FIG. 3 is a diagram illustrating an example of defining parameters of an autonomous driving robot. It is a figure which shows the calculation example of the maximum value of the shortening rate of a self-driving robot.

Embodiments of the present disclosure will be described below with reference to the drawings. In the drawings, the same or corresponding elements are given the same reference numerals, and detailed descriptions are omitted as appropriate.

(Embodiment 1)
FIG. 1 is a diagram showing the configuration of a control system 100 for multiple robots according to Embodiment 1 of the present disclosure. The control system 100 includes a control device 10 operated by an operator 1 and two

robots

20a and 20b controlled by the control device 10. The control device 10 is located at the operation base B. The robot 20a is placed in a working environment W1, and the robot 20b is placed in a working environment W2. Normally, the operation base B and the work environments W1 and W2 are assumed to be located at separate locations, but they may be located at the same location.

The control device 10 and the robot 20a can communicate with each other via wireless communication. Similarly, the control device 10 and the robot 20b can communicate with each other via wireless communication. Operator 1 can control

robots

20a and 20b by operating control device 10. In other words, the operator 1 can control the

robots

20a and 20b via the control device 10.

Although the types of

robots

20a and 20b are not particularly limited, one example is a manipulator robot used for domestic housework support. Note that the number of robots controlled by the control device 10 is not limited to two, and may be three or more. Furthermore, multiple robots may exist in one work environment.

(Configuration of control device)
FIG. 2 is a diagram showing the detailed configuration of the control device 10 and the

robots

20a and 20b. The control device 10 includes an input section 11, a simulation section 12, a display section 13,

storage sections

14a and 14b, a transmission section 15, and a reception section 16.

The input unit 11 is composed of, for example, a keyboard or a mouse, and receives operation commands from the operator 1 to the

robots

20a and 20b. Specific examples of operation commands include forward movement, backward movement, direction change, acceleration, and deceleration of the robot.

The simulation unit 12 is configured by, for example, a personal computer. The simulation unit 12 simulates the movements of the

robots

20a and 20b based on the movement commands for the

robots

20a and 20b received from the operator 1 via the input unit 11. Specifically, the simulation unit 12 includes a simulator 12a that simulates the operation of the robot 20a, and a simulator 12b that simulates the operation of the robot 20b.

FIG. 3 is a diagram showing the logical configuration of

simulators

12a and 12b and

storage units

14a and 14b, which will be described later. The simulator 12a includes a robot model 121a corresponding to the robot 20a and an environment model 122a corresponding to the work environment W1. Similarly, the simulator 12b includes a robot model 121b corresponding to the robot 20b and an environment model 122b corresponding to the work environment W2.

The simulator 12a operates the robot model 121b within the environment model 122a at a higher speed than in real time based on the operation command from the operator 1, and outputs the predicted observation value obtained as a result. Specific examples of predicted observed values include information on the internal world such as the position, velocity, and acceleration of the robot model 121a, and information on the external world such as the position, velocity, and acceleration of obstacles that exist around the robot model 121a. be.

Similarly, the simulator 12b operates the robot model 121b within the environment model 122b at a higher speed than in real time based on the operation command from the operator 1, and outputs the resulting predicted observation value. Specific examples of predicted observed values include information on the internal world such as the position, velocity, and acceleration of the robot model 121b, and information on the external world such as the position, velocity, and acceleration of obstacles that exist around the robot model 121b. be.

Within the

environment models

122a and 122b, the

robot models

121a and 121b operate faster than in real time. In detail, let Treal be the real time, that is, the time that actually elapses in the real environment, and Tsim be the simulation time that elapses within the environment model, and the following relationship holds between them.

Shortening rate Kscale=Tsim/Treal

In the above formula, the shortening rate Kscale is a real number of 1 or more. For example, when the reduction rate Kscale=10, when 100 seconds of simulation time has elapsed in the environment model, only 10 seconds have elapsed in real time.

Returning to FIG. 2, the display section 13 is composed of, for example, a liquid crystal display. The display unit 13 displays for the operator 1 the predicted observed values corresponding to the robot 20a output from the simulator 12a and the predicted observed values corresponding to the robot 20b outputted from the simulator 12b. Since the time experienced by the operator 1 is real time, for example, when the reduction rate Kscale=10, the operator 1 can experience the simulation time that passes within the environment model in 1/10 of the time. Therefore, in principle, the operator 1 can simultaneously operate up to 10 robot models within the environment model.

The

storage units

14a and 14b are configured by, for example, a memory device or the like. The storage unit 14a includes operation commands from the operator 1, predicted observed values obtained as a result of operating the robot model 121a within the environment model 122a based on the operation commands, and time information within the environment model 122a. , accumulates task information corresponding to the robot 20a (see FIG. 3). Similarly, the storage unit 14b stores operation commands from the operator 1, predicted observed values obtained as a result of operating the robot model 121b within the environment model 122b based on the operation commands, and time information within the environment model 122b. (See FIG. 3).

In FIG. 3, the task information stored first in the storage unit 14a is the task information at time Tst in the environment model 122a. The second stored task information is the task information at time Tst+ΔT in the environment model 122a. The Nth accumulated task information is the task information at time Tst+NΔT in the environment model 122a. Here, ΔT is the time that passes within the environment model 122a, that is, the simulation time.

Similarly, the task information stored first in the storage unit 14b is the task information at time Tst in the environment model 122b. The second stored task information is the task information at time Tst+ΔT in the environment model 122b. The Nth accumulated task information is the task information at time Tst+NΔT in the environment model 122b. Here, ΔT is the time elapsed within the environment model 122b, that is, the simulation time.

Returning to FIG. 2, the transmitting unit 15 is configured by, for example, a wireless communication interface, and transmits the task information stored in the storage unit 14a all at once to the corresponding robot 20a. Further, the transmitter 15 transmits the task information stored in the storage section 14b to the corresponding robot 20b all at once.

The receiving unit 16 is configured by, for example, a wireless communication interface, and receives abnormality detection information, which will be described later, from the

robot

20a or 20b. The display unit 13 displays the abnormality detection information received by the receiving unit 16 for the operator 1 .

(Robot configuration)
Next, the configurations of the

robots

20a and 20b will be explained. However, since the configurations of the

robots

20a and 20b are the same, only the robot 20a will be described here. The robot 20a includes a receiving section 21a, a storage section 22a, a control section 23a, a traveling motor 24a, an internal sensor 25a, an external sensor 26a, and an abnormality detecting section 27a.

The receiving unit 21a is configured by, for example, a wireless communication interface, and receives the task information transmitted from the control device 10 all at once. This task information has been stored in the storage section 14a of the control device 10.

The storage unit 22a is configured by, for example, a memory device, and stores the task information collectively received by the reception unit 21a. FIG. 4 is a diagram showing the logical configuration of

robots

20a and 20b and their

storage units

22a and 22b in a working environment, that is, a real environment.

In FIG. 4, the task information stored first in the storage unit 22a is the task information at time Tst in the actual environment. The second stored task information is the task information at time Tst+Δτ in the actual environment. The Nth accumulated task information is the task information at time Tst+NΔτ in the actual environment. Here, the time Tst in the actual environment and the time Tst in the environment model are equal. Further, Δτ is the time that passes in the real environment, that is, real time, and the relationship Δτ=Ksacle·ΔT holds true.

Returning to FIG. 2, the control unit 23a is configured by, for example, a microcomputer. The control unit 23a reads the task information stored in the storage unit 22a, and controls the travel motor 24a, etc. according to the operation command included in the task information, thereby adjusting the operation of the robot 20a in the real environment, that is, the work environment W1. (see Figure 4). Specific examples of the movements of the robot 20a include forward movement, backward movement, direction change, acceleration, and deceleration of the robot 20a.

Furthermore, the control unit 23a acquires actual observed values obtained as a result of the operation of the robot 20a via the internal world sensor 25a and the external world sensor 26a. Specific examples of actual observed values include information on the internal world such as the position, velocity, and acceleration of the robot 20a, and information on the external world such as the position, velocity, and acceleration of obstacles existing around the robot 20a.

The abnormality detection unit 27a is configured by, for example, a microcomputer. The anomaly detection unit 27a determines the predicted observed value and the actual observed value based on the comparison between the predicted observed value included in the task information and the actual observed value acquired via the internal sensor 25a and the external sensor 26a. Detects a deviation, that is, an abnormality (see FIG. 4). For example, if the difference between the position and speed of the surrounding obstacles in the predicted observed values and the position and speed of the surrounding obstacles in the actual observed values is greater than or equal to a predetermined threshold, it is determined that there is an abnormality. When detecting an abnormality, the abnormality detection unit 27a transmits abnormality detection information including the details of the abnormality to the control device 10 via wireless communication.

(Operation of control system)
Next, the operation of the control system 100 for multiple robots according to the first embodiment will be described with reference to the flowchart in FIG. 5. The operation of the control system 100 is divided into an operation phase of steps S11 to S16 executed by the control device 10 and an execution phase of steps S21 to S27 executed by the

robots

20a and 20b.

In step S11 of the execution phase, the input unit 11 of the control device 10 receives one operation command from the operator 1 to the robot 20a. Furthermore, the input unit 11 of the control device 10 receives one operation command from the operator 1 to the robot 20b.

In step S12, the simulation unit 12 of the control device 10 simulates the movements of the

robots

20a and 20b faster than in real time, based on the movement command received in step S11 above. In detail, the simulator 12a operates the robot model 121a within the environment model 122a at a higher speed than in real time based on one operation command, and outputs one predicted observation value obtained as a result. Similarly, the simulator 12b operates the robot model 121b within the environment model 122b at a higher speed than in real time based on one operation instruction, and outputs one predicted observed value obtained as a result.

In step S13, the display unit 13 of the control device 10 displays one predicted observation value corresponding to the robot 20a and one predicted observation value corresponding to the robot 20b output in step S12 above to the operator 1. indicate.

In step S14, the storage unit 14a of the control device 10 receives the one motion command for the robot 20a received in step S11 above and the robot 20a output in step S12 as a result of the one motion command. One piece of task information including one corresponding predicted observed value and time information in the environment model 122a is accumulated. Similarly, the storage unit 14b of the control device 10 corresponds to one motion command to the robot 20b received in step S11 above and the robot 20b outputted in step S12 above as a result of the one motion command. One piece of task information is accumulated, including one predicted observation value to be performed and time information in the environment model 122b.

In step S15, the input unit 11 of the control device 10 determines whether or not an operation command is subsequently received from the operator 1. If the operation command is subsequently received from the operator 1 (S15=NO), the process returns to step S11, and if not (S15=YES), the process proceeds to step S16.

In step S16, the transmitter 15 of the control device 10 transmits the task information stored in the storage unit 14a all at once to the corresponding robot 20a. Furthermore, the transmitter 15 of the control device 10 transmits the task information stored in the storage section 14b to the corresponding robot 20b all at once. Up to this point, the operation phase executed by the control device 10 has ended, and the process moves to the execution phase executed by the

robots

20a and 20b.

In step S21 of the execution phase, the receiving unit 21a of the robot 20a receives task information from the control device 10 all at once. Similarly, the receiving unit 21b of the robot 20b receives task information all at once from the control device 10.

In step S22, the storage unit 22a of the robot 20a stores the task information collectively received in step S21 above. At this time, time information Tst+nΔT in the environment model 122a included in the task information is converted to time information Tst+nΔτ of the actual environment. However, n is a natural number from 0 to N, and the relationship Δτ=Kscale·ΔT holds true.

Similarly, the storage unit 22b of the robot 20b stores the task information received all at once in step S21 above. At this time, time information Tst+nΔT in the environment model 122b included in the task information is converted to time information Tst+nΔτ of the actual environment. However, n is a natural number from 0 to N, and the relationship Δτ=Kscale·ΔT holds true.

In step S23, the control unit 23a of the robot 20a retrieves one piece of task information from the storage unit 22a, and controls the traveling motor 24a and the like according to one operation command included in the one task information, thereby controlling the real environment. That is, the operation of the robot 20a in the work environment W1 is controlled. Specific examples of the movements of the robot 20a include forward movement, backward movement, direction change, acceleration, and deceleration of the robot 20a.

Similarly, the control unit 23b of the robot 20b retrieves one piece of task information from the storage unit 22b, and controls the traveling motor 24b etc. according to one operation command included in the one task information, thereby controlling the real environment, i.e. The operation of the robot 20b in the work environment W2 is controlled. Specific examples of the operations of the robot 20b include forward movement, backward movement, direction change, acceleration, deceleration, etc. of the robot 20b.

In step S24, the control unit 23a of the robot 20a acquires one actual observation value obtained as a result of the operation in step S23 above, via the internal world sensor 25a and the external world sensor 26a. A specific example of one actual observation value is information on the inner world such as the position, velocity, and acceleration of the robot 20a, and information on the outside world such as the position, velocity, and acceleration of obstacles that exist around the robot 20a. be.

Similarly, the control unit 23b of the robot 20b obtains one actual observation value obtained as a result of the operation in step S23 above via the internal world sensor 25b and the external world sensor 26b. A specific example of one actual observation value is information on the internal world such as the position, velocity, and acceleration of the robot 20b, and information on the external world such as the position, velocity, and acceleration of obstacles that exist around the robot 20b. be.

In step S25, the abnormality detection unit 27a of the robot 20a detects one predicted observation value included in one piece of task information extracted in step S23 above and one actual observation value acquired in step S24 above. Based on the comparison, the deviation between the predicted observed value and the actual observed value, that is, the presence or absence of an abnormality, is determined. For example, if the difference between the position and speed of the surrounding obstacles in the predicted observed values and the position and speed of the surrounding obstacles in the actual observed values is greater than or equal to a predetermined threshold, it is determined that there is an abnormality.

If an abnormality is detected (S25=YES), the abnormality detection unit 27a transmits abnormality detection information including the details of the abnormality to the control device 10 via wireless communication (step S27), and Control ends (END). After this, control of the robot 20a shifts to real-time control by the operator 1. Thereby, the operator 1 can control the obstacle avoidance operation in real time, for example. On the other hand, if no abnormality is detected (S25=NO), the process advances to step S26.

Similarly, the abnormality detection unit 27b of the robot 20b detects one predicted observation value included in one task information extracted in step S23 above and one actual observation value acquired in step S24 above. Based on the comparison, the deviation between the predicted observed value and the actual observed value, that is, the presence or absence of an abnormality, is determined. For example, if the difference between the position and speed of the surrounding obstacles in the predicted observed values and the position and speed of the surrounding obstacles in the actual observed values is greater than or equal to a predetermined threshold, it is determined that there is an abnormality.

If an abnormality is detected (S25=YES), the abnormality detection unit 27b transmits abnormality detection information including the details of the abnormality to the control device 10 via wireless communication (step S27), and Control ends (END). After this, control of the robot 20b shifts to real-time control by the operator 1. Thereby, the operator 1 can control the obstacle avoidance operation in real time, for example. On the other hand, if no abnormality is detected (S25=NO), the process advances to step S26.

In step S26, the control unit 23a of the robot 20a determines whether task information remains in the storage unit 22a. If the task information remains in the storage unit 22a (S26=NO), the process returns to step S23, and if the storage unit 22a is empty (S26=YES), the control of the robot 20a ends ( END).

Similarly, the control unit 23b of the robot 20b determines whether task information remains in the storage unit 22b. If the task information remains in the storage unit 22b (S26=NO), the process returns to step S23, and if the storage unit 22b is empty (S26=YES), the control of the robot 20b ends ( END).

As described above, the control system for multiple robots according to the first embodiment includes multiple robot models and environment models corresponding to the multiple robots. The control device causes the plurality of robot models to operate within the environment model at a higher speed than in real time based on the movement commands for the plurality of robots received from the operator, and obtains predicted observed values obtained as a result. . The control device accumulates task information including operation commands and predicted observed values, and transmits the accumulated task information to a plurality of robots. Due to these features, the control system according to the first embodiment allows one operator to control a plurality of robots.

Each robot accumulates task information received from the control device and operates in the real environment according to operation instructions included in the accumulated task information. As a result, compared to sequentially transmitting task information from the control device to each robot, the influence of communication delays between the control device and the robots can be minimized.

The upper time chart in FIG. 6 is a time chart for collectively transmitting and executing task information accumulated by the control device to the robot in the control system according to the first embodiment. The lower time chart in FIG. 6 is a time chart in which task information is sequentially transmitted and executed from the control device to the robot, as seen in the prior art. As is clear from the comparison between the two, in the control system according to the first embodiment, the influence of communication delay between the control device and the robot can be minimized.

Each robot acquires actual observed values obtained as a result of operating in a real environment, and detects anomalies based on a comparison between the predicted observed values and the actual observed values. This allows each robot to detect deviations between predicted observed values and actual observed values, that is, abnormalities, in real time.

If an abnormality is detected, control of the robot where the abnormality has been detected shifts to real-time control by the operator. This allows the operator to control the robot in real time when an abnormality is detected.

The relationship of shortening rate Kscale=Tsim/Treal holds between the real time Treal experienced by the operator and the simulation time Tsim that elapses within the environment model, and the shortening rate Kscale is a real number of 1 or more. This allows the operator to experience the simulation time that passes within the environment model in a time of 1/K scale. For example, when the reduction rate Kscale=10, the operator's restraint time required to operate one robot model within the environment model for 100 seconds is 100 seconds/10=10 seconds. In other words, one operator can operate up to 10 robot models simultaneously. This allows one operator to efficiently control a plurality of robots.

The reduction rate Kscale may be determined based on the frequency of occurrence of disturbance elements present in the robot's working environment, that is, the actual environment. For example, an obstacle that has a relatively high possibility of causing a deviation between the predicted observed value and the actual observed value is a disturbance element that occurs frequently. If such disturbance elements that occur frequently exist, by setting the reduction rate Kscale small, it is possible to increase the time from when an abnormality is detected to when the control shifts to real-time control and the operator takes action. be able to.

The shortening rate Kscale may be determined based on the spatial density of disturbance elements present in the robot's working environment, that is, the actual environment. For example, a work environment in which a relatively large number of obstacles are present is an environment with a high spatial density of disturbance elements. When the spatial density of disturbance elements is high in this way, by setting the shortening rate Kscale small, it is possible to increase the time margin from when an abnormality is detected to when the control shifts to real-time control and the operator takes action. can.

The reduction rate Kscale may be determined based on the difficulty of dealing with disturbance elements present in the robot's work environment, that is, the actual environment. For example, an obstacle whose speed changes relatively rapidly is a disturbance element that is difficult to deal with. If such a disturbance element that is difficult to deal with exists, by setting the reduction rate Kscale small, the time margin from when an abnormality is detected until real-time control is started and the operator takes action can be increased. You can take it.

The reduction rate Kscale may be determined based on a combination of the frequency of occurrence, spatial density, and difficulty of dealing with disturbance elements present in the robot's working environment, that is, the actual environment. For example, as shown in FIG. 7, the environmental complexity may be defined based on a combination of the spatial density of disturbance elements and the difficulty of handling, and the shortening rate Kscale may be determined based on the environmental complexity.

If the robot is a household robot (home mobile manipulator) operated in a private home, the reduction rate Kscale may be determined based on the time attribute of the private home. Specific examples of attributes of time at a private home include parent and child going out (daytime on weekdays), child going out (daytime on holiday), and parent and child staying home (nighttime), and the reduction rate Kscale is set to be small in accordance with this order.

If the robot is a cart-type robot (cart-type mobile manipulator) operated in a factory or warehouse, the reduction rate Kscale may be determined based on the attribute of the type of factory or warehouse. Specific examples of attributes of the type of factory or warehouse include unmanned factory, unmanned warehouse, manned factory, and manned warehouse, and the reduction rate Kscale is set to be small in this order.

If the robot is a store robot (store mobile manipulator) operated in a retail store, the reduction rate Kscale may be determined based on the time attribute of the retail store. Specific examples of the time attributes of a retail store include non-business hours, business hours (off-season hours), business hours (busy hours), etc., and the reduction rate Kscale is set to be smaller in accordance with this order.

If the robot is a self-driving robot that travels on a road, the reduction rate Kscale may be determined based on the attributes of the type of road that the robot travels on. Specific examples of road type attributes include expressways, arterial roads, city roads, private roads, parking lots, etc., and the reduction rate Kscale is set to be small in this order.

(Modified example)
In the first embodiment described above, the two

simulators

12a and 12b simulate the operations of the two

robots

20a and 20b, respectively. In other words, as shown in FIG. 8, there was a one-to-one correspondence between the simulator and the robot. However, one simulator may simulate the movements of multiple robots. In other words, the simulator and the robot may correspond to each other in a 1:N ratio. In the example of FIG. 9, the operations of two

robots

20a and 20b placed in the same working environment W1 are simulated by one simulator 12a. This makes it possible to save computational resources required for simulation.

(formal expression)
In order to consider the maximum value of the shortening rate Kscale, we will formally express the idea of considering the operator's reaction speed limit. Since there is a limit to human reaction speed, there is a limit to the simulation speed reduction rate Kscale beyond which the operator cannot fully react.

As mentioned above, the shortening rate Kscale can be expressed as follows using the real time Treal that actually elapses in the real environment and the simulation time Tsim that elapses within the environment model.

In addition, in order to ensure that the operation of the robot is not hindered, the operator can recover from or avoid the abnormality by detecting and predicting the abnormality and then performing appropriate operations on the robot. There needs to be. This can be expressed as follows, where Dthr is the range in which a certain object (person/object) can cause significant disturbance to the robot.

However, in the above equation, the positions of the robot and object at time T1 when the abnormality is predicted are respectively PfollowerT1 and PojectT1, the positions of the robot and object at subsequent time t are Pfollower(t) and Poject(t), and the speed is are expressed as Vfollower(Oopt(t), t) and Voject(t). Moreover, Oopt(t) indicates the appropriate operation content when the operator deals with an abnormality.

In reality, it is not possible to start an appropriate operation immediately after detecting the object; instead, the appropriate operation must be started after a delay time Td, and there is a difference between simulation time and real time based on equation (1). Therefore, increasing the shortening rate Kscale as much as possible without interfering with robot operation comes down to the following problem.

(Specific example: self-driving robot)
In a self-driving robot in which the operator controls the speed of the robot, as shown in Figure 10, parameters such as the width of the safety area (safety area boundary), the operator's field of view, the operator's avoidance maneuver, and the operator's reaction speed are controlled. defined.

By using the parameters shown in FIG. 10, the above equations (3) and (4) can be specifically written as follows.

Each value of the parameters in FIG. 10 can be set based on the attribute of the road type, for example, expressway, main road, city road, etc. Further, it is assumed that the object is a moving truck, and a collision with an oncoming vehicle (object) is abnormal. At this time, if the operator does not perform any operation to change the direction of travel until he notices the abnormality, Omax is a positive constant. Furthermore, if the oncoming vehicle is in front of the robot (car) to be operated, then P'follower (T, Kscale) < Object (T), and the constraint equation can be transformed as follows.

Here, when the indefinite integral of Oopt(t) is set as Sopt(t), it is decomposed into the form of a definite integral as follows.

Here, it is assumed that the operator moves straight toward the object, that is, does not change the direction of movement until the operator performs the optimal operation, so Sopt (T1 + KTd) = Omax (T1 + KTd), The following relationship holds true.

Applying this relationship to the above constraint formula results in the following.

In particular, when T1=0, the following results.

Here, assuming that Dthr=2.0m, Voject_max=Omax=maximum travel speed, and Td=0.1 seconds, G2(T) is as follows.

Since time T is a time later than T1 and T1=0, the range is T≧T1=0. Therefore, from equation (5), the maximum value that K can take is G2(0).

When G2(0) is calculated for expressways, arterial roads, and roads in urban areas, it becomes as shown in FIG. 11.

Although several embodiments have been described, these embodiments are presented as examples and are not intended to limit the scope of the embodiments. Various omissions, substitutions, changes, and combinations can be made without departing from the gist of the embodiments. These embodiments and their modifications are included within the scope and gist of the embodiments as well as within the scope of the claims and their equivalents.

Note that the technology according to the present disclosure is not limited to a specific standard, and the illustrated settings may be changed as appropriate. Note that each of the embodiments described above shows an example for embodying the present disclosure, and the present disclosure can be implemented in various other forms. For example, various modifications, substitutions, omissions, combinations, etc. are possible without departing from the gist of the present disclosure. Forms in which such modifications, substitutions, omissions, or combinations are made are also included within the scope of the present disclosure, as well as within the scope of the invention described in the claims and its equivalents.

Furthermore, the processing procedure described in the present disclosure may be regarded as a method having a series of these procedures. Alternatively, it may be regarded as a program for causing a computer to execute these series of procedures, or a recording medium that stores the program. Further, the processing described above is executed by a processor such as a CPU of a computer. Furthermore, the type of recording medium is not particularly limited because it does not affect the embodiments of the present disclosure.

Note that each component shown in the present disclosure may be realized by software or hardware. For example, each component may be a software module implemented by software such as a microprogram, and each component may be implemented by a processor executing the software module. Alternatively, each component may be realized by a circuit block on a semiconductor chip (die), for example, an integrated circuit such as an ASIC (Application Specific Integrated Circuit) or an FPGA (Field Programmable Gate Array). Further, the number of constituent elements and the number of hardware realizing the constituent elements do not need to match. For example, one processor or circuit may implement multiple components. Conversely, one component may be implemented by multiple processors or circuits.

Note that the types of processors described in this disclosure are not limited. For example, it may be a CPU, an MPU (Micro Processing Unit), a GPU (Graphics Processing Unit), or the like.

Note that the present disclosure can also have the following configuration.
[1] (How to control multiple robots)
a step (S11) of receiving operation commands from an operator to a plurality of robots;
A plurality of robot models and an environment model corresponding to the plurality of robots are used to operate the plurality of robot models within the environment model at a higher speed than in real time based on the movement command, and the resulting result is obtained. a step (S12) of obtaining predicted observed values;
a step (S13) of displaying the predicted observed value to the operator;
accumulating task information including the operation command and the predicted observation value (S14);
A method for controlling a plurality of robots, including a step (S16) of transmitting the accumulated task information to the plurality of robots.
[2]
a step of receiving the task information (S21);
accumulating the received task information (S22);
The method for controlling a plurality of robots according to [1], further comprising a step (S23) of causing the plurality of robots to operate in a real environment according to the operation instructions included in the accumulated task information.
[3]
acquiring actual observed values obtained as a result of operating the plurality of robots in the actual environment (S24);
The method for controlling a plurality of robots according to [2], further comprising a step (S25) of detecting an abnormality based on a comparison between the predicted observed value and the actual observed value.
[4]
The method for controlling a plurality of robots according to [3], further comprising, when the abnormality is detected, shifting control of the robot in which the abnormality has been detected to real-time control by the operator.
[5]
Between the real time Real experienced by the operator and the simulation time Tsim that passes within the environment model,
Shortening rate Kscale=Tsim/Treal
The method for controlling a plurality of robots according to any one of [1] to [4], wherein the following relationship holds true and the reduction rate Kscale is a real number of 1 or more.
[6]
The method for controlling a plurality of robots according to [5], wherein the shortening rate is determined based on at least one of the frequency of occurrence, spatial density, and difficulty of dealing with disturbance elements existing in the actual environment.
[7]
The method for controlling a plurality of robots according to [5], wherein the robot is a domestic robot operated in a private home, and the reduction rate is determined based on a time attribute of the private home.
[8]
The method for controlling a plurality of robots according to [5], wherein the robot is a cart type robot operated in a factory or warehouse, and the reduction rate is determined based on attributes of the type of the factory or warehouse.
[9]
The method for controlling a plurality of robots according to [5], wherein the robot is a store robot operated in a store, and the reduction rate is determined based on a time attribute of the store.
[10]
The method for controlling a plurality of robots according to [10], wherein the robot is a self-driving robot that runs on a road, and the reduction rate is determined based on attributes of the type of road.
[11] (Control device for multiple robots)
an input unit (11) that receives operation commands from the operator (1) to the plurality of robots (20a, 20b);
The robot models include a plurality of robot models (121a, 121b) and an environment model (122a, 122b) corresponding to the plurality of robots, and the plurality of robot models are moved within the environment model at a speed faster than in real time based on the operation instruction. a simulation unit (12) that operates to output predicted observed values obtained as a result;
a display unit (13) that displays the predicted observed value to the operator;
a storage unit (14a, 14b) that stores task information including the operation command and the predicted observation value;
A control device for a plurality of robots, comprising a transmitting section (15) that transmits the task information stored in the storage section to the plurality of robots.
[12]
The simulation unit includes a plurality of simulators (12a, 12b),
The control device for a plurality of robots according to [11], wherein each of the plurality of simulators simulates the operation of one or more of the plurality of robot models.
[13] (Control system for multiple robots)
including the plurality of robots (20a, 20b) and a control device (10) for the plurality of robots,
The control device includes:
an input unit (11) that receives operation commands from an operator (1) to the plurality of robots;
The robot model includes a plurality of robot models (121a, 121b) and an environment model (122a, 122b) corresponding to the plurality of robots, and the operation of the plurality of robot models is performed in real time within the environment model based on the movement command. a simulation unit (12) for simulating at high speed and outputting the predicted observed values obtained as a result;
a display unit (13) that displays the predicted observed value to the operator;
a first storage unit (14a, 14b) that stores task information including the operation command and the predicted observation value;
a transmitting unit (15) configured to transmit the task information stored in the first storage unit to the plurality of robots;
Each of the plurality of robots is
a receiving unit (21a, 21b) that receives the task information from the control device;
a second storage unit (22a, 22b) that stores the task information received by the reception unit;
a control unit (23a, 23b) for operating the machine in a real environment according to the operation command included in the task information stored in the second storage unit;
Control system for multiple robots.

1 Operator 10 Control device 11 Input section 12 Simulation section 12a Simulator 12b Simulator 12c Simulator 12d Simulator 13 Display section 14a Accumulation section 14b Accumulation section 15 Transmission section 16 Receiving section 20a Robot 20b Robot 20c Robot 20d Robot 21a Receiving section 21b Receiving section 22a Accumulation Part 22b Accumulation part 23a Control part 23b Control part 24a Travel motor 24b Travel motor 25a Internal world sensor 25b Internal world sensor 26a Outside world sensor 26b Outside world sensor 27a Abnormality detection part 27b Abnormality detection part 100 Control system 121a Robot model

121b Robot model

122a Environment model 122b Environment model B Operation base W1 Work environment W2 Work environment W3 Work environment

Claims

receiving operation instructions from an operator to the plurality of robots;
A plurality of robot models and an environment model corresponding to the plurality of robots are used to operate the plurality of robot models within the environment model at a higher speed than in real time based on the movement command, and the resulting result is obtained. obtaining predicted observed values for
displaying the predicted observed value to the operator;
accumulating task information including the action instructions and the predicted observed values;
A method for controlling a plurality of robots, the method comprising: transmitting the accumulated task information to the plurality of robots.
receiving the task information;
accumulating the received task information;
2. The method of controlling a plurality of robots according to claim 1, further comprising the step of causing the plurality of robots to operate in a real environment according to the operation instructions included in the accumulated task information.
acquiring actual observed values obtained as a result of operating the plurality of robots in the actual environment;
3. The method of controlling a plurality of robots according to claim 2, further comprising the step of detecting an abnormality based on a comparison between the predicted observed value and the actual observed value.
4. The method for controlling a plurality of robots according to claim 3, further comprising, when the abnormality is detected, shifting control of the robot in which the abnormality has been detected to real-time control by the operator.
Between the real time Real experienced by the operator and the simulation time Tsim that passes within the environment model,
Shortening rate Kscale=Tsim/Treal
2. The method of controlling a plurality of robots according to claim 1, wherein the following relationship holds true and the reduction rate Kscale is a real number of 1 or more.
6. The method for controlling a plurality of robots according to claim 5, wherein the shortening rate is determined based on at least one of the frequency of occurrence, spatial density, and difficulty of dealing with disturbance elements existing in the actual environment.
6. The method of controlling a plurality of robots according to claim 5, wherein the robot is a domestic robot operated in a private home, and the reduction rate is determined based on a time attribute of the private home.
6. The method of controlling a plurality of robots according to claim 5, wherein the robot is a cart type robot operated in a factory or warehouse, and the reduction rate is determined based on attributes of the type of the factory or warehouse.
The method of controlling a plurality of robots according to claim 5, wherein the robot is a store robot operated in a store, and the reduction rate is determined based on a time attribute of the store.
6. The method of controlling a plurality of robots according to claim 5, wherein the robot is a self-driving robot that travels on a road, and the reduction rate is determined based on attributes of the type of road.
an input unit that receives operation commands from an operator to the plurality of robots;
comprising a plurality of robot models and an environment model corresponding to the plurality of robots, the plurality of robot models are operated within the environment model at a higher speed than in real time based on the movement instruction, and predictive observation obtained as a result; a simulation section that outputs values;
a display unit that displays the predicted observed value toward the operator;
an accumulation unit that accumulates task information including the operation command and the predicted observation value;
A control device for a plurality of robots, comprising: a transmitting section that transmits the task information stored in the storage section to the plurality of robots.
The simulation unit includes a plurality of simulators,
12. The control device for a plurality of robots according to claim 11, wherein each of the plurality of simulators simulates an operation of one or more of the plurality of robot models.
including the plurality of robots and a control device for the plurality of robots,
The control device includes:
an input unit that receives operation commands from an operator to the plurality of robots;
The method includes a plurality of robot models and an environment model corresponding to the plurality of robots, and simulates the motions of the plurality of robot models in the environment model at a higher speed than in real time based on the motion instructions, and obtains a resultant result. a simulation unit that outputs predicted observed values;
a display unit that displays the predicted observed value toward the operator;
a first storage unit that stores task information including the operation command and the predicted observation value;
a transmitting unit that transmits the task information stored in the first storage unit to the plurality of robots,
Each of the plurality of robots is
a receiving unit that receives the task information from the control device;
a second storage unit that stores the task information received by the reception unit;
a control unit that operates the machine in a real environment according to the operation command included in the task information stored in the second storage unit;
Control system for multiple robots.