WO2021038842A1 - 情報処理装置、制御方法及び記憶媒体 - Google Patents

情報処理装置、制御方法及び記憶媒体 Download PDF

Info

Publication number
WO2021038842A1
WO2021038842A1 PCT/JP2019/034182 JP2019034182W WO2021038842A1 WO 2021038842 A1 WO2021038842 A1 WO 2021038842A1 JP 2019034182 W JP2019034182 W JP 2019034182W WO 2021038842 A1 WO2021038842 A1 WO 2021038842A1
Authority
WO
WIPO (PCT)
Prior art keywords
robot
target
logical expression
information processing
information
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Ceased
Application number
PCT/JP2019/034182
Other languages
English (en)
French (fr)
Japanese (ja)
Inventor
大山　博之
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
NEC Corp
Original Assignee
NEC Corp
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by NEC Corp filed Critical NEC Corp
Priority to PCT/JP2019/034182 priority Critical patent/WO2021038842A1/ja
Priority to JP2021541931A priority patent/JP7264253B2/ja
Priority to US17/636,073 priority patent/US12521881B2/en
Priority to EP19943354.1A priority patent/EP4023396A4/en
Publication of WO2021038842A1 publication Critical patent/WO2021038842A1/ja
Anticipated expiration legal-status Critical
Ceased legal-status Critical Current

Links

Images

Classifications

    • B—PERFORMING OPERATIONS; TRANSPORTING
    • B25—HAND TOOLS; PORTABLE POWER-DRIVEN TOOLS; MANIPULATORS
    • B25J—MANIPULATORS; CHAMBERS PROVIDED WITH MANIPULATION DEVICES
    • B25J9/00—Program-controlled manipulators
    • B25J9/16—Program controls
    • B25J9/1656—Program controls characterised by programming, planning systems for manipulators
    • B25J9/1661—Program controls characterised by programming, planning systems for manipulators characterised by task planning, object-oriented languages
    • B—PERFORMING OPERATIONS; TRANSPORTING
    • B25—HAND TOOLS; PORTABLE POWER-DRIVEN TOOLS; MANIPULATORS
    • B25J—MANIPULATORS; CHAMBERS PROVIDED WITH MANIPULATION DEVICES
    • B25J9/00—Program-controlled manipulators
    • B25J9/16—Program controls
    • B25J9/1694—Program controls characterised by use of sensors other than normal servo-feedback from position, speed or acceleration sensors, perception control, multi-sensor controlled systems, sensor fusion
    • B25J9/1697—Vision controlled systems
    • G—PHYSICS
    • G05—CONTROLLING; REGULATING
    • G05B—CONTROL OR REGULATING SYSTEMS IN GENERAL; FUNCTIONAL ELEMENTS OF SUCH SYSTEMS; MONITORING OR TESTING ARRANGEMENTS FOR SUCH SYSTEMS OR ELEMENTS
    • G05B2219/00—Program-control systems
    • G05B2219/30—Nc systems
    • G05B2219/40—Robotics, robotics mapping to robotics vision
    • G05B2219/40014—Gripping workpiece to place it in another place
    • G—PHYSICS
    • G05—CONTROLLING; REGULATING
    • G05B—CONTROL OR REGULATING SYSTEMS IN GENERAL; FUNCTIONAL ELEMENTS OF SUCH SYSTEMS; MONITORING OR TESTING ARRANGEMENTS FOR SUCH SYSTEMS OR ELEMENTS
    • G05B2219/00—Program-control systems
    • G05B2219/30—Nc systems
    • G05B2219/40—Robotics, robotics mapping to robotics vision
    • G05B2219/40108—Generating possible sequence of steps as function of timing and conflicts
    • G—PHYSICS
    • G05—CONTROLLING; REGULATING
    • G05B—CONTROL OR REGULATING SYSTEMS IN GENERAL; FUNCTIONAL ELEMENTS OF SUCH SYSTEMS; MONITORING OR TESTING ARRANGEMENTS FOR SUCH SYSTEMS OR ELEMENTS
    • G05B2219/00—Program-control systems
    • G05B2219/30—Nc systems
    • G05B2219/40—Robotics, robotics mapping to robotics vision
    • G05B2219/40114—From vision detected initial and user given final state, generate tasks

Definitions

  • the present invention relates to a technical field of an information processing device, a control method, and a storage medium that perform processing related to a task to be performed by a robot.
  • Patent Document 1 When a task to be made to work by a robot is given, a control method for controlling the robot necessary to execute the task has been proposed.
  • Patent Document 1 when a plurality of articles are gripped by a robot having a hand and stored in a container, a combination of the order in which the hands hold the articles is determined, and the storage is based on an index calculated for each combination.
  • a robot control device for determining the order of articles to be processed is disclosed.
  • Patent Document 1 does not disclose any method for determining the operation of the robot in consideration of the constraint conditions.
  • An object of the present invention is to provide an information processing device, a control method, and a storage medium capable of suitably generating information on robot control in consideration of constraints to be satisfied in a given task in view of the above-mentioned problems. Is the main issue.
  • One aspect of the information processing device is an information processing device, which is a logical expression conversion unit that converts a target task, which is a task to be performed by a robot, into a logical formula based on time phase logic, and should be satisfied in the execution of the target task. It has a constraint condition information acquisition unit that acquires constraint condition information indicating a constraint condition, and a constraint condition addition unit that generates a target logical expression that is a logical expression in which a proposition representing the constraint condition is added to the logical expression.
  • One aspect of the control method is a control method executed by an information processing apparatus, in which a target task, which is a task to be performed by a robot, is converted into a logical formula based on temporal logic, and restrictions to be satisfied in the execution of the target task Constraint condition information indicating a condition is acquired, and a target logical formula which is a logical formula in which a proposition representing the constraint condition is added to the logical formula is generated.
  • One aspect of the storage medium is a logical expression conversion unit that converts a target task, which is a task to be performed by a robot, into a logical expression based on time phase logic, and constraint condition information indicating a constraint condition to be satisfied in executing the target task. It is a storage medium that stores a constraint condition information acquisition unit to be acquired and a program that functions a computer as a constraint condition addition unit that generates a target logical expression that is a logical expression in which a proposition representing the constraint condition is added to the logical expression. ..
  • the configuration of the robot control system is shown.
  • the hardware configuration of the information processing device is shown.
  • An example of the data structure of application information is shown.
  • a bird's-eye view of the work space is shown. It is a functional block composition diagram of the target logical formula generation part.
  • the first display example of the task input screen is shown.
  • a second display example of the task input screen is shown.
  • This is an example of a flowchart showing the details of the process of step S13 of FIG.
  • FIG. 1 shows the configuration of the robot control system 100 according to the first embodiment.
  • the robot control system 100 mainly includes an information processing device 1, an input device 2, a display device 3, a storage device 4, a robot 5, and a measuring device 7.
  • the information processing device 1 sets the target task in a sequence for each time step (time step) of a simple task that the robot 5 can accept. Is converted, and the sequence is supplied to the robot 5.
  • a simple task (command) that can be accepted by the robot 5 is also referred to as a "subtask”.
  • the information processing device 1 is electrically connected to the input device 2, the display device 3, and the storage device 4. For example, the information processing device 1 receives an input signal "S1" for designating a target task from the input device 2. Further, the information processing device 1 transmits a display signal "S2" to the display device 3 for displaying the task to be executed by the robot 5. Further, the information processing device 1 transmits a control signal "S3" related to the control of the robot 5 to the robot 5. For example, the information processing device 1 transmits a sequence of subtasks (also referred to as “subtask sequence”) to be executed for each robot hand 52 as a control signal S3 to the robot 5. Further, the information processing device 1 receives the output signal “S4” from the measuring device 7.
  • subtask sequence also referred to as “subtask sequence”
  • the input device 2 is an interface that accepts user input, and corresponds to, for example, a touch panel, a button, a keyboard, a voice input device, and the like.
  • the input device 2 supplies the input signal S1 generated based on the user's input to the information processing device 1.
  • the display device 3 is, for example, a display, a projector, or the like, and performs a predetermined display based on the display signal S2 supplied from the information processing device 1. As will be described later, for example, the display device 3 displays an input screen (also referred to as a “task input screen”) for designating information about the target task based on the display signal S2.
  • an input screen also referred to as a “task input screen”
  • the storage device 4 has an application information storage unit 41.
  • the application information storage unit 41 stores application information necessary for generating a sequence of subtasks from a target task. Details of the application information will be described later.
  • the storage device 4 may be an external storage device such as a hard disk connected to or built in the information processing device 1, or may be a storage medium such as a flash memory. Further, the storage device 4 may be a server device that performs data communication with the information processing device 1. In this case, the storage device 4 may be composed of a plurality of server devices.
  • the robot 5 operates based on the control signal S3 transmitted from the information processing device 1.
  • the robot 5 shown in FIG. 1 has a plurality of (two) robot arms 52 capable of gripping an object as control targets, and picks and places (picks up) an object 61 existing in the work space 6. Process to move).
  • the robot 5 has a robot control unit 51.
  • the robot control unit 51 controls the operation of each robot arm 52 based on the subtask sequence designated for each robot arm 52 by the control signal S3.
  • the work space 6 is a work space in which the robot 5 executes a target task.
  • the measuring device 7 is a camera, a range sensor, a sonar, or a combination of these, one or a plurality of external world sensors that measure the work space 6 as a measurement target range.
  • the measuring device 7 supplies the generated output signal S4 to the information processing device 1.
  • the output signal S4 may be image data captured in the work space 6 or point cloud data indicating the position of an object in the work space 6.
  • the configuration of the robot control system 100 shown in FIG. 1 is an example, and various changes may be made to the configuration.
  • the robot 5 may include only one robot arm 52 or three or more robot arms 52.
  • the information processing apparatus 1 generates a subtask sequence to be executed for each control target of the robot 5 based on the target task, and sends a control signal S3 indicating the subtask sequence to the control target of the target. It is transmitted to the robot 5 that has it.
  • the measuring device 7 may be a part of the robot 5.
  • the robot control unit 51 may be configured separately from the robot 5.
  • the input device 2 and the display device 3 may be configured as the same device (for example, a tablet terminal) as the information processing device 1 depending on the mode such as being built in the information processing device 1.
  • the information processing device 1 may be composed of a plurality of devices. In this case, the plurality of devices constituting the information processing device 1 exchange information necessary for executing the pre-assigned process between the plurality of devices.
  • the robot 5 may have at least a part of the functions of the information processing device 1.
  • FIG. 2 shows the hardware configuration of the information processing device 1.
  • the information processing device 1 includes a processor 11, a memory 12, and an interface 13 as hardware.
  • the processor 11, the memory 12, and the interface 13 are connected via the data bus 19.
  • the processor 11 executes a predetermined process by executing the program stored in the memory 12.
  • the processor 11 is a processor such as a CPU (Central Processing Unit) and a GPU (Graphics Processing Unit).
  • the memory 12 is composed of various types of memory such as a RAM (Random Access Memory) and a ROM (Read Only Memory). Further, the memory 12 stores a program for the information processing apparatus 1 to execute a predetermined process. Further, the memory 12 is used as a working memory and temporarily stores information and the like acquired from the storage device 4. The memory 12 may function as a storage device 4. Similarly, the storage device 4 may function as the memory 12 of the information processing device 1. The program executed by the information processing device 1 may be stored in a storage medium other than the memory 12.
  • the interface 13 is an interface for electrically connecting the information processing device 1 and another device.
  • the interface 13 connects an interface for connecting the information processing device 1 and the input device 2, an interface for connecting the information processing device 1 and the display device 3, and the information processing device 1 and the storage device 4.
  • the interface 13 includes an interface for connecting the information processing device 1 and the robot 5, and an interface for connecting the information processing device 1 and the measuring device 7.
  • These connections may be wired connections or wireless connections.
  • the interface for connecting the information processing device 1 and the storage device 4 may be a communication interface for transmitting and receiving data to and from the storage device 4 by wire or wirelessly under the control of the processor 11.
  • the information processing device 1 and the storage device 4 may be connected by a cable or the like.
  • the interface 13 includes an interface compliant with USB, SATA (Serial AT Attainment), etc. for exchanging data with the storage device 4.
  • the hardware configuration of the information processing device 1 is not limited to the configuration shown in FIG.
  • the information processing device 1 may include at least one of an input device 2, a display device 3, and a storage device 4. Further, the information processing device 1 may be connected to or built in a sound output device such as a speaker. In these cases, the information processing device 1 may be a tablet terminal or the like in which an input function and an output function are integrated with the main body.
  • FIG. 3 shows an example of the data structure of the application information stored in the application information storage unit 41.
  • the application information storage unit 41 provides abstract state designation information I1, constraint condition information I2, operation limit information I3, subtask information I4, abstract model information I5, and object model information I6. Including.
  • Abstract state specification information I1 is information that specifies the abstract state that needs to be defined when generating a subtask sequence.
  • This abstract state is an abstract state of an object in the work space 6, and is defined as a proposition used in a target logical formula described later.
  • the abstract state specification information I1 specifies the abstract state that needs to be defined for each type of target task.
  • the target task may be various types of tasks such as pick-and-place, capture of moving objects, and screwdriver.
  • Constraint information I2 is information indicating the constraint conditions when executing the target task.
  • the constraint condition information I2 is, for example, a constraint condition that the robot 5 (robot arm 52) must not touch an obstacle and a constraint condition that the robot arms 52 must not touch each other when the target task is pick and place. And so on.
  • the constraint condition information I2 may be information that records suitable constraint conditions for each type of target task.
  • the operation limit information I3 indicates information regarding the operation limit of the robot 5 controlled by the information processing device 1.
  • the operation limit information I3 is, for example, information that defines the maximum leaching speed of the robot arm 52 in the case of the robot 5 shown in FIG.
  • Subtask information I4 indicates information on subtasks that can be accepted by the robot 5. For example, when the target task is pick-and-place, the subtask information I4 defines leaching, which is the movement of the robot arm 52, and glassing, which is the gripping by the robot arm 52, as subtasks. The subtask information I4 may indicate information on subtasks that can be used for each type of target task.
  • Abstract model information I5 is information related to a model (also referred to as an "abstract model") that abstracts the dynamics in the workspace 6.
  • the abstract model is represented by a model in which the dynamics of reality are abstracted by a hybrid system, as will be described later.
  • the abstract model information I5 includes information indicating the conditions for switching the dynamics in the above-mentioned hybrid system. The switching condition corresponds to, for example, in the case of the pick-and-place shown in FIG. 1, the condition that the object 61 cannot be moved unless it is gripped by the hand of the robot arm 52.
  • Abstract model information I5 has information about an abstract model suitable for each type of target task.
  • the object model information I6 is information about the object model of each object (in the example of FIG. 1, the robot arm 52, the object 61, the obstacle 62, etc.) to be recognized from the output signal S4 generated by the measuring device 7.
  • the above-mentioned object model is, for example, a learning model in machine learning such as a neural network. This learning model is learned to output, for example, the type and position of the object to be measured by the output signal S4, and the parameters of the learned learning model are recorded in the object model information I6.
  • the object model information I6 may include CAD data of the object to be recognized.
  • the application information storage unit 41 may store various information related to the generation process of the subtask sequence in addition to the above-mentioned information.
  • the application information storage unit 41 includes a dynamic model of the robot 5 itself, an abstracted dynamic model of the robot arm 52, a simple model for calculating the time required to grip the object 61 by the robot arm 52, and the like. It may be.
  • FIG. 4 is an example of a functional block of the information processing device 1.
  • the processor 11 of the information processing device 1 has an abstract state setting unit 31, a target logical formula generation unit 32, a time step logical formula generation unit 33, an abstract model generation unit 34, and a control input generation unit 35. And a subtask sequence generation unit 36.
  • FIG. 4 shows an example of data that is exchanged between blocks, but the present invention is not limited to this.
  • the abstract state setting unit 31 generates information (also referred to as “measurement information Im”) indicating the measurement result in the work space 6 based on the output signal S4 supplied from the measurement device 7. Specifically, when the abstract state setting unit 31 receives the output signal S4, the abstract state setting unit 31 refers to the object model information I6 and the like, and the type of each object in the work space 6 related to the execution of the target task (object 61). , Obstacles 62, etc.) and position, etc. are recognized, and this recognition result is generated as measurement information Im. The abstract state setting unit 31 supplies the generated measurement information Im to the abstract model generation unit 34.
  • the abstract state setting unit 31 is an abstraction in the workspace 6 that needs to be considered when executing the target task based on the above-mentioned measurement information Im and the abstract state designation information I1 acquired from the application information storage unit 41. Set the state.
  • the abstract state setting unit 31 defines a proposition for expressing each abstract state by a logical expression.
  • the abstract state setting unit 31 supplies information indicating the set abstract state (also referred to as “abstract state setting information Is”) to the target logical expression generation unit 32.
  • the target logical expression generation unit 32 When the target logical formula generation unit 32 receives the input signal S1 related to the target task from the input device 2, the target logical expression generation unit 32 sets the target task indicated by the input signal S1 as a time phase representing the final achievement state based on the abstract state setting information Is. It is converted into a logical expression (also referred to as "target logical expression Ltag"). In this case, the target logical expression generation unit 32 adds the constraint conditions to be satisfied in the execution of the target task to the target logical expression Ltag by referring to the constraint condition information I2 from the application information storage unit 41. Then, the target logical formula generation unit 32 supplies the generated target logical formula Ltag to the time step logical formula generation unit 33. Further, the target logical formula generation unit 32 generates a display signal S2 for displaying a task input screen that receives an input related to the target task, and supplies the display signal S2 to the display device 3.
  • a logical expression also referred to as "target logical expression Ltag"
  • the time step logical formula generation unit 33 converts the target logical formula Ltag supplied from the target logical formula generation unit 32 into a logical formula (also referred to as “time step logical formula Lts”) representing the state at each time step. To do. Then, the time step logical formula generation unit 33 supplies the generated time step logical formula Lts to the control input generation unit 35.
  • the abstract model generation unit 34 generates an abstract model " ⁇ " that abstracts the actual dynamics in the workspace 6 based on the measurement information Im and the abstract model information I5 stored in the application information storage unit 41.
  • the abstract model generation unit 34 regards the target dynamics as a hybrid system in which continuous dynamics and discrete dynamics are mixed, and generates an abstract model ⁇ based on the hybrid system. The method of generating the abstract model ⁇ will be described later.
  • the abstract model generation unit 34 supplies the generated abstract model ⁇ to the control input generation unit 35.
  • the control input generation unit 35 satisfies the time step logical expression Lts supplied from the time step logical expression generation unit 33 and the abstract model ⁇ supplied from the abstract model generation unit 34, and optimizes the evaluation function for each time step.
  • the control input to the robot 5 is determined.
  • the control input generation unit 35 supplies the subtask sequence generation unit 36 with information indicating the control input for each time step to the robot 5 (also referred to as “control input information Ic”).
  • the subtask sequence generation unit 36 generates a subtask sequence based on the control input information Ic supplied from the control input generation unit 35 and the subtask information I4 stored in the application information storage unit 41, and the control signal S3 indicating the subtask sequence. Is supplied to the robot 5.
  • the abstract state setting unit 31 outputs measurement information Im indicating the measurement result (type, position, etc.) of an object in the work space 6 based on the output signal S4 supplied from the measurement device 7. At the same time as generating, the abstract state in the workspace 6 is set.
  • the abstract state setting unit 31 refers to the abstract state designation information I1 and recognizes the abstract state to be set in the workspace 6.
  • the abstract state to be set in the workspace 6 differs depending on the type of the target task. Therefore, when the abstract state to be set for each type of the target task is defined in the abstract state specification information I1, the abstract state setting unit 31 specifies the abstract state corresponding to the target task specified by the input signal S1. Refer to the information I1 and recognize the abstract state to be set.
  • FIG. 5 shows a bird's-eye view of the work space 6.
  • two robot arms 52a and 52b, four objects 61a to 61d, and an obstacle 62 are present.
  • the abstract state setting unit 31 first analyzes the output signal S4 received from the measuring device 7 using the object model information I6 and the like, so that the state of the object 61, the existence range of the obstacle 62, and the goal Recognize the existence range of the area G set as a point.
  • the abstract state setting unit 31 recognizes the position vectors “x 1 ” to “x 4 ” at the centers of the objects 61a to 61d as the positions of the objects 61a to 61d.
  • the abstract state setting unit 31 recognizes the position vector “x r1 ” of the robot hand 53a that grips the object and the position vector “x r2 ” of the robot hand 53b as the positions of the robot arm 52a and the robot arm 52b. To do.
  • the abstract state setting unit 31 recognizes the existence range of the obstacle 62, the existence range of the region G, and the like, such as the postures of the objects 61a to 61d (unnecessary because the object is spherical in the example of FIG. 5).
  • the abstract state setting unit 31 recognizes the position vectors of the obstacle 62 and each vertex of the area G. Then, the abstract state setting unit 31 generates these recognition results based on the output signal S4 as measurement information Im.
  • the abstract state setting unit 31 determines the abstract state to be defined in the target task by referring to the abstract state designation information I1. In this case, the abstract state setting unit 31 recognizes the object and the area existing in the work space 6 based on the measurement information Im, and the recognition result (for example, the number of each type of the object and the area) and the constraint regarding the object and the area. Based on the condition information I2, a proposition indicating an abstract state is determined.
  • abstract state setting unit 31 refers to the abstract state designation information I1, recognizes the abstract state should be defined, the proposition representing the abstract state (g i in the above example, o i, h) Is defined according to the number of objects 61, the number of robot arms 52, the number of obstacles 62, and the like. Then, the abstract state setting unit 31 supplies the information indicating the proposition representing the abstract state to the target logical expression generation unit 32 as the abstract state setting information Is.
  • FIG. 6 is a functional block configuration diagram of the target logical formula generation unit 32.
  • the target logical expression generation unit 32 functionally includes an input reception unit 321, a logical expression conversion unit 322, a constraint condition information acquisition unit 323, and a constraint condition addition unit 324.
  • the input receiving unit 321 receives the input of the input signal S1 that specifies the type of the target task and the final state of the object to be worked by the robot. Further, the input receiving unit 321 transmits the display signal S2 of the task input screen that accepts these inputs to the display device 3.
  • the logical expression conversion unit 322 converts the target task specified by the input signal S1 into a logical expression using temporal logic.
  • temporal logic There are various existing techniques for converting a task expressed in natural language into a logical expression. For example, it is assumed that the logical expression conversion unit 322 is given the target task of "finally the object 2 exists in the region G" in the example of FIG.
  • logical expression conversion unit 322 a linear logical expression of the desired task (LTL: Linear Temporal Logic) the operator corresponding to "eventually"" ⁇ ", propositions defined by abstract state setting unit 31 "g i To generate the logical expression " ⁇ g 2".
  • LTL Linear Temporal Logic
  • the formula conversion unit 322 is an operator of any temporal logic other than the operator " ⁇ " (logical product “ ⁇ ”, OR “ ⁇ ”, negative “ ⁇ ”, logical inclusion “ ⁇ ”, always “ ⁇ ”, next“ ⁇ ”, until“ U ”, etc.) may be used to express the logical expression. Further, the logical expression may be expressed by using any time phase logic such as MTL (Metric Temporal Logic) or STL (Signal Temporal Logic), not limited to the linear temporal logic.
  • the constraint condition information acquisition unit 323 acquires the constraint condition information I2 from the application information storage unit 41.
  • the constraint condition information acquisition unit 323 has a constraint condition corresponding to the type of the target task specified by the input signal S1.
  • Information I2 is acquired from the application information storage unit 41.
  • the constraint condition addition unit 324 generates a target logical expression Ltag by adding the constraint condition indicated by the constraint condition information I2 acquired by the constraint condition information acquisition unit 323 to the logical expression generated by the logical expression conversion unit 322.
  • constraint condition information I2 includes two constraint conditions corresponding to pick and place, "the robot arms 52 do not interfere with each other" and "the object i does not interfere with the obstacle O"
  • the constraint condition The addition unit 324 converts these constraints into a logical expression.
  • constraint addition section 324 uses the proposition "o i” and the proposition "h” which is defined by the abstract state setting unit 31 in the description of FIG. 5, the two constraints mentioned above, respectively below Convert to a logical expression of. ⁇ ⁇ h ⁇ i ⁇ ⁇ o i
  • the constraint condition addition unit 324 adds the logical expressions of these constraint conditions to the logical expression " ⁇ g 2" corresponding to the target task "finally the object 2 exists in the area G". By doing so, the following target formula Ltag is generated. ( ⁇ g 2 ) ⁇ ( ⁇ ⁇ h) ⁇ ( ⁇ i ⁇ ⁇ o i )
  • the constraint conditions corresponding to pick and place are not limited to the above two, and "the robot arm 52 does not interfere with the obstacle O" and "a plurality of robot arms 52 do not grab the same object". , "Objects do not touch each other” and other constraints exist. Similarly, such a constraint condition is also stored in the constraint condition information I2 and reflected in the target logical expression Ltag.
  • FIG. 7 shows a first display example of the task input screen.
  • the input receiving unit 321 generates the display signal S2 and transmits the display signal S2 to the display device 3, so that the display device 3 displays the task input screen shown in FIG. 7.
  • the task input screen shown in FIG. 7 mainly has a task type designation field 15, an image display field 16, an estimated work time designation field 17, a calculation work time display area 18, and a decision button 20.
  • the input reception unit 321 accepts an input for designating the type of the target task in the task type designation field 15.
  • the task type designation field 15 is a pull-down menu format input field, and the input reception unit 321 displays a list of acceptable target task type candidates in the task type designation field 15 so as to be selectable. ..
  • pick and place is designated as the type of the target task in the task type designation field 15.
  • the input reception unit 321 uses the CAD model of the object 61 stored in the object model information I6 in the image display field 16 to display a CAD image that reproduces the environment in the work space 6.
  • the input receiving unit 321 generates a CAD image that reproduces the environment in the work space 6 based on, for example, the measurement information Im generated by the abstract state setting unit 31 and the object model information I6 that records the CAD data. To do.
  • the input receiving unit 321 accepts an input for designating the final position of each object 61 based on a touch panel operation on the image display field 16 or a drag-and-drop operation with a mouse. Then, the input receiving unit 321 recognizes the target task based on the input signal S1 indicating the input in the task type designation field 15 and the image display field 16.
  • the animation in which the robot 5 executes the target task may be displayed on the image display field 16. ..
  • the input receiving unit 321 may display an animation on the image display field 16 showing changes in the work space 6 until the object 61 is carried by the robot 5 to the designated final position.
  • the input receiving unit 321 operates the robot 5 and changes the position of each object 61 for each time step based on the control input information Ic or the subtask sequence obtained based on the target logical expression Ltag obtained by converting the target task, for example. Etc. are recognized. Then, the input reception unit 321 displays the animation generated based on the recognition result on the image display field 16.
  • the input reception unit 321 may display a two-dimensional image representing a bird's-eye view of the work space 6 instead of displaying the CAD image of the work space 6 on the image display field 16. Even in this case, the input receiving unit 321 can suitably accept the designation of the final position of each object 61 based on the touch panel operation on the image display field 16 or the drag-and-drop operation by the mouse.
  • the input reception unit 321 accepts the input of the estimated work time of the target task in the estimated work time designation field 17.
  • the input receiving unit 321 supplies the estimated time specified in the estimated working time designation field 17 to the time step logical formula generation unit 33.
  • the time step logical formula generation unit 33 can preferably determine the target number of time steps from the notified estimated time.
  • the input reception unit 321 displays the estimated work time of the target task calculated by the information processing device 1 in the calculation work time display area 18.
  • the input receiving unit 321 displays the above-mentioned estimated time by, for example, calculating the required number of time steps based on the target logical expression Ltag obtained by converting the target task specified on the task input screen.
  • the input reception unit 321 calculates that an error has occurred when the designated target task cannot be achieved, for example, when the control input generation unit 35 cannot generate the control input information Ic based on the target task. Display on 18.
  • the input receiving unit 321 prompts the re-input of the position of the object on the task input screen.
  • the input receiving unit 321 detects that the enter button 20 is selected, the input receiving unit 321 supplies the information of the target task recognized based on the input signal S1 indicating the input contents on the task input screen to the logical expression conversion unit 322. .. After that, the information processing device 1 generates a control signal S3 and transmits the generated control signal S3 to the robot 5 to cause the robot 5 to execute a target task.
  • FIG. 8 shows a second display example of the task input screen.
  • the input receiving unit 321 transmits the display signal S2 to the display device 3, so that the display device 3 displays the task input screen shown in FIG.
  • the input receiving unit 321 tasks the table column 16A for designating the final position for each object instead of the image display column 16 of the first display example for displaying the CAD image related to the work space 6. It is displayed on the input screen.
  • the table column 16A has each item of "object”, "initial position", and "final position".
  • the identification label of each labeled object 61 is displayed.
  • a coordinate value indicating the initial position of each object 61 in the work space 6 is displayed. This coordinate value is a coordinate value defined in the work space 6.
  • the item "final position” is an input field, and each object 61 can be input.
  • the input receiving unit 321 receives an input that specifies a coordinate value indicating the final position or a defined area name (“area G” in FIG. 8).
  • the input receiving unit 321 displays an image schematically showing the work space 6 (for example, an image displayed in the image display field 16) when an arbitrary input field of the item "final position" is selected. , An input for designating the final position of the target object 61 in the image may be accepted.
  • Target logical expression generation unit 33 determines the number of time steps (also referred to as “target time step number”) for completing the target task, and the target logical expression Ltag is determined by the target number of time steps. Determine a combination of propositions that represent the state at each time step that satisfies. Since there are usually a plurality of these combinations, the time step logical expression generation unit 33 generates a logical expression in which these combinations are combined by a logical sum as a time step logical expression Lts.
  • the above combination is a candidate for a logical expression representing a sequence of actions instructing the robot 5, and is also referred to as "candidate ⁇ " hereafter.
  • the time step logical formula generation unit 33 is supplied with "( ⁇ g 2 ) ⁇ ( ⁇ ⁇ h) ⁇ ( ⁇ i ⁇ ⁇ o i )" as the target logical formula Ltag from the target logical formula generation unit 32.
  • the time step logical expression generating unit 33 the proposition "g i” an extended proposition "g i, k” to include the notion of time step is used.
  • the proposition "gi , k " is a proposition that "the object i exists in the region G in the time step k".
  • ⁇ g2 and 3 can be rewritten as shown in the following equation.
  • the above-mentioned target logical formula Ltag is represented by the logical sum ( ⁇ 1 ⁇ ⁇ 2 ⁇ ⁇ 3 ⁇ ⁇ 4 ) of the four candidates “ ⁇ 1 ” to “ ⁇ 4” shown below.
  • the time step logical expression generating unit 33 defines a logical sum of the four candidate phi 1 ⁇ phi 4 as a time step formulas Lts.
  • the time step formulas Lts is either at least four candidate phi 1 ⁇ phi 4 is true if that is true.
  • the time step logical formula generation unit 33 determines the feasibility of the generated candidate by referring to the operation limit information I3, and excludes the candidate determined to be unrealizable. For example, the time step logical formula generation unit 33 recognizes the distance that the robot hand can move per time step based on the operation limit information I3. Further, the time step logical formula generation unit 33 recognizes the distance between the object to be moved (object 2) and the robot hand based on the position vector of each object and the robot hand indicated by the measurement information Im. Then, the time step logical formula generation unit 33 determines the feasibility based on these distances.
  • the time step logical expression generating unit 33 both the robot hand 53a and the robot hand 53b is, if the distance to the object 2 is determined to longer than the moving distance per one time step, the above-mentioned candidate phi 3 and candidate phi 4 is determined to not realized.
  • the time step logical expression generating unit 33 excludes the candidate phi 3 and candidate phi 4 from time step formulas Lts.
  • the time step logical formula Lts is the logical sum ( ⁇ 1 ⁇ ⁇ 2 ) of the candidate ⁇ 1 and the candidate ⁇ 2.
  • the time step logical formula generation unit 33 preferably reduces the processing load of the subsequent processing unit by excluding the unrealizable candidates from the time step logical formula Lts with reference to the operation limit information I3. be able to.
  • the time step logical formula generation unit 33 determines the target number of time steps based on, for example, the estimated work time specified by user input (see the estimated work time designation column 17 in FIGS. 7 and 8). In this case, the time step logical formula generation unit 33 calculates the target number of time steps from the above-mentioned estimated time based on the information of the time width per time step stored in the memory 12 or the storage device 4. In another example, the time step logical formula generation unit 33 previously stores information associated with the target number of time steps suitable for each type of target task in the memory 12 or the storage device 4, and refers to the information. By doing so, the target number of time steps is determined according to the type of target task to be executed.
  • the time step logical expression generation unit 33 sets the target number of time steps to a predetermined initial value. Then, the time step logical formula generation unit 33 gradually increases the target number of time steps until the time step logical formula Lts in which the control input generation unit 35 can determine the control input is generated. In this case, the time step logical formula generation unit 33 determines the target time step number when the optimum solution cannot be derived as a result of the control input generation unit 35 performing the optimization process according to the set target time step number. Add only a number (integer of 1 or more).
  • the time step logical expression generation unit 33 specifies the initial value of the target time step number as a value smaller than the number of time steps corresponding to the work time of the target task expected by the user (for example, in the expected work time designation column 17). It is recommended to set it to half of the estimated time. As a result, the time step logical expression generation unit 33 preferably suppresses setting an unnecessarily large target number of time steps.
  • the time step logical formula generation unit 33 sets the initial value of the target time step number to a small value, and gradually increases the target time step number until a solution in the optimization process of the control input generation unit 35 exists. ..
  • the time step logical formula generation unit 33 can set the target number of time steps as small as possible within the range in which the solution in the optimization process of the control input generation unit 35 exists. Therefore, in this case, it is possible to reduce the processing load in the optimization process and shorten the time required for the robot 5 to achieve the target task.
  • the abstract model generation unit 34 generates the abstract model ⁇ based on the measurement information Im and the abstract model information I5.
  • the abstract model information I5 information necessary for generating the abstract model ⁇ is recorded for each type of the target task. For example, when the target task is pick-and-place, a general-purpose abstraction that does not specify the position and number of objects, the position of the area where the objects are placed, the number of robots 5 (or the number of robot arms 52), etc.
  • the model is recorded in the abstract model information I5.
  • the abstract model generation unit 34 refers to the position and number of objects indicated by the measurement information Im, the position of the area on which the objects are placed, and the robot 5 with respect to the abstract model of the general-purpose format recorded in the abstract model information I5.
  • An abstract model ⁇ is generated by reflecting the number of units.
  • the dynamics in the work space 6 are frequently switched. For example, in pick and place, when the robot arm 52 grabs the object i, the object i moves, but when the robot arm 52 does not grab the object i, the object i moves. Absent.
  • the action of grabbing the object i is abstractly expressed by the logical variable “ ⁇ i”.
  • the abstract model generation unit 34 can determine the abstract model ⁇ to be set for the work space 6 shown in FIG. 5 by the following equation (1).
  • u j indicates a control input for controlling the robot hand j
  • "I" is the unit matrix. Shown.
  • the control input is assumed to be speed as an example here, it may be acceleration.
  • ⁇ j, i is a logical variable that becomes “1” when the robot hand j grabs the object i, and becomes "0” in other cases.
  • x r1 " and “x r2 " indicate the position vector of the robot hand j
  • x 1 " to "x 4 indicate the position vector of the object i.
  • equation (1) is a difference equation showing the relationship between the state of the object at the time step k and the state of the object at the time step k + 1. Then, in the above equation (1), the gripping state is represented by a logical variable that is a discrete value, and the movement of the object is represented by a continuous value, so that the equation (1) represents a hybrid system. ..
  • Equation (1) considers only the dynamics of the robot hand, which is the hand of the robot 5 that actually grips the object, rather than the detailed dynamics of the entire robot 5. As a result, the amount of calculation of the optimization process can be suitably reduced by the control input generation unit 35.
  • the abstract model generation unit 34 has abstract model information I5 and measurement information even when the position and number of objects, the area where the objects are placed (area G in FIG. 5), the number of robots 5 and the like fluctuate. By combining with Im, it is possible to determine the abstract model ⁇ suitable for the environment of the target workspace 6.
  • the abstract model generation unit 34 generates a model of a mixed logical dynamic (MLD: Mixed Logical Dynamic) system or a hybrid system combining Petri net, an automaton, etc., instead of the model shown in the equation (1). May be good.
  • MLD Mixed Logical Dynamic
  • Control input generation unit 35 is based on the time step logical formula Lts supplied from the time step logical formula generation unit 33 and the abstract model ⁇ supplied from the abstract model generation unit 34.
  • the control input for each time step for the robot 5 for each optimal time step is determined.
  • the control input generation unit 35 defines an evaluation function for the target task, and solves an optimization problem that minimizes the evaluation function with the abstract model ⁇ and the time step logical expression Lts as constraints.
  • the evaluation function is determined in advance for each type of target task, and is stored in the memory 12 or the storage device 4, for example.
  • the control input generation unit 35 when the pick and place purposes task, the control input generation unit 35, the minimum distance "d k" control input "u k" between the target point carrying the object and the object of interest to carry (That is, the evaluation function is defined so as to minimize the energy consumed by the robot 5).
  • the above-mentioned distance d k corresponds to the distance between the object 2 and the area G in the case of the target task that “the object 2 finally exists in the area G”.
  • control input generation unit 35 defines a sum of the square of the square and the control input u k of the distance d k of the total time steps as the evaluation function, the abstract model ⁇ and timestep formulas Lts (i.e. candidate phi i Solve the constrained mixed integer optimization problem shown in the following equation (2) with the constraint condition (logical sum).
  • T is the number of time steps to be optimized, may be the target number of time steps, or may be a predetermined number smaller than the target number of time steps, as will be described later.
  • the control input generation unit 35 approximates the logical variable to a continuous value (referred to as a continuous relaxation problem). As a result, the control input generation unit 35 can suitably reduce the amount of calculation.
  • STL linear logic formula
  • LTL linear logic formula
  • the control input generation unit 35 sets the time step number T of the equation (2) used for optimization to a value smaller than the target time step number (for example). It may be set to the above-mentioned threshold value). In this case, the control input generation unit 35 is, for example, each time a predetermined time step number has elapsed, by solving an optimization problem based on the equation (2), to determine the control input u k sequentially.
  • the control input generation unit 35 for each predetermined event corresponding to the intermediate state for achieving the desired state task, solve the optimization problem based on the equation (2), to determine the control input u k to be used You may.
  • the control input generation unit 35 sets the number of time steps until the next event occurs to the number of time steps T in the equation (2).
  • the above-mentioned event is, for example, an event in which the dynamics in the workspace 6 are switched. For example, when pick and place is the target task, it is determined as an event that the robot 5 grabs the object, the robot 5 finishes carrying one of the plurality of objects to be carried to the destination, and the like. Be done.
  • the event is determined in advance for each type of target task, for example, and information for identifying the event for each type of target task is stored in the storage device 4. Details of the processing for determining a control input u k for each event will be described later in the flowchart of FIG. 11.
  • the number of time steps T in the equation (2) can be reduced to preferably reduce the amount of calculation of the optimization problem.
  • Subtask sequence generation unit 36 generates a subtask sequence based on the control input information Ic supplied from the control input generation unit 35 and the subtask information I4 stored in the application information storage unit 41. To do. In this case, the subtask sequence generation unit 36 recognizes the subtask that can be accepted by the robot 5 by referring to the subtask information I4, and converts the control input for each time step indicated by the control input information Ic into the subtask.
  • the subtask information I4 is a function indicating two subtasks, that is, the movement of the robot hand (reaching) and the grasping of the robot hand (grasping), as the subtasks that the robot 5 can accept when the target task is pick and place.
  • the function "Move” representing leaching takes, for example, the initial state of the robot 5 before the execution of the function, the final state of the robot 5 after the execution of the function, and the time required to execute the function as arguments. It is a function.
  • the function "Grasp” representing glassing is, for example, a function that takes as arguments the state of the robot 5 before the execution of the function, the state of the object to be grasped before the execution of the function, and the logical variable ⁇ .
  • the function "Grasp” indicates that the operation of grasping is performed when the logical variable ⁇ is "1", and the operation of releasing when the logical variable ⁇ is "0" is performed.
  • the subtask sequence generation unit 36 determines the function "Move” based on the trajectory of the robot hand determined by the control input for each time step indicated by the control input information Ic, and the control input information Ic determines the function "Grasp". It is determined based on the transition of the logical variable ⁇ for each time step shown.
  • the subtask sequence generation unit 36 generates a subtask sequence composed of the function "Move” and the function "Grasp", and supplies the control signal S3 indicating the subtask sequence to the robot 5. For example, when the target task is "finally the object 2 exists in the area G", the subtask sequence generation unit 36 uses the function “Move”, the function "Grasp”, and the function "Grasp” for the robot hand closest to the object 2. Generate a subtask sequence of the function "Move” and the function "Grasp". In this case, the robot hand closest to the object 2 moves to the position of the object 2 by the function "Move”, grasps the object 2 by the function "Grasp", and moves to the area G by the function "Move". The object 2 is placed in the region G by the function "Grasp".
  • FIG. 9 is an example of a flowchart showing an outline of robot control processing executed by the information processing apparatus 1 in the first embodiment.
  • the abstract state setting unit 31 of the information processing device 1 generates measurement information Im indicating the measurement result of an object in the work space 6 and sets the abstract state based on the output signal S4 supplied from the measurement device 7. (Step S11).
  • the target logical formula generation unit 32 determines the target logical formula Ltag from the target task designated by the input signal S1 or the like (step S12). In this case, the target logical expression generation unit 32 adds the constraint condition in the execution of the target task to the target logical expression Ltag by referring to the constraint condition information I2.
  • the process of step S12 may be executed before step S11.
  • the time step logical formula generation unit 33 converts the target logical formula Ltag into the time step logical formula Lts representing the state at each time step (step S13).
  • the time step logical formula generation unit 33 determines the target number of time steps, and the logical sum of the candidates ⁇ representing the state at each time step such that the target number of time steps satisfies the target logical formula Ltag is the time step logic. Generated as the formula Lts.
  • the time step logical formula generation unit 33 determines the feasibility of each candidate ⁇ by referring to the operation limit information I3, and time steps the candidate ⁇ determined to be infeasible. Exclude from the formula Lts.
  • the abstract model generation unit 34 determines the abstract model ⁇ suitable for the target task based on the measurement information Im generated in step S11 and the abstract model information I5 (step S14). Then, the control input generation unit 35 satisfies the abstract model ⁇ and the time step logical formula Lts, and determines the control input for optimizing the evaluation function (step S15). Then, the subtask sequence generation unit 36 determines the subtask sequence from the control input determined by the control input generation unit 35, and outputs the control signal S3 indicating the subtask sequence to the robot 5 (step S16).
  • FIG. 10 is an example of a flowchart showing the details of the process of step S13 of FIG.
  • the time step logical expression generation unit 33 sets the target number of time steps to the initial value (step S21).
  • This initial value may be determined based on user input, or may be a value stored in advance in the memory 12 or the storage device 4.
  • the initial value is preferably set to a value smaller than the number of time steps expected to be required to execute the target task (for example, half the number of time steps specified by the user).
  • the time step logical formula generation unit 33 determines a logical formula candidate ⁇ representing a state for each time step that satisfies the target logical formula Ltag with the target number of time steps set in step S21 (step S22). Then, the time step logical formula generation unit 33 determines the feasibility of each candidate ⁇ by referring to the operation limit information I3, and excludes the candidate ⁇ determined to be infeasible (step S23).
  • the time step logical formula generation unit 33 determines whether or not the optimization solution in step S15 of FIG. 9 exists (step S24). In this case, the time step logical formula generation unit 33 satisfies the abstract model ⁇ determined in step S14 and one of the executable candidates ⁇ , and controls and generates the control input to the robot 5 that optimizes the evaluation function. It is determined whether or not the unit 35 can be derived. Then, when the solution of the optimization in step S15 exists (step S24; Yes), the time step logical formula generation unit 33 generates the abstract model generation unit 34 with the time step logical formula Lts in which the candidate ⁇ s are ORed by OR. Is output to (step S25).
  • step S15 when the optimization solution in step S15 does not exist (step S24; No), the time step logical expression generation unit 33 determines that the target task cannot be executed with the currently set target number of time steps. Therefore, in this case, the time step logical expression generation unit 33 adds the target number of time steps by a predetermined value (an integer of 1 or more) (step S26). Then, the time step logical formula generation unit 33 executes the processes of steps S22 to S24 again.
  • a predetermined value an integer of 1 or more
  • the time step logical expression generation unit 33 can preferably set the target time step number so that the target time step number becomes an appropriate value.
  • FIG. 11 is a modification of the robot control process shown in the flowchart of FIG. 9, and is an example of a flowchart showing the robot control process that determines a subtask sequence each time an event occurs.
  • the abstract state setting unit 31 and the target logical formula generation unit 32 perform the same processing as in steps S11 and S12 of FIG. 9, respectively, in step S31 and step S32, respectively.
  • the time step logical formula generation unit 33 converts the target logical formula Ltag into the time step logical formula Lts until the next event occurs (step S33). In this case, the time step logical formula generation unit 33 determines the target number of time steps required until the next event occurs, and the candidate ⁇ representing the state at each time step such that the target logical formula Ltag is satisfied by the target number of time steps. Is generated as the time step formula Lts.
  • the target number of time steps in this case may be determined based on, for example, the method for determining the number of target time steps shown in FIG. 10, or may be set to an appropriate value stored in advance in the storage device 4 or the like.
  • the abstract model generation unit 34 determines the abstract model ⁇ suitable for the target task based on the measurement information Im generated in step S31 and the abstract model information I5 (step S34). If the number or type of the object does not change during the operation of the robot 5, the process of step S34 may be executed only once, and does not need to be executed every time an event occurs. Therefore, if the abstract model generation unit 34 has already executed step S34, the abstract model generation unit 34 may output the abstract model ⁇ obtained in the previous step S34.
  • control input generation unit 35 satisfies the abstract model ⁇ and the time step logical formula Lts, and determines the control input for optimizing the evaluation function (step S35). In this case, the control input generation unit 35 determines the control inputs for the number of time steps required until the next event occurs. Then, the subtask sequence generation unit 36 determines the subtask sequence from the control input determined by the control input generation unit 35, and outputs the control signal S3 indicating the subtask sequence to the robot 5 (step S36).
  • the information processing device 1 determines whether or not the target task has been completed (step S37).
  • the information processing device 1 determines whether or not the target task is completed by recognizing the state of the object or the like based on the output signal S4 supplied from the measuring device 7, for example.
  • the information processing device 1 determines that the target task has been completed when the robot 5 notifies that the subtask sequence has been normally completed. Then, when the target task is completed (step S37; Yes), the information processing apparatus 1 ends the processing of the flowchart.
  • step S38 determines whether or not the next event has occurred.
  • the information processing device 1 determines whether or not an event has occurred by recognizing the state of the object or the like based on the output signal S4 supplied from the measuring device 7, for example.
  • the information processing apparatus 1 determines that an event has occurred when the robot 5 notifies the robot 5 of the normal end of the subtask sequence until the next event occurs.
  • step S38; Yes the information processing apparatus 1 returns the process to step S33.
  • the information processing apparatus 1 re-executes the subtask sequence generation process required for the next event occurrence in steps S33 to S36.
  • step S38 when the event has not occurred (step S38; No), the information processing apparatus 1 returns the process to step S37.
  • FIG. 12 is a schematic configuration diagram of the information processing device 1A according to the second embodiment.
  • the information processing apparatus 1A mainly includes a logical expression conversion unit 322A, a constraint condition information acquisition unit 323A, and a constraint condition addition unit 324A.
  • the logical formula conversion unit 322A converts the target task, which is a task to be performed by the robot, into a logical formula based on the temporal logic.
  • the constraint condition information acquisition unit 323A acquires the constraint condition information I2 indicating the constraint conditions to be satisfied in the execution of the target task.
  • the constraint condition addition unit 324A generates a target logical expression Ltag, which is a logical expression in which a proposition representing the above constraint condition is added to the logical expression generated by the logical expression conversion unit 322A.
  • the information processing device 1A when the target task to be worked by the robot is given, the information processing device 1A preferably generates a target logical expression Ltag that clearly indicates the constraint conditions to be satisfied in the execution of the target task. be able to.
  • FIG. 13 is a schematic configuration diagram of the information processing device 1B according to the third embodiment.
  • the information processing apparatus 1B mainly includes an abstract model information acquisition unit 34X, a measurement information acquisition unit 34Y, and an abstract model generation unit 34Z.
  • the abstract model information acquisition unit 34X acquires the abstract model information I5 relating to the abstract model that abstracts the dynamics in the workspace 6 of the robot 5 that executes the target task.
  • the measurement information acquisition unit 34Y acquires the measurement information Im indicating the measurement result in the work space 6.
  • the abstract model generation unit 34Z generates the abstract model ⁇ based on the abstract model information I5 and the measurement information Im.
  • the abstract model information acquisition unit 34X, the measurement information acquisition unit 34Y, and the abstract model generation unit 34Z are realized by, for example, the abstract model generation unit 34 in the first embodiment.
  • the information processing apparatus 1B preferably generates an abstract model ⁇ that simply represents an actual dynamic model of the robot when a target task to be made to work by the robot is given. be able to.
  • a logical formula conversion unit that converts a target task, which is a task to be performed by a robot, into a logical formula based on temporal logic
  • a constraint condition information acquisition unit that acquires constraint condition information indicating the constraint conditions to be satisfied in the execution of the target task
  • a constraint condition addition part that generates a target logical expression, which is a logical expression in which a proposition representing the constraint condition is added to the logical expression
  • Addendum which further has an abstract state setting unit that defines an abstract state, which is an abstract state of an object in the work space, as a proposition to be used in the target logical formula, based on the measurement result in the work space where the robot works.
  • the information processing apparatus according to 1.
  • the abstract state setting unit recognizes an object and a region existing in the work space based on the measurement result, and determines the proposition based on the recognition result regarding the object and the region and the constraint condition information.
  • the information processing apparatus according to 2.
  • Appendix 5 Further provided with an input receiving unit that accepts input for designating the type of the target task and the final state of the object to be worked on by the robot.
  • the information processing device according to any one of Supplementary note 1 to 4, wherein the input receiving unit transmits a display signal of an input screen that receives the input to the display device.
  • Appendix 6 The information processing device according to Appendix 5, wherein the input receiving unit further receives input regarding an estimated time required to execute the target task.
  • Appendix 7 The information processing device according to Appendix 5 or 6, wherein the input receiving unit transmits a display signal for displaying an animation in which the robot executes the target task based on the input to the display device.
  • Appendix 8 Any one of Appendix 1 to 7, further comprising a time step logical expression generator that generates a time step logical expression that is a logical expression representing the state of each time step in order to execute the target task from the target logical expression.
  • a time step logical expression generator that generates a time step logical expression that is a logical expression representing the state of each time step in order to execute the target task from the target logical expression.
  • the time step logical expression generation unit generates a time step logical expression in which the candidates are combined by a logical sum when a plurality of candidates exist as a logical expression representing the state of each time step, as described in Appendix 8.
  • Information processing device generates a time step logical expression in which the candidates are combined by a logical sum when a plurality of candidates exist as a logical expression representing the state of each time step, as described in Appendix 8.
  • the time step logical expression generation unit acquires the operation limit information regarding the operation limit of the robot, and excludes the infeasible candidate from the time step logical expression based on the operation limit information, as described in Appendix 9. Information processing equipment.
  • Appendix 11 The information processing apparatus according to any one of Appendix 8 to 10, further comprising a subtask sequence generation unit that generates a sequence of subtasks that are tasks of a unit that the robot can accept based on the time step logical formula.
  • Appendix 12 An abstract model generator that creates an abstract model that abstracts the dynamics in the workspace in which the robot works.
  • a control input generator that determines a control input for each time step for controlling the robot based on the abstract model and the time step logical formula.
  • [Appendix 13] It is a control method executed by the information processing device. Convert the objective task, which is the task to be made to work by the robot, into a logical formula based on temporal logic, Acquire constraint condition information indicating the constraint conditions to be satisfied in the execution of the target task, and obtain A control method for generating a target logical expression, which is a logical expression in which a proposition representing the constraint condition is added to the logical expression.
  • a logical formula conversion unit that converts a target task, which is a task to be performed by a robot, into a logical formula based on temporal logic
  • a constraint condition information acquisition unit that acquires constraint condition information indicating the constraint conditions to be satisfied in the execution of the target task
  • a storage medium in which a program for operating a computer as a constraint condition addition unit for generating a target logical expression, which is a logical expression in which a proposition representing the constraint condition is added to the logical expression, is stored.

Landscapes

  • Engineering & Computer Science (AREA)
  • Robotics (AREA)
  • Mechanical Engineering (AREA)
  • Manipulator (AREA)
  • Numerical Control (AREA)
PCT/JP2019/034182 2019-08-30 2019-08-30 情報処理装置、制御方法及び記憶媒体 Ceased WO2021038842A1 (ja)

Priority Applications (4)

Application Number Priority Date Filing Date Title
PCT/JP2019/034182 WO2021038842A1 (ja) 2019-08-30 2019-08-30 情報処理装置、制御方法及び記憶媒体
JP2021541931A JP7264253B2 (ja) 2019-08-30 2019-08-30 情報処理装置、制御方法及びプログラム
US17/636,073 US12521881B2 (en) 2019-08-30 2019-08-30 Information processing device, control method, and storage medium
EP19943354.1A EP4023396A4 (en) 2019-08-30 2019-08-30 INFORMATION PROCESSING DEVICE, CONTROL METHOD AND STORAGE MEDIA

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
PCT/JP2019/034182 WO2021038842A1 (ja) 2019-08-30 2019-08-30 情報処理装置、制御方法及び記憶媒体

Publications (1)

Publication Number Publication Date
WO2021038842A1 true WO2021038842A1 (ja) 2021-03-04

Family

ID=74685362

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/JP2019/034182 Ceased WO2021038842A1 (ja) 2019-08-30 2019-08-30 情報処理装置、制御方法及び記憶媒体

Country Status (4)

Country Link
US (1) US12521881B2 (https=)
EP (1) EP4023396A4 (https=)
JP (1) JP7264253B2 (https=)
WO (1) WO2021038842A1 (https=)

Cited By (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
WO2024180756A1 (ja) * 2023-03-02 2024-09-06 日本電気株式会社 制御システム、制御方法、および記録媒体

Families Citing this family (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
WO2020097486A1 (en) 2018-11-08 2020-05-14 SafeAI, Inc. Performing tasks using autonomous machines
JP7452619B2 (ja) * 2020-02-25 2024-03-19 日本電気株式会社 制御装置、制御方法及びプログラム
US11713059B2 (en) * 2021-04-22 2023-08-01 SafeAI, Inc. Autonomous control of heavy equipment and vehicles using task hierarchies
CN114297213B (zh) * 2021-12-29 2026-01-23 天翼物联科技有限公司 实现可灵活配置结算因子的结算方法、装置、计算机设备及存储介质

Citations (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JP2012125871A (ja) * 2010-12-15 2012-07-05 Mitsubishi Electric Corp ロボット制御設定支援装置
JP2014240110A (ja) * 2013-06-12 2014-12-25 三菱電機株式会社 ビンピッキング性能評価装置及び方法
JP2018051684A (ja) 2016-09-28 2018-04-05 ファナック株式会社 複数個の物品をひとつずつ最適な経路で取り出す搬送装置
WO2018194094A1 (ja) * 2017-04-19 2018-10-25 株式会社安川電機 プログラミング支援装置、ロボットシステム、プログラミング支援方法及びプログラム生成方法
JP2018167361A (ja) * 2017-03-30 2018-11-01 株式会社安川電機 ロボット動作指令生成方法、ロボット動作指令生成装置及びコンピュータプログラム

Family Cites Families (7)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
WO2009144826A1 (ja) * 2008-05-30 2009-12-03 富士通株式会社 検査用ファイル生成プログラム、検査用ファイル生成装置および検査用ファイル生成方法
KR102023588B1 (ko) * 2016-03-03 2019-10-14 구글 엘엘씨 로봇 파지용 심층 기계 학습 방법 및 장치
US11429118B2 (en) * 2018-07-18 2022-08-30 The Trustees Of The University Of Pennsylvania Control of multi-drone fleets with temporal logic objectives
US11586914B2 (en) * 2019-01-11 2023-02-21 Arizona Board Of Regents On Behalf Of Arizona State University Systems and methods for evaluating perception systems for autonomous vehicles using quality temporal logic
WO2022074824A1 (ja) * 2020-10-09 2022-04-14 日本電気株式会社 時相論理式生成装置、時相論理式生成方法及び記憶媒体
US20240042617A1 (en) * 2021-03-23 2024-02-08 Nec Corporation Information processing device, modification system, information processing method, and non-transitory computer-readable medium
US20250282382A1 (en) * 2024-03-08 2025-09-11 Toyota Research Institute, Inc. Determining a weight set to personalize a rule that governs a manner of a performance of a change of a movement of an automated vehicle

Patent Citations (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JP2012125871A (ja) * 2010-12-15 2012-07-05 Mitsubishi Electric Corp ロボット制御設定支援装置
JP2014240110A (ja) * 2013-06-12 2014-12-25 三菱電機株式会社 ビンピッキング性能評価装置及び方法
JP2018051684A (ja) 2016-09-28 2018-04-05 ファナック株式会社 複数個の物品をひとつずつ最適な経路で取り出す搬送装置
JP2018167361A (ja) * 2017-03-30 2018-11-01 株式会社安川電機 ロボット動作指令生成方法、ロボット動作指令生成装置及びコンピュータプログラム
WO2018194094A1 (ja) * 2017-04-19 2018-10-25 株式会社安川電機 プログラミング支援装置、ロボットシステム、プログラミング支援方法及びプログラム生成方法

Non-Patent Citations (2)

* Cited by examiner, † Cited by third party
Title
KUMAZAWA, TSUTOMU ET AL.: "A Study on How to Elicit Anticipated Scenarios from Use Cases for Safety Requirements Analysis", IPSJ SIG TECHNICAL REPORT, vol. 2007, no. 33, pages 191 - 198 *
See also references of EP4023396A4

Cited By (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
WO2024180756A1 (ja) * 2023-03-02 2024-09-06 日本電気株式会社 制御システム、制御方法、および記録媒体

Also Published As

Publication number Publication date
JP7264253B2 (ja) 2023-04-25
JPWO2021038842A1 (https=) 2021-03-04
EP4023396A4 (en) 2022-09-07
US12521881B2 (en) 2026-01-13
US20220355475A1 (en) 2022-11-10
EP4023396A1 (en) 2022-07-06

Similar Documents

Publication Publication Date Title
JP7264253B2 (ja) 情報処理装置、制御方法及びプログラム
JP7452619B2 (ja) 制御装置、制御方法及びプログラム
JP7416197B2 (ja) 制御装置、制御方法及びプログラム
JP7601215B2 (ja) 制御装置、制御方法及びプログラム
JP7750282B2 (ja) 制御装置、制御方法及びプログラム
JP7364032B2 (ja) 制御装置、制御方法及びプログラム
JP7456552B2 (ja) 情報処理装置、情報処理方法及びプログラム
JP7609169B2 (ja) 制御装置、制御方法及び、プログラム
JP7276466B2 (ja) 情報処理装置、制御方法及びプログラム
JP7468694B2 (ja) 情報収集装置、情報収集方法及びプログラム
JP7435814B2 (ja) 時相論理式生成装置、時相論理式生成方法及びプログラム
JP7435815B2 (ja) 動作指令生成装置、動作指令生成方法及びプログラム
JP7323045B2 (ja) 制御装置、制御方法及びプログラム
JP7448024B2 (ja) 制御装置、制御方法及びプログラム
JP7485058B2 (ja) 判定装置、判定方法及びプログラム
WO2021171352A1 (ja) 制御装置、制御方法及び記録媒体
JP7687389B2 (ja) 動作計画装置、動作計画方法及びプログラム
JP7416199B2 (ja) 制御装置、制御方法及びプログラム
WO2022074827A1 (ja) 命題設定装置、命題設定方法及び記憶媒体

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 19943354

Country of ref document: EP

Kind code of ref document: A1

ENP Entry into the national phase

Ref document number: 2021541931

Country of ref document: JP

Kind code of ref document: A

NENP Non-entry into the national phase

Ref country code: DE

ENP Entry into the national phase

Ref document number: 2019943354

Country of ref document: EP

Effective date: 20220330

WWG Wipo information: grant in national office

Ref document number: 17636073

Country of ref document: US