WO2023037634A1 - 指令値生成装置、方法、及びプログラム - Google Patents
指令値生成装置、方法、及びプログラム Download PDFInfo
- Publication number
- WO2023037634A1 WO2023037634A1 PCT/JP2022/015158 JP2022015158W WO2023037634A1 WO 2023037634 A1 WO2023037634 A1 WO 2023037634A1 JP 2022015158 W JP2022015158 W JP 2022015158W WO 2023037634 A1 WO2023037634 A1 WO 2023037634A1
- Authority
- WO
- WIPO (PCT)
- Prior art keywords
- command value
- robot
- generator
- state data
- unit
- Prior art date
Links
- 238000000034 method Methods 0.000 title claims description 60
- 230000033001 locomotion Effects 0.000 claims description 84
- 230000009471 action Effects 0.000 claims description 34
- 238000001514 detection method Methods 0.000 claims description 23
- 230000005856 abnormality Effects 0.000 claims description 12
- 238000012937 correction Methods 0.000 claims description 6
- 230000002146 bilateral effect Effects 0.000 claims description 5
- 238000005457 optimization Methods 0.000 claims description 4
- 230000008569 process Effects 0.000 description 51
- 238000012545 processing Methods 0.000 description 26
- 238000010586 diagram Methods 0.000 description 23
- 238000003860 storage Methods 0.000 description 23
- 230000006870 function Effects 0.000 description 10
- 238000005516 engineering process Methods 0.000 description 5
- 238000013486 operation strategy Methods 0.000 description 5
- 238000004088 simulation Methods 0.000 description 5
- 238000013528 artificial neural network Methods 0.000 description 4
- 230000007704 transition Effects 0.000 description 4
- 238000004364 calculation method Methods 0.000 description 3
- 238000004891 communication Methods 0.000 description 3
- 238000003780 insertion Methods 0.000 description 3
- 230000037431 insertion Effects 0.000 description 3
- 230000004048 modification Effects 0.000 description 3
- 238000012986 modification Methods 0.000 description 3
- 230000008859 change Effects 0.000 description 2
- 238000006243 chemical reaction Methods 0.000 description 2
- 239000011248 coating agent Substances 0.000 description 2
- 238000000576 coating method Methods 0.000 description 2
- 238000013135 deep learning Methods 0.000 description 2
- 230000010354 integration Effects 0.000 description 2
- 238000004519 manufacturing process Methods 0.000 description 2
- 238000003825 pressing Methods 0.000 description 2
- 239000000758 substrate Substances 0.000 description 2
- 230000001133 acceleration Effects 0.000 description 1
- 239000003990 capacitor Substances 0.000 description 1
- 230000008602 contraction Effects 0.000 description 1
- 230000012447 hatching Effects 0.000 description 1
- 238000003384 imaging method Methods 0.000 description 1
- 239000000463 material Substances 0.000 description 1
- 230000009291 secondary effect Effects 0.000 description 1
- 239000004065 semiconductor Substances 0.000 description 1
- 239000007787 solid Substances 0.000 description 1
Images
Classifications
-
- B—PERFORMING OPERATIONS; TRANSPORTING
- B25—HAND TOOLS; PORTABLE POWER-DRIVEN TOOLS; MANIPULATORS
- B25J—MANIPULATORS; CHAMBERS PROVIDED WITH MANIPULATION DEVICES
- B25J9/00—Programme-controlled manipulators
- B25J9/16—Programme controls
- B25J9/1656—Programme controls characterised by programming, planning systems for manipulators
-
- G—PHYSICS
- G05—CONTROLLING; REGULATING
- G05B—CONTROL OR REGULATING SYSTEMS IN GENERAL; FUNCTIONAL ELEMENTS OF SUCH SYSTEMS; MONITORING OR TESTING ARRANGEMENTS FOR SUCH SYSTEMS OR ELEMENTS
- G05B19/00—Programme-control systems
- G05B19/02—Programme-control systems electric
- G05B19/42—Recording and playback systems, i.e. in which the programme is recorded from a cycle of operations, e.g. the cycle of operations being manually controlled, after which this record is played back on the same machine
Definitions
- the present disclosure relates to a command value generation device, a command value generation method, and a command value generation program.
- a feedback control system is configured to feed back data acquired by various sensors to the control of the robot.
- Non-Patent Document 1 For example, manually guide the motion of a robot arm capable of controlling external force, record the position of the robot and the acting external force at that time, and output the command value for the motion of the robot arm so as to reproduce the recorded information. A technique for doing so has been proposed (Non-Patent Document 1).
- Non-Patent Document 2 a technology has been proposed that uses deep learning to generate a multi-modal integration model of sensor data such as voice and images and data acquired from robot motion, and outputs command values for robot motion from this model.
- the user In order to configure the feedback control system, the user must define and implement the sensor feature values and the method of generating command values for the robot based on them. Also, even if the definition is correct, if the gain adjustment from the sensor feature value to the command value to the robot is inappropriate, the task will not succeed, and there is a problem with the definition, implementation, or adjustment. It is also difficult to determine whether
- the force control system being executed uses the logging data of the position and force when a person teaches with help as it is as a command value input of the force control system. Low robustness.
- Non-Patent Document 2 does not use a force sensor and cannot robustly execute applications such as assembly and pick-and-place.
- the present disclosure has been made in view of the above points, and aims to configure a feedback control system for causing a robot to robustly perform work on an object that can be in various states.
- a command value generation device provides a command value for causing a robot to perform work on an object, and the above command value when manually teaching the operation of the robot during the work state data representing the state of the robot, comprising at least motion data representing the motion of the robot, position and orientation data representing the relative position and orientation of the robot and the object, and an acquisition unit configured to acquire a plurality of types of state data including external force data representing an external force applied to an object; a generation unit that generates a generator that generates a command value for causing the robot to perform an action corresponding to the state data.
- the generator may generate the generator by determining parameters in the generator based on optimization. Thereby, robustness can be improved more.
- the command value generation device includes a reception for receiving a selection of a portion of the state data to be used for generation by the generator from among the state data acquired by the acquisition unit for each of a plurality of teachings. , wherein the generator utilizes selected portions of the state data to generate the generator. This makes it possible to eliminate parts of the state data that are not suitable for generation by the generator.
- the receiving unit receives selection of a type of the state data to be used for generation by the generator from among the plurality of types of the state data obtained by the obtaining unit, and the generating unit receives the selected type
- the generator is generated by optimizing parameters for generating a command value that can reproduce the state represented by the selected type of state data based on the state data and the command value of good too. This makes it possible to generate a generator that generates convincing command values.
- the generation unit may receive correction of the generated parameter of the generator. As a result, it is possible to correct in advance parameters that are clearly inappropriate, parameters that are not in line with the user's intention, and the like.
- the parameters of the generator include an upper limit value of the command value and a target value of the operation with respect to the command value, and the generator fixes the upper limit value and the target value to specified values
- the generator may be generated by optimizing other parameters. As a result, it is possible to generate a generator capable of outputting a command value for realizing the robot motion desired by the user.
- the command value generation device is generated when the state data in which a perturbation term is added to a parameter that may vary in the work is input to the generator generated by the generation unit.
- an instruction unit that determines whether the robot is operable based on the command value, and instructs the acquisition unit to acquire the command value generated when the perturbation term is added and the state data if the robot is not operable; may be configured to include Thereby, it is possible to automatically determine whether or not sufficient state data for generation of the generator has been acquired by manual teaching.
- the generation unit performs at least one of deleting a part of the state data used for generating the generator and adding the state data newly acquired by the acquisition unit to generate the generator. You can rerun the generation. As a result, when an unintended operation is executed based on the generated command value, the generation of the generator is redone, thereby improving the quality of the operation by the feedback control system.
- the acquisition unit acquires an image of a work area including the object at the time of the teaching, and sets parameters for recognizing the work area based on the image acquired by the acquisition unit.
- the obtaining unit obtains a camera for capturing the image and the object, which are calculated based on a preset size of the object and a size on the image of the object recognized from the image. You can also get the distance to an object. As a result, the distance to the object can be accurately obtained without using a special sensor.
- Manual teaching of the robot's motion may be performed by direct teaching, remote control from a controller, or remote control using a teaching device connected to the robot via bilateral control.
- command value generation device may include a control unit that outputs the command value generated by the generator to control the robot.
- the command value generation device estimates the state data by inputting the command value generated by the generator into the generator and calculating back, and estimating the state data, the estimated state data, and the A detection unit may be included that detects an abnormality that occurs during the work by the robot by comparing the status data acquired by the acquisition unit.
- the acquisition unit provides command values for causing a robot to perform work on an object, State data representing a state, comprising at least motion data representing the motion of the robot, position and orientation data representing the relative position and orientation of the robot and the object, and the object moving during the work.
- State data representing a state
- a plurality of types of state data including external force data representing the external force received is acquired, and the generation unit generates the input state based on the command value and the state data acquired at the corresponding time by the acquisition unit
- a method for generating a generator for generating a command value for causing the robot to perform an action corresponding to data is generating a generator for generating a command value for causing the robot to perform an action corresponding to data.
- the command value generation program provides a computer with a command value for causing a robot to perform work on an object, and the state of the robot when manually instructed to operate the robot during the work.
- a computer with a command value for causing a robot to perform work on an object, and the state of the robot when manually instructed to operate the robot during the work.
- command value generation device it is possible to configure a feedback control system for causing a robot to robustly perform work on an object that can be in various states.
- FIG. 1 is a schematic diagram of a robot control system according to first to third and fifth embodiments;
- FIG. FIG. 4 is a diagram for explaining an example of manual teaching of a motion to a robot;
- FIG. 10 is a diagram for explaining another example of manually teaching a robot an action;
- FIG. 10 is a diagram for explaining another example of manually teaching a robot an action;
- FIG. 4 is a diagram showing an example of work for explaining the requirements of the sensor;
- It is a block diagram which shows the hardware constitutions of a command value production
- It is a block diagram showing an example of functional composition of a command value generation device concerning the 1st and 3rd embodiments. It is a figure for demonstrating the conversion from sensor data to state data.
- FIG. 4 is a diagram for explaining a generator;
- FIG. FIG. 4 is a diagram for explaining feedback control using a command value generated by a generator;
- FIG. 4 is a flowchart showing the flow of learning processing in the first embodiment;
- 4 is a flowchart showing the flow of control processing;
- It is a block diagram showing an example of functional composition of a command value generation device concerning a 2nd embodiment.
- 9 is a flowchart showing the flow of learning processing in the second embodiment; It is a figure which shows an example of a kind selection screen.
- FIG. 11 is a diagram showing an example of a schematic configuration of a generator in a third embodiment;
- FIG. 11 is a flowchart showing the flow of learning processing in the third embodiment; It is a schematic diagram of a robot control system according to a fourth embodiment. It is a block diagram showing an example of functional composition of a command value generating device concerning a 4th embodiment.
- FIG. 14 is a flowchart showing the flow of learning processing in the fourth embodiment;
- FIG. 4 is a diagram for explaining determination of a motion target based on a distance to an object; It is a block diagram showing an example of functional composition of a command value generating device concerning a 5th embodiment. It is a figure for demonstrating the process of a detection part. 4 is a flowchart showing the flow of detection processing;
- the robot control system 1 includes a command value generation device 10, a robot 40, and a sensor group 50. As shown in FIG. 1, the robot control system 1 according to the first embodiment includes a command value generation device 10, a robot 40, and a sensor group 50. As shown in FIG. 1, the command value generation device 10, a robot 40, and a sensor group 50.
- the robot 40 includes a robot arm 42 and a hand section 44.
- the robot arm 42 includes links and joints that connect the links and rotate or linearly expand and contract by being driven by a motor.
- the motor is driven according to the command value output from the command value generation device 10, and the rotation angle or expansion/contraction state of the joint is changed.
- the hand unit 44 is controlled to assume a specified position and a specified posture in the three-dimensional space.
- the hand part 44 is a tool provided at the tip of the robot arm 42 and capable of gripping an object 90 .
- the hand unit 44 may be, for example, an articulated multi-fingered robot hand, a gripper-type robot hand, a suction pad, or the like.
- the robot 40 may be a tool suitable for the task, such as having a nozzle serving as a discharge port for the coating material instead of the hand part 44 .
- the robot 40 is configured to be able to be instructed to operate manually.
- the manual motion teaching may be, for example, direct teaching, that is, teaching with the help of a human hand to teach the motion, as shown in FIG.
- a person may operate the controller 60 to teach the motion of the robot 40 by remote control.
- teaching may be performed by remote control using a teaching device 62 connected to the robot 40 by bilateral control. Through bilateral control, the force applied by the teaching device 62 is transmitted to the robot 40 , and the external force acting on the robot 40 is transmitted to the teaching device 62 .
- the sensor group 50 includes multiple types of sensors, and the sensor data acquired by each sensor is output to the command value generation device 10 as time-series data.
- the sensor group 50 is conceptually represented by one block near the hand unit 44, but each sensor included in the sensor group 50 is located at a position corresponding to the type and function of the sensor. be provided.
- the work shown in FIG. 5 is a work of gripping the main object 90A with the hand portion 44 and fitting the main object 90A and the sub-object 90B.
- insertion of a connector For example, insertion of a substrate into a housing, insertion of an electrolytic capacitor into a substrate, and the like.
- it is considered to use an operation strategy that aligns the position and posture of the main object 90A by contacting the sides and surfaces of the main object 90A and the sub object 90B.
- Such fitting using contact makes it easy to match the position and orientation of the main object 90A to the fitting position even when the resolution of the sensor or actuator is rough.
- the "action strategy” is the execution order of "action primitives” for executing "actions".
- "Movement” is a movement that has a purpose by itself, and is a movement with a purpose, such as “grab”, “move”, “engage”, etc. realized by “movement strategy” .
- a “movement primitive” is a minimum unit of movement of a robot, such as “hold”, “move”, and “apply”.
- the “movement primitive” here is a set target, for example, “lay diagonally”, “move to the position where the corner enters the hole", and “apply to the edge". In the example of FIG.
- each portion indicated by a dashed line corresponds to an "action”
- each portion indicated by a one-dot chain line corresponds to an “action strategy”
- each portion indicated by a two-dot chain line corresponds to an "action primitive”.
- the execution order of the "actions” is referred to as an "action sequence”.
- Fig. 5 shows an example of an operation sequence including each operation of "1. Grab”, “2. Move”, and “3. Fit”.
- action primitives of "1.1. Move onto the main object with rough positioning” and “1.2. Grab the main object” are defined.
- “1.1. Move onto the main object with rough positioning” is a movement to move the hand unit 44 to a position where the main object 90A can be grasped.
- “1.2. Grasping the main object” is a movement for grasping the main object 90A with the hand unit 44 .
- the action primitive “2.1. Move above the hole with rough positioning” is defined.
- “2.1. Rough positioning move to the top of the hole” is a movement to move the main object 90A in the X-axis direction and the Y-axis direction in accordance with the fitting position.
- Tracing is a movement to move the main object 90A in the X-axis direction and the Y-axis direction while maintaining constraint with the surface of the sub-object 90B.
- Searching motion is a movement of tracing the surface of the sub-object 90B with the main object 90A and searching for the fitting position.
- XY posture correction while tracing corrects the position and posture of the main object 90A in the X-axis direction and the Y-axis direction using the alignment of the main object 90A with the sub-object 90B. It's movement.
- Insert into a hole is a motion to move the main object 90A downward in the Z-axis direction while maintaining restraint with the inner circumference of the fitting position (hole) of the sub-object 90B.
- Engagement Relief Operation is a movement that changes the posture of the main object 90A so as to eliminate entrapment.
- the sensor requirements necessary to control the robot 40 to execute each of the above motion primitives are as follows.
- Regarding “1.1. Move to the main object by rough positioning” the positions of the hand unit 44 and the main object 90A in each of the X, Y, and Z axial directions and the rotational direction about the Z axis It is possible to recognize attitude errors.
- Regarding “1.2. Gripping the main object” it is possible to recognize the gripping force by the hand unit 44 .
- Regarding “2.1. Move to the top of the hole by rough positioning” the fitting between the main object 90A and the sub object 90B is performed in each of the X, Y, and Z axial directions and in the rotational direction about the Z axis. It is possible to recognize the positional error with respect to the position.
- the sensors that meet the above requirements include the following.
- a vision sensor, a 3D sensor, or the like capable of detecting the position of an object in a three-dimensional space can be applied.
- a force sensor, a pad sensor, a difference between a command value and a hand encoder, or the like can be applied as a sensor capable of recognizing the gripping force.
- the pad sensor is a sensor for detecting the amount of deformation of the suction pad in the case of the hand unit 44 that sucks and grips the object 90, and is, for example, a proximity sensor or a pressure sensor.
- a force sensor, a pad sensor, or the like can be applied as a sensor capable of detecting the reaction force in each axial direction. Also, as a sensor capable of recognizing the posture of the main object 90A, an arm encoder, a hand encoder, a pad sensor, or the like can be applied.
- the arm encoder is an encoder that detects the rotation angle of each joint of the robot arm 42
- the hand encoder is an encoder that detects the rotation angle of each joint of the hand unit 44, the opening degree of the gripper, and the like. .
- the hand part 44 is of a multi-fingered multi-joint type and one target 90 is the work target.
- the sensors included in the sensor group 50 are a vision sensor, an arm encoder, a hand encoder, and a pad sensor provided at the tip of the hand portion 44 will be described.
- the vision sensor is attached to the tip of the hand portion 44 so that the imaging direction is parallel to the Z-axis of the coordinate system of the hand portion 44 . Thereby, the distance between the vision sensor and the object is regarded as the distance between the tip of the hand part 44 and the object.
- the sensors included in the sensor group 50 are not limited to this example, and the relative position and orientation between the target object 90 and the hand unit 44, the relative position between the main target object 90A and the sub-target object 90B, and the target object A sensor capable of detecting an external force acting on 90 may be included.
- FIG. 6 is a block diagram showing the hardware configuration of the command value generation device 10 according to the first embodiment.
- the command value generation device 10 includes a CPU (Central Processing Unit) 12, a memory 14, a storage device 16, an input/output I/F (Interface) 18, an input/output device 20, a storage medium reader 22, and a communication I/F 24.
- Each component is communicatively connected to each other via a bus 26 .
- the storage device 16 stores a command value generation program including a learning program for executing learning processing, which will be described later, and a control program for executing control processing.
- the CPU 12 is a central processing unit that executes various programs and controls each configuration. That is, the CPU 12 reads a program from the storage device 16 and executes the program using the memory 14 as a work area. The CPU 12 performs control of the above components and various arithmetic processing according to programs stored in the storage device 16 .
- the memory 14 is composed of RAM (Random Access Memory) and temporarily stores programs and data as a work area.
- the storage device 16 is composed of ROM (Read Only Memory), HDD (Hard Disk Drive), SSD (Solid State Drive), etc., and stores various programs including an operating system and various data.
- the input/output I/F 18 is an interface for connecting each of the robot 40 and the sensor group 50 to the command value generation device 10 .
- Sensor data output from each sensor included in sensor group 50 is input to command value generation device 10 via input/output I/F 18 .
- the command value generated by the command value generating device 10 is output to the robot 40 via the input/output I/F 18 .
- the input/output device 20 is, for example, an input device such as a keyboard and a mouse for performing various inputs, and an output device such as a display and a printer for outputting various information. By adopting a touch panel display as an output device, it may be made to function as an input device.
- the storage medium reading device 22 reads data stored in various storage media such as CD (Compact Disc)-ROM, DVD (Digital Versatile Disc)-ROM, Blu-ray Disc, USB (Universal Serial Bus) memory, etc. writes data to the
- the communication I/F 24 is an interface for communicating with other devices, and uses standards such as Ethernet (registered trademark), FDDI, and Wi-Fi (registered trademark), for example.
- FIG. 7 is a block diagram showing an example of the functional configuration of the command value generating device 10.
- the command value generation device 10 includes an acquisition unit 31, a reception unit 32, a generation unit 33, and a control unit 38 as functional configurations.
- Each functional configuration is realized by the CPU 12 reading out a command value generating program stored in the storage device 16, developing it in the memory 14, and executing it.
- the acquisition unit 31 acquires command values for causing the robot 40 to perform work on the object 90, and state data representing the state of the robot 40 when the motion of the robot 40 during the work is manually taught.
- the state data is, for example, motion data representing the motion of the robot 40 during the teaching, position/orientation data representing the relative position and relative posture between the robot 40 and the object, and external force applied to the object during the operation. Includes multiple types of data such as external force data.
- the acquisition unit 31 acquires sensor data from each of the sensors included in the sensor group 50 and converts the sensor data into state data.
- the acquisition unit 31 acquires an image, which is sensor data of the vision sensor, and calculates the position and orientation of the object 90 within the image, that is, on the XY plane.
- the acquisition unit 31 also acquires sensor data from the arm encoder, and based on this sensor data and the kinematics information of the robot arm 42, the posture (roll angle, pitch angle, yaw angle). Further, the acquisition unit 31 acquires sensor data of the hand encoder, and the sensor data and the hand unit 4 4, the posture of the tip of the hand unit 44 (hereinafter referred to as "fingertip") relative to the hand is calculated.
- the acquisition unit 31 also acquires sensor data from the pad sensor, and calculates the external force acting on each fingertip from this sensor data.
- the acquisition unit 31 also calculates the orientation of the object 90 gripped by the hand unit 44 with respect to the fingertip from the sensor data.
- the acquisition unit 31 resolves the distance from the hand to the target object 90 in each of the X, Y, and Z axial directions based on the calculated position of the target object 90 in the image and the posture of the hand. A value is obtained as relative position data of the object 90 with respect to the hand.
- the acquisition unit 31 also calculates the orientation of the fingertip in the absolute coordinate system based on the orientation of the hand and the orientation of the fingertip with respect to the hand.
- the acquiring unit 31 acquires, as external force data, values obtained by resolving the external force acting on each fingertip in the directions of the X, Y, and Z axes based on the posture of the fingertip in the absolute coordinate system.
- the acquisition unit 31 acquires the orientation of the object 90, which is specified based on the orientation of the object 90 gripped by the hand unit 44 with respect to the fingertip, the orientation of the hand, and the orientation of the fingertip with respect to the hand.
- the object 90 is obtained as object relative posture data.
- the acquisition unit 31 also calculates the angular velocity of rotation of each joint of the robot arm 42 and the hand unit 44 from the sensor data of the arm encoder and the hand encoder.
- the acquisition unit 31 also calculates the speed of the hand based on the sensor data of the arm encoder and the Jacobian of the robot arm 42 .
- the acquisition unit 31 also calculates the fingertip velocity based on the sensor data of the hand encoder and the Jacobian of the hand unit 44 .
- the acquisition unit 31 acquires the speed of the hand and fingertips as motion data.
- each sensor data is time-series data
- each of the converted relative position data, relative attitude data, external force data, and motion data is also time-series data.
- the receiving unit 32 receives a selection of a portion of the state data to be used for generating a generator, which will be described later, from the state data obtained by the obtaining unit 31 for each of the multiple manual teachings.
- the state data part is the state data acquired for each of the partial teachings selected from multiple teachings, and the state data acquired for one teaching, the portion included in the specified time range including both.
- the receiving unit 32 displays a partial selection screen 70 as shown in FIG. 9, and receives information (hereinafter referred to as "selected information") selected as a portion of the state data used for generating the generator.
- the partial selection screen 70 shown in FIG. 9 includes a selection area 73 for selecting the type of state data. Further, the partial selection screen 70 selects whether or not to adopt the state data for each of the plurality of teachings for the type of state data selected in the selection area 73 as state data to be used for generation of the generator. includes a selection area 71 for In the example of FIG.
- the selection area 71 includes a "number of trials” that identifies each of a plurality of teachings, a “time” at which the teaching was performed, a “target” to be checked if adopted, and a "partial adoption”. Each item of is included.
- the partial selection screen 70 also includes a display area 72 in which each type of state data selected in the selection area 73 is displayed in graph form.
- the state data graph selected in the selection area 71 is highlighted.
- the graph of selected state data is represented by a solid line, and the graph of other state data is represented by dotted lines.
- the teaching of the number of trials selected in the selection area 71 is indicated by hatching.
- the display area 72 also includes a slide bar (broken line in FIG. 9) for designating the start time and end time of the time range to be selected. A time range is selected by sliding the slide bar.
- the partial selection screen 70 also includes a display area 74 that displays the time range designated by the slide bar within the display area 72 .
- the partial selection screen 70 also includes a display area 75 in which the image acquired by the vision sensor at the time specified in the display area 72 (black triangle in FIG. 9) is displayed. Thereby, the user can refer to the image displayed in the display area 75 and select the time range.
- "partial adoption" in the display area 71 is "yes", and if no time range is specified, it is "no".
- the generation unit 33 generates a generator based on the state data of the portion indicated by the selection information received by the reception unit 32 among the state data acquired by the acquisition unit 31 and the command value for the corresponding time. .
- the generator generates and outputs a command value for causing the robot 40 to perform an action corresponding to the input state data.
- the generation unit 33 generates a generator by learning the weight of each layer of the neural network as a parameter using a plurality of combinations of command values and state data.
- the control unit 38 outputs command values generated by the generator generated by the generation unit 33 to control the operation of the robot 40 .
- the control unit 38 receives the state data T(t) from the acquisition unit 31 and inputs it to the generator as shown in the control phase in the lower diagram of FIG. From this, the command value (in this case, the command speed) at time t corresponding to the current state indicated by the state data T(t) from the generator Cv ⁇ (in FIG. )”)(t) is output from the generator.
- the control unit 38 outputs this command value Cv ⁇ (t) to each motor M of the robot 40 . Accordingly, the robot 40 operates by driving each motor based on the command value.
- the sensor data group S(t) at time t is the state data T(t) at time t, namely relative position data Tp(t), relative attitude data T ⁇ (t), and external force data Tf(t). , and motion data Tv(t), which are input to the generator.
- the generator generates a command value Cv ⁇ (t) at time t based on the input state data T(t), and the command value Cv ⁇ (t) is output to each motor M of the robot 40.
- the robot 40 operates by driving the motor M based on the command value Cv ⁇ (t).
- the actual angular acceleration Sea(t+1), the actual angular velocity Se ⁇ (t+1), and the actual angle Seq(t+1) are acquired as the actual movement Se(t+1) by the arm encoder and the hand encoder.
- This actual operation Se(t+1) and the sensor data of the vision sensor and the pad sensor acquired at time t+1 form the sensor data group S(t+1) at the next time t+1.
- the CPU 12 reads out a learning program from the storage device 16, develops it in the memory 14, and executes it, so that the CPU 12 functions as each functional component of the command value generation device 10, and the learning process is executed.
- the CPU 12 reads the control program from the storage device 16, develops it in the memory 14, and executes it, so that the CPU 12 functions as each functional configuration of the command value generation device 10, and the control process is executed.
- FIG. 12 is a flowchart showing the flow of learning processing executed by the CPU 12 of the command value generation device 10.
- the learning process is performed for each motion in the motion sequence that the robot 40 is caused to perform.
- step S11 the control unit 38 controls the robot 40 so that it assumes the starting position and posture of the movement targeted for learning processing.
- the control unit 38 may set the end position and orientation of the motion for which the learning process was performed previously among the motions in the motion sequence as the start position and posture of the target motion of the current learning process.
- step S12 the acquisition unit 31 determines whether or not an instruction to start teaching has been given by the user, for example, by pressing a button indicating start of teaching. If instructed to start teaching, the process proceeds to step S13, and if not instructed, the determination of this step is repeated. After instructing the start of teaching, the user manually teaches the robot 40 the target motion.
- step S ⁇ b>13 the acquisition unit 31 acquires the command value corresponding to the motion to be taught, and acquires sensor data from each sensor included in the sensor group 50 .
- step S14 the obtaining unit 31 determines whether or not the end of teaching has been instructed by, for example, pressing a button indicating the end of teaching by the user. If instructed to end teaching, the process proceeds to step S15, and if not instructed, the process returns to step S13.
- step S15 the acquisition unit 31 converts the sensor data acquired in step S13 into state data.
- step S16 the acquisition unit 31 determines whether or not teaching has been completed a predetermined number of times. When the teaching for the predetermined number of times has been completed, the process proceeds to step S17, and when the teaching has not been completed, the process returns to step S11.
- step S17 the reception unit 32 displays the partial selection screen 70 and receives selection information of the state data used for generating the generator.
- step S18 the generation unit 33 generates a generator using the portion of the state data indicated by the selection information and the corresponding command value, and the learning process ends.
- FIG. 13 is a flowchart showing the flow of control processing executed by the CPU 12 of the command value generating device 10.
- step S21 the acquisition unit 31 acquires sensor data from each sensor included in the sensor group 50.
- step S22 the acquisition unit 31 converts the sensor data acquired in step S21 into state data.
- step S23 the control unit 38 receives the state data from the acquisition unit 31 and inputs it to the generator to generate a command value.
- step S24 the controller 38 outputs the generated command value to each motor of the robot 40, and the process returns to step S21.
- the command value generation device manually generates command values for causing the robot to perform work on an object and the motion of the robot during the work.
- Acquire state data representing the state of the robot at the time of teaching by .
- the state data includes multiple types of data, including motion data representing the motion of the robot, position and orientation data representing the relative position and orientation between the robot and the object, and external force data representing the external force that the object receives during work. be.
- the command value generator generates a command value for causing the robot to perform an action corresponding to the input state data, based on the command value and the state data acquired at the corresponding time. This makes it possible to configure a feedback control system for causing the robot to robustly perform work on an object that can be in various states.
- the force control system being executed uses the logging data of the position and force when a person teaches with help as it is as a command value input of the force control system.
- Low robustness In order to improve robustness, it is necessary to internally estimate a feature quantity that accurately expresses the current state and generate a command value based on the feature quantity, but the technology described in Non-Patent Document 1 , that structure does not exist.
- a neural network is a model that has a structure that can internally have feature values. The neural network transforms the data from the input layer by changing the network weights so that features appear in the hidden layers.
- the command value generation device in the above embodiment can configure a sensor feedback system that can ensure more robustness.
- the command value generation device accepts selection of whether or not to adopt each of the state data acquired by teaching multiple times and the time range of the state data to be adopted, and uses the state data of the selected part Create a generator.
- the command value generation device accepts selection of whether or not to adopt each of the state data acquired by teaching multiple times and the time range of the state data to be adopted, and uses the state data of the selected part Create a generator.
- the command value generation device may collect the compliance parameter together with the external force data when the work is performed by the teaching device. Then, the command value generation device uses the collected external force data and the compliance parameter to learn the weight of each layer of the generator configured with the autoencoder as in the above embodiment as a parameter, and uses the external force data as an input. , may generate a generator that outputs the compliance parameter. Thereby, even if it is necessary to change the compliance parameter of the hand dynamically according to the situation, the change can be automatically performed.
- the command value generating device deletes part of the state data used to generate the generator and adds newly obtained state data to generate the generator. You can try again. Specifically, when deleting a part of the state data, after generating the generator, the user confirms the operation of the robot based on the command value output from the generator, and selects a part as shown in FIG. You can select the state data to be deleted from a screen similar to the screen. When adding state data, the state data to be added may be obtained by manually teaching again about the unnatural portion of the motion sequence. As a result, when an unintended operation is executed based on the generated command value, the generation of the generator is redone, thereby improving the quality of the operation by the feedback control system.
- the robot control system 2 includes a command value generation device 210, a robot 40, and a sensor group 50.
- FIG. 14 is a block diagram showing an example of the functional configuration of the command value generation device 210.
- the command value generation device 210 includes an acquisition unit 231, a generation unit 33, an instruction unit 234, and a control unit 38 as a functional configuration.
- Each functional configuration is realized by the CPU 12 reading out a command value generating program stored in the storage device 16, developing it in the memory 14, and executing it.
- the hardware configuration of the command value generation device 210 is the same as the hardware configuration of the command value generation device 10 according to the first embodiment shown in FIG. 6, so description thereof will be omitted.
- the instruction unit 234 determines whether the robot 40 is operable based on the command value generated when the state data including the perturbation term is input to the generator generated by the generation unit 33 .
- Perturbation terms relate to parameters that may vary in assembly or pick-and-place applications. For example, it relates to parameters such as the assumed size and mass of the pick-and-place object 90, the initial position, the target position, and the coefficient of friction of the object 90 and the like.
- the instruction unit 234 adds or subtracts a value corresponding to the size of the object 90 to or from at least one of the relative position data and the relative orientation data as the perturbation term.
- the value corresponding to the size of the object 90 may be designated, for example, as a ratio to the size of the object 90, or may be designated as a specific numerical value such as "10 mm".
- the instruction unit 234 executes a motion simulation of the robot 40 based on the command value generated with the perturbation term added, and determines whether or not the robot 40 is operable. Determination of whether or not the robot is operable may be performed by determining whether or not the work to be executed in a series of motion sequences is completed, or by setting a motion target value and determining whether or not the motion target value has been achieved. It may be determined whether
- the instruction unit 234 determines that it is not operable, it instructs the acquisition unit 231 to acquire the command value and state data generated when the perturbation term is taken into account. Specifically, the instruction unit 234 displays the trajectory of the robot 40 with the perturbation term added to the user by, for example, displaying it on a display device, and controls the robot 40 so as to assume the starting position and posture of the trajectory.
- the control unit 38 is instructed to
- FIG. 15 is a flowchart showing the flow of learning processing executed by the CPU 12 of the command value generation device 210.
- FIG. When the CPU 12 reads the learning program from the storage device 16, expands it in the memory 14, and executes it, the CPU 12 functions as each functional component of the command value generating device 210, and the learning process shown in FIG. 15 is executed.
- the same step numbers are assigned to the same processes as the learning process (FIG. 12) in the first embodiment, and detailed description thereof is omitted.
- step S11 the control unit 38 controls the robot 40 so that it assumes the starting position and posture of the movement targeted for learning processing.
- step S200 an acquisition process is performed.
- the acquisition process is the same as steps S12 to S15 of the learning process shown in FIG.
- step S16 the acquisition unit 231 determines whether or not teaching for a predetermined number of times has been completed. If the teaching for the predetermined number of times has been completed, the process proceeds to step S211, and if not, the process returns to step S11.
- step S211 the generation unit 33 generates a generator using the acquired state data and command value.
- step S212 the instruction unit 234 simulates the motion of the robot 40 based on the command value generated when state data including the perturbation term is input to the generated generator.
- step S213 the instruction unit 234 determines whether or not the robot 40 is operable when the perturbation term is taken into consideration as a result of the simulation. If it is operable, the learning process ends, and if it is not operable, the process proceeds to step S214.
- step S214 the instruction unit 234 displays the trajectory of the robot 40 with the perturbation term added to the user by, for example, displaying it on a display device, and controls the robot 40 so as to assume the starting position and posture of the trajectory. is instructed to the control unit 38, and the process returns to step S200.
- the command value generating device generates a command value when the state data including the perturbation term is input to the generated generator. Based on this, it is determined whether the robot is operable. Then, if the device is not operable, the command value generation device instructs the acquisition unit to acquire the command value and state data generated when the perturbation term is added. Thereby, it is possible to automatically determine whether or not sufficient state data for learning of the generator has been obtained by manual teaching. Therefore, even a user who is unfamiliar with the robot system can determine whether or not the data necessary for generating the generator has been collected.
- the robot control system 3 includes a command value generation device 310, a robot 40, and a sensor group 50.
- a command value is generated by inputting a plurality of types of state data into the generator, and the inside of the generator is in a black box state, and there are cases where the generated command value is not convincing. be. Therefore, in the third embodiment, a generator is generated that generates a command value in relation to state data selected by the user. In addition, in the third embodiment, the generator also generates information as to whether or not the target motion is achieved based on the generated command value.
- FIG. 7 is a block diagram showing an example of the functional configuration of the command value generation device 310.
- the command value generation device 310 includes an acquisition unit 31, a reception unit 332, a generation unit 333, and a control unit 38 as a functional configuration.
- Each functional configuration is realized by the CPU 12 reading out a command value generating program stored in the storage device 16, developing it in the memory 14, and executing it.
- the hardware configuration of the command value generation device 310 is the same as the hardware configuration of the command value generation device 10 according to the first embodiment shown in FIG. 6, so description thereof will be omitted.
- the receiving unit 332 displays, for example, a type selection screen 80 as shown in FIG. 16, and receives selection of the type of state data to be used for generating the generator from among the plurality of types of state data obtained by the obtaining unit 31. .
- a type selection screen 80 As shown in FIG. 16, for each type of state data, "command value object" is checked when it is used for generating a command value, and when it is used for determination of achievement of an operation goal.
- Each item of "determination target" is included.
- the generation unit 333 learns parameters of generators including a command value generator that generates a command value and a determiner that determines achievement of an operation target, based on the selected type of state data and command value. Specifically, the generation unit 333 optimizes parameters for generating command values that can reproduce the state represented by the selected type of state data, based on the selected type of state data and the command value. By doing so, a command value generator is generated.
- the command value generator may be, for example, a regression equation representing the relationship between the selected type of state data and the command value.
- the generation unit 333 may include the upper limit value of the command value in the parameter of the generator.
- the generation unit 333 generates state data of the selected type and a flag indicating whether or not the state indicated by the state data indicates that the target motion has been achieved (hereinafter referred to as "motion target achievement flag"). Generate a determiner by learning the relationship between .
- the generation unit 333 may include the operation target value in the parameters of the generator.
- the generating unit 333 reduces the coefficient of the unselected state data in the optimization of the internal parameters of each of the command value generator and the determiner, thereby generating a command value generator and a command value generator according to the selected state data.
- Generate a classifier Generate a classifier.
- the generation unit 333 sets at least one of the specified upper limit value of the command value and the operation target value to the specified value. fixed. After that, the generator 333 generates a generator by optimizing other parameters. As a result, it is possible to generate a generator capable of outputting a command value for realizing the robot motion desired by the user.
- FIG. 17 shows an example of the schematic configuration of the generator in the third embodiment.
- the input state data and the upper limit of the command value are input to the command value generator.
- the command value generator generates and outputs a command value according to the state data.
- the input state data and the operation target value are input to the determiner.
- the determiner outputs an action target achievement flag corresponding to the state data.
- FIG. 18 is a flow chart showing the flow of learning processing executed by the CPU 12 of the command value generating device 310.
- FIG. When the CPU 12 reads the learning program from the storage device 16, develops it in the memory 14, and executes it, the CPU 12 functions as each functional configuration of the command value generating device 310, and the learning process shown in FIG. 18 is executed.
- the same step numbers are assigned to the same processes as the learning process (FIG. 15) in the second embodiment, and detailed description thereof will be omitted.
- step S311 the reception unit 332 displays the type selection screen 80, and among the plurality of types of state data acquired by the acquisition unit 31, state data used for learning of the command value generator and Accepts selection of the type of state data to be used.
- step S312 the generation unit 333 instructs to generate a command value based on the state data of the type selected as the state data used for learning of the command value generator and the command value based on the operation data.
- Optimize value generator parameters the generation unit 333 optimizes the parameters of the determiner that generates the action goal achievement flag according to the state data of the type selected as the state data used for learning of the determiner. Thereby, a generator including a command value generator and a determiner is generated. Then, the learning process ends.
- control process is the same as in the first embodiment, detailed description will be omitted.
- the motion target achievement flag output from the generator indicates that the motion target has not been achieved
- the motion of the robot 40 may be controlled to stop,
- the command value may be corrected in the direction of achieving the operation target value.
- the command value generating device generates generators using the type of state data selected by the user. This makes it possible to output a command value that is more convincing to the user than a command value generated by a generator whose content is a black box.
- a partial selection screen as shown in FIG. may be accepted.
- a user interface may be provided that allows the user to check and modify the parameters of the generated generator.
- a screen that allows direct modification of the parameters of the generator may be displayed.
- a simulation image of the operation based on the command value output from the generated generator may be displayed. The user can check the operation using the simulation image, make modifications such as slowing down the operation speed, and reflect the modification of the parameters corresponding to this, such as lowering the upper limit value, in the generator. Further, it is also possible to display a simulation image of the operation based on the command value output from the generator after the parameter correction so that the content of the correction can be confirmed. As a result, it is possible to correct in advance parameters that are clearly inappropriate, parameters that are not in line with the user's intention, and the like.
- the robot control system 4 includes a command value generation device 410, a robot 40, and a sensor group 50.
- the sensor group 50 includes vision sensors. Note that the sensor group 50 includes sensors other than vision sensors as in the above-described embodiments, but FIG. 19 shows only vision sensors as sensors included in the sensor group 50 .
- FIG. 20 is a block diagram showing an example of the functional configuration of the command value generation device 410.
- the command value generation device 410 includes an acquisition unit 431, a generation unit 33, a setting unit 435, and a control unit 38 as functional configurations.
- Each functional configuration is realized by the CPU 12 reading out a command value generating program stored in the storage device 16, developing it in the memory 14, and executing it.
- the hardware configuration of the command value generation device 410 is the same as the hardware configuration of the command value generation device 10 according to the first embodiment shown in FIG. 6, so description thereof will be omitted.
- the acquisition unit 431 acquires an image of the work area including the target object during manual teaching of the operation of the robot 40 . Specifically, the acquisition unit 431 acquires the image acquired by the vision sensor.
- the acquisition unit 431 calculates the distance between the vision sensor and the object 90 based on the preset size of the object 90 and the size on the image of the object 90 recognized from the acquired image. do.
- the acquisition unit 431 stores the calculated distance between the vision sensor and the object 90 and the position coordinates of the hand of the robot 40 when the image used for calculating the distance was acquired as a set. Based on this stored information and the time-series data of the position coordinates of the hand acquired during manual teaching, the acquisition unit 431 acquires the time-series data of the distance to the object 90 as one of the state data. get as one.
- setting unit 435 sets parameters for image recognition based on the image acquired by the acquisition unit 431 .
- Setting parameters for image recognition includes optimization of parameters of a recognition model such as a neural network for recognizing an object from an image, and calibration of internal and external parameters of a vision sensor camera.
- FIG. 21 is a flow chart showing the flow of learning processing executed by the CPU 12 of the command value generating device 410.
- FIG. When the CPU 12 reads out the learning program from the storage device 16, expands it in the memory 14, and executes it, the CPU 12 functions as each functional component of the command value generating device 410, and the learning process shown in FIG. 21 is executed.
- the same step numbers are assigned to the same processes as the learning process (FIG. 15) in the second embodiment, and detailed description thereof will be omitted.
- step S411 the acquisition unit 431 calculates the distance between the vision sensor and the object 90 based on the preset size of the object 90 and the acquired image. Then, the acquisition unit 431 stores the calculated distance and the position coordinates of the hand of the robot 40 at that time as a set. This information need only be stored the first time this step is executed. Thereafter, when this step is executed, the distance to the object is acquired as one of the state data based on this stored information and the position coordinates of the hand acquired during manual teaching. .
- step S412 the setting unit 435 sets parameters for image recognition based on the image acquired by the acquisition unit 431, and the learning process ends.
- the command value generation device determines the distance to the object based on the image of the vision sensor and the preset size of the object. Time-series data is acquired as one of state data. In order to recognize the distance in the Z-axis direction, triangulation is necessary unless a special sensor such as a depth sensor is used. Cannot perceive distance. In the fourth embodiment, the distance to the object can be obtained with high accuracy without using a special sensor. Accordingly, it is possible to generate a command value capable of appropriately executing an operation that requires an accurate grasp of the distance in the Z-axis direction, such as an operation of gripping an object with the hand unit.
- the state data for generating the determiner is the distance to the object. You may choose. In this case, as shown in FIG. 22, it is sufficient to generate a determiner that outputs a flag indicating achievement of the motion target when the distance to the object is equal to or less than a threshold value set as the motion target value. This makes it possible to robustly perform the operation of gripping the object with the hand unit.
- the command value generation device sets parameters for image recognition based on the image acquired by the vision sensor.
- the command value generation device sets parameters for image recognition based on the image acquired by the vision sensor.
- the robot control system 5 includes a command value generation device 510, a robot 40, and a sensor group 50.
- FIG. 23 is a block diagram showing an example of the functional configuration of the command value generation device 510.
- the command value generation device 510 includes an acquisition unit 31, a generation unit 33, a control unit 38, and a detection unit 539 as functional configurations. Each functional configuration is realized by the CPU 12 reading out a command value generating program stored in the storage device 16, developing it in the memory 14, and executing it.
- the hardware configuration of the command value generation device 510 is the same as the hardware configuration of the command value generation device 10 according to the first embodiment shown in FIG. 6, so description thereof will be omitted.
- the command value generation program according to the fifth embodiment includes a detection program for executing detection processing, which will be described later, in addition to the learning program and the control program.
- the detection unit 539 estimates the state data by inputting the command value generated by the generator into the generator and calculating back, and compares the estimated state data with the state data acquired by the acquisition unit 31. to detect an abnormality that occurs while the robot 40 is working.
- the detection unit 539 inputs state data from time t ⁇ n to t+k, which is time-series data, to the generator in the control phase.
- Time t ⁇ n to time t are past times based on time t
- time t to time t+k are future times based on time t. Therefore, the hatched portion in FIG. 24 corresponds to future data based on time t.
- the detection unit 539 inputs the command values from time t to time t+k output from the generator to the generator and performs back calculation to obtain the state data from time t to time t+k (FIG. 24 inside) is generated.
- the detection unit 539 converts the generated state data from time t to time t+k and the state data from time t to time t+k (The difference from the two-dot chain line) is compared with a predetermined threshold value. When the difference is equal to or greater than the threshold, the detection unit 539 detects an abnormality and controls the state so as to proceed to the state transition at the time of occurrence of the abnormality in the operation strategy flow chart. If there is no state transition at the time of occurrence of an abnormality in the operation strategy flow chart, the detection unit 539 notifies the user of the occurrence of an abnormality.
- FIG. 25 is a flowchart showing the flow of detection processing executed by the CPU 12 of the command value generating device 510. As shown in FIG. When the CPU 12 reads the detection program from the storage device 16, expands it in the memory 14, and executes it, the CPU 12 functions as each functional configuration of the command value generation device 510, and the detection process shown in FIG. 25 is executed.
- step S511 the detection unit 539 inputs the command value output to the robot 40 to the generator and performs back calculation to estimate state data.
- step S ⁇ b>512 the detection unit 539 calculates the difference between the estimated state data and the state data acquired by the acquisition unit 31 .
- step S513 the detection unit 539 determines whether or not the difference calculated in step S512 is equal to or greater than a predetermined threshold. If the difference is greater than or equal to the threshold, the process proceeds to step S514, and if the difference is less than the threshold, the process returns to step S511.
- step S514 in the operation strategy, if the corresponding state is a state that transitions to the next state when an abnormality occurs, the detection unit 539 notifies the control unit 38 to complete execution of the operation, and an abnormality occurs. Proceed to the processing in case of If the transition destination when the abnormality occurs is not determined, the detection unit 539 notifies the user that the abnormality has been detected, and the detection process ends.
- the command value generation device inputs the command value to the generator that generates the command value from the state data, and performs back calculation to estimate the state data. and the actual state data. Then, when the difference between the two is large, an abnormality is detected. This makes it possible to detect anomalies without creating complex anomaly detection conditions.
- the processing executed by the CPU by reading the software (program) in each of the above embodiments may be executed by various processors other than the CPU.
- the processor is a PLD (Programmable Logic Device) whose circuit configuration can be changed after manufacturing, such as an FPGA (Field-Programmable Gate Array), and an ASIC (Application Specific Integrated Circuit) to execute specific processing.
- a dedicated electric circuit or the like which is a processor having a specially designed circuit configuration, is exemplified.
- processing may be performed by one of these various processors, or by a combination of two or more processors of the same or different type (for example, multiple FPGAs, a combination of a CPU and an FPGA, etc.). may be executed.
- the hardware structure of these various processors is an electric circuit in which circuit elements such as semiconductor elements are combined.
- the command value generation program is pre-stored (installed) in the storage device, but the present invention is not limited to this.
- the program may be provided in a form stored in a storage medium such as a CD-ROM, DVD-ROM, Blu-ray disc, USB memory, or the like. Also, the program may be downloaded from an external device via a network.
Landscapes
- Engineering & Computer Science (AREA)
- Physics & Mathematics (AREA)
- General Physics & Mathematics (AREA)
- Automation & Control Theory (AREA)
- Robotics (AREA)
- Mechanical Engineering (AREA)
- Manipulator (AREA)
Abstract
Description
際の前記ロボットの状態を表す状態データであって、少なくとも、前記ロボットの動作を表す動作データ、前記ロボットと前記対象物との相対位置及び相対姿勢を表す位置姿勢データ、並びに、前記作業の間に前記対象物が受ける外力を表す外力データを含む複数種類の状態データとを取得する取得部と、前記取得部により対応する時間に取得された前記指令値と前記状態データとに基づいて、入力された前記状態データに対応する動作を前記ロボットに実行させるための指令値を生成する生成器を生成する生成部と、を含んで構成される。これにより、様々な状態を取り得る対象物に対する作業をロボットにロバストに実行させるためのフィードバック制御系を構成することができる。
得し、前記取得部により取得された前記画像に基づいて、前記作業領域を認識するためのパラメータを設定する設定部を含んで構成されてもよい。これにより、生成器の生成と共に、認識用のパラメータの設定も行うことができる。
図1に示すように、第1実施形態に係るロボット制御システム1は、指令値生成装置10と、ロボット40と、センサ群50とを含んで構成される。
教示は、例えば、図2に示すように、ダイレクトティーチング、すなわち、人の手で動作を伝える手伝えによる教示としてよい。また、例えば、図3に示すように、人がコントローラ60を操作して、ロボット40の動作を遠隔操作することにより教示してもよい。また、例えば、図4に示すように、ロボット40とバイラテラル制御で接続されたティーチング用機器62を用いた遠隔操作により教示してもよい。バイラテラル制御により、ティーチング用機器62が加えた力はロボット40に伝わり、ロボット40に作用した外力はティーチング用機器62に伝わる。
90AをX軸方向及びY軸方向へ移動させる動きである。「3.3.探り動作」は、主対象物90Aで副対象物90B表面をなぞりながら嵌合位置を探る動きである。「3.4.なぞりながらXY姿勢補正」は、主対象物90Aの副対象物90Bとのアライメントを利用して、主対象物90AのX軸方向及びY軸方向の位置と姿勢とを補正する動きである。「3.5.穴に挿入」は、副対象物90Bの嵌合位置(穴)の内周との拘束を保って主対象物90AをZ軸方向下方へ移動させる動きである。「3.6.噛みこみ解消動作」は、噛みこみを解消するように主対象物90Aの姿勢を変化させる動きである。
4のキネマティクス情報とに基づいて、手先に対するハンド部44の先端(以下、「指先」という)の姿勢を算出する。また、取得部31は、パッドセンサからセンサデータを取得し、このセンサデータから各指先に作用する外力を算出する。また、取得部31は、このセンサデータから、ハンド部44に把持された対象物90の、指先に対する姿勢を算出する。
指定された時間範囲を表示する表示領域74を含む。また、部分選択画面70は、表示領域72において指定された時間(図9中の黒三角)におけるビジョンセンサで取得された画像が表示される表示領域75を含む。これにより、ユーザは、表示領域75に表示された画像を参考にして、時間範囲を選択することができる。時間範囲が指定された状態データを含む試行回数の教示については、表示領域71の「部分採用」が「あり」となり、時間範囲が指定されていない場合は「なし」となる。
習処理及び制御処理の各々について詳述する。
次に、第2実施形態について説明する。なお、第2実施形態に係るロボット制御システムにおいて、第1実施形態に係るロボット制御システム1と同様の構成については、同一符号を付して詳細な説明を省略する。
行する。
次に、第3実施形態について説明する。なお、第3実施形態に係るロボット制御システムにおいて、第1実施形態に係るロボット制御システム1と同様の構成については、同一符号を付して詳細な説明を省略する。
333は、指令値の上限値を生成器のパラメータに含めてもよい。
納得性の高い指令値を出力することができる。
次に、第4実施形態について説明する。なお、第4実施形態に係るロボット制御システムにおいて、第1実施形態に係るロボット制御システム1と同様の構成については、同一符号を付して詳細な説明を省略する。
像から認識するために、事前にビジョンセンサによる画像に基づいて、認識用のパラメータを設定する必要がある。そこで、設定部435は、取得部431により取得された画像に基づいて、画像認識用のパラメータを設定する。画像認識用のパラメータの設定は、画像から対象物を認識するためのニューラルネットワーク等の認識モデルのパラメータの最適化と、ビジョンセンサのカメラの内部パラメータ及び外部パラメータのキャリブレーションとを含む。
パラメータの設定も行うことができ、ユーザの手間を削減することができる。また、副次的な効果として、設定したパラメータによる動作確認も容易に実行することができる。
次に、第5実施形態について説明する。なお、第5実施形態に係るロボット制御システムにおいて、第1実施形態に係るロボット制御システム1と同様の構成については、同一符号を付して詳細な説明を省略する。
る。次に、ステップS513で、検知部539が、上記ステップS512で算出した差が、予め定めた閾値以上か否かを判定する。差が閾値以上の場合には、ステップS514へ移行し、差が閾値未満の場合には、ステップS511に戻る。ステップS514では、動作戦略において、該当状態が、異常が発生した時に次の状態に遷移する状態であれば、検出部539が制御部38にその動作の実行を完了するように伝え、異常が発生した場合の処理に進む。異常が発生した時の遷移先が決まっていない状態である場合は、検知部539が、異常が検知されたことをユーザに報知し、検知処理は終了する。
10、210、310、410、510 指令値生成装置
12 CPU
14 メモリ
16 記憶装置
18 入出力I/F
20 入出力装置
22 記憶媒体読取装置
24 通信I/F
26 バス
31、231、431 取得部
32、332 受付部
33、333 生成部
38 制御部
234 指示部
435 設定部
539 検知部
40 ロボット
42 ロボットアーム
44 ハンド部
50 センサ群
60 コントローラ
62 ティーチング用機器
70 部分選択画面
80 種類選択画面
90 対象物
90A 主対象物
90B 副対象物
Claims (15)
- 対象物への作業をロボットに実行させるための指令値と、前記作業時の前記ロボットの動作を手動により教示した際の前記ロボットの状態を表す状態データであって、少なくとも、前記ロボットの動作を表す動作データ、前記ロボットと前記対象物との相対位置及び相対姿勢を表す位置姿勢データ、並びに、前記作業の間に前記対象物が受ける外力を表す外力データを含む複数種類の状態データとを取得する取得部と、
前記取得部により対応する時間に取得された前記指令値と前記状態データとに基づいて、入力された前記状態データに対応する動作を前記ロボットに実行させるための指令値を生成する生成器を生成する生成部と、
を含む指令値生成装置。 - 前記生成部は、前記生成器内のパラメータを最適化に基づいて決定することにより前記生成器を生成する請求項1に記載の指令値生成装置。
- 複数回の教示の各々について前記取得部により取得された前記状態データのうち、前記生成器の生成に利用する前記状態データの部分の選択を受け付ける受付部を含み、
前記生成部は、選択された前記状態データの部分を利用して前記生成器を生成する
請求項1又は請求項2に記載の指令値生成装置。 - 前記受付部は、前記取得部により取得された複数種類の前記状態データのうち、前記生成器の生成に利用する前記状態データの種類の選択を受け付け、
前記生成部は、選択された種類の前記状態データと前記指令値とに基づいて、選択された種類の前記状態データが表す状態を再現可能な指令値を生成するためのパラメータを最適化することにより、前記生成器を生成する
請求項3に記載の指令値生成装置。 - 前記生成部は、生成した前記生成器のパラメータの修正を受け付ける請求項4に記載の指令値生成装置。
- 前記生成器のパラメータは、前記指令値の上限値、及び前記指令値に対する動作の目標値を含み、
前記生成部は、前記上限値及び前記目標値を指定された値に固定すると共に、他のパラメータを最適化することにより前記生成器を生成する
請求項4又は請求項5に記載の指令値生成装置。 - 前記生成部により生成された前記生成器に、前記作業においてばらつく可能性があるパラメータに摂動項を加味した前記状態データを入力した場合に生成される指令値に基づいて前記ロボットが動作可能か否か判定し、動作可能ではない場合、前記摂動項を加えた場合に生成される指令値及び前記状態データの取得を前記取得部に指示する指示部を含む請求項1~請求項6のいずれか1項に記載の指令値生成装置。
- 前記生成部は、前記生成器の生成に利用された前記状態データの一部を削除、及び新たに前記取得部により取得された前記状態データの追加の少なくとも一方を行って前記生成器の生成を再実行する請求項1~請求項7のいずれか1項に記載の指令値生成装置。
- 前記取得部は、前記教示の際に、前記対象物を含む作業領域を撮影した画像を取得し、
前記取得部により取得された前記画像に基づいて、前記作業領域を認識するためのパラメータを設定する設定部を含む
請求項1~請求項8のいずれか1項に記載の指令値生成装置。 - 前記取得部は、予め設定された前記対象物のサイズと、前記画像から認識される前記対象物の画像上でのサイズとに基づいて算出される、前記画像を撮影するカメラと前記対象物との距離を取得する請求項9に記載の指令値生成装置。
- 前記ロボットの動作の手動による教示は、ダイレクトティーチング、コントローラからの遠隔操作、又は前記ロボットとバイラテラル制御で接続されたティーチング用機器を用いた遠隔操作により実行される請求項1~請求項10のいずれか1項に記載の指令値生成装置。
- 前記生成器により生成される指令値を出力して前記ロボットを制御する制御部を含む請求項1~請求項11のいずれか1項に記載の指令値生成装置。
- 前記生成器により生成された指令値を前記生成器に入力して逆算することにより、前記状態データを推定し、推定された前記状態データと、前記取得部により取得される前記状態データとを比較して、前記ロボットによる作業中に発生する異常を検知する検知部を含む請求項12に記載の指令値生成装置。
- 取得部が、対象物への作業をロボットに実行させるための指令値と、前記作業時の前記ロボットの動作を手動により教示した際の前記ロボットの状態を表す状態データであって、少なくとも、前記ロボットの動作を表す動作データ、前記ロボットと前記対象物との相対位置及び相対姿勢を表す位置姿勢データ、並びに、前記作業の間に前記対象物が受ける外力を表す外力データを含む複数種類の状態データとを取得し、
生成部が、前記取得部により対応する時間に取得された前記指令値と前記状態データとに基づいて、入力された前記状態データに対応する動作を前記ロボットに実行させるための指令値を生成する生成器を生成する
指令値生成方法。 - コンピュータを、
対象物への作業をロボットに実行させるための指令値と、前記作業時の前記ロボットの動作を手動により教示した際の前記ロボットの状態を表す状態データであって、少なくとも、前記ロボットの動作を表す動作データ、前記ロボットと前記対象物との相対位置及び相対姿勢を表す位置姿勢データ、並びに、前記作業の間に前記対象物が受ける外力を表す外力データを含む複数種類の状態データとを取得する取得部、及び、
前記取得部により対応する時間に取得された前記指令値と前記状態データとに基づいて、入力された前記状態データに対応する動作を前記ロボットに実行させるための指令値を生成する生成器を生成する生成部
として機能させるための指令値生成プログラム。
Priority Applications (2)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
CN202280058959.3A CN117881507A (zh) | 2021-09-07 | 2022-03-28 | 指令值生成装置、方法、以及程序 |
EP22866966.9A EP4400274A1 (en) | 2021-09-07 | 2022-03-28 | Command value generating device, method, and program |
Applications Claiming Priority (2)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
JP2021145670A JP2023038776A (ja) | 2021-09-07 | 2021-09-07 | 指令値生成装置、方法、及びプログラム |
JP2021-145670 | 2021-09-07 |
Publications (1)
Publication Number | Publication Date |
---|---|
WO2023037634A1 true WO2023037634A1 (ja) | 2023-03-16 |
Family
ID=85507366
Family Applications (1)
Application Number | Title | Priority Date | Filing Date |
---|---|---|---|
PCT/JP2022/015158 WO2023037634A1 (ja) | 2021-09-07 | 2022-03-28 | 指令値生成装置、方法、及びプログラム |
Country Status (4)
Country | Link |
---|---|
EP (1) | EP4400274A1 (ja) |
JP (1) | JP2023038776A (ja) |
CN (1) | CN117881507A (ja) |
WO (1) | WO2023037634A1 (ja) |
Cited By (1)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
CN116494249A (zh) * | 2023-06-26 | 2023-07-28 | 极限人工智能(北京)有限公司 | 协作机械臂实时手动控制装置、控制方法及协作系统 |
Families Citing this family (1)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
US20230271319A1 (en) * | 2022-02-28 | 2023-08-31 | Denso Wave Incorporated | Method of generating a learning model for transferring fluid from one container to another by controlling robot arm based on a machine-learned learning model, and a method and system for weighing the fluid |
Citations (6)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
JP2018190241A (ja) * | 2017-05-09 | 2018-11-29 | オムロン株式会社 | タスク実行システム、タスク実行方法、並びにその学習装置及び学習方法 |
WO2019107454A1 (ja) * | 2017-11-28 | 2019-06-06 | 川崎重工業株式会社 | 技能伝承機械装置 |
JP2020082332A (ja) * | 2018-11-30 | 2020-06-04 | オムロン株式会社 | 制御装置、制御方法、及び制御プログラム |
JP2020192641A (ja) * | 2019-05-28 | 2020-12-03 | 川崎重工業株式会社 | 制御装置、制御システム、機械装置システム及び制御方法 |
JP2020192640A (ja) * | 2019-05-28 | 2020-12-03 | 川崎重工業株式会社 | 制御装置、制御システム、機械装置システム及び制御方法 |
JP2021030360A (ja) * | 2019-08-22 | 2021-03-01 | オムロン株式会社 | モデル生成装置、モデル生成方法、制御装置及び制御方法 |
-
2021
- 2021-09-07 JP JP2021145670A patent/JP2023038776A/ja active Pending
-
2022
- 2022-03-28 EP EP22866966.9A patent/EP4400274A1/en active Pending
- 2022-03-28 CN CN202280058959.3A patent/CN117881507A/zh active Pending
- 2022-03-28 WO PCT/JP2022/015158 patent/WO2023037634A1/ja active Application Filing
Patent Citations (6)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
JP2018190241A (ja) * | 2017-05-09 | 2018-11-29 | オムロン株式会社 | タスク実行システム、タスク実行方法、並びにその学習装置及び学習方法 |
WO2019107454A1 (ja) * | 2017-11-28 | 2019-06-06 | 川崎重工業株式会社 | 技能伝承機械装置 |
JP2020082332A (ja) * | 2018-11-30 | 2020-06-04 | オムロン株式会社 | 制御装置、制御方法、及び制御プログラム |
JP2020192641A (ja) * | 2019-05-28 | 2020-12-03 | 川崎重工業株式会社 | 制御装置、制御システム、機械装置システム及び制御方法 |
JP2020192640A (ja) * | 2019-05-28 | 2020-12-03 | 川崎重工業株式会社 | 制御装置、制御システム、機械装置システム及び制御方法 |
JP2021030360A (ja) * | 2019-08-22 | 2021-03-01 | オムロン株式会社 | モデル生成装置、モデル生成方法、制御装置及び制御方法 |
Non-Patent Citations (2)
Title |
---|
INSTITUTE OF ROBOTICS AND MECHATRONICS-SARA, 18 June 2021 (2021-06-18), Retrieved from the Internet <URL:https://www.dlr.de/rm/en/desktopdefault.aspx/tabid-11709/#gallery/29681> |
KUNIAKI NODA: "MULTIMODAL INTEGRATION FOR ROBOT SYSTEMS USING DEEP LEARNING", WASEDA UNIVERSITY DOCTORAL DISSERTATION, July 2015 (2015-07-01) |
Cited By (2)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
CN116494249A (zh) * | 2023-06-26 | 2023-07-28 | 极限人工智能(北京)有限公司 | 协作机械臂实时手动控制装置、控制方法及协作系统 |
CN116494249B (zh) * | 2023-06-26 | 2023-12-19 | 极限人工智能(北京)有限公司 | 协作机械臂实时手动控制装置、控制方法及协作系统 |
Also Published As
Publication number | Publication date |
---|---|
JP2023038776A (ja) | 2023-03-17 |
CN117881507A (zh) | 2024-04-12 |
EP4400274A1 (en) | 2024-07-17 |
Similar Documents
Publication | Publication Date | Title |
---|---|---|
WO2023037634A1 (ja) | 指令値生成装置、方法、及びプログラム | |
US10618164B2 (en) | Robot system having learning control function and learning control method | |
JP5754454B2 (ja) | ロボットピッキングシステム及び被加工物の製造方法 | |
Neto et al. | High‐level robot programming based on CAD: dealing with unpredictable environments | |
US11040451B2 (en) | Teaching device and teaching method | |
US20150239127A1 (en) | Visual debugging of robotic tasks | |
JP2021000678A (ja) | 制御システムおよび制御方法 | |
JP6973119B2 (ja) | ロボット制御装置及びロボットシステム | |
US10406688B2 (en) | Offline programming apparatus and method having workpiece position detection program generation function using contact sensor | |
KR20080088165A (ko) | 로봇 캘리브레이션 방법 | |
JP2021505416A (ja) | マニピュレータの制御システムおよび制御方法 | |
JP6322949B2 (ja) | ロボット制御装置、ロボットシステム、ロボット、ロボット制御方法及びロボット制御プログラム | |
Chico et al. | Hand gesture recognition and tracking control for a virtual UR5 robot manipulator | |
US12059814B2 (en) | Object-based robot control | |
Jha et al. | Imitation and supervised learning of compliance for robotic assembly | |
CN110794969A (zh) | 面向非接触力反馈的自然人机交互方法 | |
JP5806105B2 (ja) | パラメータ補正方法および制御システム | |
Shauri et al. | Sensor integration and fusion for autonomous screwing task by dual-manipulator hand robot | |
CN112533739B (zh) | 机器人控制装置、机器人控制方法以及存储介质 | |
JP6743431B2 (ja) | 制御装置、及びロボットシステム | |
US20240227180A9 (en) | Robot system, learning apparatus, information processing apparatus, learned model, control method, information processing method, method for manufacturing product, and recording medium | |
JP4808920B2 (ja) | 自動機械の制御装置 | |
WO2021210514A1 (ja) | ロボットの制御装置及び制御方法、ロボットシステム、ロボットの動作プログラムを生成する装置及び方法 | |
WO2023145309A1 (ja) | 動作経路生成装置、方法、及びプログラム | |
CN110666796B (zh) | 带可调电磁力反馈的自然人机交互方法 |
Legal Events
Date | Code | Title | Description |
---|---|---|---|
121 | Ep: the epo has been informed by wipo that ep was designated in this application |
Ref document number: 22866966 Country of ref document: EP Kind code of ref document: A1 |
|
WWE | Wipo information: entry into national phase |
Ref document number: 18687674 Country of ref document: US Ref document number: 202280058959.3 Country of ref document: CN |
|
WWE | Wipo information: entry into national phase |
Ref document number: 2022866966 Country of ref document: EP |
|
NENP | Non-entry into the national phase |
Ref country code: DE |
|
ENP | Entry into the national phase |
Ref document number: 2022866966 Country of ref document: EP Effective date: 20240408 |