JP5733518B2 - Motion prediction control apparatus and method - Google Patents

Motion prediction control apparatus and method Download PDF

Info

Publication number
JP5733518B2
JP5733518B2 JP2011116663A JP2011116663A JP5733518B2 JP 5733518 B2 JP5733518 B2 JP 5733518B2 JP 2011116663 A JP2011116663 A JP 2011116663A JP 2011116663 A JP2011116663 A JP 2011116663A JP 5733518 B2 JP5733518 B2 JP 5733518B2
Authority
JP
Japan
Prior art keywords
robot
control
internal state
data storage
time
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Active
Application number
JP2011116663A
Other languages
Japanese (ja)
Other versions
JP2012245568A (en
Inventor
周平 江本
周平 江本
俊寛 林
俊寛 林
肇 坂野
肇 坂野
藤井 正和
正和 藤井
光治 曽根原
光治 曽根原
Original Assignee
株式会社Ihi
株式会社Ihi
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by 株式会社Ihi, 株式会社Ihi filed Critical 株式会社Ihi
Priority to JP2011116663A priority Critical patent/JP5733518B2/en
Priority claimed from CN201280022935.9A external-priority patent/CN103517789B/en
Publication of JP2012245568A publication Critical patent/JP2012245568A/en
Application granted granted Critical
Publication of JP5733518B2 publication Critical patent/JP5733518B2/en
Active legal-status Critical Current
Anticipated expiration legal-status Critical

Links

Description

  The present invention relates to a motion prediction control apparatus and method for predicting motion of a workpiece, a robot, etc., among automatic devices using a robot or the like.

In an automatic device that handles a moving workpiece with a robot, the position of the workpiece is measured by a visual sensor such as a camera installed on the robot or outside the robot, or a distance measuring sensor such as a laser range finder (LRF). Based on this, it is necessary to control the robot arm to follow up.
Also, even in the case of a mobile robot that moves by itself, it is necessary to measure the position of the mobile robot with sensors installed on the robot or outside the robot and control the robot based on the measurement results. It is done. As a method of measuring the position of the robot, there are a method of measuring a relative position with respect to a landmark with a sensor on the robot, and a method of directly measuring the position of the robot with a sensor installed outside the robot.
Hereinafter, a moving work or a mobile robot is simply referred to as a “moving body”.

  In the robot motion prediction control described above, the object (work or landmark) that changes from moment to moment and the relative position between the robots are measured at a constant period (eg, 30 fps camera), and based on the measurement result, for example, the workpiece A movement command is output to the robot to bring the robot closer to the robot.

However, even if the robot is operated with the measured workpiece position as a target, the robot may not be able to follow the workpiece that is moving due to control delays such as sensor measurement delay, data acquisition delay, and robot operation delay.
In the case of a visual sensor, since the sensor measurement cycle is generally longer than the robot control cycle (eg, 4 ms), the robot cannot obtain the latest measurement results for each control cycle. Become. In particular, if the image processing takes time or the work is out of the field of view of the camera, the measurement result update period becomes longer and is not constant. As described above, in a device that handles a moving workpiece or a mobile robot, there is a problem that the tracking performance of the robot is deteriorated due to a control delay.

  In order to solve the above problems, various control means have already been proposed (for example, Patent Documents 1 to 3).

The “three-dimensional motion prediction device” of Patent Document 1 estimates the motion parameter from the position data of the measured object that performs simple vibration, predicts the future position, and measures the target by a manipulator based on the position information. Holds the body.
The state estimation means of Patent Document 2 estimates the internal state of the system that performed the observation based on observation signals input in time series by observation. The internal state means state variables such as the position, posture, and deflection angle of the object.
The “motion prediction device” of Patent Document 3 performs automatic tracking and motion prediction by using both background processing and foreground processing.

Japanese Patent Application Laid-Open No. 07-019818, “3D Motion Prediction Device” Japanese Patent No. 4072017, “State Estimation Device, Method and Program, Current State Estimation Device and Future State Estimation Device” Japanese Patent No. 4153625, “Motion Estimation Device”

  According to Patent Document 1 described above, by estimating the motion state of the object from the observation history of measuring the object (object to be measured), it is possible to supplement the data while the measured value is not obtained, It is possible to predict a gripping point when gripping an object with a robot.

  A Bayesian filter such as a Kalman filter or a particle filter is generally used for estimating the motion state. In the Bayes filter, as disclosed in Patent Document 2, an “internal state” representing a motion state, a “state transition equation”, and an “observation equation” are defined.

The Bayes filter is roughly divided into the following two processes.
(1) “Prediction process” for predicting an internal state at an arbitrary time.
(2) “Update process” in which a measurement value is predicted from an internal state and compared with an actual measurement value to correct the internal state (described in Patent Document 2 as “filter estimation means”).

When the robot uses the state estimation result for control, the following two configurations are conceivable.
A The above algorithm is implemented in the robot control device, and the prediction process (1) is performed every robot control cycle, and when the sensor measurement value is obtained, the update process (2) is performed.
B As in Patent Document 1, a system including a state estimation device and a robot is used, and the robot reads a predicted value from the estimation device and uses it for control.

  When controlling the motion of an automatic machine such as a robot, it is required to calculate a control command value at a constant control cycle (eg, 4 ms cycle). On the other hand, the prior art has not proposed a specific method for configuring a system using motion state estimation in consideration of time constraints.

For example, with the means A described above, the update process may not be completed within the robot control cycle. The update process of the state estimator generally takes time, especially when there are many types of measurement values, and particularly when using a particle filter or other complicated processes.
In addition, the means of Patent Document 2 can reduce the amount of calculation according to the purpose by dividing the prediction process and the update process according to the types of observation values and state variables. When all values are to be reflected, all processes need to be connected after all, so the amount of calculation required for estimation does not decrease.

  In the method B described above, the robot controller does not need to perform the update process, but after the robot requests the predicted value, the estimation apparatus must complete the prediction process within a certain time and return the predicted value. I must. In order to design a state estimation apparatus that satisfies this requirement, the following problems must be considered.

(1) When the number of robots increases or when the load of update processing is high, the arithmetic processing capability of the estimation device is distributed. Even in such a situation, the prediction process must be completed within a certain time.
(2) Since the robot control cycle is different for each robot, the time constraint until the predicted value is returned from the request on the robot side is not uniquely determined.
(3) A certain amount of time is required for communication between the robot control device and the state estimation device. The time required for this communication varies depending on the communication method (wired / wireless).

  The present invention has been developed to solve the above-described problems. That is, the object of the present invention is to predict the relative motion between the object and the robot based on the measurement result (for example, position and posture) of the object (work or landmark) and the relative position between the robots. Provided is a motion prediction control apparatus and method capable of controlling a robot by calculating a control command value at a control cycle determined for each robot without being affected by an estimated calculation amount or a data communication time. There is.

According to the present invention, a motion prediction control apparatus that predicts a relative motion with an object or robot by measuring the object or robot, and controls the robot based on a prediction result,
Measuring device for measuring relative relationship between object and robot, state estimating device for predicting and updating the internal state of the object and robot, data storage device for storing the internal state, and robot control for controlling the robot An apparatus,
The measurement device, the state estimation device, and the data storage device are non-real-time processing systems that are executed at a timing different from the control cycle of the robot,
The robot control device is a real-time processing system controlled by a robot control cycle,
The state estimating device has a total prediction unit that predicts the internal state, the old internal state stored in the data storage device, and an update unit that updates the new internal state calculated,
The state transition equation is a relational expression X (t + Δt) = f (X (t)) indicating how the internal state X changes with time.
The overall prediction unit performs a calculation of causing the internal state X (tx) of the model time tx recorded in the data storage device to transition to the measured time ty using the state transition equation.
The update unit predicts a measurement value at the measurement time ty using the internal state X (ty) at the measurement time ty in the data storage device and the observation equation h (X), and the predicted value and the measurement value To correct the internal state X (ty) and record it in the data storage device,
The observation equation is a relational expression that associates the internal state with the measured value,
The robot control device has an on-robot prediction unit that predicts a predicted position necessary for the control in the control period of the robot,
The predicted position is a current position or a future position of a workpiece or a robot,
The on-robot prediction unit performs a calculation to transition the position of the workpiece or the robot at the model time tx to the specified time using the state transition equation among the internal states recorded in the data storage device. ,
(A) at any time that does not depend on the control period, the measuring device, the state estimating device, and by the data storage device, to measure the relative relationship between the object and the robot, the object and the measurement time of the robot ty the predicted internal state X (ty), and updates the internal state of predictive, and stored in the state transition equations and co used in the prediction,
(B) by the robot controller, based on the latest internal state stored in the data storage device, in the control period, predicting the predicted position, to control the robot in real time, that motion prediction, wherein A control device is provided.

According to the present invention, there is provided a motion prediction control method for predicting a relative motion with an object or a robot by measuring the object or the robot, and controlling the robot based on a prediction result,
Measuring device for measuring relative relationship between object and robot, state estimating device for predicting and updating the internal state of the object and robot, data storage device for storing the internal state, and robot control for controlling the robot An apparatus,
The measurement device, the state estimation device, and the data storage device are non-real-time processing systems that are executed at a timing different from the control cycle of the robot,
The robot control device is a real-time processing system controlled by a robot control cycle,
The state estimating device has a total prediction unit that predicts the internal state, the old internal state stored in the data storage device, and an update unit that updates the new internal state calculated,
The state transition equation is a relational expression X (t + Δt) = f (X (t)) indicating how the internal state X changes with time.
The overall prediction unit performs a calculation of causing the internal state X (tx) of the model time tx recorded in the data storage device to transition to the measured time ty using the state transition equation.
The update unit predicts a measurement value at the measurement time ty using the internal state X (ty) at the measurement time ty in the data storage device and the observation equation h (X), and the predicted value and the measurement value To correct the internal state X (ty) and record it in the data storage device,
The observation equation is a relational expression that associates the internal state with the measured value,
The robot control device has an on-robot prediction unit that predicts a predicted position necessary for the control in the control period of the robot,
The predicted position is a current position or a future position of a workpiece or a robot,
The on-robot prediction unit performs a calculation to transition the position of the workpiece or the robot at the model time tx to the specified time using the state transition equation among the internal states recorded in the data storage device. ,
(A) at any time that does not depend on the control period, the measuring device, the state estimating device, and by the data storage device, to measure the relative relationship between the object and the robot, the object and the measurement time of the robot ty the predicted internal state X (ty), and updates the internal state of predictive, and stored in the state transition equations and co used in the prediction,
(B) by the robot controller, based on the latest internal state stored in the data storage device, in the control period, predicting the predicted position, to control the robot in real time, that motion prediction, wherein A control method is provided.

According to the apparatus and method of the present invention, the relative state (for example, relative position) between the object and the robot is measured by the measuring device, the state estimating device, and the data storage device, and the internal state between the object and the robot is determined. predict, updates the internal state of predictive, since stored in the state transition equations and co used in the prediction, the object (workpiece and landmarks) and measurement results of the relative relationship between the robot (for example, position, and orientation) on the basis of the The motion of a moving body (work or mobile robot) can be predicted.

  Further, the robot control device has an on-robot prediction unit that predicts a predicted value necessary for robot control in the robot control cycle, and the control cycle is based on the latest internal state stored in the data storage device. Because the predicted value is predicted and the robot is controlled in real time, the control command value is calculated at the control cycle determined for each robot without being affected by the amount of state estimation calculation or the time required for data communication. The robot can be controlled.

That is, the robot controller can operate without being affected by the time required for the state estimation update process.
Also, when the number of measuring devices and robot control devices increases and the amount of calculation for state estimation increases, each robot control device performs prediction calculation independently, so that the time required for prediction processing does not increase. Therefore, it is not necessary to review the design of arithmetic processing capacity when the system is changed.
Furthermore, the time limit of the prediction process can be set for each robot control device. Further, the accuracy of the predicted value to be calculated, the type of predicted value, and the like can be set for each robot control device. Therefore, in consideration of the time required for the prediction process, the control cycle, and the like, the respective robots can realize contrivances such as changing the accuracy of the prediction calculation or calculating only necessary variables in the internal state.

In addition, the data storage device transfers the latest internal state stored in the data storage device to the storage unit on the robot at a timing that does not depend on the request from the robot, thereby enabling communication between the robot control device and the state estimation device. Even if time is required or the time required for communication is not constant, data necessary for the prediction process can be referred to immediately. In this case, the data referred on the robot does not necessarily become the latest value due to the influence of communication delay, but the prediction process can be completed within a certain time.

1 is a diagram illustrating a first embodiment of a robot system including a motion prediction control device according to the present invention. 1 is a first embodiment diagram of a motion prediction control apparatus according to the present invention. It is a 2nd embodiment figure of a motion prediction control device by the present invention. It is 2nd Embodiment of the robot system provided with the motion prediction control apparatus by this invention. It is a 3rd embodiment figure of a robot system provided with a motion prediction control device by the present invention.

  Hereinafter, preferred embodiments of the present invention will be described in detail with reference to the accompanying drawings. In addition, the same code | symbol is attached | subjected to the common part in each figure, and the overlapping description is abbreviate | omitted.

FIG. 1 is a diagram showing a first embodiment of a robot system provided with a motion prediction control apparatus according to the present invention.
In this figure, 1 is a workpiece, 2 is a movable robot, 3 is a robot arm, 4 is a hand, 5a is a first camera fixed to the hand 4, 5b is a second camera fixed to an external fixed position, 6 is a fixed landmark, and 10 is a motion prediction control apparatus of the present invention. Hereinafter, the workpiece 1 and the landmark 6 are collectively referred to as an “object”.
This robot system measures the workpiece 1 moving while pendulum movement and the fixed landmark 6 with the cameras 5a and 5b, measures the robot 2 with the camera 5b, and controls the robot 2 to follow the workpiece 1, The workpiece 1 is held by the hand 4. In this example, the position of the landmark 6 need not be known.

FIG. 2 is a diagram showing a first embodiment of the motion prediction control apparatus according to the present invention.
In this figure, the motion prediction control apparatus 10 of the present invention includes a measurement device 12, a state estimation device 14, a data storage device 16, and a robot control device 20. The devices 12, 14 and 20 are connected to the data storage device 16.

The measuring device 12 measures the relative relationship between the object and the robot 2. A measurement result Y such as the relative position or posture between the object and the robot measured by the measurement device 12 is defined as a measurement value Y. The measurement value Y is defined according to the type of measurement device and the object to be measured. The measured value Y differs depending on the object to be measured (whether it is workpiece 1, landmark 6 or mobile robot 2) and the type of measuring device.
For example, the relative relationship is information such as position, posture, speed, angular velocity, acceleration, angular acceleration, and the like.

In this example, the measurement device 12 includes three measurement devices (first measurement device 12a, second measurement device 12b, and third measurement device 12c) connected to the first camera 5a, the second camera 5b, and the robot control device 20, respectively. ).
That is, in this example, two cameras and two measuring devices are used, the images of the object are taken from the first camera 5a and the second camera 5b, and the measurement value Y of the object is obtained by image processing. The measuring device 12b also captures the image of the robot 2 and obtains the measured value Y of the robot 2 by image processing. The measuring device 12c measures the amount of movement of the tire of the robot 2 and the amount of rotation of the arm joint with an encoder, and obtains the measured value Y of the robot 2. The obtained object and the measured value Y of the robot 2 are stored in the data storage device 16.
Note that the measuring device 12 is not limited to this configuration, and a device that can measure the relative relationship between the object and the robot 2, such as a laser radar, an acceleration sensor, a gyro sensor, or a speed sensor, may be used.

The state estimation device 14 is implemented with a state estimation algorithm based on the Kalman filter algorithm. In this algorithm, the swing angle θ of the pendulum on which the workpiece 1 is suspended, Δθ, fulcrum positions x, y, and z, the positions Xl, Yl, and Zl of the landmark 6, the postures RXl, RYl, RZl, and the position of the robot 2 State variables such as Xr, Yr, Zr, and postures RXr, RYr, RZr are collectively set as an internal state X.
Further, a state transition equation X (t + Δt) = f (X (t)) indicating how the internal state X changes as time advances, and an observation equation Y = h that associates the internal state X with the measured value Y. Define (X). An observation equation is defined for each type of measurement value Y.
This algorithm manages the error distribution C ov X of the internal state X, the error distribution C ov Y of the measurement value, and the like, and the accuracy of state estimation can be obtained from these data. In the following example, these accuracy index values are collectively referred to as E.

  The state estimation device 14 includes an overall prediction unit 15 a that predicts the internal state of the object and the robot 2, and an update unit 15 b that updates the internal state stored in the data storage device 16. Predict and update internal state.

  The overall prediction unit 15a performs a calculation for causing the internal state X recorded in the data storage device 16 to transition to a specified time using the state transition equation f (X).

  The update unit 15b calculates a predicted value of the measurement value Y of the target object or the robot 2 using the internal state X in the data storage device 16 and the observation equation h (X). The input measurement value Y and the predicted value are compared, and the internal state X is updated and recorded in the data storage device 16. At this time, the state estimation accuracy index E is simultaneously updated and recorded in the data storage device 16.

The measurement device 12, the state estimation device 14, and the data storage device 16 described above are non-real-time processing systems that are executed at a timing different from the control cycle of the robot 2. The processing interval of this non-real time processing system may be longer than the control cycle of the robot 2 and the cycle may not be constant. This processing interval is, for example, 30 to 80 ms.
Therefore, with the above-described configuration, the measurement device 12, the state estimation device 14, and the data storage device 16 measure the relative relationship (for example, relative position and posture) between the object and the robot at a timing different from the control cycle of the robot 2. and predicts the internal state of the object and the robot 2, and updates the internal state of predictive, and stores the state transition equations and co used in the prediction.

  In this example, the robot control device 20 includes a reception unit 21, a command value calculation unit 22, a transmission unit 23, and an on-robot prediction unit 24.

The receiving unit 21 receives position information (such as encoder information) from the robot 2.
The command value calculation unit 22 calculates a command value for the robot 2 from the prediction value obtained by the on-robot prediction unit 24. The predicted value is, for example, the current position or future position of the workpiece 1 or the robot 2.
The transmission unit 23 transmits the calculated command value to the robot 2.
The on-robot prediction unit 24 predicts a prediction value necessary for controlling the robot 2 at a predetermined control cycle.

That is, the robot control apparatus 20 includes an on-robot prediction unit 24 that implements a prediction process for obtaining a prediction value necessary for controlling the robot 2.
The on-robot prediction unit 24 only needs to be able to predict the internal quantity (predicted value) related to the control of the robot 2, and does not need to predict all the internal quantities.

The robot control device 20 transmits an operation command to the robot 2 at a constant cycle (for example, 4 ms) and receives encoder information of the robot 2 and the like. The received position information is acquired by the above-described measuring device 12 c and becomes the measurement value Y of the mobile robot 2.
The on-robot prediction unit 24 is used to predict current and future motion states of the target object and the robot 2 and use them to calculate command values.
The on-robot prediction unit 24 calculates the state quantity related to the robot control among the internal states X recorded in the data storage device 16 by using the state transition equation f (X) until the specified time. To do.

  The data storage device 16 stores data such as the internal state X, the model time tx of the internal state, the measured value Y, the measured time ty, the state transition equation f, the observation equation h, and the state estimation accuracy index value E. Hold.

The robot control device 20 described above is a real-time processing system controlled by the control cycle of the robot 2. This control cycle is a fixed cycle shorter than the processing interval of the non-real time processing system described above. This control cycle is, for example, 3 to 4 ms.
Therefore, with the above-described configuration, the robot control device 20 predicts a predicted value necessary for the control in the control cycle of the robot 2 based on the latest internal state stored in the data storage device 16, and makes the robot 2 in real time. To control.

Hereinafter, operation | movement of the motion prediction control apparatus 10 of this invention is demonstrated.
The measuring device 12 measures the object and the robot 2 at an arbitrary timing, and records the measurement result in the data storage device 16.
When a new measurement result is recorded in the data storage device 16, the state estimation device 14 estimates an internal state through prediction processing and update processing. The calculated internal state and the state transition equation used for estimation are recorded in the data storage device 16.

The robot control device 20 controls the robot 2 at a constant control cycle. At this time, the predicted position of the object and the robot 2 at an arbitrary time is calculated using the on-robot prediction unit 24 and used for control. The on-robot prediction unit 24 performs a prediction process using the latest internal state or state transition model recorded in the data storage device 16.
The robot controller 20 operates as a real-time processing system that performs processing at a constant control cycle. Each of the other devices may be a non-real-time processing system that does not have a fixed period time constraint.

Hereinafter, the operation of the motion prediction control apparatus 10 of the present invention described above will be described.
The two measuring devices (the first measuring device 12a and the second measuring device 12b) repeat the following (1) to (3) at an arbitrary timing.
(1) A shutter signal is transmitted to the cameras 5a and 5b at an arbitrary timing, and a captured image is acquired.
(2) The position / posture information (measurement value Y) of the object and the robot 2 is acquired from the obtained image by image processing. For example, a white area in the image is extracted and the center of gravity is obtained.
(3) The calculated measurement value Y, the shutter time (measured value time ty), and the error distribution C ov Y of the measurement value Y are recorded in the data storage device.

The measuring device 12c repeats the following (a1) to (a3) at an arbitrary timing.
(A1) Acquire encoder information of the robot controller 20 at an arbitrary timing.
(A2) The movement amount and joint angle of the tire are obtained from the obtained encoder information, and the position / posture information (measured value Y) of the mobile robot 2 is obtained.
(A3) The calculated measurement value Y, the measurement time of the encoder, and the error distribution C ov Y of the measurement value Y are recorded in the data storage device.

The state estimation device 14 performs the following (4) to (6) every time the measurement value Y is updated.
(4) The data storage device 16 is monitored, and if a new measurement value Y is recorded, Y, ty, X, tx, f, h, E are read from the data storage device.
(5) Since the internal state X indicates the internal state at the time tx, the internal state at the time ty is predicted using the overall prediction unit 15a.
(6) Using the update unit 15b, the measured value at the time ty is predicted, the predicted value and the measured value Y are compared, and the internal state X is corrected. The corrected internal state X and the new model time tx (= ty) are recorded in the data storage device 16. Further, the update unit 15 b calculates an error distribution C ov X of the internal state X and an accuracy index E such as a difference between the measured value and the predicted value, and records it in the data storage device 16.

The robot controller 20 repeats the following (7) to (10) at an arbitrary cycle.
(7) Receive encoder information from the robot 2. Communication with the robot 2 is managed by the robot 2 so as to be performed, for example, at a cycle of 4 ms. Therefore, the robot controller must wait until data is received and complete the transmission process (11) within 4 ms after receiving the data.
(8) The current position of the workpiece and the robot and the future position are calculated using the robot prediction unit 24. When performing this calculation, the on-robot prediction unit 24 refers to the latest internal state or the like in the data storage device 16.
(9) A target trajectory is calculated so that the arm hand approaches the predicted position of the workpiece 1. Further, the state estimation accuracy index value E is evaluated, and when the state estimation is performed with high accuracy, the arm target trajectory and the hand opening / closing command value are calculated so as to perform the gripping operation to the future workpiece position. .
(10) The calculated target trajectory and hand opening / closing command value are transmitted to the robot 2.

FIG. 3 is a diagram showing a second embodiment of the motion prediction control apparatus according to the present invention.
This example is suitable when the time required for communication between the robot control device 20 and the data storage device 16 is long.
That is, in this example, the data storage device 16 has a function of transferring the latest internal state stored in the data storage device to the on-robot storage unit 26 at a timing that does not depend on a request from the robot.

With this configuration, the data storage device 16 transfers the internal state and state transition model recorded in the data storage device 16 to the on-robot storage unit 26. The on-robot prediction unit 24 performs processing with reference to data in the on-robot storage unit 26.
The transfer timing may be when the internal state X of the data storage device 16 is updated (immediately after (6)).
Data transfer is performed when it is detected that new data has been recorded in the data storage unit (immediately after (6)) or at regular intervals. Alternatively, it may be performed at both timings described above.

  With this configuration, the contents of the data storage device 16 are sequentially transferred to the on-robot storage unit 26, and the on-robot prediction unit 24 refers to the on-robot storage unit 26. Therefore, it is necessary to communicate in the calculation of (8) described above. There is no. Therefore, even when communication using a long communication time such as wireless communication is used, the influence of communication time can be avoided.

FIG. 4 is a diagram showing a second embodiment of a robot system provided with a motion prediction control apparatus according to the present invention.
This example is a robot to which the robot 2 is fixed, and is different from the first embodiment in that there is no landmark 6. Other configurations are the same as those of the first embodiment.
That is, this example is a case in which the workpiece 1 moving with the arm 3 of the fixed robot 2 is gripped, and is different from the first embodiment in that it is not necessary to estimate the position and posture of the robot 2.

FIG. 5 is a diagram showing a third embodiment of a robot system provided with a motion prediction control apparatus according to the present invention.
This example is a case in which the robot 2 estimates its own position from the landmark 6 whose position is known. The first embodiment is not required to estimate the workpiece or the position and orientation of the landmark 6. Is different.

In this example, the mobile robot 2 measures the landmark 6 whose position is known, and estimates the self-position of the robot 2. In this example, the estimated internal quantity X is the origin position (X r , Y r , Z r ) of the robot 2 and the posture (R xr , R yr , R zr ) of the robot 2. In addition, the steer angle, speed, angular velocity, etc. of the wheel may be estimated.

  As the measurement value used in the update process, the rotation amount (θ) of the wheel recorded by the robot controller as a result of measuring the landmark 6 as a result of the camera 5a and measuring the position of the robot 2 as the camera 5b is used.

In addition, this invention is not limited to embodiment mentioned above.
For example, even when there are a plurality of robots and workpieces, the motion state estimation of the workpieces and the motion state estimation of the robots are combined. Therefore, according to the present invention described above, it is possible to realize a system that takes into account time constraints for robot control.
In the above-described embodiments, the pendulum work and the wheel-type mobile robot are shown. However, the form in which each moves is not limited thereto. For example, the workpiece may be one that moves on a conveyor, one that floats on water, or one that is flying. The robot may be a crawler type or a type that moves on a rail. In addition, “moving” includes cases in which the base sways or passively moves even with a fixed arm.

Also, each device need not be separated into separate processing devices. For example, a configuration in which a plurality of programs are processed in parallel on one PC may be used as the measurement program and the state estimation program.
However, the robot control device (program) is a real-time processing system that performs processing under the constraint of a fixed period. In the real-time OS, a non-real-time processing program and a real-time processing program can be paralleled.
The data storage device 16 may be a shared memory in which a plurality of devices and programs read and write the same data. Therefore, it is not necessary to use an independent device, and it is sufficient that the device is prepared in a memory space such as the state estimation device 14 or the measurement device 12.

According to the apparatus and method of the present invention described above, the measurement device 12, the state estimation device 14, and the data storage device 16 measure the relative relationship (for example, relative position) between the object and the robot, and predicts the internal state of, and updates the internal state of predictive, since stored in the state transition equations and co used in the prediction, the object (workpiece and landmarks) and measurement results of the relative relationship between the robot (for example, position, and orientation ) To predict the motion of the moving body (work or mobile robot).

  In addition, the robot control device 20 has an on-robot prediction unit 24 that predicts a predicted value necessary for controlling the robot 2 in the control cycle of the robot 2, and the latest internal state stored in the data storage device 16 is updated. Based on the control cycle, the predicted value is predicted and the robot 2 is controlled in real time. Therefore, the control cycle determined for each robot is not affected by the calculation amount of state estimation or the time required for data communication. Thus, the control command value can be calculated and the robot 2 can be controlled.

That is, the robot controller 20 can operate without being affected by the time required for the state estimation update process.
Also, when the number of measuring devices 12 and robot control devices 20 increases and the amount of calculation for state estimation increases, the time required for the prediction processing does not increase because each robot control device 20 performs prediction calculation independently. Therefore, it is not necessary to review the design of arithmetic processing capacity when the system is changed.
Furthermore, the time constraint of the prediction process can be set for each robot control device 20. Also, the accuracy of the predicted value to be calculated, the type of predicted value, and the like can be set for each robot control device 20. Therefore, in consideration of the time required for the prediction process, the control period, and the like, the respective robots 2 can realize contrivances such as changing the accuracy of the prediction calculation or calculating only necessary variables in the internal state.

  Further, the data storage device 16 transfers the latest internal state stored in the data storage device 16 to the on-robot storage unit 24 at a timing that does not depend on the request from the robot 2, so that the state estimation with the robot control device 20 is performed. Even when the communication between the devices 14 takes a long time or when the required communication time is not constant, the data necessary for the prediction process can be referred to immediately. In this case, the data referred to on the robot 2 does not necessarily become the latest value due to the influence of communication delay, but the prediction process can be completed within a certain time.

  In addition, this invention is not limited to embodiment mentioned above, is shown by description of a claim, and also includes all the changes within the meaning and range equivalent to description of a claim.

1 work (object), 2 robot,
3 Robot arms, 4 hands,
5a first camera, 5b second camera,
10 motion prediction control device, 12 measuring device,
12a 1st measuring device, 12b 2nd measuring device,
14 state estimation apparatus, 15a whole prediction part,
15b update unit, 16 data storage device,
20 robot controller, 21 receiver,
22 command value calculation unit, 23 transmission unit,
24 Prediction unit on the robot,
26 Robot storage unit

Claims (6)

  1. A motion prediction control device that predicts relative motion with an object or robot by measuring the object or robot, and controls the robot based on the prediction result,
    Measuring device for measuring relative relationship between object and robot, state estimating device for predicting and updating the internal state of the object and robot, data storage device for storing the internal state, and robot control for controlling the robot An apparatus,
    The measurement device, the state estimation device, and the data storage device are non-real-time processing systems that are executed at a timing different from the control cycle of the robot,
    The robot control device is a real-time processing system controlled by a robot control cycle,
    The state estimating device has a total prediction unit that predicts the internal state, the old internal state stored in the data storage device, and an update unit that updates the new internal state calculated,
    The state transition equation is a relational expression X (t + Δt) = f (X (t)) indicating how the internal state X changes with time.
    The overall prediction unit performs a calculation of causing the internal state X (tx) of the model time tx recorded in the data storage device to transition to the measured time ty using the state transition equation.
    The update unit predicts a measurement value at the measurement time ty using the internal state X (ty) at the measurement time ty in the data storage device and the observation equation h (X), and the predicted value and the measurement value To correct the internal state X (ty) and record it in the data storage device,
    The observation equation is a relational expression that associates the internal state with the measured value,
    The robot control device has an on-robot prediction unit that predicts a predicted position necessary for the control in the control period of the robot,
    The predicted position is a current position or a future position of a workpiece or a robot,
    The on-robot prediction unit performs a calculation to transition the position of the workpiece or the robot at the model time tx to the specified time using the state transition equation among the internal states recorded in the data storage device. ,
    (A) at any time that does not depend on the control period, the measuring device, the state estimating device, and by the data storage device, to measure the relative relationship between the object and the robot, the object and the measurement time of the robot ty the predicted internal state X (ty), and updates the internal state of predictive, and stored in the state transition equations and co used in the prediction,
    (B) by the robot controller, based on the latest internal state stored in the data storage device, in the control period, predicting the predicted position, to control the robot in real time, that motion prediction, wherein Control device.
  2. The robot control device includes a robot on a storage unit that stores the latest internal state stored in the data storage device,
    The motion prediction according to claim 1, wherein the data storage device transfers the latest internal state stored in the data storage device to a storage unit on the robot at a timing not depending on a request from the robot. Control device.
  3.   The motion prediction control apparatus according to claim 1, wherein the relative relationship to be measured is a relative position or posture between an object and a robot.
  4.   The motion prediction control apparatus according to claim 1, wherein the relative relationship to be measured is a relative speed or a posture change between an object and a robot.
  5. A motion prediction control method that predicts relative motion with an object or robot by measuring the object or robot, and controls the robot based on the prediction result,
    Measuring device for measuring relative relationship between object and robot, state estimating device for predicting and updating the internal state of the object and robot, data storage device for storing the internal state, and robot control for controlling the robot An apparatus,
    The measurement device, the state estimation device, and the data storage device are non-real-time processing systems that are executed at a timing different from the control cycle of the robot,
    The robot control device is a real-time processing system controlled by a robot control cycle,
    The state estimating device has a total prediction unit that predicts the internal state, the old internal state stored in the data storage device, and an update unit that updates the new internal state calculated,
    The state transition equation is a relational expression X (t + Δt) = f (X (t)) indicating how the internal state X changes with time.
    The overall prediction unit performs a calculation of causing the internal state X (tx) of the model time tx recorded in the data storage device to transition to the measured time ty using the state transition equation.
    The update unit predicts a measurement value at the measurement time ty using the internal state X (ty) at the measurement time ty in the data storage device and the observation equation h (X), and the predicted value and the measurement value To correct the internal state X (ty) and record it in the data storage device,
    The observation equation is a relational expression that associates the internal state with the measured value,
    The robot control device has an on-robot prediction unit that predicts a predicted position necessary for the control in the control period of the robot,
    The predicted position is a current position or a future position of a workpiece or a robot,
    The on-robot prediction unit performs a calculation to change the position of the workpiece or the robot at the model time tx from the internal state recorded in the data storage device to the specified time using the state transition equation. ,
    (A) at any time that does not depend on the control period, the measuring device, the state estimating device, and by the data storage device, to measure the relative relationship between the object and the robot, the object and the measurement time of the robot ty the predicted internal state X (ty), and updates the internal state of predictive, and stored in the state transition equations and co used in the prediction,
    (B) by the robot controller, based on the latest internal state stored in the data storage device, in the control period, predicting the predicted position, to control the robot in real time, that motion prediction, wherein Control method.
  6. The motion prediction control method according to claim 5 , wherein the control cycle is a constant cycle shorter than a processing interval of the measuring device.
JP2011116663A 2011-05-25 2011-05-25 Motion prediction control apparatus and method Active JP5733518B2 (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
JP2011116663A JP5733518B2 (en) 2011-05-25 2011-05-25 Motion prediction control apparatus and method

Applications Claiming Priority (5)

Application Number Priority Date Filing Date Title
JP2011116663A JP5733518B2 (en) 2011-05-25 2011-05-25 Motion prediction control apparatus and method
CN201280022935.9A CN103517789B (en) 2011-05-12 2012-04-24 motion prediction control device and method
EP12782445.6A EP2708334B1 (en) 2011-05-12 2012-04-24 Device and method for controlling prediction of motion
PCT/JP2012/060918 WO2012153629A1 (en) 2011-05-12 2012-04-24 Device and method for controlling prediction of motion
US14/078,342 US9108321B2 (en) 2011-05-12 2013-11-12 Motion prediction control device and method

Publications (2)

Publication Number Publication Date
JP2012245568A JP2012245568A (en) 2012-12-13
JP5733518B2 true JP5733518B2 (en) 2015-06-10

Family

ID=47466538

Family Applications (1)

Application Number Title Priority Date Filing Date
JP2011116663A Active JP5733518B2 (en) 2011-05-25 2011-05-25 Motion prediction control apparatus and method

Country Status (1)

Country Link
JP (1) JP5733518B2 (en)

Families Citing this family (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JP5740433B2 (en) * 2013-05-24 2015-06-24 日本電信電話株式会社 Delay compensation apparatus, method, program, and recording medium

Family Cites Families (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JPH0719818A (en) * 1993-06-30 1995-01-20 Kajima Corp Three-dimensional movement predicting device
JPH07239236A (en) * 1994-02-28 1995-09-12 Hitachi Ltd Method and apparatus for measurement of quantity of state of moving body and calculation device of attitude angle of moving body
JPH1124718A (en) * 1997-07-07 1999-01-29 Toshiba Corp Device and method for controlling robot
JP3988255B2 (en) * 1998-05-20 2007-10-10 株式会社安川電機 Position command method and apparatus in numerical controller
JP2011209203A (en) * 2010-03-30 2011-10-20 Sony Corp Self-position estimating device and self-position estimating method

Also Published As

Publication number Publication date
JP2012245568A (en) 2012-12-13

Similar Documents

Publication Publication Date Title
US9821463B2 (en) Virtual safety cages for robotic devices
US20180250825A1 (en) Robot system
EP3011362B1 (en) Systems and methods for tracking location of movable target object
Du et al. A markerless human–robot interface using particle filter and Kalman filter for dual robots
DE112013003209B4 (en) Robot control device and robot control method
Schmid et al. Stereo vision based indoor/outdoor navigation for flying robots
Schmid et al. Stereo vision and imu based real-time ego-motion and depth image computation on a handheld device
US9437005B2 (en) Information processing apparatus and information processing method
US10037028B2 (en) Systems, devices, and methods for on-board sensing and control of micro aerial vehicles
JP5632470B2 (en) System and method for remote coordinated vehicle
EP1555508B1 (en) Measuring system
US8285417B2 (en) Robot and task execution system
CN104057447B (en) The manufacture method of robot picking up system and machined object
KR101730221B1 (en) Automatic guided vehicle and method for operating an automatic guided vehicle
US20140032012A1 (en) Tracking on-road vehicles with sensors of different modalities
JP6717974B2 (en) Sensor trajectory planning for vehicles
Lynen et al. A robust and modular multi-sensor fusion approach applied to mav navigation
Armesto et al. Fast ego-motion estimation with multi-rate fusion of inertial and vision
WO2015073322A1 (en) System for robotic 3d printing
JP6151159B2 (en) Cooperative transfer robot system
KR102079940B1 (en) Mobile robot having function estimating friction coefficient and method of estimating the same
US8793069B2 (en) Object recognition system for autonomous mobile body
CN103020952B (en) Messaging device and information processing method
EP2865495B1 (en) Control synchronization for high-latency teleoperation
US20150213617A1 (en) Method and apparatus for estimating position

Legal Events

Date Code Title Description
A621 Written request for application examination

Free format text: JAPANESE INTERMEDIATE CODE: A621

Effective date: 20140219

A131 Notification of reasons for refusal

Free format text: JAPANESE INTERMEDIATE CODE: A131

Effective date: 20141216

A521 Written amendment

Free format text: JAPANESE INTERMEDIATE CODE: A523

Effective date: 20150213

TRDD Decision of grant or rejection written
A01 Written decision to grant a patent or to grant a registration (utility model)

Free format text: JAPANESE INTERMEDIATE CODE: A01

Effective date: 20150318

A61 First payment of annual fees (during grant procedure)

Free format text: JAPANESE INTERMEDIATE CODE: A61

Effective date: 20150331

R151 Written notification of patent or utility model registration

Ref document number: 5733518

Country of ref document: JP

Free format text: JAPANESE INTERMEDIATE CODE: R151

R250 Receipt of annual fees

Free format text: JAPANESE INTERMEDIATE CODE: R250

R250 Receipt of annual fees

Free format text: JAPANESE INTERMEDIATE CODE: R250