JP2022090923A - Picking system control device and robot, and picking system - Google Patents

Abstract

To provide a picking system control device which can automatically correct a position of fingertips while a robot is performing a series of picking operation and place operation.SOLUTION: A control device according to the present invention controls a picking system 100a which comprises a robot 102 equipped with fingertips for grasping a work-piece 107, and a sensor 103. The control device comprises: a fingertip recognition part 1020 which obtains a fingertip position 111 on the basis of information of the sensor 103; a work-piece recognition part 1030 which obtains a work-piece position 113 on the basis of information of the sensor 103; a deviation relative amount calculation part 1060 which calculates a difference between the fingertip position 111 and the work-piece position 113 as a deviation relative amount 114; a correction function update part 1080 which calculates a correction function that represents a relationship between the work-piece position 113 and a correction amount of the fingertip position 111 on the basis of data of a plurality of sets of the deviation relative amount 114 and the work-piece position 113 corresponding to the deviation relative amount 114; and a target position correction part 1100 which calculates the correction amount of the fingertip position 111 by using the work-piece position 113 and the correction function.SELECTED DRAWING: Figure 5

Description

The present invention relates to a picking system that grips an object, a control device that controls the picking system, and a robot included in the picking system.

In recent years, due to the decrease in the working population, industries such as the manufacturing industry are required to solve labor shortages and improve productivity. As one of the means for solving such a problem, it is expected to utilize a picking system equipped with an autonomous robot in which the robot recognizes, judges and operates by itself. In the picking system, a robot equipped with a hand performs a pick-and-place operation. The pick-and-place is a series of operations including a picking operation of taking out a work in a box or the like and gripping the work, and a place operation of placing the gripped work in a desired position. In order to realize this pick-and-place operation, the picking system needs to recognize the position of the work by the sensor, grasp the work by hand based on the recognition result, and carry the grasped work to the desired position. be. However, depending on the positional relationship between the robot and the sensor, the model of the robot and the hand, and the positional relationship between the robot and the hand and the global coordinate system, the coordinate system assumed by the robot and the coordinate system of the actual device are between. The picking operation may fail due to a deviation (difference).

Patent Document 1 discloses an example of a technique for calibrating such a deviation. The calibration device described in Patent Document 1 includes a representative point angle correction amount data table including the position of each point in the arm operating range and the optimum angle correction amount corresponding to each position, and from this data table. Multiple angle correction amount data near the target position and posture are extracted, the angle correction amount of the target position and posture is calculated by interpolation calculation using the extracted angle correction amount data, and the calculated correction amount is added to the target joint angle. And calibrate the arm hand position.

Japanese Unexamined Patent Publication No. 2002-219674.

In the calibration device described in Patent Document 1, angle correction amount data is obtained using an angle correction amount corresponding to a reference point (position of each point in the arm operating range), and calculation is performed using this angle correction amount data. By correcting the position of the hand of the arm with the corrected amount, the deviation between the coordinate system assumed by the robot and the coordinate system of the actual device is corrected, and the accuracy of the picking operation is improved.

However, in the technique described in Patent Document 1, it is necessary to measure the angle correction amount corresponding to the discretized reference point with an arbitrary fixed length. This measurement work is not a work performed during the picking operation, but a work required only for calibration. Therefore, in the conventional technique such as Patent Document 1, it is necessary to interrupt the picking operation and perform the calibration work, which causes a problem that the pick-and-place operation is increased and the personnel cost is increased.

An object of the present invention is a picking system capable of automatically correcting the position of a hand while the robot is performing a series of picking operations and place operations, a control device for controlling the picking system, and the picking system. Is to provide a robot equipped with.

The control device according to the present invention controls a picking system including a robot having a hand for gripping a work and a sensor, and a hand recognition unit that acquires a hand position, which is the position of the hand, based on the information acquired by the sensor. And, based on the information acquired by the sensor, the work recognition unit that acquires the work position, which is the position of the work, and the deviation relative amount calculation unit that calculates the difference between the hand position and the work position as the deviation relative amount. And, a plurality of sets of data of the shift relative amount and the work position corresponding to the shift relative amount are input, and the relationship between the work position and the correction amount of the hand position is based on the input data. Using the correction function update unit that calculates the correction function representing the above, the work position, and the correction function, the correction amount of the hand position for correcting the difference is calculated, and the target position correction that corrects the hand position is performed. It has a part.

The robot according to the present invention has a hand for gripping the work, is provided in the picking system, and is controlled by the control device according to the present invention.

The picking system according to the present invention includes a robot having a hand for gripping a work, a sensor, and a control device according to the present invention.

According to the present invention, a picking system capable of automatically correcting the position of a hand while the robot is performing a series of picking operations and place operations, a control device for controlling the picking system, and the picking system are provided. A robot can be provided.

The figure which shows the structure of the picking system by Example 1 of this invention. The figure which shows the work housed in a work container. The figure which shows the hand position of a robot. The figure which shows the work position. The functional block diagram of the picking system according to Example 1. FIG. The figure for demonstrating the processing of the deviation relative quantity calculation part. The figure which shows the deviation relative amount. The figure which shows the example of the data which a shift relative quantity storage part stores. The figure which shows the example of the flowchart of the process which the picking system by Example 1 executes a series of picking operation and place operation. The figure which shows the example of the flowchart of the process of the picking operation executed by the picking system by Example 1. FIG. The figure which shows the example of the flowchart of the process of calculating the correction function and updating the correction function executed by the picking system according to the first embodiment. The figure which shows the structure of the picking system by Example 2 of this invention. The functional block diagram of the picking system according to the second embodiment. The figure which shows the robot which attached the marker. The functional block diagram of the picking system according to Example 3 of this invention. The figure which shows the plurality of sensors provided in the picking system according to Example 4 of this invention. The functional block diagram of the picking system according to Example 4. FIG.

The picking system according to the present invention is provided with the control device according to the present invention, and automatically corrects the position (hand position) of the robot's hand while the robot performs a series of picking movements and place movements (pick and place). Then, the hand position can be matched with the position (work position) of the work which is the object of picking. The picking system and the control device according to the present invention have a work position and a hand position based on a plurality of sets of a shift relative amount which is a difference between the hand position and the work position and a work position corresponding to the shift relative amount. A correction function representing the relationship with the correction amount is calculated, and the correction amount of the hand position is calculated using this correction function to correct the hand position.

The picking system and the control device according to the present invention are provided with such a configuration, and can be calibrated without stopping a series of picking operations and place operations of the robot, which can reduce the calibration time and personnel cost. Is. Further, since the picking system and the control device according to the present invention directly correct the difference (deviation) between the hand position and the work position, the picking system has elements (for example, a robot hand or a sensor such as a camera). There is no need to calibrate individually. Therefore, as compared with a system or device that calibrates each element individually, it is possible to reduce the man-hours for calibration and reduce the possibility that a calculation error occurs during the execution of calibration.

The robot according to the present invention is controlled by the control device according to the present invention.

The picking system, the control device for controlling the picking system, and the robot included in the picking system according to the embodiment of the present invention will be described with reference to the drawings.

FIG. 1 is a diagram showing a configuration of a picking system 100a according to a first embodiment of the present invention. The picking system 100a includes a control device 101, a robot 102, and a sensor 103. The control device 101, the robot 102, and the sensor 103 are connected to each other by a wired connection or a wireless connection, respectively.

The picking system 100a performs a picking operation for gripping the work 107 and a place operation for placing the gripped work 107 at a desired position.

The control device 101 controls the picking system 100a by controlling the robot 102 and the sensor 103.

The robot 102 includes an arm and a hand 102a located on the arm, and the hand 102a grips the work 107 to perform a picking operation, and also performs a place operation to place the gripped work 107 at a place location 105. .. The hand 102a can be provided with an arbitrary configuration as long as the work 107 can be gripped, and can be configured by, for example, a suction hand that sucks the work 107, a finger gripper that sandwiches the work 107 between fingers, and the like.

The sensor 103 can be configured by any sensor as long as it can recognize an object around the robot 102, and can be configured by, for example, a camera. The picking system 100a may include one or more sensors 103.

The picking operation of the robot 102 will be described. The work 107 is housed in the work container 106 and is carried near the robot 102 by the supply conveyor 104. The work 107 is an object to be gripped by the robot 102, and may not be housed in the work container 106.

The sensor 103 acquires information on an object existing around the robot 102.

The control device 101 recognizes the work 107 based on the information acquired by the sensor 103, and obtains the trajectory of the hand 102a based on the result of recognizing the work 107. Then, the control device 101 outputs a command to move the hand 102a along the obtained trajectory to the robot 102 to control the robot 102.

Based on the command received from the control device 101, the robot 102 moves the hand (the portion of the hand 102a that grips the work 107) to the target position, performs a picking operation of gripping the work 107 with the hand 102a, and grips the work 107. A place operation is performed in which the work 107 is placed at the place location 105.

The control device 101 can perform calibration (correction of the position of the hand of the robot 102) online while the robot 102 is performing a series of picking operations and place operations.

As shown in FIG. 1, the control device 101 may be installed outside the robot 102 or inside the robot 102. Further, the control device 101 may be mounted on a network such as a cloud.

The sensor 103 can be installed at any place, and may be fixed to the ceiling or wall of the room in which the robot 102 is installed, or may be installed in the robot 102.

FIG. 2 is a diagram showing a work 107 housed in a work container 106. The work 107 is housed in the work container 106 carried by the supply conveyor 104. The work 107 is housed in the work container 106 in an arbitrary form. For example, as shown in FIG. 2, a plurality of the work 107 may be randomly arranged, or a plurality of the work 107 may be arranged in an orderly manner. The work 107 may be an object of the same type or an object of a different type. Further, the work 107 may not be housed in the work container 106 but may be directly mounted on the supply conveyor 104 and carried.

FIG. 3 is a diagram showing a hand position 111 of the robot 102. The hand position 111 is the position of the hand of the robot 102. The hand of the robot 102 is a portion that grips the work 107 of the hand 102a, and is, for example, a tip portion of the hand 102a.

The hand position 111 can be represented by at least one of the hand position and the posture. The position of the hand can be represented by coordinates (x, y, z). The posture of the hand can be expressed by an angle (roll angle, pitch angle, and yaw angle). FIG. 3 shows an example of a vector nt (position of the hand represented by coordinates) indicating the hand position 111.

FIG. 4 is a diagram showing the work position 113. The work position 113 is a position that typically represents the position of the work 107, and is, for example, a gripping point of the work 107. The gripping point of the work 107 is a portion where the robot 102 grips the work 107, and is predetermined according to the work 107.

The work position 113 can be represented by at least one of the position and the posture of the work 107. The position of the work 107 can be represented by coordinates (x, y, z). The posture of the work 107 can be represented by an angle (roll angle, pitch angle, and yaw angle). FIG. 4 shows an example of a vector wn indicating the work position 113 (the position of the work 107 represented by coordinates).

In the following, for the sake of simplicity, the hand position 111 is represented by the hand position (coordinates), and the work position 113 is represented by the position (coordinates) of the work 107.

The control device 101 includes a hand recognition unit 1020 and a work recognition unit 1030, as will be described later. In this embodiment, the sensor 103 recognizes the hand position 111 and the work position 113 (FIGS. 3 and 4).

The hand recognition unit 1020 acquires the hand position 111 based on the sensor data which is the information acquired by the sensor 103. The hand recognition unit 1020 can acquire the hand position 111 by any method. For example, the hand recognition unit 1020 can acquire the hand position 111 by image recognition processing using sensor data such as an RGB image or a distance image of the sensor 103 configured by the camera. Specifically, for example, the hand recognition unit 1020 may store the hand image of the robot 102 as a master image in advance, and acquire the hand position 111 by pattern matching that matches the master image with the sensor data. can.

The work recognition unit 1030 acquires the work position 113 based on the sensor data of the sensor 103. The work recognition unit 1030 can acquire the work position 113 by any method. For example, the work recognition unit 1030 can acquire the work position 113 by pattern matching in the same manner as the hand recognition unit 1020.

FIG. 5 is a functional block diagram of the picking system 100a according to the present embodiment. The picking system 100a includes a control device 101, a robot 102, and a sensor 103.

The control device 101 includes a correction function update trigger generation unit 1010, a hand recognition unit 1020, a work recognition unit 1030, a deviation relative amount calculation unit 1060, a deviation relative amount storage unit 1070, a correction function update unit 1080, and correction. It includes a function storage unit 1090, a target position correction unit 1100, an orbit planning unit 1040, and a control command unit 1050.

FIG. 5 will be described with reference to a configuration in which the control device 101 of the picking system 100a according to the present embodiment recognizes the work 107, obtains the trajectory of the hand 102a, and controls the robot 102.

The correction function update trigger generation unit 1010 generates a correction function update trigger, and outputs the generated correction function update trigger to the work recognition unit 1030 and the hand recognition unit 1020. The correction function update trigger is a trigger for the control device 101 to start the process of updating the correction function.

The correction function is a function for correcting the deviation (difference) between the work position 113 and the actual hand position 111, and is a function for outputting the correction amount of the hand position 111 with the work position 113 as an argument. The correction function is based on discrete data (as described later, a plurality of sets of data of a shift relative amount which is a difference between the hand position 111 and the work position 113 and a work position 113 corresponding to this shift relative amount). It is calculated and represents a continuous relationship between the work position 113 and the correction amount of the hand position 111.

By using the correction function, the correction amount of the hand position 111 with respect to the work position 113 can be continuously obtained. The correction amount of the hand position 111 is a correction amount for correcting the deviation (difference) between the work position 113 and the actual hand position 111 so that the hand position 111 matches the work position 113. The correction amount of the hand position 111 can be expressed by at least one of the coordinates (x, y, z) and the angle (roll angle, pitch angle, and yaw angle) corresponding to each of the position and the posture.

The deviation between the work position 113 and the actual hand position 111 is caused by the deviation between the coordinate system of the actual device and the coordinate system assumed by the robot 102, for example, the positional relationship between the robot 102 and the sensor 103. , The model of the robot 102 and the hand 102a, and the like.

The control device 101 moves the hand to the target hand position 111 (target position), but the hand position 111 that the hand actually reaches does not match the work position 113, and the actual hand position 111 and the work position 113. There may be a gap (difference) between and. Therefore, the control device 101 corrects the target hand position 111 by using the correction function to change the hand position 111, so that the hand position 111 actually reached by the hand matches the work position 113.

By using the correction function, the control device 101 is used not only for the work position 113 acquired in the past by the sensor 103 (that is, measured in the past by the sensor 103), but also within the range defined in the correction function. For example, the correction amount of the hand position 111 can be obtained even for the work position 113 that the sensor 103 has not acquired in the past.

As described above, the hand recognition unit 1020 inputs the sensor data of the sensor 103, and acquires the hand position 111 based on the sensor data. While the correction function update trigger is input from the correction function update trigger generation unit 1010, the hand recognition unit 1020 outputs the acquired hand position 111 to the shift relative amount calculation unit 1060 for calibration.

As described above, the work recognition unit 1030 inputs the sensor data of the sensor 103, acquires the work position 113 based on the sensor data, and recognizes the work 107. The work recognition unit 1030 outputs the acquired work position 113 to the target position correction unit 1100. Further, the work recognition unit 1030 outputs the acquired work position 113 to the shift relative amount calculation unit 1060 for calibration while the correction function update trigger is input from the correction function update trigger generation unit 1010.

The deviation relative amount calculation unit 1060 inputs the hand position 111 and the work position 113, calculates the difference between the hand position 111 and the work position 113 as the deviation relative amount, and outputs the calculated deviation relative amount to the deviation relative amount storage unit 1070. do. The deviation relative amount calculation unit 1060 also outputs the work position 113 to the deviation relative amount storage unit 1070.

The deviation relative amount storage unit 1070 associates the deviation relative amount input from the deviation relative amount calculation unit 1060 with the work position 113 when the deviation relative amount is obtained, and uses the data as a set of the deviation relative amount and the work position 113. Remember. The shift relative amount storage unit 1070 stores a plurality of sets of data of the shift relative amount and the work position 113 corresponding to the shift relative amount. In the shift relative amount storage unit 1070, when the robot 102 performs a series of picking operations and place operations and the hand recognition unit 1020 and the work recognition unit 1030 are operating, the data of the set of the shift relative amount and the work position 113 is performed. Will accumulate.

The correction function update unit 1080 inputs a plurality of sets of data of the work position 113 corresponding to the deviation relative amount and the deviation relative amount from the deviation relative amount storage unit 1070, and the work corresponding to the input deviation relative amount and the deviation relative amount. A correction function is calculated based on a plurality of sets of data at the position 113.

The correction function storage unit 1090 inputs and stores the correction function from the correction function update unit 1080. When there is a correction function already stored, the correction function storage unit 1090 overwrites the stored correction function with the correction function newly input from the correction function update unit 1080. The control device 101 can update the correction function by overwriting the existing correction function with the new correction function.

The target position correction unit 1100 has a correction function stored in the work position 113 acquired by the work recognition unit 1030 and the correction function storage unit 1090 in order to correct the deviation between the actual hand position 111 and the work position 113. Is used to correct the hand position 111 acquired by the hand recognition unit 1020. The target position correction unit 1100 outputs the corrected hand position 111 to the trajectory planning unit 1040 as a new target hand position 111.

In the picking operation, the trajectory planning unit 1040 sets the corrected hand position 111 as a new target position, calculates a trajectory for moving the hand of the robot 102 to this new target position, and uses the calculated trajectory as the control command unit. Output to 1050. Further, the trajectory planning unit 1040 calculates the trajectory of the hand for placing the work 107 held by the robot 102 at the place location 105 in the place operation, and outputs the obtained trajectory to the control command unit 1050.

The control command unit 1050 controls the robot 102 by transmitting the track input from the track planning unit 1040 to the robot 102 as a track command.

The robot 102 operates the hand 102a based on the trajectory command input from the control command unit 1050.

With the above configuration, in the picking system 100a according to the present embodiment, the control device 101 works with the actual hand position 111 (target hand position 111) while the robot 102 performs a series of picking movements and place movements. By correcting the error of the hand position 111 caused by the deviation (difference) from the position 113, the hand position 111 of the robot 102 can be automatically corrected. The robot 102 can recognize the position of the work 107, perform a picking operation for gripping the work 107 with high accuracy, and perform a place operation for placing the gripped work 107 at the place place 105.

Next, the process of calculating the correction function by the picking system 100a according to the present embodiment will be described with reference to FIG.

Although the control device 101 can periodically update the correction function, it is preferable to start the process for updating the correction function at the timing when the correction function update trigger generation unit 1010 generates the correction function update trigger. The timing at which the correction function update trigger generation unit 1010 generates the correction function update trigger can be arbitrarily determined. For example, it can be determined that the correction function update trigger is generated when a predetermined fixed time elapses or when the frequency of picking operation failures reaches a predetermined value. The frequency of picking operation failures can be detected, for example, from changes in air pressure during suction when the hand 102a is composed of a suction hand.

In this embodiment, when the correction function update trigger generation unit 1010 generates the correction function update trigger, the process for updating the correction function is executed. The processing for updating the correction function includes a hand recognition unit 1020 for acquiring the hand position 111, a work recognition unit 1030 for acquiring the work position 113, a shift relative amount calculation unit 1060 for calculating the shift relative amount, and a shift relative. A shift relative quantity storage unit 1070 that stores a plurality of sets of quantities and corresponding work positions 113 and a correction function update unit 1080 that calculates a correction function mainly execute the functions.

The hand recognition unit 1020, the work recognition unit 1030, the deviation relative amount calculation unit 1060, and the deviation relative amount storage unit 1070 perform processing for acquiring data on a plurality of sets of the deviation relative amount and the corresponding work position 113. Run. These processes are executed immediately after the correction function update trigger is generated and the picking operation is completed (before the place operation is executed).

Specifically, these processes are executed as follows. The hand recognition unit 1020 acquires the hand position 111, and the work recognition unit 1030 acquires the work position 113. The deviation relative amount calculation unit 1060 calculates the difference (relative deviation amount) between the hand position 111 and the work position 113 as the deviation relative amount based on the hand position 111 and the work position 113. The details of the processing of the deviation relative amount calculation unit 1060 will be described later with reference to FIG. The shift relative amount storage unit 1070 associates the shift relative amount with the work position 113 when the shift relative amount is obtained, and stores a plurality of sets of the shift relative amount and the corresponding work position 113.

The correction function update unit 1080 stores the correction function (hand position with respect to the work position 113) when the set of the shift relative amount and the corresponding work position 113 is stored in the shift relative amount storage unit 1070 by a predetermined number or more. A function that gives a correction amount of 111) is calculated. The correction function update unit 1080 inputs data of a plurality of sets of the shift relative amount and the work position 113 from the shift relative quantity storage unit 1070, and based on the input data of the plurality of sets, the work position 113 and the hand position 111. Calculate a correction function that expresses the relationship between the correction amounts of. The correction amount of the hand position 111 has a value that makes the deviation relative amount zero. For example, the correction function update unit 1080 calculates the correction function by calculating the parameters of the correction function by using interpolation calculation or function fitting by the least squares method using a plurality of sets of the deviation relative amount and the work position 113. do.

The correction function storage unit 1090 inputs the correction function calculated by the correction function update unit 1080, and stores the input correction function. When there is a correction function already stored, the correction function storage unit 1090 overwrites the stored correction function with the correction function newly input from the correction function update unit 1080. The correction function storage unit 1090 updates the correction function in this way.

By executing the above processing, the control device 101 changes the actual hand position 111 (target hand position 111) when the robot 102 moves the hand based on the acquired work position 113 to the work position 113. The deviation relative amount indicating how much the deviation is with respect to the deviation is stored, and the actual hand position 111 (target hand position 111) and the work position are stored based on the data of a plurality of sets of the deviation relative amount and the work position 113. A correction function for correcting the deviation from 113 can be calculated.

FIG. 6 is a diagram for explaining the processing of the shift relative amount calculation unit 1060. Hereinafter, the processing of the shift relative amount calculation unit 1060 will be described. FIG. 6 shows an example when the hand position 111 and the work position 113 are offset amounts in the coordinate system of the sensor 103.

The deviation relative amount calculation unit 1060 inputs the hand position 111 and the work position 113, and calculates the difference between the hand position 111 and the work position 113 as the deviation relative amount. The vector cn indicating the relative amount of deviation can be represented by the difference (cn = tn−wn) between the vector tun indicating the hand position 111 and the vector wn indicating the work position 113. That is, the deviation relative amount has the difference between the coordinate components of the hand position 111 and the work position 113 as components. When the hand position 111 and the work position 113 are represented by postures, that is, angles (roll angle, pitch angle, and yaw angle), the displacement relative amount is a component of the angle between the hand position 111 and the work position 113, respectively. Has the difference of as a component.

FIG. 7 is a diagram showing a shift relative amount 114. The deviation relative amount 114 is the difference between the hand position 111 and the work position 113. FIG. 7 shows the shift relative amount 114 immediately after the end of the picking operation (immediately before the place operation).

The robot 102 is controlled by the control device 101 and moves a hand to a target position to perform a picking operation. Ideally, the hand position 111 and the work position 113 that have moved to the target position coincide with each other. However, a deviation (difference) occurs between the coordinate system assumed by the robot 102 (the coordinate system used by the robot 102) and the coordinate system of the actual device, and this deviation accumulates and the hand position 111 is the work position. It does not match 113. Examples of causes of this deviation include the joint angle of the robot 102, the position of the sensor 103, the dimensions of the hand 102a, the accuracy with which the work recognition unit 1030 acquires the work position 113, and the hand recognition unit 1020 with the hand position 111. Precision can be included.

The deviation relative amount 114 is the difference between the hand position 111 and the work position 113 thus generated. Therefore, if the control device 101 uses the shift relative amount 114 as the correction amount of the hand position 111 and corrects the target position of the hand (target hand position 111) by the shift relative amount 114, the hand position 111 becomes the work position 113. The hand position 111 can be controlled accurately so as to match.

FIG. 8 is a diagram showing an example of data stored by the shift relative quantity storage unit 1070. The shift relative amount storage unit 1070 stores the shift relative amount 114 in association with the work position 113 when the shift relative amount 114 is obtained. FIG. 8 shows the values of the respective coordinates (x, y, and z) of the work position 113 in the coordinate system set in the sensor 103 as the work position 113. The deviation relative amount 114 indicates the value of each coordinate component (x, y, and z) of the deviation of the hand position 111 from the work position 113. That is, the deviation relative amount 114 shown in FIG. 8 indicates how far the work position 113 and the hand position 111 are from each coordinate. Although the work position 113 and the deviation relative amount 114 are represented by positions (coordinates) in FIG. 8, the work position 113 and the deviation relative amount 114 may be represented by the position and the posture (angle).

FIG. 9 is a diagram showing an example of a flowchart of a process in which the picking system 100a according to the present embodiment executes a series of picking operations and place operations.

In S100, the control device 101 receives sensor data from the sensor 103. The sensor data is information acquired by the sensor 103, and is data that can acquire the hand position 111 and the work position 113. Examples of sensor data can include RGB images, distance images, point groups, distance information, and data read by the sensor 103 from the barcode attached to the work 107.

In S110, the control device 101 causes the robot 102 to perform a picking operation. Details of the processing of S110 will be described later with reference to FIG.

In S120, the control device 101 determines whether or not the correction function update trigger generation unit 1010 has output the correction function update trigger. If the correction function update trigger is not output, it is assumed that there is no command to update the correction function, and the process of S165 is executed. When the correction function update trigger is output, it is assumed that there is a command to update the correction function, and the process of S130 is executed.

In S130, the hand recognition unit 1020 acquires the hand position 111, and the work recognition unit 1030 acquires the work position 113 using the current sensor data acquired from the sensor 103. Next, the shift relative amount calculation unit 1060 calculates the shift relative amount 114 using the hand position 111 and the work position 113.

In S140, the shift relative amount storage unit 1070 stores the shift relative amount 114 calculated by the shift relative amount calculation unit 1060 in association with the work position 113 when the shift relative amount 114 is obtained. That is, the shift relative amount storage unit 1070 stores a set of the shift relative amount 114 and the work position 113 corresponding to the shift relative amount 114.

In S165, the control device 101 causes the robot 102 to perform a place operation of moving the gripped work 107 to the place place 105. In the place operation, the trajectory planning unit 1040 calculates the trajectory for placing the work 107 held by the robot 102 at the place location 105, and the control command unit 1050 controls the robot 102 based on the calculated trajectory. Will be executed. The place operation can be performed, for example, using existing techniques.

In S170, the control device 101 determines whether or not a series of picking operations and place operations have all been completed. If the series of picking operation and place operation are not all completed and there is a work 107 to perform the picking operation and place operation, the process returns to S100 and the process of the flowchart of FIG. 9 is repeated. When the series of picking operation and place operation are all completed, the processing of the flowchart of FIG. 9 is terminated.

FIG. 10 is a diagram showing an example of a flowchart of a picking operation process executed by the picking system 100a according to the present embodiment. This process is the process in which the robot 102 executes the picking operation in S110 of FIG.

In S111, the work recognition unit 1030 acquires the work position 113.

In S112, the target position correction unit 1100 determines whether or not the correction function stored in the correction function storage unit 1090 exists. If the stored correction function exists, the process of S113 is executed, and if the stored correction function does not exist, the process of S115 is executed.

In S113, the target position correction unit 1100 acquires the correction function stored in the correction function storage unit 1090. The target position correction unit 1100 inputs the work position 113 acquired in S111 into the acquired correction function, and calculates the required correction amount of the hand position 111. As described above, the correction function is a function that outputs the correction amount of the hand position 111 with respect to the work position 113.

In S114, the target position correction unit 1100 corrects the hand position 111 by using the correction amount of the hand position 111 calculated in S113. The target position correction unit 1100 sets the corrected hand position 111 as a new target hand position 111.

In S115, the orbit planning unit 1040 calculates a trajectory for moving the hand of the robot 102 to the target hand position 111 (target position). The target position is the corrected hand position 111 (new target hand position 111) when the correction function stored in the correction function storage unit 1090 exists in S112. The control command unit 1050 controls the robot 102 by using the track calculated by the track planning unit 1040 as a track command. The robot 102 moves the hand to the target position and grips the work 107 with the hand 102a. The control of the robot 102 can be performed, for example, by using an existing technique.

It should be noted that the robot 102 can also grip the work 107 with the hand 102a before executing the place operation by the process of S165 of FIG. 9 instead of S115.

FIG. 11 is a diagram showing an example of a flowchart of a process of calculating a correction function and updating the correction function, which is executed by the picking system 100a according to the present embodiment. This process is executed independently of the picking operation and the place operation processing shown in the flowchart of FIG.

As described above, the correction function update unit 1080 uses the data stored in the shift relative quantity storage unit 1070, that is, a plurality of sets of the work positions 113 corresponding to the shift relative quantity 114 and the shift relative quantity 114 to perform the correction function. Is calculated. The correction function update unit 1080 executes the process of calculating and updating the correction function when the picking system 100a is executing a series of picking operations and place operations (processing of the flowchart of FIG. 9 is being executed). When), the number of data stored in the shift relative quantity storage unit 1070 becomes a predetermined number or more.

In S161, in the correction function update unit 1080, the number of sets of the work position 113 corresponding to the deviation relative amount 114 and the deviation relative amount 114 stored in the deviation relative amount storage unit 1070 becomes a predetermined number or more. Determine if it has become. This predetermined number is the minimum number of data required to calculate the correction function, and can be arbitrarily determined according to the method for calculating the correction function and the required correction accuracy. When the number of sets of the work positions 113 corresponding to the deviation relative amount 114 and the deviation relative amount 114 is a predetermined number or more, the process of S162 is executed.

In S162, the correction function update unit 1080 calculates the correction function from a plurality of sets of the work position 113 corresponding to the shift relative amount 114 and the shift relative amount 114. As a method of calculating the correction function, for example, function fitting by the least squares method can be mentioned.

The method of function fitting will be specifically described using equations (1) to (7).

Equation (1) indicates that n is an integer of 0 or more. n is an index for distinguishing the hand position 111 and the work position 113 according to the timing at which they are grasped, and corresponds to the number of sets of the work position 113 corresponding to the shift relative amount 114 and the shift relative amount 114. do. Equation (2) is a vector indicating the hand position 111. Equation (3) is a vector indicating the work position 113. Equation (4) is a vector showing the deviation relative amount 114. Equations (5), (6) and (7) represent each coordinate component (x, y, and z) of the deviation relative quantity 114. Each coordinate component of the deviation relative quantity 114 can be defined by a function with the work position 113 as an argument.

In the function fitting, the functions represented by the equations (5), (6) and (7) are obtained as the correction functions. The correction function increases the number of picking operations so that the number of samples indicated by n increases, and the more the number of sets of the work position 113 corresponding to the shift relative amount 114 and the shift relative amount 114 increases, the higher the accuracy becomes. Get better.

In the formulas (1) to (7), the case where the hand position 111 and the work position 113 are represented only by the position is shown. Even when the hand position 111 and the work position 113 are represented by positions and postures (roll angle, pitch angle, and yaw angle), the correction function can be calculated by the above processing.

The correction function can be calculated by a method other than function fitting. For example, the correction function can be calculated by interpolation operations such as linear interpolation.

In S162, the correction function storage unit 1090 stores the correction function calculated by the correction function update unit 1080. When there is a correction function already stored, the correction function storage unit 1090 overwrites the existing correction function with the correction function newly calculated by the correction function update unit 1080 and updates the existing correction function.

As described above, the picking system 100a according to the present embodiment calculates a correction function based on the work position 113 and the deviation relative amount 114 obtained from the past operation results, and uses this correction function to determine the hand position of the robot 102. By automatically correcting the 111, the deviation between the actual hand position 111 (target hand position 111) and the work position 113 can be automatically corrected at a higher speed than the sequential correction.

In the correction function update unit 1080, in S161 of the flowchart shown in FIG. 11, the number of data (a set of the work position 113 corresponding to the deviation relative amount 114 and the deviation relative amount 114) stored in the deviation relative amount storage unit 1070 is If the number does not exceed a predetermined number, the process of S162 is not executed and the correction function is not calculated. That is, if the hand recognition unit 1020 and the work recognition unit 1030 are not operating, the correction function update unit 1080 does not increase the number of data stored in the shift relative quantity storage unit 1070 and does not exceed a predetermined number. The function cannot be calculated. Therefore, the picking system 100a according to the present embodiment automatically corrects the position of the hand while the robot 102 is performing a series of picking operations and place operations.

After the correction function update unit 1080 calculates the correction function, the shift relative amount storage unit 1070 preferably erases the stored data (a set of the work position 113 corresponding to the shift relative amount 114 and the shift relative amount 114). The shift relative quantity storage unit 1070 can adjust the number of data to be stored in this way.

The picking system 100a according to the present embodiment has the above-described configuration, and can automatically and accurately correct the position of the hand of the robot 102 while the robot 102 is performing a series of picking operations and place operations.

The picking system according to the second embodiment of the present invention will be described. The picking system according to the present embodiment further includes a log generation unit and a display unit in the picking system 100a according to the first embodiment. Hereinafter, the picking system according to the present embodiment will mainly be described with a configuration different from that of the picking system 100a according to the first embodiment.

FIG. 12 is a diagram showing the configuration of the picking system 100b according to the second embodiment of the present invention. The picking system 100b according to the present embodiment further includes a display unit 201 in the picking system 100a (FIG. 1) according to the first embodiment. The display unit 201 is connected to the control device 101 by a wired connection or a wireless connection.

FIG. 13 is a functional block diagram of the picking system 100b according to the present embodiment. The control device 101 further includes a log generation unit 2010 in the control device 101 (FIG. 5) of the picking system 100a according to the first embodiment. The log generation unit 2010 generates a log and outputs the generated log to the display unit 201.

The log generation unit 2010 generates logs of the correction function input from the correction function update unit 1080, the deviation relative amount 114 input from the deviation relative amount calculation unit 1060, and the work position 113 one by one. In this log, the correction function, the deviation relative amount 114, and all or part of the work position 113 are recorded. The log generation unit 2010 inputs a correction function from the correction function update unit 1080, and also inputs a deviation relative amount 114 and a work position 113 from the deviation relative amount calculation unit 1060, and inputs a correction function, a deviation relative amount 114, and a work position 113. Generate a log of the above in any format that is easy for the user to understand. This log may contain additional information about the correction function, such as the update time of the correction function.

The display unit 201 displays the log generated by the log generation unit 2010. The display unit 201 can be configured by any display device, for example, a monitor or the like. The user can obtain information about the shift relative amount 114 and the correction function by looking at the log displayed on the display unit 201.

The picking system 100b according to the present embodiment can show the user how much the correction amount (shift relative amount 114) of the current hand position 111 and the deviation relative amount 114 are improved by updating the correction function. This allows the user to obtain reference information about the improvement of the picking system 100b. For example, the user recalibrates each part of the robot 102 to prevent the picking operation from failing when the correction is too large, or picking when the effect of updating the correction function is small. It is possible to consider reviewing the overall configuration of the system 100b.

The picking system according to the third embodiment of the present invention will be described. In the picking system according to the present embodiment, the marker is attached to the robot 102 in the picking system 100a according to the first embodiment. Hereinafter, the picking system according to the present embodiment will mainly be described with a configuration different from that of the picking system 100a according to the first embodiment.

In the picking system 100a according to the first embodiment, as described with reference to FIG. 3, the sensor 103 recognizes the hand position 111, and the hand recognition unit 1020 acquires the hand position 111 based on the sensor data. In the configuration of the first embodiment, when the hand position 111 is hidden by the posture of the robot 102, or when the hand is made of a blunt object such as a bellows, it is difficult for the sensor 103 to recognize the hand position 111. There is.

In the picking system according to the present embodiment, the robot 102 is provided with a marker at a position easily recognized by the sensor 103. The marker has a known distance from the hand position 111. In the picking system according to the present embodiment, the sensor 103 recognizes the marker, and the hand position 111 can be acquired based on the sensor data that recognizes the marker.

FIG. 14 is a diagram showing a robot 102 to which the marker 301 is attached. The marker 301 can be attached to any position of the robot 102 as long as it is easily recognized by the sensor 103. The marker 301 can have an arbitrary configuration as long as the sensor 103 can recognize it, and can be configured with an existing marker for obtaining the position of the robot, for example. The distance 303 from the marker 301 to the hand position 111 of the robot 102 is accurately measured in advance.

As will be described later, the control device 101 includes a marker recognition unit and a hand position estimation unit, the marker recognition unit acquires the position of the marker 301 in the coordinate system of the sensor 103, and the hand position estimation unit uses the distance 303. Estimate the hand position 111. The position of the marker 301 can be represented by at least one of the coordinates (x, y, z) and the angle (roll angle, pitch angle, and yaw angle) corresponding to each of the position and the posture.

FIG. 15 is a functional block diagram of the picking system 100c according to the present embodiment. The control device 101 does not include the hand recognition unit 1020 but includes the marker recognition unit 3010 and the hand position estimation unit 3020 in the control device 101 (FIG. 5) of the picking system 100a according to the first embodiment.

The sensor 103 recognizes the position of the marker 301 and the work position 113.

The marker recognition unit 3010 acquires the position of the marker 301 based on the sensor data (information that the sensor 103 recognizes the marker 301) input from the sensor 103, and outputs the acquired position of the marker 301 to the hand position estimation unit 3020. do.

The hand position estimation unit 3020 has acquired the distance 303 from the marker 301 to the hand position 111 of the robot 102 in advance by inputting by the user or the like. The hand position estimation unit 3020 estimates the hand position 111 using the distance 303 and the position of the marker 301 input from the marker recognition unit 3010. The hand position estimation unit 3020 outputs the estimated hand position 111 to the shift relative amount calculation unit 1060.

In the picking system 100c according to the present embodiment, even when the sensor 103 is difficult to recognize the hand position 111, the hand position 111 can be accurately acquired by the sensor 103 recognizing the marker.

The picking system according to the fourth embodiment of the present invention will be described. The picking system according to the present embodiment includes a plurality of sensors in the picking system 100a according to the first embodiment. Hereinafter, the picking system according to the present embodiment will mainly be described with a configuration different from that of the picking system 100a according to the first embodiment.

The picking system according to the present embodiment is provided with a plurality of sensors, so that even if it is difficult for one sensor to recognize both or one of the hand position 111 and the work position 113, both the hand position 111 and the work position 113 can be accurately recognized. Can be obtained. Hereinafter, as an example of the picking system according to the present embodiment, a picking system including two sensors will be described.

FIG. 16 is a diagram showing a plurality of sensors included in the picking system according to the present embodiment. The picking system according to this embodiment includes a work recognition sensor 401 and a hand recognition sensor 402. The work recognition sensor 401 and the hand recognition sensor 402 have the same configuration as the sensor 103 described in the first embodiment, and are connected to the control device 101 and the robot 102 by a wired connection or a wireless connection.

The work recognition sensor 401 is a sensor for recognizing the work position 113, and is installed in a place where the work position 113 can be easily recognized. The hand recognition sensor 402 is a sensor for recognizing the hand position 111, and is installed in a place where the hand position 111 can be easily recognized. The positional relationship between the work recognition sensor 401 and the hand recognition sensor 402 is accurately measured in advance.

The control device 101 acquires in advance the positional relationship between the work recognition sensor 401 and the hand recognition sensor 402 by input from the user or the like.

FIG. 17 is a functional block diagram of the picking system 100d according to the present embodiment. The picking system 100d includes a work recognition sensor 401 and a hand recognition sensor 402 in place of one sensor 103 in the picking system 100a (FIG. 5) according to the first embodiment.

The work recognition sensor 401 outputs sensor data to the work recognition unit 1030. The hand recognition sensor 402 outputs sensor data to the hand recognition unit 1020.

Similar to the picking system 100a according to the first embodiment, the shift relative amount calculation unit 1060 inputs the hand position 111 and the work position 113, and calculates the difference between the hand position 111 and the work position 113 as the shift relative amount 114. When calculating the deviation relative amount 114, the deviation relative amount calculation unit 1060 uses the positional relationship between the work recognition sensor 401 and the hand recognition sensor 402.

In the picking system 100d according to the present embodiment, even if it is difficult for one sensor to recognize both or one of the hand position 111 and the work position 113, by providing the plurality of sensors 401 and 402, the hand position 111 and the work position 113 can be recognized. Both can be obtained accurately.

The present invention is not limited to the above embodiment, and various modifications are possible. For example, the above-described embodiment has been described in detail in order to explain the present invention in an easy-to-understand manner, and the present invention is not necessarily limited to the embodiment including all the described configurations. Further, it is possible to replace a part of the configuration of one embodiment with the configuration of another embodiment. It is also possible to add the configuration of another embodiment to the configuration of one embodiment. Further, it is possible to delete a part of the configuration of each embodiment and add / replace another configuration.

100a, 100b, 100c, 100d ... Picking system, 101 ... Control device, 102 ... Robot, 102a ... Hand, 103 ... Sensor, 104 ... Supply conveyor, 105 ... Place location, 106 ... Work container, 107 ... Work, 111 ... Hand Position, 113 ... Work position, 114 ... Relative amount of deviation, 201 ... Display unit, 301 ... Marker, 303 ... Distance, 401 ... Work recognition sensor, 402 ... Hand recognition sensor, 1010 ... Correction function update trigger generation unit, 1020 ... Hand recognition unit, 1030 ... Work recognition unit, 1040 ... Orbital planning unit, 1050 ... Control command unit, 106 ... Shift relative amount calculation unit, 1070 ... Shift relative amount storage unit, 1080 ... Correction function update unit, 1090 ... Correction function Storage unit, 1100 ... Target position correction unit, 2010 ... Log generation unit, 3010 ... Marker recognition unit, 3020 ... Hand position estimation unit.

Claims (8)

Controls a picking system with a sensor and a robot with a hand to grip the work,
Based on the information acquired by the sensor, the hand recognition unit that acquires the hand position, which is the position of the hand, and the hand recognition unit.
A work recognition unit that acquires the work position, which is the position of the work, based on the information acquired by the sensor.
A shift relative amount calculation unit that calculates the difference between the hand position and the work position as a shift relative amount,
A plurality of sets of data of the shift relative amount and the work position corresponding to the shift relative amount are input, and the relationship between the work position and the correction amount of the hand position is shown based on the input data. A correction function updater that calculates the correction function, and
Using the work position and the correction function, a correction amount of the hand position for correcting the difference is calculated, and a target position correction unit for correcting the hand position, and a target position correction unit.
A control device characterized by comprising. A shift relative quantity storage unit for storing a plurality of the sets of the shift relative amount and the work position corresponding to the shift relative amount is provided.
The correction function update unit calculates the correction function when the number of the set stored in the deviation relative quantity storage unit is equal to or more than a predetermined number.
The control device according to claim 1. The correction function update unit calculates the correction function by using the function fitting by the least squares method.
The control device according to claim 1. A log generation unit for generating a log in which all or a part of the correction function, the relative amount of deviation, and the work position is recorded is provided.
The control device according to claim 1. A marker is attached to the robot,
A hand that estimates the hand position using the marker recognition unit that acquires the position of the marker, the distance from the marker to the hand position, and the position of the marker based on the information that the sensor recognizes the marker. Position estimation unit and
The control device according to claim 1. With the hand for gripping the work,
Provided in the picking system
A robot characterized by being controlled by the control device according to claim 1. With the robot having the hand that grips the work,
With the sensor
The control device according to claim 1 and
A picking system characterized by being equipped with. The sensor includes a work recognition sensor for recognizing the work position and a hand recognition sensor for recognizing the hand position.
The picking system according to claim 7.

Images