JP2013052490A - Workpiece takeout device - Google Patents

Abstract

PROBLEM TO BE SOLVED: To provide a workpiece takeout device that reduces failures of taking out a workpiece from a plurality of workpieces and also reduces a retry operation.SOLUTION: A measuring characteristic determination part 52 estimates the easy-to-grip performance of a part of the workpiece measured by a sensor 2 based on information on success or a failure of past takeout in a DB1. A grip operation calculation part 53 and an opening/closing operation calculation part 54 decide not to preferentially grip a workpiece hard in gripping but adjust at least one of opening/closing amount, operation speed and gripping force of a hand to perform careful takeout operation when there is no workpiece for gripping.

Description

  The present invention relates to a technique for picking up a workpiece from a large number of workpieces stacked in bulk.

  In order to reduce unsuccessful removal from a large number of workpieces, a workpiece unloading apparatus that determines a measurement failure or unsuccessful failure, performs NG workpiece registration, and prevents a workpiece at a coarse position registered as an NG workpiece from being removed. (See, for example, Patent Document 1 below).

Japanese Patent No. 4226623 Japanese Patent No. 2555824 Japanese Patent No. 3930490 JP 2011-93058 A

  However, when the rough position of the workpiece that has failed to be taken out is registered as NG by preventing the same failure as in the conventional technique, NG registration increases and the number of workpieces that can be taken out decreases. As a result, the workpiece can be seen but cannot be taken out, and there is a problem that the number of retries increases.

  The present invention has been made to solve the above-described problems, and an object of the present invention is to provide a work picking apparatus that achieves both low retries and a high picking success rate, and improves work efficiency and consequently production efficiency. And

  The present invention includes at least a DB for storing measurement features and success / failure of a workpiece, a sensor for measuring a stacked workpiece, a hand for gripping a stacked workpiece, a robot for moving the hand to a gripping position and posture, An information processing unit that calculates at least one motion control information among the gripping position and orientation and the opening / closing amount, gripping force, and motion speed from the measurement data, and commands the robot to perform a gripping operation according to the motion control information of the information processing unit 5 A control unit that commands the hand to open and close, a measurement feature extraction unit that extracts a measurement feature for calculating a gripping position and orientation from measurement data, and the measurement from the information in the DB A measurement feature identification unit for identifying a feature; a gripping operation calculation unit for calculating a gripping position and orientation of a hand that grips a workpiece from the measurement feature and the information of the DB; An opening / closing operation calculating unit that calculates parameters related to opening / closing of the hand from the DB information, and the measurement feature identifying unit includes past measurement features stored in the DB and success / failure of extraction. Based on the information, the measurement feature extraction unit extracts the measurement feature extraction success rate, and the opening / closing operation calculation unit calculates the opening / closing amount / grip force / motion of the hand according to the estimated extraction success rate. A workpiece picking device characterized in that at least one of the speeds is adjusted.

  In this invention, even in the situation where there is only work that is difficult to grasp in the measured scene, the retrieval is succeeded by improving the operation without retrying, so the retry is small and the retrieval success rate is high. There is no high work efficiency and hence production efficiency.

It is a block diagram which shows the workpiece removal apparatus by Embodiment 1 of this invention. It is a figure for demonstrating operation | movement of the measurement feature extraction part in this invention. It is an image figure of one method of P / N calculation in the measurement feature identification part in this invention. It is an operation | movement flowchart figure of the information processing part 5 in this invention. It is a block diagram which shows the workpiece taking-out apparatus by Embodiment 2 of this invention. It is a block diagram which shows the workpiece removal apparatus by Embodiment 3 of this invention. It is a block diagram which shows the workpiece taking-out apparatus by Embodiment 4 of this invention.

  In the workpiece picking device according to the present invention, the ease of grasping the measurement feature (feature information obtained from the appearance and shape of the workpiece) is estimated based on the past success / failure information. If there is no other measurement feature that seems to be easy to grip, it will automatically adjust at least one of the opening / closing amount, gripping force, and operating speed of the hand and change it to a more polite motion. The work that seems to be difficult is taken out.

  Hereinafter, a workpiece picking apparatus according to the present invention will be described with reference to the drawings according to each embodiment. In each embodiment, the same or corresponding parts are denoted by the same reference numerals, and redundant description is omitted.

Embodiment 1 FIG.
FIG. 1 is a block diagram showing a workpiece picking apparatus according to Embodiment 1 of the present invention. The workpiece picking device includes at least a DB (database) 1, a sensor 2, a hand 3, a robot 4, an information processing unit 5, and a control unit 6. The information processing unit 5 includes at least a measurement feature extraction unit 51, a measurement feature identification unit 52, a gripping operation calculation unit 53, and an opening / closing operation calculation unit 54. For example, the workpieces 7 are piled up in the supply box 8.

  An operation flow according to the configuration of FIG. 1 will be described. The sensor 2 measures the workpiece 7 stacked in the supply box 8 and transmits the measurement data to the information processing unit 5. In the information processing unit 5, first, the measurement feature extraction unit 51 extracts the work appearance and shape features (referred to as measurement features) from the measurement data. The measurement feature identification unit 52 estimates the ease of grasping the measurement feature extracted by the measurement feature extraction unit 51 from the relationship between the past measurement feature stored in the DB 1 and the success or failure of extraction (success or not).

  The gripping motion calculation unit 53 calculates the gripping position and orientation by a method according to the ease of gripping the estimated measurement feature. The opening / closing operation calculation unit 54 also uses an opening / closing parameter (opening / closing amount / opening / closing (gripping) force / opening / closing (gripping) operation speed of the hand 3 including one or more of the opening / closing (gripping) operation speed) in a method according to the ease of grasping the estimated measurement feature. Adjustment parameters (which means adjustment parameters related to opening and closing).

The grip position / posture and opening / closing parameter information calculated by the information processing unit 5 are sent to the control unit 6.
The grip position / posture information becomes operation control information of the robot 4 for the hand 3 to take the grip position / posture in the control unit 6, and is transmitted to the robot 4 as a gripping operation. The opening / closing parameter becomes operation control information for the hand 3 to grip the workpiece 7 by the control unit 6, and is transmitted to the hand 3 as an opening / closing operation. The robot 4 and the hand 3 hold the workpiece in accordance with information from the control unit 6.

  Next, the explanation for each component will be supplemented. The DB 1 stores and stores at least two of the measurement features extracted from the workpiece shape information (measurement data) from the sensor 2 measured in the past and the success / failure data of the measurement features extracted from the hand 3. Work shape information and hand shape information represented by CAD data, edges, point groups, circles, ellipses, surfaces, curved surfaces, sets of parameters thereof, geometric relationships, and the like may be stored.

  The sensor 2 measures a workpiece 7 stacked in a supply box 8 and may be installed on the robot 4 or may be installed on another fixed jig or movable slider. For measuring two-dimensional images and three-dimensional data of workpieces stacked. For two-dimensional, a camera, for three-dimensional, for example, binocular / multi-eye stereo system, multi-baseline stereo, laser, An active method such as a spatial coding method with a projector and a projector and a camera, a time-of-flight method that measures distance from the return time of irradiated light, a factorization method that uses robot motion with a single eye, Structure from Motion (Method of simultaneously restoring the 3D shape of the scene and the position of the camera from multiple images taken while changing the viewpoint of the camera, hereinafter referred to as SFM or Structure and Motion (hereinafter also referred to as SAM)), motion stereo , Volumetric intersection method and focus type Shape from Focus (Shape from Focus method is based on in-focus information (information indicating whether the subject is in focus or not) (Shape from Defocus (Shape From Focus / Defocus method): Focus evaluation value is calculated for each block from multifocal images taken with different focal points, and the distance is estimated from this) Alternatively, an apparatus using a light cutting method using line light may be used.

  The format of the three-dimensional information measured by the sensor 2 is an image format in which each pixel has height information of the measurement object as a distance image even if X, Y, Z, total three-dimensional position information is used as point cloud data. It may be used. The coordinate system at the time of measurement may be a camera coordinate system, a robot coordinate system, or a work coordinate system. The position and distance may be expressed in units of meters or pixels.

  The hand 3 grips and removes the workpieces 7 stacked in bulk. As the hand shape, for example, a sandwiching type that sandwiches and grips an object, an adsorption type that sucks onto a surface, and a spreading type that extends into a hole of an object and lifts it up may be used. Further, the number of fingers (nails) and suction portions of the hand 3 that come into contact with the work 7 may be any number.

  The robot 4 moves the hand 3 to a position / posture for gripping the workpiece 7 or moves after taking out. The robot 4 may be anything such as a scalar type, a vertical articulated type, or a parallel link type. The degree of freedom of the robot 4 may be any degree as long as the operation of gripping the workpiece can be performed. The operation of taking out the workpiece refers to the operation from grasping the workpiece 7 stacked in bulk with the hand 3 and taking out the workpiece 7 from the supply box 8. The operation after the take-out operation may be performed by transferring the workpiece 7 onto a belt conveyor (not shown), palletizing, assembling, or the like.

  The information processing unit 5 includes a measurement feature extraction unit 51, a measurement feature identification unit 52, a gripping operation calculation unit 53, and an opening / closing operation calculation unit 54. This may be realized by a PC (Personal Computer) or an FPGA (Field-Programmable Gate Array) board.

The measurement feature extraction unit 51 extracts the appearance and shape features of the workpiece 7 from the measurement data measured by the sensor 2. Here, this is called a measurement feature.
For example, in the case of a two-dimensional sensor, the measurement features include the spatial arrangement of the texture on the surface of the workpiece, the complexity of the texture, the amount of edges, the area of the area surrounded by the edges, the principal axis direction, the local edge direction, etc. It may be used.
In the case of a three-dimensional sensor, the main normal direction of the surface, the distribution of the normal direction of the surface, a histogram thereof, flatness, etc. may be used. In addition, the peripheral complexity may be quantified based on the texture state or height distribution variation, and may be used as a measurement feature.
At least one of these measurement features is stored in the DB 1 as a multidimensional feature vector. The measurement feature may be vector quantized.

  2A and 2B are diagrams for explaining the operation of the measurement feature extraction unit 51 when a distance image is used as measurement data. FIG. 2A is a flowchart, and FIG. 2B is a description corresponding to each step of FIG. FIG. When the edge is detected from the distance image, the shape edge of the object is extracted (step ST1). When the region delimited by the edge is divided (segmented) (step ST2), a partial surface region of the workpiece 7 can be extracted. A candidate is selected from here (step ST3). Candidate selection can be calculated based on information such as height and whether there is a surface in the vicinity. In FIG. 2, the surface at the highest position is extracted as a candidate. From the extracted candidate (segment S1), information such as the area of the region, the principal axis direction, the principal normal direction and the area of the surface is extracted to obtain a multidimensional feature vector (step ST4), which is used as a measurement feature. With this method, measurement features such as surfaces can be extracted without having workpiece shape information.

  The measurement feature identification unit 52 estimates whether or not the extracted measurement feature is easy to grasp based on the past measurement feature and its success or failure (whether it is successful). This can be solved as a pattern identification problem in which whether a newly extracted measurement feature is classified into two classes of extraction success or failure is determined using a past measurement feature group in the multidimensional feature vector space. At this time, the neural network, SVM (support vector machine), k-neighbor classifier, and Bayesian classification may be used as the classifier. Whether or not it is easy to grasp may be expressed as a binary value that is easy to grasp or difficult to grasp from the information of whether it is classified as success or failure of extraction. You may calculate from the degree Sp close | similar to the class 1 (removal completion) and the degree Sn close | similar to a class 2 (rejection failure) obtained from a discriminator.

Sp and Sn are, for example, past measurement features classified into class 2 with Sp as the Euclidean distance between the past measurement features classified into class 1 and the extracted measurement features by the nearest neighbor search in the multidimensional feature vector space. And the extracted Euclidean distance of the measurement feature can be calculated as Sn. This may be normalized by the difference in the maximum expected Euclidean distance.
The following values are defined in terms of whether the measurement feature sample (Positive sample) successfully extracted or the measurement feature sample (Negative sample) failed to be extracted.

P / N = S _p / S _n (1)

  Here, P / N means Positive Negative ratio. The ease of grasping the measurement feature can be quantified by P / N. For a workpiece having a high P / N, a gripping operation and an opening / closing operation are calculated with an emphasis on high-speed processing, and for a workpiece having a low P / N, a gripping operation and an opening / closing operation are calculated with an emphasis on a polite operation. P / N may be obtained in other ways based on measurement characteristics.

  FIG. 3 is an image diagram of one method of P / N calculation. At this time, the DB1 stores multidimensional feature vectors that represent extraction success and extraction failure based on past measurement features and their success or failure. In this case, multidimensional vectors of past measurement features are classified into two classes by success or failure, and then a multidimensional feature vector represented by the average value or median value of each feature vector of each class can be created. At this time, all feature vectors may be used, or the amount of calculation may be reduced by focusing on the features with less variation among the features in the class and not using other features. These representative multidimensional feature vector computation processes may be performed by a DB management processing unit (not shown) that manages information in the DB 1 connected to the DB 1, or any one of the information processing units 5. You may make it carry out by a part.

In FIG. 3, the similarity between the multidimensional feature vector representing each class and the extracted multidimensional feature vector is evaluated, and these are set as Sp and Sn. The respective similarities Sp and Sn are the extracted multi-dimensional feature vectors of the measurement features as vectors v _i (i = 1, 2,..., N), the multi-dimensional feature vectors representing successful extraction as vectors p _i , and the extraction failure. Is a multidimensional feature vector representing the vector n _i ,

  Can be calculated as By adding Equation (1) from Equations (2) and (3), an index for determining whether or not it is easy to grasp is added to the measurement feature measured by the sensor while the workpiece is repeatedly taken out. Will be able to. This can be calculated without any prior work shape information.

The measurement feature identification unit 52 not only quantifies the ease of grasping the measurement feature, but also evaluates the similarity between the measurement feature that has been successfully extracted in the past and the extracted measurement feature, and omits subsequent processing. Make it possible. For example, P / N of past measurement features and multidimensional feature vectors are stored in DB1. At that time, first, P / N of the extracted measurement feature is calculated. When the P / N is estimated to be high and easy to grip, a past measurement feature having a close P / N is selected. The degree of similarity Sb between the selected multi-dimensional feature vector b _i of the past measurement feature and the extracted multi-dimensional feature vector v _i of the measurement feature is newly calculated.

  The reason for calculating P / N without calculating equation (4) from the beginning is that the processing time becomes enormous if many accumulated measurement features in the past and equation (4) are calculated. . The calculations of equations (1), (2), and (3) do not change even if the information stored in DB1 increases. By calculating P / N first, the processing time is not greatly increased even if the past information stored in DB1 increases.

  If it is determined that the Sb calculated by Expression (4) is high and the appearance and shape are similar to the past measurement characteristics, the calculation processing of the gripping motion calculation unit 53 and the opening / closing operation calculation unit 54 is omitted, and the past The gripping position / posture and open / close parameters when the measurement feature is successfully gripped are used. Thereby, processing time can be shortened and speeded up. Note that the gripping position / posture and opening / closing parameter data when the past measurement features are successfully gripped are stored in, for example, the DB 1.

  The gripping operation calculation unit 53 calculates a gripping position / orientation from the extracted measurement features. This may be a method of recognizing the position and orientation of the workpiece by matching (searching) the measurement feature with the workpiece shape information in DB1, and calculating the gripping position and orientation corresponding to the workpiece. Interference may be taken into account. Matching is realized by chamfer matching of edge features if it is a two-dimensional image, or by ICP (Iterative Closest Point) method if it is a three-dimensional point group.

  As a method for matching workpiece shape information and measurement data, many methods have been proposed as techniques relating to the extraction of a large number of workpieces. For example, there are examples of processing in Patent Document 2 and Patent Document 3 described above. Any method can be used as long as it is realized by this configuration.

For example, in Patent Document 2, a work in a tray is imaged by an image input camera, and the object recognition apparatus processes the video signal to obtain a line segment image from the contour line. The line image and the matching model are collated, and the position of the workpiece or a specific part of the workpiece is recognized. In collation corresponding to a specific part of the workpiece, the collation time is extremely short.
For example, in Patent Document 3, an entire model of a workpiece and several partial models are taught together with a mutual positional relationship. The robot is moved to the measurement position and the entire model is detected by the sensor. The existence area of each partial model is calculated, and detection of each partial model is attempted within that range. A three-dimensional position / posture of the workpiece corresponding to the detected overall model is acquired, and a reference gripping posture is determined.

  Further, the gripping position / orientation may be calculated without using the workpiece shape information by matching the measurement feature with the hand shape information in the DB 1. Several methods for matching hands have been proposed in the past. For example, there is an example of processing in Patent Document 4 described above, and any method may be used as long as the configuration is realized by this configuration.

  For example, in Patent Document 4, three-dimensional information of a supply unit is generated from three-dimensional measurement data of a supply unit of a workpiece gripped by a robot, and a gripping region including a gripping mechanism region and a gripping portion region that are stored in advance is used. An area where an object exists in the entire partial area and no object exists in the gripping mechanism area is extracted from the three-dimensional information as a grippable area.

  In addition, as another method of hand matching, for example, if the hand is a suction type, a candidate is a place where the matching between the model and the feature point group is high, the feature surface is small, or a hole is formed in the feature surface. If the match level with the model is low, the matching score is lowered from the candidate. Such matching is realized by matching points such as ICP (Iterative Closest Point) if it is a 3D model, by template matching if it is a 2D model, or by matching by convolution processing considering the model as a filter. Can do. In addition, interference with the surrounding environment can be taken into consideration at the same time in the case of the sandwiching type or the extension type. That is, for example, in the case of the pinching type, when the measurement data is included in the area defined by the depth of entry and the vertical and horizontal widths of the hand, this is realized by lowering the matching score. The hand matching control unit calculates the gripping position / posture having the highest matching score for each of the extracted features, and uses the plurality of gripping positions (error → position) and postures as gripping position / posture candidates. Alternatively, for a single extracted feature, a plurality of gripping position / postures whose matching score is higher than a predetermined threshold are calculated, and the plurality of gripping position / postures are set as gripping position / posture candidates. By such processing, the gripping position / posture can be dynamically defined even if the workpiece shape is unknown. Any method may be used as long as it is realized by this configuration.

  As described above, the calculation of the gripping position and orientation is realized by matching workpiece shape information and measurement data, or matching hand shape information and measurement data. However, for example, based on workpiece shape matching, processing time is required, and prior adjustment is required for each part. In addition, there is a problem that there is a possibility of gripping failure based on hand shape matching.

  Therefore, we usually focus on speed and calculate only by matching the measurement characteristics and hand shape information in DB1. However, if P / N is low, gripping may not be stable. Search for position and orientation. As a result, it becomes possible to search the grip position / posture with higher accuracy, or to grasp a position that is easy to grip based on the workpiece shape.

  Even if the workpiece shape is not used, it is usually omitted. However, when the P / N is low, priority is given to the case where the rotation of the hand model and the orthogonality between the workpiece shape edges are high (for example, above a predetermined threshold). A gripping process can be added. As a result, it is possible to reduce the number of cases in which the workpiece is missed by the gripping operation. Further, by grasping a portion close to the center of gravity of the extracted segment (for example, meaning S1 in FIG. 2), it is possible to reduce the possibility of the workpiece falling without being stabilized after the grasping. As described above, based on the estimation result of the ease of grasping by the measurement feature identifying unit 52, the operation can be easily performed by increasing the number of processes.

  The opening / closing operation calculation unit 54 adjusts gripping parameters such as the opening / closing amount, force (gripping force), and speed (operation speed) of the hand 3. In a pinch-shaped hand, what to do with the opening width immediately before gripping is important when considering peripheral interference. Stores hand shape information in a state where the hand is opened with different opening / closing amounts in DB1, and matches the measurement characteristics and the surrounding measurement data with the hand shape data of different hand opening / closing amounts, and the opening / closing with the highest evaluation value Select the amount. The hand shape data matching method is realized by the method described above as “another method of hand matching method”. Basically, the operating speed and gripping force are set to the maximum in order to increase the tact time. However, when it is estimated that the extraction success rate is low, the operation speed and the gripping force are lowered. As a result, the operation becomes polite, and it is possible to prevent the workpiece 3 from being missed by flipping off the workpiece during the opening / closing operation of the hand 3.

  The description of the processing in the information processing unit 5 has been continued. Here, the entire processing is summarized. FIG. 4 is an operation flowchart of the information processing unit 5 when a distance image is used as measurement data.

  In the measurement feature extraction unit 51, when an edge is detected from the distance image, the shape edge of the object is extracted (step ST1), and when the region delimited by this edge is segmented (step ST2), a part of the surface of the workpiece 7 is obtained. Regions can be extracted. From here, candidate selection is performed by calculation based on information such as height (distance) and whether or not there is a surface in the vicinity (step ST3). For example, the surface at the highest position (the distance from the sensor 2 is short) is selected as a candidate. From the extracted candidates, information such as area, principal axis direction, principal normal direction and area of the surface is extracted to calculate a multidimensional feature vector (step ST4), and this is used as a measurement feature.

The measurement feature identification unit 52 performs feature vector characteristic evaluation by P / N of the above-described equation (1) of the multidimensional feature vector that is the measurement feature (step ST5). When P / N is low (when it is less than a predetermined threshold), it is determined that it is difficult to grasp, and when P / N is high (when it is equal to or greater than a predetermined threshold), it is determined that it is easy to grasp (step ST6).
When the gripping motion calculation unit 53 determines that the P / N is low and it is difficult to grip, a gripping position / posture search process is added. The gripping position / orientation search is usually realized by hand model matching according to the method shown as “another method of hand matching” described above (step ST8). In addition, as an additional evaluation process, the hand model is preferentially gripped where the orthogonality (degree) between the rotation of the hand model and the workpiece shape edge is high. Also, by grasping the portion near the center of gravity of the extracted segment, the possibility of dropping after grasping is reduced. In addition, if the system allows adjustment depending on the workpiece, the grip position / posture search may be performed with higher accuracy using the workpiece shape information (step ST9).

  When the P / N is low in this way, the opening / closing operation calculation unit 54 reduces the opening / closing amount of the hand 3 (step ST12), and reduces the opening / closing (gripping) operation speed and gripping force for optimization (step ST13). . This reduces the occurrence of unsuccessful picking, such as flipping the workpiece during the gripping approach or dropping the workpiece after gripping because the gripping position and posture are unstable.

  Further, in the measurement feature identification unit 52, when P / N is high in step ST6, processing with an emphasis on speed is performed. Therefore, first, it is determined whether or not there is a similar high P / N measurement feature (step ST7). If not, it is determined that there is no similar measurement feature that has been successfully extracted in the past. In this case, the gripping motion calculation unit 53 performs processing focusing on speed, performs gripping position / posture search (step ST10), and does not perform additional evaluation processing. The opening / closing operation calculation unit 54 sets the opening / closing (gripping) speed and gripping force to the maximum (step ST15) only by calculating the opening / closing amount (step ST14).

  If there is a similar high P / N measurement feature in step ST7 of the measurement feature identification unit 52, it is determined that there is a similar measurement feature that has been successfully extracted in the past. In this case, the gripping motion calculation unit 53 uses the past gripping position / posture of similar measurement features (step ST11), and the opening / closing motion calculation unit 54 calculates the opening / closing amount / opening / closing (gripping) motion of the past similar measurement features. By using the speed and gripping force (step ST16), the processing is omitted and the processing speed is increased.

  Through the processing of the information processing unit 5 described above, the gripping position / posture and the open / close parameters according to the appearance of the workpiece to be gripped are output to the control unit 6, and the control unit 6 converts these into motion control information for the robot and hand. Then, the operation control of the robot 4 and the hand 3 is performed to realize the workpiece picking operation.

  Based on the opening / closing amount of the hand 3, it is possible to determine whether the workpiece has been successfully taken out or failed. After gripping the workpiece, if the opening / closing amount is 0 (closed), it is determined that the hand 3 does not grip the workpiece, and information indicating that the workpiece is unsuccessful is transmitted to the DB 1. If the opening / closing amount of the hand 3 is not 0, information indicating that the workpiece has been successfully taken out is transmitted to the DB 1. At this time, advance information such as successful removal may be stored in the DB 1 in advance based on the size of the workpiece to be grasped if the opening / closing amount is M or more. The above-described DB management processing unit (not shown) reflects the extracted multi-dimensional feature vector of the measurement feature in the multi-dimensional feature vector representing the successful extraction when the workpiece is successfully extracted. This can be obtained by the following formula when the workpiece is taken out i (i = 1, 2,..., N).

The feature vector p _i which represents the current extraction success, are added divided by the difference between the feature vector v _i of extraction successful measurement features (p _i -v _i) the number of attempts l (el). p _{i + 1} is not limited to equation (5), but may be obtained from p _i and v _i .

When the workpiece extraction fails, the extracted multi-dimensional feature vector v _i of the measurement feature is reflected in the multi-dimensional feature vector n _i representing the extraction failure. This can be obtained by the following formula when the workpiece is taken out i (i = 1, 2,..., N) times.

p _{i + 1} is not limited to equation (6), but may be obtained from n _i and v _i .

  In the initial state where extraction is started, the relationship between extraction success and measurement characteristics is not sufficiently learned. In such an initial state, the grip position / posture and open / close parameter calculation at the time of classification that the P / N in FIG. 4 is high and there is no similar high P / N measurement feature (ST10, ST14, ST15 in FIG. 4). To do. If the number of failure cases increases, the identification performance of the measurement feature identification unit 52 increases, but until then, only by storing data in DB1, the P / N is always high, and similar high P / N measurement features are present. You may identify that there is no. For example, the identification may be started by the measurement feature identification unit 52 when the number of trials exceeds n.

  By using the present invention according to the first embodiment, extraction failures can be automatically reduced while extraction is continued. Further, the processing can be omitted and the speed can be increased. The number of areas that cannot be extracted does not increase due to NG determination (success / unsuccessful workpiece extraction), and retries resulting from the increase do not increase. When using a gripping position and orientation search that does not use a workpiece shape, such as hand model matching in the method described above as “another method of hand matching”, the object does not have shape information of the workpiece and Even in an unknown state, the extracted measurement features can be identified and appropriately extracted as the extraction continues.

Embodiment 2. FIG.
FIG. 5 is a block diagram showing a workpiece picking apparatus according to Embodiment 2 of the present invention. In this embodiment, the extraction failure classification unit 55 performs extraction failure classification based on the opening / closing amount of the hand 3 and stores it in the DB 1. By using not only extraction success / failure information but also extraction failure classification information, calculation of gripping position / posture and open / close parameters is omitted, or only parameters corresponding to the cause of failure are calculated.

  The take-out failure classification unit 55 receives as an input the opening / closing amount of the hand 3 at the time of taking out the workpiece, and classifies whether the take-out failure has been missed, has been flipped by an open / close operation, or has dropped the workpiece after gripping.

  When the user does not grasp the object, the opening / closing amount changes smoothly during the opening / closing operation, and the opening / closing amount becomes 0 after the opening / closing operation is completed. That the opening / closing amount changes smoothly means whether, for example, if the opening / closing movement is realized by trapezoidal control, the opening / closing amount is opened / closed at a speed change equal to that of the trapezoidal control. If the workpiece is gripped in the middle and the opening / closing operation is temporarily stopped, it does not fall under gripping. This is confirmed by sampling the opening / closing amount during the opening / closing operation and comparing it with the model of the opening / closing movement. For this reason, a model of the opening / closing motion is stored in advance in DB1 for the determination of failure. In addition, information such as the multi-dimensional feature vector of the measurement feature at that time, failure to extract, and failure to grasp is collectively stored in DB1 (failure classification).

  Grabbing means that the hand is not in contact with the workpiece and the hand is swung. Since the swing of the hand occurs due to a search error of the gripping position / orientation, the measurement feature identification unit 52 performs measurement similar to the measurement feature that is gripped by the similarity evaluation as in the above equation (4). When the feature is extracted, the gripping motion calculation unit 53 increases the number of gripping position / orientation search processes to perform a more precise search. On the other hand, as the opening / closing parameters not related to the grasping of the opening / closing operation calculation unit 54, the parameters at the time of the grasping are diverted. This increases the success rate of gripping while omitting the processing and increasing the speed.

  In the flipping of the workpiece, the opening / closing operation is temporarily stopped during the opening / closing operation, and the opening / closing amount becomes 0 after the opening / closing operation is completed. Bounce-off means a state where the workpiece is in contact with the workpiece but cannot be gripped. In DB1, information such as multi-dimensional feature vectors of measurement features, extraction failure, and skipping is collectively stored (failure classification).

  The flipping may be improved by changing the gripping position / posture, but may be easily prevented by reducing the opening / closing (operation) speed and opening / closing force of the hand. Therefore, when a measurement feature similar to the measurement feature that has been flipped is extracted by the measurement feature identification unit 52 by the similarity evaluation as in the above equation (4), the opening / closing operation calculation unit 54 opens and closes the hand ( Only the gripping force and the opening / closing (gripping) speed are reduced, and the other parameters are the same as those used for flipping. This increases the success rate of gripping while omitting the processing and increasing the speed. If it still fails, the DB 1 accumulates together the information that the multi-dimensional feature vector of the measurement feature, the extraction failure, the flipping, and the open / close parameter adjustment completed. After that, when a measurement feature similar to the measurement feature that performed the flipping is extracted, the gripping position / posture and the opening / closing amount are searched with high accuracy by increasing the number of processes. This increases the gripping success rate.

  The drop after gripping means a case where the opening / closing amount is not zero at the end of the opening / closing operation, but thereafter the opening / closing amount becomes zero during the transfer of lifting the workpiece from the supply box 8. For this measurement, the amount of opening and closing of the hand from the start of the gripping operation to the completion of transfer is sampled.

  The fall after gripping is greatly influenced by the gripping position and posture. In particular, this is likely to occur when a part of the workpiece away from the center of gravity position is gripped. Accordingly, when a measurement feature similar to the measurement feature that has dropped after gripping is extracted by the similarity evaluation as in the above equation (4) by the measurement feature identification unit 52, the gripping operation calculation unit 53 performs the gripping position. If the number of posture search processes is increased, especially if the workpiece shape is not used, the grip position / posture is corrected so as to approach the center of gravity of the extracted segment. When the workpiece shape is used, the gripping position / posture is corrected so as to approach the workpiece center of gravity. The gripping position / orientation correction is realized by adding the Euclidean distance from the center of gravity to the evaluation value at the time of hand model matching. As for other parameters, the parameters at the time of taking out and dropping are used. As a result, the success rate of gripping is increased while the processing is omitted and the processing speed is increased.

  As described above, according to the second embodiment, the extraction failure classification unit 55 classifies the cause of failure based on the opening / closing amount of the hand 3, and only the parameter corresponding to the cause of failure is readjusted, thereby omitting the processing. However, the success rate of gripping can be increased while speeding up.

Embodiment 3 FIG.
FIG. 6 is a block diagram showing a workpiece picking apparatus according to Embodiment 3 of the present invention. In this embodiment, the success / failure extraction and failure classification are realized based on the measurement information of the second sensor 9 installed outside the supply box 8.

  In the second sensor 9, whether or not the workpiece is gripped is realized by measuring whether the hand is fully closed after the opening / closing operation. This is realized by the difference from the data when the hand 3 passes through a certain position of the supply box 8 after gripping and is measured by the second sensor 9 and is not gripped. In this case, data obtained by measuring a hand not gripped in DB1 is stored. Thereby, the success or failure of extraction can be determined.

  In addition, if the removal fails, it is determined whether the workpiece stacked in the supply box 8 vibrates or moves during the opening / closing operation. Thus, it is possible to determine whether the retrieval has failed. This determination is realized by measuring the bulk workpiece just before the gripping approach and determining whether there is a change in position of the bulk workpiece measured during gripping. Change area detection based on difference, edge detection and edge movement determination Optical flow determination or the like may be used.

  When swinging without interfering with the workpiece, there is a high possibility that there is an error in the height information due to the disturbance of the measurement data in the first sensor 2. Therefore, when a measurement feature similar to the measurement feature that has been swung is extracted by the measurement feature identification unit 52 based on the similarity evaluation as in the above formula (4), a measurement command is issued to the first sensor 2. Increase the number of feeds and measurements. By performing smoothing using a plurality of measurement data, the measurement data is prevented from being disturbed, and the measurement data is used to increase the gripping success rate.

  Interference with the work is greatly affected by the amount of opening and closing of the hand. Therefore, when the measurement feature similar to the measurement feature that interfered with the workpiece is extracted by the similarity evaluation as in the above formula (4) by the measurement feature identification unit 52, the opening / closing operation calculation unit 54 determines the opening / closing amount of the hand. In order to make the calculation more accurate, the number of hand models having different opening / closing amounts is increased to realize the hand model matching by the above-mentioned “another method of hand matching method”. For other parameters, the parameters at the time of workpiece interference are used.

  As described above, according to the third embodiment, based on the measurement data of the second sensor 9, the failure failure classification unit 55 classifies the failure cause, and only the parameter corresponding to the failure cause is readjusted. It is possible to increase the success rate of gripping while omitting the processing and increasing the speed.

Embodiment 4 FIG.
FIG. 7 is a block diagram showing a workpiece picking apparatus according to Embodiment 4 of the present invention. In this embodiment, the extraction failure classification unit 55 performs failure classification based on the opening / closing amount of the hand 3 and the measurement data of the second sensor 9. In this embodiment, the workpiece take-out failure can be classified into four types: idle swinging, unreasonable gripping, flipping off, and dropping after gripping.
In the case of idling, the smoothing result of multiple measurements by the measurement command is used as measurement data by the method described so far.
In the case where the object is not grasped by interference, the opening / closing operation calculation unit 54 adjusts the opening / closing amount of the hand with higher accuracy.
When flipping, the opening / closing operation calculating unit 54 adjusts the opening / closing amount and opening / closing speed of the hand.
When dropping after gripping, the gripping motion calculation unit 53 adjusts the gripping position and orientation.
As described above, according to the fourth embodiment, by increasing the failure classification and adjusting only suitable parameters, it is possible to increase the success rate of grasping while omitting the processing and increasing the speed.

  The present invention is not limited to the above-described embodiments, and it goes without saying that all possible combinations of the features of these embodiments are included.

  1 DB, 2 (first) sensor, 3 hand, 4 robot, 5 information processing unit, 6 control unit, 7 work, 8 supply box, 9 second sensor, 51 measurement feature extraction unit, 52 measurement feature identification unit , 53 Grasping operation calculation unit, 54 Opening / closing operation calculation unit, 55 Extraction failure classification unit.

Claims (9)

DB that stores at least the measurement features and success / failure of the workpiece,
A sensor that measures the stacked workpieces;
A hand for gripping the stacked workpieces;
A robot that moves the hand to the holding position and posture;
An information processing unit that calculates at least one motion control information among the gripping position and orientation and the opening / closing amount, gripping force, and motion speed from the measurement data;
A control unit that commands a gripping operation to the robot according to the operation control information of the information processing unit, and commands an opening / closing operation to a hand;
With
The information processing unit
A measurement feature extraction unit that extracts measurement features for calculating a gripping position and orientation from measurement data;
A measurement feature identifying unit for identifying the measurement feature from the information in the DB;
A gripping operation calculator that calculates a gripping position and orientation of a hand that grips a workpiece from the measurement feature and the information of the DB;
An opening / closing operation calculating unit for calculating a parameter related to opening / closing of the hand from the measurement feature, gripping position / posture, and information of the DB;
Including
The measurement feature identification unit estimates the success rate of extraction of the measurement feature extracted by the measurement feature identification unit based on the past measurement feature stored in the DB and information on the success or failure of the extraction,
The opening / closing operation calculation unit adjusts at least one of the opening / closing amount, gripping force, and operation speed of the hand according to the estimated extraction success rate.
A workpiece picking device characterized by the above. The information processing unit further includes a take-out failure classifying unit that classifies the cause of take-out failure from at least one state of the hand and the work and stores it in the DB as failure class information,
Based on the past measurement characteristics and failure classification information stored in the DB, the gripping position / posture calculated by the gripping motion calculation unit and the opening / closing amount / grip force / motion speed calculated by the opening / closing motion calculation unit Adjust the parameters that affect the failure,
The workpiece picking apparatus according to claim 1.   When the measurement feature extracted by the measurement feature extraction unit is identified as similar to the past measurement feature by the measurement feature identification unit, and when it is estimated that the past measurement feature has a high grasp success rate, The gripping motion calculation unit and the opening / closing motion calculation unit use at least one of the gripping position / posture, the opening / closing amount, the gripping force, and the operating speed, the same as when gripping a similar measurement feature in the past. The workpiece picking device according to claim 1 or 2.   When the measurement feature extracted by the measurement feature extraction unit is estimated by the measurement feature identification unit to have a low success rate, the number of gripping position / posture searches performed by the gripping motion calculation unit is increased. The workpiece picking device according to any one of claims 1 to 3.   When the measurement feature extracted by the measurement feature extraction unit is estimated by the measurement feature identification unit to have a low success rate, at least one of gripping force / speed is reduced by the gripping operation identification unit. The workpiece take-out apparatus according to any one of claims 1 to 4.   6. The workpiece picking apparatus according to claim 1, wherein the information on the success or failure of the picking is estimated from an opening / closing amount of the hand.   7. The workpiece picking apparatus according to claim 2, wherein the failure classification information is estimated from an opening / closing amount of the hand.   8. The apparatus according to claim 1, further comprising a second sensor that measures an operation of the hand at the moment of extraction, wherein the success or failure of the extraction is estimated from measurement data obtained by the second sensor. The workpiece take-out device according to item.   9. The apparatus according to claim 2, further comprising a second sensor that measures the movement of the hand at the moment of extraction, wherein the failure classification is estimated from measurement data obtained by the second sensor. The workpiece take-out device according to item.

Images