JP5716433B2 - Shape recognition device, shape recognition method, and program thereof - Google Patents

Description

  The present invention relates to a shape recognition apparatus, a shape recognition method, and a program for recognizing the position and orientation of an arbitrary recognition object.

  A picking system that autonomously recognizes a plurality of regularly placed recognition objects and picks up the recognized recognition objects one by one by an industrial robot has become widespread. Further, in recent years, even when the recognition target object is not regularly placed, it is possible to specify the position and orientation of the recognition target object and pick the recognition target object according to the position and orientation. It has become like this.

  For example, a three-dimensional model having a specific shape (cylindrical shape or conical shape) is stored in advance, and an image feature (feature amount) of a recognition target corresponding to the three-dimensional model is extracted from an arbitrary projection image by a two-dimensional camera. A technique is disclosed that corrects the three-dimensional position and orientation of the held three-dimensional model, compares the feature quantities of the recognition object and the three-dimensional model, and determines the degree of matching (for example, Patent Document 1). In addition, a technique (for example, Patent Document 2) that extracts image features from an image in which the brightness of a projected image is converted and emphasizes a specific hue to obtain a feature amount, or a slit light projection pattern by irradiating a recognition object with slit light. A technique for extracting a contour line based on the above and making it a feature quantity (for example, Patent Document 3), and a technique for extracting a pair of parallel line segments corresponding to the outline of an object having a cylindrical shape part and making it a feature quantity ( For example, Patent Document 4) is also known.

  Also, a technique for projecting slit light onto the surface of the recognition object and recognizing the shape of the recognition object based on the similarity between the two-dimensional spatial frequency spectrum of the slit image and the two-dimensional spatial frequency spectrum of a known standard object Is also disclosed (for example, Patent Document 5).

JP 09-178426 A JP 11-051611 A JP-A-10-160464 Japanese Patent Laid-Open No. 01-0708106 JP-A-60-191373

  However, in the techniques based on Patent Documents 1 to 5 described above, the shape of the recognition object is limited to a very simple shape such as a cylindrical shape or a conical shape. Therefore, in the techniques based on Patent Documents 1 to 5, if the recognition target object is not rounded, has no clear straight line or edge, or has irregularities formed on its surface, The object could not be recognized accurately.

  The shape recognition in the picking system is not limited to the simple shape described above, and it is desired that recognition objects of various shapes can be recognized. In order to grasp such various shapes uniformly, the inventor of the present application tried to characterize the shape by a longitudinal axis caused by, for example, deviation of the shape of the recognition object.

  However, the recognition object having the longitudinal axis and the three-dimensional model thereof have the same longitudinal axis as the convergence point where the pattern matching converges (the point where the degree of coincidence between the recognition object and the three-dimensional model is maximized). There are two postures arranged symmetrically, and if the pattern matching converges to the correct one of them, an accurate recognition result can be obtained, but if the pattern matching converges to the wrong posture, the recognition object 3 Although the dimensional model is the same, it may be erroneously determined to be different, or the threshold that can be regarded as the same will not be reached, and processing time may be wasted.

  In view of such problems, the present invention has an object to provide a shape recognition device, a shape recognition method, and a program thereof that can accurately extract not only a simple shape but also a recognition object.

In order to solve the above-described problem, a shape recognition apparatus according to the present invention includes a model holding unit that holds three-dimensional model information indicating a three-dimensional model having a longitudinal axis, and recognition target information that indicates a three-dimensional shape of a recognition target. Posture candidate information indicating two posture candidates in which the recognition target object or the three-dimensional model is symmetrically arranged along the longitudinal axis based on the recognition target information acquisition unit to be acquired, the recognition target information, and the three-dimensional model information. The position and orientation of the recognition target object based on the posture candidate information generation unit to be generated and the posture candidate information and three-dimensional model information of the recognition target object, or based on the recognition target information and the posture candidate information of the three-dimensional model. A recognition object specifying unit for specifying

  The shape recognition device further includes an axis specifying unit that specifies one or more axes of the recognition target based on the recognition target information and the three-dimensional model information, and the posture candidate information generation unit is specified by the axis specifying unit. Alternatively, posture candidate information indicating two posture candidates for one or more axes may be generated.

In order to solve the above problems, another shape recognition apparatus of the present invention shows a three-dimensional model having a plurality of axes that obtain a high degree of coincidence between shapes before and after rotation at one or more rotation angles. Based on the recognition target information or the three-dimensional model information based on the recognition target information and the three-dimensional model information, the model holding unit that holds the three-dimensional model information, the recognition target information acquisition unit that acquires the recognition target information indicating the three-dimensional shape of the recognition target object, Based on the posture candidate information generation unit that generates posture candidate information indicating the same number of posture candidates as the number of axes rotated by the angle formed by the axes of the three-dimensional model, the posture candidate information of the recognition object, and the three-dimensional model information Or a recognition object specifying unit that specifies the position and orientation of the recognition object based on the recognition object information and the attitude candidate information of the three-dimensional model.

  The shape recognition device includes one or more shape recognition devices for the recognition target based on the recognition target information and the three-dimensional model information, and further includes an axis specifying unit for specifying an axis. Posture candidate information indicating the same number of posture candidates as the number of axes may be generated along one or a plurality of axes specified by the specifying unit.

  The recognition target specifying unit may specify the position and orientation of the recognition target using ICP.

In order to solve the above-mentioned problem, the shape recognition method of the present invention previously holds 3D model information indicating a 3D model having a longitudinal axis, acquires recognition object information indicating the 3D shape of the recognition object, Based on the recognition target information and the three-dimensional model information, posture candidate information indicating two posture candidates in which the recognition target object or the three-dimensional model is symmetrically arranged along the longitudinal axis is generated, and the posture candidate of the recognition target object The position and orientation of the recognition target are specified based on the information and the three-dimensional model information, or based on the recognition target information and the posture candidate information of the three-dimensional model.

In order to solve the above-mentioned problem, another shape recognition method of the present invention is a three-dimensional model showing a three-dimensional model having a plurality of axes that obtain a high degree of coincidence between shapes before and after rotation at one or a plurality of rotation angles. The model information is held in advance, the recognition target information indicating the three-dimensional shape of the recognition target object is acquired, and the angle formed by the axes of the recognition target object or the three-dimensional model is determined based on the recognition target information and the three-dimensional model information . Posture candidate information indicating the same number of posture candidates as the number of axes rotated by an angle is generated, and based on the posture candidate information of the recognition target and the three-dimensional model information, or the recognition target information and the posture candidate information of the three-dimensional model Based on the above, the position and orientation of the recognition object are specified.

In order to solve the above problem, a computer uses a recognition target information acquisition unit that acquires recognition target information indicating a three-dimensional shape of a recognition target object, a recognition target information, and a three-dimensional model indicating a three-dimensional model having a longitudinal axis. A posture candidate information generating unit that generates posture candidate information indicating two posture candidates in which a recognition object or a three-dimensional model is symmetrically arranged along the longitudinal axis based on the information, and posture candidate information of the recognition target And a program for functioning as a recognition target specifying unit for specifying the position and orientation of a recognition target based on the recognition target information and the three-dimensional model posture candidate information Is done.

In order to solve the above problem, the computer is configured to recognize a recognition target information acquisition unit that acquires recognition target information indicating a three-dimensional shape of the recognition target object, recognition target information, and at one or a plurality of rotation angles. And the axis rotated by the angle of the angle formed by the axes of the recognition object or the three-dimensional model based on the three-dimensional model information indicating the three-dimensional model having a plurality of axes that obtain a high degree of coincidence with the shape after rotation Based on the posture candidate information generating unit that generates posture candidate information indicating the same number of posture candidates, and the posture candidate information and the three-dimensional model information of the recognition target object, or the recognition target information and the posture candidate information of the three-dimensional model and A program for functioning as a recognition object specifying unit that specifies the position and orientation of a recognition object based on the above is provided.

  According to the present invention, not only a simple shape but also a recognition object can be accurately extracted.

It is explanatory drawing which showed the rough connection relation of the picking system. It is explanatory drawing for demonstrating the outline | summary of a two-stage pattern matching. It is the functional block diagram which showed the schematic structure of the shape recognition apparatus in 1st Embodiment. It is explanatory drawing which illustrated the three-dimensional model which has a longitudinal axis. It is explanatory drawing for demonstrating the processing operation of a central control part. It is the flowchart which showed the flow of the process of the shape recognition method. It is explanatory drawing for demonstrating the identification process of a recognition target object. It is the functional block diagram which showed the schematic structure of the shape recognition apparatus in 2nd Embodiment. It is explanatory drawing for demonstrating the recognition target object from which a matching degree becomes high by giving rotation.

  Hereinafter, preferred embodiments of the present invention will be described in detail with reference to the accompanying drawings. The dimensions, materials, and other specific numerical values shown in the embodiments are merely examples for facilitating the understanding of the invention, and do not limit the present invention unless otherwise specified. In the present specification and drawings, elements having substantially the same function and configuration are denoted by the same reference numerals, and redundant description is omitted, and elements not directly related to the present invention are not illustrated. To do.

(First Embodiment: Picking System 100)
FIG. 1 is an explanatory diagram showing a schematic connection relationship of the picking system 100. The picking system 100 includes a three-dimensional shape measuring device 110, a shape recognition device 120, a robot control device 130, and a robot 140.

  The three-dimensional shape measuring apparatus 110 is composed of, for example, a laser scanner type three-dimensional distance sensor, and a plurality of recognition objects (workpieces) 112 on a measurement surface (surface) of a plurality of recognition objects (workpieces) 112 stacked irregularly. The three-dimensional coordinates of the detection point are detected. Specifically, the three-dimensional distance sensor as the three-dimensional shape measurement apparatus 110 includes a Z-axis coordinate indicating the distance from the three-dimensional shape measurement apparatus 110 of a plurality of detection points on the measurement surface and a surface perpendicular to the detection direction. The X and Y axis coordinates on the (XY plane) are combined to form three-dimensional coordinates, and recognition target information that is information indicating the three-dimensional coordinates of all of the plurality of detection points is generated. The recognition target information generated in this way is transmitted to the shape recognition device 120. Here, three-dimensional coordinates at a plurality of detection points on the recognition target object are transmitted as the recognition target information. However, the present invention is not limited to this, and various information representing the three-dimensional shape of the recognition target object is transmitted. Can do. Moreover, although the three-dimensional distance sensor is mentioned as the three-dimensional shape measuring apparatus 110, it is possible to employ | adopt various existing sensors which can grasp | ascertain a three-dimensional shape.

  The shape recognition device 120 extracts the recognition target object 112 based on the recognition target information received from the three-dimensional shape measurement device 110, and the recognition target 112 is indicated by the three-dimensional model 122 by pattern matching or the like. The object 112 is recognized and the position and orientation of the recognition object 112 are specified.

  The robot control device 130 receives information indicating the position and orientation of the one recognition object 112 recognized as described above from the shape recognition device 120, and issues a control command for causing the robot 140 to pick the recognition object 112. . In response to the control command, the robot 140 picks the instructed one recognition object 112 through the suction band 142. With such a picking system 100, for example, it is possible to automate the work of aligning the recognition objects 112 stacked irregularly at a predetermined position.

Such shape recognition in the picking system 100 is not limited to a simple shape such as a cylindrical shape or a conical shape, and it is desired that recognition objects 112 having various shapes can be recognized. However, it is difficult to uniformly process all recognition objects 112 having different shapes. Therefore, the inventor of the present application has focused on the fact that the shape can be characterized by the longitudinal axis caused by the deviation of the shape of the recognition object 112 in order to capture such various shapes uniformly. A general recognition object 112 has a longitudinal axis (longitudinal direction) except for a recognition object that is completely symmetric like a sphere. The longitudinal axis refers to an axis that has become longer due to a deviation in shape .

  However, even if the longitudinal axis is specified, the recognition object 112 having the longitudinal axis is not completely point-symmetric with respect to the longitudinal axis direction (the shapes directed in the forward direction and the reverse direction are completely coincident). There are two different postures in the forward direction and the backward direction, and which of the two posture candidates is the true posture is not known without waiting for the result of pattern matching. The two orientation candidates exist in this way. If the longitudinal axes of the recognition object 112 and the three-dimensional model 122 are matched, the matching degree of pattern matching is high even if the recognition subject 112 is in the wrong orientation. caused by.

  Here, focusing on the effects of the matching error in the longitudinal axis direction and the matching error in the short axis direction (axis perpendicular to the longitudinal axis) in pattern matching, the matching error in the short axis direction is much less during posture convergence. Although there is no great influence, the matching error in the longitudinal axis direction has a great influence, and once it starts to converge to the wrong convergence point (posture), it cannot be changed to the true convergence point. Therefore, if the pattern matching converges in an incorrect posture depending on the initial value and the convergence direction of pattern matching, it is erroneously determined that the recognition target object 112 and the three-dimensional model 122 are different although they are the same. Or the threshold that can be regarded as identical may not be reached, and processing time may be wasted.

  Therefore, when a plurality of pattern matching convergence points between the recognition target object 112 and the three-dimensional model 122 are assumed, the inventor of the present application relates a plurality of posture candidates that can be converged in advance with respect to the recognition target object 112 or the three-dimensional model 122. Each of the plurality of posture candidates is generated and used for pattern matching.

  For example, when the recognition object 112 has a longitudinal axis, there are two extreme points (convergence points) with high coincidence, so that the shape recognition device 120 is acquired by the three-dimensional shape measurement device 110. Two posture candidates, that is, the posture itself of the recognition target object 112 and a posture obtained by simply rotating the recognition target object 112 around the Z axis by 180 °, are prepared, and the two posture candidates of the recognition target object 112 and the three-dimensional model 122 are prepared. And the position and orientation of the recognition target object 112 are specified. Alternatively, the shape recognition apparatus 120 prepares two posture candidates, that is, the posture of the three-dimensional model 122 and a posture obtained by simply rotating the three-dimensional model 122 by 180 °, and the two posture candidates of the three-dimensional model 122 and the recognition target object. The pattern matching with 112 is executed to identify the position and orientation of the recognition object 112.

  With this configuration, in pattern matching, at least one of the two posture candidates reaches the true convergence point, so even if the other posture candidate fails to converge or falls into an erroneous determination, Without affecting the result, the position and posture of the recognition target object 112 can be accurately specified by one posture candidate.

  However, in the above-described pattern matching for preparing two posture candidates from the beginning, the pattern matching with the posture candidates must be performed twice in parallel over the entire pattern matching process, which increases the processing load. End up. In addition, in some matching methods, the method of searching for a convergence point is not simple, so both candidates may converge to an incorrect posture. Therefore, the present inventor further reduced the processing load by dividing pattern matching into two stages. That is, in the first stage, an axis on which the result of pattern matching between the recognition object 112 and the three-dimensional model 122 can converge is specified, and two orientation candidates aligned in opposite directions with respect to the specified axis are generated. In the second stage, pattern matching is strictly performed using the posture candidates. However, when the three-dimensional model having an axis is completely symmetric in the direction of the longitudinal axis, it is not necessary to create a candidate posture because both of the two candidate postures (forward direction and reverse direction) are true.

  FIG. 2 is an explanatory diagram for explaining an outline of two-stage pattern matching. For example, as shown in FIG. 2A, it is assumed that there is a recognition object 112 and a three-dimensional model 122 in the image of a “key” having a longitudinal axis. Here, in order to facilitate understanding, the longitudinal axis of the recognition object 112 is in a state in which the longitudinal axis of the three-dimensional model 122 is rotated 270 ° clockwise in FIG. The spin angle is not considered. In the pattern matching, first, the recognition target object 112 is rotated in a direction (indicated by an arrow along the locus of coincidence in FIG. 2B) in which the degree of coincidence of both increases with the three-dimensional model 122 as a reference. . When the degree of coincidence settles to an extreme value, that point is used as a convergence point for pattern matching.

  At this time, if the initial value of the pattern matching is an attitude corresponding to the rotation angle of the recognition target object 112 in FIG. 2B equal to or greater than 180 °, the pattern has a correct attitude (270 °) matching the three-dimensional model 122. Matching converges and an accurate recognition result can be obtained. However, if the initial value of pattern matching is a posture corresponding to the rotation angle of the recognition target object 112 being less than 180 °, the pattern matching will converge to an incorrect posture (90 °).

  Here, it is unclear whether the correct posture (270 °) or the wrong posture (90 °) converges, but the correct posture (270 °) and the wrong posture (90 °) are rotated by 180 ° relative to each other. It can be inferred that it is a posture. Therefore, in the first stage, the axis where the pattern matching is expected to converge is extracted, and in the second stage, the two postures that are symmetrically arranged (rotated 180 °) along the extracted axis are strict. A correct matching posture is obtained by performing pattern matching.

  For example, as shown in FIG. 2C, (1) the recognition object 112 is rotated clockwise from 0 ° around the Z axis, and the degree of coincidence becomes the first extreme value (90 °). When the previous predetermined threshold is reached, the pattern matching is once stopped. In the example of FIG. 2C, it is assumed that the predetermined threshold value is reached when the recognition target object 112 rotates by 80 °. The recognition target object 112 rotated by 80 ° is set as a second-stage posture candidate, and the recognition target object 112 rotated by 180 ° (rotated by 260 °) is also set as a posture candidate. Subsequently, (2) strict pattern matching is performed in parallel based on the two orientation candidates (80 °, 260 °) of the recognition target object 112.

  Then, the two posture candidates show extreme values at 90 ° and 270 °, respectively, and the posture of 270 ° having a high degree of coincidence is determined as the true posture of the recognition target object 112. Here, since there are two posture candidates for one longitudinal axis, pattern matching is avoided by converging pattern matching only in a wrong posture by performing pattern matching for both of them.

  In addition, since it is known that there is a relationship between the two postures, for example, a symmetrical relationship along the axis, the axis of one of the postures may be used for specifying the axis. Does not require strict pattern matching. That is, in the first stage, it is not necessary to perform strict pattern matching (extreme value search), and only one axis may be extracted with a gentle threshold.

  Here, since the optimum convergence point (extreme value) is derived for at least one posture candidate, the pattern matching is completed without deriving the other convergence point. Therefore, the position and orientation of the recognition object 112 are accurately derived. Further, since the pattern matching until the axis is extracted is not executed for each of the two extreme values, but only one of them is performed, the processing load can be reduced. The above is the outline of the pattern matching of this embodiment. Below, the specific structure of the shape recognition apparatus 120 which can extract the recognition target object 112 which has an axis | shaft exactly is shown, and the flow of a specific process (shape recognition method) is explained in full detail after that.

(Shape recognition device 120)
FIG. 3 is a functional block diagram illustrating a schematic configuration of the shape recognition device 120. The shape recognition apparatus 120 includes a model holding unit 210, a data buffer 212, and a central control unit 214. The shape recognition apparatus 120 generates two posture candidates for each axis with respect to one or a plurality of axes of the recognition target object 112. Here, in order to facilitate understanding, the shape recognition apparatus 120 has two axes for one axis (longitudinal axis). Explain with one attitude. However, it goes without saying that it is not limited to one axis.

  The model holding unit 210 includes a ROM, a nonvolatile RAM, a flash memory, an HDD (Hard Disk Drive), and the like, and is a 3D model 122 corresponding to the recognition target object 112, particularly a 3D model 122 having a longitudinal axis. Holds dimensional model information.

  FIG. 4 is an explanatory diagram illustrating a three-dimensional model having a longitudinal axis. For example, FIG. 4 (a) is a cylindrical shape, FIG. 4 (b) is a cone shape, FIG. 4 (c) is a shape combining cylinders with different diameters, FIG. 4 (d) is a rectangular parallelepiped shape, and FIG. 4 shows a shape of a slanted column, and in both cases, it is common to have a longitudinal axis 250 in the vertical direction in FIG. In addition, various recognition objects 112 having a longitudinal axis 250 such as a building pillar, a post, a telephone booth, and a utility pole are assumed.

  The data buffer 212 is configured by SRAM, DRAM, or the like, and temporarily holds recognition target information received from the three-dimensional shape measuring apparatus 110.

  The central control unit 214 includes a central processing unit (CPU), a signal processing unit (DSP: Digital Signal Processor), a ROM and a memory storing programs, a semiconductor integrated circuit including a RAM as a work area, and the like. Manage and control the entire 120. In the present embodiment, the central control unit 214 also functions as the recognition target information acquisition unit 230, the axis specification unit 232, the posture candidate information generation unit 234, and the recognition target object specification unit 236.

  The recognition target information acquisition unit 230 acquires recognition target information indicating the three-dimensional shape of an arbitrary recognition target object 112 from the three-dimensional shape measurement apparatus 110 and stores it in the data buffer 212.

  The axis specifying unit 232 specifies one or more axes of the recognition target object 112 based on the recognition target information and the three-dimensional model information. The axis specification by the axis specifying unit 232 corresponds to the first stage of the two-stage pattern matching described with reference to FIG. Accordingly, the axis specifying unit 232 does not perform strict pattern matching between the recognition target object 112 and the three-dimensional model 122, and specifies only the axis on which pattern matching can converge.

  Here, in the pattern matching (axis specification) between the recognition target object 112 and the three-dimensional model 122, the axis specifying unit 232 changes the position and orientation of the other with reference to either one. It is not limited whether it is set as a standard. In the present embodiment, since the three-dimensional model 122 has more information than the recognition target object 112, the position and orientation of the recognition target object 112 are changed based on the three-dimensional model 122 in order to reduce the calculation load. To do.

  For such axis identification, various methods such as ICP (Iterative Closest Points) and PCA (Principal Component Analysis) can be used. Here, ICP is used as an example. The ICP determines the position and orientation of the three-dimensional model 122 having six degrees of freedom until the total distance (norm) between corresponding points for identifying the recognition object 112 and the three-dimensional model 122 is equal to or less than a threshold value. This is a method of updating iteratively using a conjugate gradient method or the like. PCA is a method (principal component analysis) for examining how input data has a main axis in space. ICP uses, for example, matching between point groups of the recognition target object 112 and the three-dimensional model 122, unlike PCA and the like, so that the recognition target object 112 may be partially missing due to overlapping of the recognition target objects 112 or the like. Since it is strong, it is excellent in robustness, and the position and posture of the recognition target object 112 can be specified with relatively high accuracy.

  As described with reference to FIG. 2C, the axis specifying unit 232 is intended to extract an approximate axis for generating a posture candidate without performing strict pattern matching. Therefore, when using ICP as the pattern matching method, the axis specifying unit 232 loosens the threshold for the norm that is a material for determining the completion of pattern matching, and if a certain degree of coincidence can be obtained, the position of the recognition object 112 beyond that The axis (here, the longitudinal axis) is specified based on the position and orientation of the recognition object 112 at that time.

  FIG. 5 is an explanatory diagram for explaining the processing operation of the central control unit 214. For example, as illustrated in FIG. 5A, when the axis specifying unit 232 changes the position and orientation of the recognition target object 112 and obtains a certain degree of coincidence with the three-dimensional model 122, the recognition target object 112. The longitudinal axis 250 is specified by the position and orientation. Further, as illustrated in FIG. 5B, even if the axis specifying unit 232 changes the position and orientation of the recognition target object 112 and converges, as a result of the convergence, the recognition target object 112 has an incorrect posture. When a certain degree of coincidence with the dimensional model 122 is obtained, the longitudinal axis 250 is specified by the position and orientation of the recognition object 112 as in FIG. The axis specifying unit 232 is intended to specify the longitudinal axis 250 common to correct and incorrect postures, and is affected by the correctness of the postures as shown in FIGS. 5A and 5B in the first stage. Absent.

  Further, in the axis specifying unit 232, since the pattern matching threshold is set to be loose, the exact matching of the shapes is not questioned, and it is temporarily covered by another recognition object 112, and a part of the recognition object 112 is recognized. Even if it is not, the axis can be easily specified.

  The posture candidate information generation unit 234 symmetrically along the axis for each of one or more specified axes of the recognition target object 112 or the three-dimensional model 122 based on the recognition target information and the three-dimensional model information. Posture candidate information indicating the two placed posture candidates is generated. Here, it is understood in advance that there are two extreme values with respect to one longitudinal axis 250 and that the two extreme values occur in two postures in which the recognition object 112 is arranged symmetrically. Yes. Therefore, the posture candidate information generation unit 234 can generate two posture candidates close to extreme values using the specified axis. For example, when the longitudinal axis 250 of the recognition object 112 is specified as shown in FIG. 5A, two symmetrical recognitions are performed along the longitudinal axis 250 as shown in FIG. The objects 112a and 112b are generated. The two recognition objects 112a and 112b are posture candidates.

  The recognition target object specifying unit 236 is based on the posture candidate information and the three-dimensional model information of the recognition target object 112 generated by the posture candidate information generation unit 234, or the recognition target information and the posture candidate information of the three-dimensional model 122. Based on the above, the exact position and orientation of the recognition object 112 are specified. At this time, when the posture candidate of the three-dimensional model 122 is displaced with reference to the posture candidate, the movement amount and the rotation amount of the three-dimensional model 122 become the position and posture of the recognition target object 112, and the three-dimensional model 122 is recognized. When the posture candidate of the target object 112 is displaced, an amount obtained by reversing the signs of the movement amount and the rotation amount of the recognition target object 112 becomes the position and posture of the recognition target object 112. Various pattern matching methods can be used as the pattern matching method used in the recognition target specifying unit 236, and ICP is used as an example.

  For example, the recognition target specifying unit 236 performs pattern matching on each of the two posture candidates of the recognition target object 112 in parallel with the three-dimensional model, so that the posture in the longitudinal direction is substantially the same for at least one posture candidate. Pattern matching can be started, and the only additional processing is to converge the posture in the short axis direction. Therefore, as shown in FIG. 5D, when there is no overlap or lack of measurement in the recognition object 112, the extreme value of at least one posture candidate (recognition object 112a) is determined as the convergence point. Therefore, the pattern matching is completed without determining the extreme value of the other posture candidate (recognition target object 112b). At this time, the other posture candidate is automatically discarded because the degree of coincidence does not exceed a predetermined threshold. Therefore, the position and orientation of the recognition target object 112 are accurately specified. If there are overlaps or missing measurement points, the one with the higher degree of matching is selected.

  As described above, in the shape recognition device 120 described above, it is possible to obtain a correct result by performing pattern matching in parallel on two symmetric postures, without converging only on a wrong posture, It becomes possible to specify the position and orientation of the recognition object 112 accurately. Further, since the pattern matching until the axis is specified is not performed on each of the two extreme values, but only on one of them, the processing load can be reduced.

  Furthermore, in this embodiment, since ICP is used as the pattern matching method, the position and orientation can be specified even for the recognition target object 112 having various shapes without a straight line or a circular portion. In the ICP, even when a part of the recognition target object 112 is hidden, the robustness is high, so that an effective result can be obtained.

  In addition, a program that functions as the shape recognition device 120 by a computer and a storage medium that stores the program are also provided. Further, the program may be read from a storage medium and taken into a computer, or may be transmitted via a communication network and taken into a computer.

(Shape recognition method)
A shape recognition method using the shape recognition device 120 is also provided. Hereinafter, such a shape recognition method will be described in detail. FIG. 6 is a flowchart showing a processing flow of the shape recognition method. In particular, FIG. 6A shows the overall processing flow of the shape recognition method, and FIG. 6B shows a subroutine related to ICP. Here, the model holding unit 210 holds 3D model information in advance.

  The recognition target information acquisition unit 230 acquires recognition target information indicating the three-dimensional shape of an arbitrary recognition target object 112 from the three-dimensional shape measurement apparatus 110 (S300). Then, the axis specifying unit 232 performs pattern matching by ICP based on the recognition target information and the three-dimensional model information, and specifies one or a plurality of axes (longitudinal axes) of the recognition target object 112 based on a loose threshold value (S302). ).

  The flow of ICP processing by the axis specifying unit 232 will be briefly described with reference to FIG. In the recognition target information acquired from the three-dimensional shape measuring apparatus 110, three-dimensional coordinates (three-dimensional point group data) relating to a plurality of detection points arranged on the measurement surface of the recognition target object 112 are shown. The number of points in the three-dimensional point group data is determined according to the size of the recognition object 112 and the required accuracy of pattern matching. The ICP engine referred to by the axis specifying unit 232 has an obvious background measurement point such as a floor, a support column, and a container for the three-dimensional point cloud data in the recognition target information before performing the ICP repetition process. It is excluded (S350).

  The ICP engine prepares an initial posture and performs posture change processing such as rotation and parallel movement conversion on the initial posture (S352). The posture changing process can be performed on the recognition target object 112 or can be performed on the three-dimensional model 122 side. For example, the calculation processing may be reduced by moving the one with the smaller number of points. Subsequently, corresponding points between the recognition target object 112 after the posture change process and the three-dimensional model 122 are obtained (S354: pairing), and a total distance (norm) between the corresponding points is calculated (S356). It can be said that the smaller the norm, the higher the degree of coincidence of shapes. A posture with a high degree of coincidence is extracted (S358). While the degree of coincidence is less than or equal to a threshold value (norm is greater than or equal to a threshold value) (NO in S360), it is replaced with an initial posture and the above process is repeated. As described above, pattern matching is performed by repetitively updating using the conjugate gradient method or the like until the degree of coincidence exceeds the threshold (norm becomes equal to or less than the threshold) (YES in S360). For example, there are L1 norm and L2 norm as the norm taking method (metric), and the taking method suitable for the application is adopted. The position and orientation thus derived are finally converted from the coordinate system of the shape recognition device 120 to the coordinate system of the robot 140.

  In pattern matching (axis specification) using such an ICP by the axis specifying unit 232, it is only necessary to specify an approximate axial direction, so that a threshold value for ending pattern matching can be set loosely. For example, even if the degree of coincidence obtained when the axis is shifted by 30 degrees is set as a threshold value, a correct operation can be desired. Further, the axis specifying unit 232 can specify the axis even when the recognition target object 112 is covered with another recognition target object 112 and a part of the recognition target object 112 is not recognized.

  FIG. 7 is an explanatory diagram for explaining the identification process of the recognition target object 112. In IPC, if the longitudinal axis of the three-dimensional model 122 is at least twice as long as the short axis, even if 50% or less of the long axis is hidden, it is not mistaken for the short axis. Therefore, even if the recognition target object 112c is covered with other recognition target objects 112d and 112e as shown in FIG. 7, it is possible to recognize two regions (region 1 and region 2 in the figure) having a longitudinal axis. . Here, two regions are temporarily grasped independently. However, since the three-dimensional model 122 pattern-matching to the two regions is the same, as a result, two regions are obtained through one three-dimensional model 122. It is determined that the part is the same recognition target 112c. As described above, by using the ICP, the recognition target object 112 can be stably extracted even when the recognition target object 112 is inclined, overlapped, or foreign matter is mixed.

  The posture candidate information generation unit 234 is symmetrical along the axis for each axis derived by the axis specifying unit 232 of the recognition target object 112 or the three-dimensional model 122 based on the recognition target information and the three-dimensional model information. Posture candidate information indicating the two posture candidates arranged in the position is generated (S304).

  The recognition target specifying unit 236 determines the position of the recognition target based on the posture candidate information and the three-dimensional model information of the recognition target 112 or based on the recognition target information and the posture candidate information of the three-dimensional model 122. The posture is specified (S306). The recognition target specifying unit 236 can also use ICP as pattern matching. Since ICP has already been described with reference to FIG. 6B, detailed description thereof is omitted here. However, since the recognition object specifying unit 236 aims at strict pattern matching, the norm threshold is also strict, and the success or failure of pattern matching is determined when the degree of coincidence converges to the maximum value (extreme value). From these extreme values, it is determined which of the two posture candidates has a high degree of coincidence, or the degree of coincidence does not reach a predetermined threshold value, so that the recognition object 112 and the three-dimensional model 122 are different.

  Even in such a shape recognition method, by performing pattern matching in parallel on two symmetric postures, it becomes possible to obtain a correct result on either side, without converging only on a wrong posture. It becomes possible to accurately identify the position and orientation of the recognition object 112. Further, since the pattern matching until the axis is specified is not performed on each of the two extreme values, but only on one of them, the processing load can be reduced.

(Second Embodiment: Shape Recognition Device 420)
In the first embodiment, the axis specifying unit 232 specifies one or more axes of the recognition target object 112, and based on two posture candidates arranged symmetrically along the axis, the recognition target specifying unit 236 specifies the position and orientation of the recognition object 112. However, when a plurality of axes in the recognition object 112 have the same angle formed by the axes, even if two attitudes are generated for one axis, both of the two attitudes like the longitudinal axis are generated. The degree of coincidence does not increase, and the necessity for generating two postures is poor. Instead, when the recognition object 112 is appropriately rotated so that the axes are aligned, the degree of coincidence is also increased. Therefore, in the second embodiment, instead of generating two postures, an example of generating a posture rotated so as to align a plurality of axes will be described.

  FIG. 8 is a functional block diagram illustrating a schematic configuration of the shape recognition device 420 according to the second embodiment. The shape recognition apparatus 420 includes a model holding unit 210, a data buffer 212, and a central control unit 414. The central control unit 414 includes a recognition target information acquisition unit 230, an axis specifying unit 232, and posture candidates. It also functions as the information generation unit 434 and the recognition target specifying unit 436. Since the model holding unit 210, the data buffer 212, the recognition target information obtaining unit 230, and the axis specifying unit 232 already described as the components in the first embodiment have substantially the same functions, redundant description is omitted. Here, the posture candidate information generation unit 434 and the recognition target object specifying unit 436 of the central control unit 414 that are different in configuration will be mainly described.

  The posture candidate information generation unit 434 generates an axis along one or more axes derived by the axis specifying unit 232 of the recognition target object 112 or the three-dimensional model 122 based on the recognition target information and the three-dimensional model information. Posture candidate information indicating the same number of posture candidates as the number of coincidence when rotating by the angle formed by each other is generated.

  FIG. 9 is an explanatory diagram for explaining the recognition object 112 whose degree of coincidence is increased by rotation. For example, the recognition object 112 in FIG. 9A is formed with an angle formed by three axes 450 of 120 °. The recognition object 112 has three extreme values due to its rotation. Therefore, the posture candidate information generation unit 434 generates posture candidate information indicating three posture candidates rotated by 120 ° along the three axes derived by the axis specifying unit 232, for example. Similarly, the recognition object 112 in FIG. 9B has four radial axes with equal angles, and the recognition object 112 in FIG. 9C has six radial axes with equal angles. Posture candidate information indicating four and six posture candidates, respectively, is generated. When the recognition object 112 has high symmetry, the shapes before and after the rotation may completely match at some rotation angles. In this case, the answer is correct regardless of the angle of rotation, so either of the candidate postures may be omitted.

  The recognition target object specifying unit 436 determines the position of the recognition target object based on the posture candidate information and the three-dimensional model information of the recognition target object 112, or based on the recognition target information and the posture candidate information of the three-dimensional model 122. Identify posture.

  In the shape recognition device 420, as in the shape recognition device 120 of the first embodiment, any one of the shape recognition devices 420 can be performed by performing pattern matching in parallel on a plurality of postures whose extreme values are assumed to be high. Thus, it is possible to obtain a correct result, and it is possible to accurately specify the position and posture of the recognition target object 112 without converging to an incorrect posture.

  Further, similarly to FIG. 6, using the shape recognition device 420, a shape recognition method for performing shape recognition, a program that functions as the shape recognition device 420 by a computer, and a storage medium that stores the program are also provided. Further, the program may be read from a storage medium and taken into a computer, or may be transmitted via a communication network and taken into a computer.

  As mentioned above, although preferred embodiment of this invention was described referring an accompanying drawing, it cannot be overemphasized that this invention is not limited to this embodiment. It will be apparent to those skilled in the art that various changes and modifications can be made within the scope of the claims, and these are naturally within the technical scope of the present invention. Is done.

  Note that each step in the shape recognition method of the present specification does not necessarily have to be processed in time series in the order described in the flowchart, and may include processing in parallel or by a subroutine.

  The present invention can be used for a shape recognition device, a shape recognition method, and a program for recognizing the position and posture of an arbitrary recognition target.

DESCRIPTION OF SYMBOLS 100 ... Picking system 112 ... Recognition object 120, 420 ... Shape recognition apparatus 122 ... Three-dimensional model 210 ... Model holding part 230 ... Recognition object information acquisition part 232 ... Axis specification part 234, 434 ... Posture candidate information generation part 236, 436 ... recognition target part

Claims (9)

A model holding unit for holding 3D model information indicating a 3D model having a longitudinal axis;
A recognition target information acquisition unit for acquiring recognition target information indicating a three-dimensional shape of the recognition target;
Based on the recognition target information and the three-dimensional model information, a posture candidate that generates posture candidate information indicating two posture candidates in which the recognition target or the three-dimensional model is arranged symmetrically along the longitudinal axis An information generator,
Recognition that identifies the position and orientation of the recognition object based on the recognition object posture candidate information and the 3D model information, or based on the recognition object information and the 3D model posture candidate information The object identification part;
A shape recognition device comprising: An axis specifying unit that specifies one or more axes of the recognition object based on the recognition object information and the three-dimensional model information;
The shape recognition apparatus according to claim 1, wherein the posture candidate information generation unit generates posture candidate information indicating two posture candidates for each of the one or a plurality of axes specified by the axis specifying unit. . A model holding unit for holding three-dimensional model information indicating a three-dimensional model having a plurality of axes that obtain a high degree of coincidence between shapes before and after rotation at one or a plurality of rotation angles;
A recognition target information acquisition unit for acquiring recognition target information indicating a three-dimensional shape of the recognition target;
Based on the recognition target information and the three-dimensional model information, generate posture candidate information indicating the same number of posture candidates as the number of rotations of the recognition target object or the angle formed by the axes of the three-dimensional model. A posture candidate information generation unit to perform,
Recognition that identifies the position and orientation of the recognition object based on the recognition object posture candidate information and the 3D model information, or based on the recognition object information and the 3D model posture candidate information The object identification part;
A shape recognition device comprising: An axis specifying unit that specifies one or more axes of the recognition object based on the recognition object information and the three-dimensional model information;
The posture candidate information generation unit generates posture candidate information indicating the same number of posture candidates as the axis along the one or a plurality of axes specified by the axis specifying unit. Shape recognition device.   The shape recognition apparatus according to any one of claims 1 to 4, wherein the recognition object specifying unit specifies the position and orientation of the recognition object using ICP. 3D model information indicating a 3D model having a longitudinal axis is stored in advance,
Obtaining recognition object information indicating the three-dimensional shape of the recognition object;
Based on the recognition target information and the three-dimensional model information, generate posture candidate information indicating two posture candidates in which the recognition target or the three-dimensional model is symmetrically arranged along the longitudinal axis ,
Identifying the position and orientation of the recognition object based on the posture candidate information of the recognition object and the 3D model information or based on the recognition object information and the posture candidate information of the 3D model. A shape recognition method characterized by the above. At one or a plurality of rotation angles, three-dimensional model information indicating a three-dimensional model having a plurality of axes that obtain a high degree of coincidence between the shapes before and after the rotation is previously held,
Obtaining recognition object information indicating the three-dimensional shape of the recognition object;
Based on the recognition target information and the three-dimensional model information, generate posture candidate information indicating the same number of posture candidates as the number of rotations of the recognition target object or the angle formed by the axes of the three-dimensional model. And
Identifying the position and orientation of the recognition object based on the posture candidate information of the recognition object and the 3D model information or based on the recognition object information and the posture candidate information of the 3D model. A shape recognition method characterized by the above. Computer
A recognition target information acquisition unit for acquiring recognition target information indicating a three-dimensional shape of the recognition target;
Based on the recognition target information and three-dimensional model information indicating a three-dimensional model having a longitudinal axis, two posture candidates in which the recognition target object or the three-dimensional model is arranged symmetrically along the longitudinal axis A posture candidate information generating unit that generates posture candidate information to be shown;
Recognition that identifies the position and orientation of the recognition object based on the recognition object posture candidate information and the 3D model information, or based on the recognition object information and the 3D model posture candidate information The object identification part;
Program to make it function. Computer
A recognition target information acquisition unit for acquiring recognition target information indicating a three-dimensional shape of the recognition target;
The recognition target object based on the recognition target information and three-dimensional model information indicating a three-dimensional model having a plurality of axes that obtain a high degree of coincidence between shapes before and after rotation at one or a plurality of rotation angles. Alternatively, a posture candidate information generation unit that generates posture candidate information indicating posture candidates equal to the number of the axes rotated by an angle formed by the axes of the three-dimensional model;
Recognition that identifies the position and orientation of the recognition object based on the recognition object posture candidate information and the 3D model information, or based on the recognition object information and the 3D model posture candidate information The object identification part;
Program to make it function.

Images