JP2010243405A - Image processing marker, image processing apparatus for detecting position and attitude of marker displayed object, and image processing program - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To obtain a good accuracy of detection without greatly enlarging components of an image processing marker to be displayed on an object, and reduce erroneous detections to expedite processing. <P>SOLUTION: Assuming the marker M to be displayed on an object industrial part W, three marker components m1 to m3 such as a true circle or an ellipse are disposed in association with the corners of a triangle, respectively. An additional component m4 such as the true circle or the ellipse is displayed in association with a weighted center of gravity of the triangle. The three marker components m1 to m3 are identified based on image data photographed with a camera C, and a position and attitude of the marker M, that is, the part W are detected based on a shape, size and position of a corresponding triangle in an image plane, and CAD data, and so on of the preset part W and the triangle. <P>COPYRIGHT: (C)2011,JPO&amp;INPIT

Description

  The present invention relates to an image processing technique used for positioning and handling of industrial parts, for example, and particularly relates to a marker displayed on an object in order to facilitate processing.

  Conventionally, when bin picking, which is important in the automatic assembly of industrial products and industrial parts, is put into practical use, the products, parts, etc. (objects) are photographed with, for example, a digital camera, and the position and orientation of the images are highly accurate. Various image processing techniques for detecting at high speed have been proposed.

  That is, for example, when shooting parts stacked on a transfer pallet from the top in a factory assembly line, the distance to the camera changes depending on the height of the stacked parts, the size of the image changes, and the parts change. As they are stacked and slanted, their appearance, that is, the shape of the image changes.

  Therefore, for example, in the method for recognizing the three-dimensional position and orientation described in Patent Document 1, a large circle part (a circular part having a predetermined diameter) and a small circle part (an index having a specific positional relationship with the target part) are used. In addition, an ellipse corresponding to the great circle part is extracted from the image data taken by the camera from an oblique direction, and an inclination angle from the center to the small circle part is obtained.

  Then, based on the information on the ellipse shape, size, position, and relative positional relationship with the small circle, and the reference information on the dimension shape of the marker stored in advance, the reference coordinate system A coordinate transformation matrix representing the position and orientation of the marker viewed from a certain camera is obtained. This coordinate transformation matrix represents the three-dimensional position and orientation of the marker with respect to the reference coordinate system, and thus the target object. In this document, the robot hand is controlled based on the detected position and orientation. I am doing so.

JP 7-098208 A

  By the way, as described above, in the conventional example, attention is paid to the fact that the great circle part, which is one of the constituent elements of the marker, becomes an elliptical image due to the inclination of the part. And the detection accuracy depends on the size of the great circle portion.

  In other words, the shape and size of the ellipse in the image data changes depending on the inclination of the surface on which the marker is displayed and the distance from the camera. Based on this, the position and orientation of the object are detected. In order to ensure the required accuracy, the large circle portion must be enlarged to some extent, but it may be difficult to display such a large circle portion on the part.

  In addition, in general, image data captured by a camera includes, in addition to the marker components such as the large circle and small circle, the contour lines of a plurality of overlapping parts, scratches on the surface of the parts, foreign objects In addition to the above-described images, optical noise and electrical noise may also appear, which may cause an error in the discrimination between such noise and marker components.

  Therefore, even if the position and orientation of the marker are detected from the identified components, the robot hand is not immediately controlled based on this, but the contour of the object is estimated once from the position and orientation of the marker. It is necessary to verify whether the result is correct, for example, against the contour of the object in the image data. And if there are many misidentifications of a marker, the processing operation for verification will be repeated and processing time will become long.

  In view of these points, the object of the present invention is to devise a configuration of an image processing marker displayed on an object such as an industrial part, and to obtain a good detection accuracy without enlarging the component. And speeding up the process by reducing false detections.

  In the invention according to claim 1 of the present application in order to achieve the above object, three marker components correspond to the vertices of a triangle, respectively, as markers for displaying on the object and detecting the position and orientation by image processing. (Claim 1). This does not mean that the marker consists of only three marker components, and may include other components including the additional components described below.

  If an object on which such a marker is displayed is photographed by a camera or the like as in the above-described conventional example (Patent Document 1) and three marker components are identified in the image data, the marker projected on the image plane The triangle shape, its size and position can be specified. Then, based on these pieces of information and reference information relating to preset triangle dimensions, for example, the triangles in the reference coordinate system are projected into a triangular shape on the image plane as in the conventional example. By obtaining the transformation matrix for this, the position and orientation of the marker (ie, the object) can be detected.

  Thus, in order to increase the accuracy in detecting the position and orientation, it is desirable that the marker is large as described above. Therefore, the three marker components are arranged at a vertex of a large triangle more than a certain distance from each other. It is good to do. Even in such a case, the individual marker components do not become large, and therefore, the meaning is different from the enlargement of the large circle portion of the conventional example, and the marker of the present invention can be displayed on the object without any problem.

  Specifically, a perfect circle, an ellipse, or the like is preferable as the marker component, and the center thereof may be positioned at the apex of the triangle (claim 2). This is because a perfect circle and an ellipse become only a true circle and an ellipse regardless of the inclination, and the logic for identifying them becomes simple. Strictly speaking, perfect circles and ellipses are slightly distorted by perspective, but this distortion can be ignored in practice. Other than that, a geometrical shape such as a parabola or a quadratic curve, which can be easily identified from noise and can easily identify one corresponding point, may be used. Although it is somewhat disadvantageous for discrimination from noise, it may be a part of an ellipse or a parabola.

  In order to display such a marker on an object, for example, when the object is a metal part formed by press molding, it is easy to mark the marker at the time of molding, which is advantageous in terms of cost. . On the other hand, if the marker is printed on the object, the marker component can be a colored point. However, in this case, there is a problem that the cost becomes high.

  By the way, as described above, the posture of the marker cannot be specified as one by only the triangular shape or the size of the marker specified on the image plane, and there are cases where it is specified in two or three ways. For example, when an inclined marker looks like a regular triangle, three vertices cannot be distinguished, so three postures are conceivable. Therefore, it is preferable to have a shape in which three points can be easily distinguished even if they are oblique, such as a right triangle that is considerably longer than the other two sides.

  Alternatively, if a perfect circle or ellipse is used as the marker component as described above, it is possible to distinguish them by making their sizes different from each other, and if using a colored point, It is also possible to distinguish by making the colors different.

  Preferably, another component is added so as to have a predetermined positional relationship with respect to the three marker components (Claim 3), and in this way, relative to the added component According to the positional relationship, the three marker components can be distinguished from each other. In addition, since it is possible to immediately determine whether or not the marker component has been mistakenly identified based on the presence or absence of the additional component, it contributes to speeding up the processing.

  If the additional component is arranged at the center of gravity of the triangle, the marker component cannot be distinguished depending on the positional relationship with this, and therefore it is preferable to arrange the additional component at a position other than the center of gravity (Claim 4). For example, a weighted center of gravity can be considered. Even if the additional component is located at the center of gravity, it is possible to determine the misidentification of the marker component as described above.

  Preferably, the additional component is arranged inside a triangle corresponding to the three marker components (claim 5). This is because if the additional component is arranged outside the triangle, the identification of the additional component may be delayed due to the limitation of the size of the window (processing range) for performing one processing operation of image processing. This is because there is a possibility that the process becomes higher and the processing is delayed.

  From another viewpoint, the present invention is an image processing apparatus for detecting the position and orientation of an object on which a marker as described above is displayed, and extracts an edge image from image data of the object photographed by a camera or the like Edge extracting means, marker component identifying means for identifying images of three marker components from the extracted edge image data, and a shape of the corresponding triangular image plane from the images of the three marker components A position / orientation detection unit that identifies the size and position, and detects the position and orientation of the marker based on the information and reference information relating to a preset shape and size of the triangle. (Claim 6).

  Preferably, the marker component identifying means identifies not only three marker component images but also additional component images, and the position and orientation detection means also considers the position of the additional component in the image plane. Thus, the position and orientation of the marker are detected (claim 7). In this case, as described above with regard to the invention of claim 3, the position and orientation of the marker, and hence the object, can be detected more quickly.

  More preferably, the contour estimation means for estimating the contour of the object in the image plane from the position and orientation of the marker detected by the position and orientation detection means, and the contour of the object in the data of the estimated contour and edge image And a correction means for correcting the detection result of the position and orientation so that the positional deviation between the two becomes small (Claim 8). Detection accuracy is improved.

  From another viewpoint, the present invention is a program (image processing program) for configuring the image processing apparatus as described above by a computer apparatus, and the invention according to claim 9 inputs image data obtained by photographing an object. An image data input step for receiving the image data, an edge extraction step for extracting an edge image from the image data, and a marker component identification step for identifying images of three marker components from the edge image data. The shape, size and position of the corresponding triangle on the image plane are identified from the images of the two marker components, and the marker is based on these information and reference information relating to the preset shape and size of the triangle. The position and orientation are detected (position and orientation detection step).

  Preferably, in the marker component identification step, an image of an additional component is also identified in addition to the three marker component images, and in the position and orientation detection step, the position of the additional component on the image plane is also taken into account. And detecting the position and orientation of the marker. Thus, the same effect as that of the third and seventh aspects of the invention can be obtained.

  More specifically, in the marker component identification step, a window having a predetermined number of pixels in the image data is scanned to select three images that are estimated as marker components, and each of them identifies a triangle located at the vertex. To do. If an image having a predetermined positional relationship with respect to the three vertices of the triangle is identified as an additional component, the process proceeds to a position / orientation detection step. If any of the marker components is not identified, the marker component is selected again.

  Preferably, a contour estimation step for estimating the contour of the object in the image plane from the position and orientation of the marker detected in the position and orientation detection step, and the contour thus estimated is used for the object in the edge image data. And a correction step of correcting the detection result of the marker position and orientation so that the positional deviation between the two becomes small as compared with the contour. By so doing, the detection accuracy of the position and orientation of the object is improved as in the case of the eighth aspect of the invention.

  As described above, the image processing marker according to the present invention includes three components that can be distinguished from noise, and each of them is arranged in association with a vertex of a triangle. Even if the size of the marker is not so large, the required size of the entire marker can be secured. Therefore, for example, even if the target object is tilted like parts stacked in bulk, the position and posture can be detected with high accuracy based on the information of the marker image.

  In particular, if an additional component having a predetermined positional relationship is provided for the three marker components, the three marker components can be distinguished by their relative positional relationship, so that the posture of the object can be more reliably determined. In addition to being able to be detected, it is possible to determine whether a marker component is erroneously identified depending on the presence or absence of an additional component, which contributes to speeding up of processing.

1 is a diagram illustrating a schematic configuration of a system including an image processing apparatus according to an embodiment. It is a figure which shows an example of the image data of the industrial component which displayed the marker of embodiment. It is a conceptual diagram which shows the change of how a marker is seen when components become diagonal. It is a flowchart figure which shows the procedure of an image process. FIG. 3 is a diagram corresponding to FIG. 2 related to a marker of a modified example that does not use an additional component.

  Hereinafter, embodiments of the present invention will be described with reference to the drawings. In addition, the following description is only an illustration essentially and does not intend restrict | limiting this invention, its application thing, or its use.

  FIG. 1 shows an example of a robot system incorporating an image processing apparatus A according to the present invention. This system is installed in an assembly line of a factory, for example, and a component W (object: also shown in FIG. 2) stacked in a supply tray T1 is gripped by the hand of the picking robot R and adjacent. It is moved to the alignment tray T2.

  In the example of the figure, the camera 1 is disposed above the supply tray T1, and image data captured thereby is processed by a personal computer 2 (a computer device, hereinafter abbreviated as PC) according to a predetermined program, The position and orientation of the part W are detected. Data of the position and orientation of the component W thus detected is transferred to the controller C of the robot R and used for pickup control of the component W.

  FIG. 2 (a) shows an example of image data thus taken (an edge image extracted from a grayscale image as will be described later). In this example, the component W has a substantially rectangular plate shape and is rectangular. One is thicker in the direction of the short side of the surface forming a wedge, and the wedge shape becomes thinner as it reaches the other end. In the example shown in the figure, twelve parts W appear to overlap each other. On the surface of each part W, three circular marker components m1, m2, and m3 are engraved at three of the four corners. This may be stamped, for example, when the part W is press-formed.

  More specifically, as shown in FIG. 3 (a), on the surface of the part W, the upper right corner when the long side is turned sideways and the corners on both sides thereof are reversed. Then, three marker components m1 to m3 are engraved so as to be located at three places, the lower left corner and the corners on both sides, and the long side, short side, and A marker M that can recognize a right triangle corresponding to a diagonal line is formed.

  These three marker components m1 to m3 are identified from the image data, the corresponding marker M is identified in a triangular shape, its size and position, and CAD data, etc. set in advance as reference information for the marker M By collating, it is possible to detect the three-dimensional position and orientation of the marker M, that is, the component W on which the marker M is displayed.

  Specifically, first, the size of the triangle specified in this way corresponds to the distance between the component W and the camera C. If the surface on which the marker M is displayed is orthogonal to the optical axis of the camera C, that is, if the part W is not tilted, the shape of the triangle matches that of the CAD data. Therefore, the posture of the marker M, that is, the posture of the part M can be specified by collation with the CAD data.

  That is, as schematically shown in FIG. 3A, an imaginary line connecting the center point c1 of the circle of the marker component m1 and the center point c2 of the same m2 corresponds to the long side of the part W, and similarly the marker An imaginary line connecting the center point c1 of the component m1 and the center point c3 of the same m3 corresponds to the short side of the component W, and further, an imaginary line connecting the center points c2 and c3 corresponds to the diagonal line of the component W. Therefore, the position and orientation of the component W can be uniquely specified from the coordinates of the vertices c1 to c3 of the right triangle indicated by these virtual lines.

  On the other hand, when the component W is tilted, the appearance changes depending on the tilt, so the triangular shape in the image data changes, for example, as shown in FIGS. Therefore, the inclination of the component W, that is, the posture of the component W in the three-dimensional space can be detected based on such a triangular change. Since the concept of this detection is already well known (see, for example, Patent Document 1), the description is omitted. However, in order to obtain the required detection accuracy, the change in appearance due to the inclination of the marker M is used. A large marker M is desirable to some extent, and for this purpose, three marker components m1 to m3 are displayed at the corners of the part W.

  By the way, if the marker image becomes close to a regular triangle as shown in FIG. 3B as a result of the component W being inclined, it is difficult to distinguish the three vertices c1 to c3. Cannot be specified as one, and a total of three postures are estimated including another two postures indicated by broken lines in the figure. Further, even if the marker image does not have an equilateral triangle shape, there are cases where two postures can be considered as shown in FIG.

  Therefore, in this embodiment, another marker component m4 is added so as to have a predetermined positional relationship with respect to the three marker components m1 to m3. Specifically, it is associated with a weighted barycentric position of a right triangle whose apexes are the center points c1 to c3 of the circles of the respective marker components m1 to m3, and a circular additional configuration so that this position becomes the center point c4. Element m4 is arranged.

  The center point c4 of this additional component m4 is the coordinates of the center points c1 to c3 of the three marker components m1 to m3 on the image plane xy, respectively (x1, y1), (x2, y2), (x3, y3) As shown in the following formula (1), the three marker components m1 are based on the coordinates (x4, y4) of the center point c4, that is, by the relative positional relationship with the center point c4. ~ M3 can be distinguished.

  In the equation (1), the coefficients a, b, and c are coefficients representing the weights of the vertices with respect to the center of gravity, and may be set to arbitrary values different from each other. In the case of a = b = c, the center point c4 of the additional component m4 is located at the center of gravity of the triangle, and depending on the positional relationship therewith, the three marker components m1 to m3 cannot be distinguished. In other words, the additional component m4 is preferably arranged at a position other than the center of gravity of the triangle.

  In this embodiment, the three marker components m1 to m3 and the additional component m4 are identified from the image data by an image processing program described below, and the triangular shape, size and position of the marker M on the image plane ( For example, the three-dimensional position and orientation of the marker M (that is, the part W) are detected based on the information on the center of gravity of the triangle) and the information on the size and position of the marker M included in the CAD data. To do.

-Image processing procedure-
Next, the procedure of image processing in the system of this embodiment will be specifically described based on the flowchart of FIG. First, a measurement request signal is sent from the controller C of the robot R to the camera 1 based on the production management information of the factory, etc., and the data of the gray scale image photographed by the camera 1 receiving the signal is sent to the PC 2 for input. (Step S1: Image data input step).

  Then, the PC 2 that receives the input of the image data extracts an edge image from the image data (step S2: edge extraction step). This is a process for converting a grayscale image into an edge image by means of a conventionally known canny edge filter, Sobel filter, or the like. As an example, the image data as shown in FIG. 2A is obtained. As shown in the figure, the image data includes images (noise) of the surface of the component W such as scratches and foreign matters in addition to the contour line of the component W and the marker components m1 to m3.

  Subsequently, in step S3, processing for identifying the ellipse images of the marker components m1 to m3 from noise is performed (ellipse identification step). This is based on the fact that the circular marker components m1, m2, and m3 stamped on the part W appear to be elliptical even when deformed due to the inclination of the part W, and the marker components m1, m2, This identifies m3. Various methods for discriminating an elliptical image from noise have been conventionally known (see, for example, JP-A-2006-65452 and JP-A-2002-183730).

  As an example, a window having a preset number of pixels (for example, 200 × 200 pixels) is scanned in the edge image data to search for edge pixels. Then, the edge is searched for every certain angle (hereinafter, described as 15 degrees) from the edge pixel first found, and a total of 5 points including the first one point and any four points among the 12 points searched later Is temporarily estimated to substitute parameters such as the center of the ellipse.

  Next, the edge is searched for every predetermined angle (hereinafter, described as 10 degrees) around the temporarily estimated ellipse center, and 36 edge pixels are selected. Using these 36 points, an ellipse is estimated by the method of least squares, and parameters such as the center, major axis, minor axis, and axis inclination of the ellipse are determined. This procedure is repeated to detect a number of images (elliptical images) that appear to be ellipses. At this time, priority is given to the prevention of detection omissions and the criteria for determining whether or not an ellipse is set loosely. Is preferred.

  After identifying a plurality (three or more) of ellipse images in this way, in step S4, three ellipse images randomly selected from them are arranged in a triangular shape that is predetermined as the marker M. (Marker detection step). That is, first, candidates are narrowed down using information on the major axis of the selected ellipse. In consideration of the fact that the major axis of the ellipse is smaller than the diameter of the circle of the marker components m1 to m3 and becomes smaller according to the distance from the camera C, an ellipse that is too large is excluded. However, priority is given to prevention of detection omissions, and the narrowing-down conditions should be set loosely.

  Then, for any three ellipse images among the narrowed candidates, as shown by the phantom lines in FIG. 2 (b), assuming triangles whose vertices are the center points c1 to c3, weights of the triangles are assumed. The coordinates of the center of gravity position (see equation (1)) are calculated to determine whether there is another ellipse image centered on this coordinate position. If there is such an ellipse image, the three ellipse images are identified as marker components m1 to m3, the marker M is detected as shown in the figure, and the process proceeds to step S5, while the ellipse image corresponding to the weighted center of gravity position. If there is not, the process starts again from the selection of the three ellipse images.

  In step S5, the marker M is detected based on the triangle shape, size, and position information on the image plane detected as described above, and the triangle shape and size (reference information) set as CAD data. M, that is, the three-dimensional position and orientation of the component W are detected (position and orientation detection step). That is, for example, similar to the conventional example (Patent Document 1), the triangle of CAD data in the three-dimensional reference coordinate system is projected so as to be the triangular shape of the marker M on the image plane detected as described above. Find the transformation matrix for

  In step S6, the contour of the component W on the image plane is estimated using the coordinate transformation matrix obtained in step S5 (contour estimation step). In this method, the contour of the part W in the image plane is estimated by applying the projection transformation matrix to the edge data of the part W in the reference coordinate system created from CAD data or the like and mapping it on the image plane. .

  In subsequent step S7, the position and orientation detection results are compared so that the contour estimated as described above and the contour of the part W in the edge image data (step S2) are compared and the positional deviation between them is reduced. (Step S5) is corrected (correction step). Various methods for matching two contour lines on the image plane in this way have been known in the past, but here, a shift amount (error) between the two contour lines is calculated using a known distance conversion method, The contour is corrected so that this error is minimized.

  That is, using the edge image extracted in step S2, a distance map representing the distance from the edge pixel in all pixels is created, and the position of each edge pixel in the contour estimated in step S6 and the actual part W The distance to each edge pixel is obtained with reference to the distance map, and the sum of the distances is taken as an error. Then, the contour is corrected by repeating the process of reducing the error using a mathematical search method.

  In step S8, the corrected contour of the component W, that is, the data of the final detection result of the position and orientation thereof is transferred to the controller C of the robot R (data output), and the processing routine is terminated. By controlling the picking robot R based on this data, the hand can definitely pick up the component W in the supply tray T1.

  Step S3 of the flow is a marker component identification step for identifying three ellipse images from edge image data, and steps S4 and S5 are positions and orientations for detecting the position and orientation of the marker M (that is, the part W). This is a detection step.

  In the image processing system according to this embodiment, the PC 2 executes an image processing program such as the flow described above, so that the PC 2 includes an edge extraction unit, a marker component identification unit, a position and orientation detection unit, The contour estimation means and the correction means are configured.

  Therefore, according to the image processing apparatus A according to this embodiment, the image data captured by the camera 1 is processed, the triangular shape, size, and position of the marker M displayed on the component W are detected, and the CAD data By obtaining a coordinate transformation matrix between the triangular shape, the three-dimensional position and orientation of the marker M, that is, the part W can be detected.

  The markers M are arranged at the corners of the surface of the part W such that the three circular components m1 to m3 correspond to the vertices of the right triangle, so that the individual marker components m1 to m3 are small. However, the marker M as a whole becomes sufficiently large, and the posture of the component W can be detected with high accuracy by the change in the appearance.

  Therefore, when the contour of the component W is estimated from the position and orientation of the marker M as described above and correction is performed so that the positional deviation from the contour in the image data is reduced, the correction is performed because the positional deviation is relatively small. For this reason, the amount of calculation required for this is small, and the processing can be speeded up.

  In particular, in this embodiment, the additional component m4 is displayed at the weighted center of gravity of the triangle corresponding to the marker M, so that the three marker components m1 to m3 can be distinguished by their relative positional relationship. Therefore, the posture of the marker M (that is, the part W) can be detected more reliably. In addition, the misidentification of the marker components m1 to m3 can be determined based on the presence or absence of the additional component m4, which also contributes to speeding up the process.

  The additional component m4 may be displayed at an arbitrary position where the positional relationship with the three marker components m1 to m3 on the image plane can be described by mathematical formulas. In this embodiment, the additional component m4 is displayed inside the triangle. Compared with displaying outside the triangle, the processing is faster.

  That is, when an elliptical image is identified in step S3 of the flow of FIG. 4, a window of a predetermined number of pixels (eg, 200 × 200) in the image data is scanned and processed once, and this process is performed on the window. This is repeated while shifting, because if the additional component m4 is inside the triangle, the window including the three marker components m1 to m3 always includes the additional component m4.

  In addition, the structure of the marker which concerns on this invention is not limited to the said embodiment. For example, the constituent element of the marker M is not limited to a perfect circle but may be an ellipse. In any case, no matter how the part W is inclined, it becomes only a perfect circle or an ellipse, and the logic for identifying it becomes simple. In addition, a geometric shape such as a parabola or a quadratic curve, which can be distinguished from noise and can easily identify a corresponding point, or a part thereof may be used.

  In addition, the marker component is not stamped when the part W is molded, but can be printed later. In this case, the marker component can be a colored point, and if the color is different, it can be easily distinguished and identified, so the additional component m4 is not necessary.

  Further, even in the case of a component such as a perfect circle or an ellipse, the sizes can be made different from each other so that they can be distinguished. FIG. 5 shows an example of the marker M configured by three perfect circles having different sizes. In the example shown in the figure, since the three components of the marker M are close to each other, the positional deviation of the contour of the component W (indicated by the alternate long and short dash line) detected thereby is slightly larger.

  Furthermore, it is possible to make the three marker components different from each other, for example, a perfect circle, an ellipse, and a hyperbola, so that they can be distinguished from each other. In this case, additional components are unnecessary.

  Furthermore, it is not necessary to display the marker components m1 to m3 at the corners of the part W as in the above-described embodiment, and they are included in the field of view of the camera C at the same time, for example, on the same plane of the part W. What is necessary is just to display on the site | part which can be obtained. The range for displaying them may be set to an appropriate size based on the relationship with the resolution of the camera C, its imaging range, and the like.

  In addition, the marker according to the present invention may have other components including the additional component m4 as long as it has the three marker components m1 to m3 as in the above embodiment. It may have a polygonal shape as a whole including other components.

  In addition, the marker and the image processing method using the marker according to the present invention can be applied not only to the pickup work of the industrial component W as in the above embodiment, but also to other various objects. Needless to say.

  As described above, the marker according to the present invention can ensure the accuracy of detecting the position and orientation of an object without increasing the size of its constituent elements, and contributes to speeding up the processing. Is highly useful.

A Image processing device M Marker m1 to m3 Marker component m4 Additional component W Industrial parts (object)
DESCRIPTION OF SYMBOLS 1 Camera 2 Computer apparatus (PC: Edge extraction means, marker component identification means, position and orientation detection means, contour estimation means, correction means)

Claims (11)

A marker for displaying on an object and detecting its position and orientation by image processing,
The marker for image processing, wherein three marker components are arranged on the object in association with vertices of triangles, respectively.   The marker for image processing according to claim 1, wherein the marker component has an elliptical shape.   The image processing marker according to claim 1, wherein additional components are arranged so as to have a predetermined positional relationship with respect to the three marker components.   The image processing marker according to claim 3, wherein the additional component is arranged at an arbitrary position excluding a center of gravity of a triangle corresponding to the three marker components.   The image processing marker according to claim 3, wherein the additional component is arranged inside a triangle corresponding to the three marker components. An image processing device for detecting the position and orientation of an object on which a marker according to any one of claims 1 to 5 is displayed,
Edge extraction means for extracting an edge image from image data obtained by photographing the object;
Marker component identification means for identifying three marker component images from the extracted edge image data;
The shape, size and position of the corresponding triangle in the image plane of the corresponding triangle are identified from the images of the three identified marker components, and based on these information and reference information relating to the preset shape and size of the triangle An image processing apparatus comprising: position and orientation detection means for detecting the position and orientation of the marker. The marker component identification means identifies the image of the additional component in addition to the image of the three marker components,
7. The image processing apparatus according to claim 6, wherein the position / orientation detection means detects the position and orientation of the marker in consideration of the position of the additional component on the image plane. Contour estimation means for estimating the contour of the object in the image plane from the position and orientation of the marker detected by the position and orientation detection means;
A correction unit that compares the contour estimated by the contour estimation unit with the contour of the object in the edge image data and corrects the detection result of the position and orientation so as to reduce the positional deviation between the two. The image processing apparatus according to claim 6 or 7. An image processing program for detecting by a computer device the position and orientation of an object on which a marker according to any one of claims 1 to 5 is displayed,
An image data input step for accepting input of image data obtained by photographing the object;
An edge extraction step of extracting an edge image from the image data;
A marker component identification step for identifying images of three marker components from the extracted edge image data;
The shape, size and position of the corresponding triangle in the image plane of the corresponding triangle are identified from the images of the three identified marker components, and based on these information and reference information relating to the preset shape and size of the triangle And a position / orientation detection step for detecting the position and orientation of the marker. In the marker component identification step, in addition to the three marker component images, the additional component image is also identified,
The image processing program according to claim 9, wherein in the position and orientation detection step, the position and orientation of the marker are detected in consideration of the position of the additional component on the image plane. A contour estimation step for estimating the contour of the object in the image plane from the position and orientation of the marker detected in the position and orientation detection step;
A correction step of comparing the contour estimated in the contour estimation step with the contour of the object in the edge image data and correcting the detection result of the position and orientation so as to reduce the positional deviation between the two. Item 15. The image processing program according to any one of Items 9 and 10.

Images