JP2013069100A - Three-dimensional position/posture recognition device, industrial robot, three-dimensional position/posture recognition method, program and storing medium - Google Patents

Abstract

PROBLEM TO BE SOLVED: To prevent miss-matching among different parts 91a-91c in stereo matching processing.SOLUTION: Target areas R1 and R2 in which a recognition target part 91a is photographed are extracted from each of plural captured images I1 and I2 of plural parts 91a-91c including the recognition target part 91a taken from different view-point. Stereo matching processing is executed on each target area R1, R2 of plural captured images I1 and I2. That is, the target areas R1 and R2 to be subjected to the stereo matching processing are limited to the target area R1 including the recognition target part 91a in the captured images I1 and I2. Therefore, the stereo matching processing can be executed while excluding the parts 91b and 91c other than the recognition target part 91a from the target area R1. As a result, miss-matching among different parts are prevented in the stereo matching processing.

Description

  The present invention relates to a technique for obtaining the position and orientation of an object in three dimensions by performing stereo matching processing on a plurality of captured images obtained by imaging the object from different viewpoints.

  Patent Document 1 describes a three-dimensional recognition technique for recognizing the position and orientation of an object in three dimensions. In this three-dimensional recognition technique, stereo matching processing is performed on a plurality of captured images obtained by imaging an object from different viewpoints, and the position and orientation of the object in three dimensions are recognized. Specifically, in this stereo matching process, parts having similar characteristics are associated with each other from a plurality of captured images, and the position and orientation of the object in three dimensions are obtained from the parallax between these parts.

Japanese Patent Laid-Open No. 08-201041

  By the way, it is conceivable to apply the above-described technology to an industrial robot for assembling parts such as mechanical parts and electronic parts. As a specific example, there is an industrial robot that assembles parts using an arm that picks up and moves the parts. At this time, in order to reliably pick up the part, it is necessary to accurately grasp the position and orientation of the part in three dimensions. Therefore, as described above, a stereo matching process is performed on a plurality of captured images obtained by imaging components from different viewpoints to recognize the position and orientation of the components in three dimensions, and the industrial robot is based on the recognition result. It is conceivable to control the arm.

  However, such an industrial robot is required to perform an operation of selecting and picking up a recognition target part from a plurality of parts. In such a case, the recognition target component and other components are mixed in each captured image captured for the stereo matching process. For this reason, the recognition target component in one captured image and the other components in another captured image have similar features (for example, edge shapes). There is a possibility that an erroneous correspondence such that these different parts are erroneously associated with each other may occur. If such an erroneous response occurs, naturally, the position and orientation of the recognition target component in three dimensions cannot be correctly recognized.

  The present invention has been made in view of the above problems, and an object of the present invention is to provide a technique capable of suppressing the occurrence of erroneous correspondence such that association is performed between different objects in stereo matching processing.

  In order to achieve the above object, a three-dimensional position / posture recognition apparatus according to the present invention acquires an image acquisition unit that acquires a plurality of captured images obtained by imaging a plurality of objects including a recognition target object from different viewpoints, and an image acquisition unit A target region extraction unit that extracts a target region in which a recognition target object is captured from each of a plurality of captured images acquired by the camera, and a stereo matching process on the target region of each captured image extracted by the target region extraction unit, And a recognition unit for recognizing the position and orientation of the original recognition object.

  In order to achieve the above object, the industrial robot according to the present invention includes a component moving means for picking up and moving a component, the three-dimensional position / posture recognition device, and a recognition object by the three-dimensional position / posture recognition device. And a control unit that controls the component moving unit based on the result of recognizing the three-dimensional position and orientation of the component.

  The three-dimensional position / posture recognition method according to the present invention includes an image acquisition step of acquiring a plurality of captured images obtained by capturing a plurality of objects including a recognition target object from different viewpoints, From each of the plurality of captured images acquired in the acquisition step, a target region extraction step for extracting a target region in which the recognition target object is captured, and a stereo matching process on the target region of each captured image extracted in the target region extraction step And a recognition step of recognizing the position and orientation of the recognition object in three dimensions.

  In order to achieve the above object, the program according to the present invention includes an image acquisition function for acquiring a plurality of captured images obtained by capturing a plurality of objects including a recognition target object from different viewpoints, and a plurality of images acquired in an image acquisition step. The target area extraction function that extracts the target area in which the recognition target object is captured from each of the captured images, and the stereo matching processing is performed on the target area of each captured image extracted in the target area extraction step, and the recognition target in three dimensions It is characterized by having a computer realize a recognition function for recognizing the position and posture of an object.

  In order to achieve the above object, a recording medium according to the present invention records the above program and is readable by a computer.

  In the invention configured in this way (three-dimensional position / posture recognition device, industrial robot, three-dimensional position / posture recognition method, program, recording medium), a plurality of images obtained by imaging a plurality of objects including recognition objects from different viewpoints A target region in which the recognition target object is captured is extracted from each of the captured images. Then, a stereo matching process is executed for each target region of the plurality of captured images. That is, according to the present invention, the region where the stereo matching process is performed is limited to the target region including the recognition target object in the captured image. For this reason, it is possible to execute the stereo matching process by removing objects other than the recognition target object from the target region, and as a result, it is possible to suppress the occurrence of miscorrespondence such that associations are performed between different objects in the stereo matching process. Is possible.

  In the three-dimensional position / posture recognition apparatus as described above, the image acquisition unit can be configured to acquire a plurality of captured images in a state where the epipolar lines are parallel to each other. In such a configuration, the positions in the orthogonal direction orthogonal to the epipolar line are aligned among the plurality of captured images. In other words, in the orthogonal direction, the section where the recognition target object exists is aligned between the plurality of captured images. Therefore, the section where the recognition target object exists in the orthogonal direction is identified for one of the plurality of captured images, that is, for each of the plurality of captured images. Therefore, in order to simplify the extraction of the region where the recognition target object exists, the following configuration may be used.

  That is, the image acquisition unit acquires a plurality of captured images in a state in which the epipolar lines are parallel to each other, and the target region extraction unit is orthogonal to the epipolar line from one of the plurality of captured images. The three-dimensional position / posture recognition apparatus may be configured such that a section where the recognition target object is captured is identified, and a region that coincides with the section in the orthogonal direction is extracted as a target region in each of the plurality of captured images. In this way, by identifying a section in which the recognition target object in the orthogonal direction is captured from one captured image, and extracting a region matching the section and the orthogonal direction as a target area in each of the plurality of captured images, the recognition target object is Extraction of existing target areas can be easily performed.

  In such a configuration, the target region is not limited in the parallel direction parallel to the epipolar line. Therefore, although it is a rare case, when the recognition target object and another object are arranged in the parallel direction, there is a possibility that the above-described erroneous correspondence occurs between them.

  Therefore, the target area extraction unit is configured to obtain each captured image in both the parallel direction parallel to the epipolar line and the orthogonal direction orthogonal to the epipolar line based on the correspondence between the captured images of each object captured in the captured image. The three-dimensional position / posture recognition apparatus may be configured to extract a target region from each of a plurality of captured images by specifying a target region in which a recognition target object is captured. In such a configuration, the target area is specified in both the parallel direction and the orthogonal direction based on the correspondence between the plurality of captured images of each object shown in the captured image. Therefore, objects other than the recognition target object can be removed from the target area in each of these directions. As a result, it is possible to more reliably suppress the occurrence of erroneous correspondence such that correspondence is performed between different objects in the stereo matching processing.

  Note that the stereo matching process is performed by searching for corresponding points between a plurality of captured images. Therefore, the recognition unit sets a search range for searching for corresponding points between the captured images in the stereo matching process for the target region of each captured image based on the information regarding the position of the target region extracted by the target region extraction unit. In addition, a three-dimensional position / posture recognition apparatus may be configured. In such a configuration, it is possible to appropriately set the search range and reduce the processing time required for searching for corresponding points.

  Note that the extraction of the target region in both the parallel direction and the orthogonal direction as described above, and the subsequent stereo matching processing can be easily performed by performing the epipolar lines of each captured image in parallel. Therefore, the image acquisition unit may configure the three-dimensional position / posture recognition apparatus so as to acquire a plurality of captured images in a state where the epipolar lines are parallel to each other.

  It is possible to suppress the occurrence of miscorrespondence such that association is performed between different objects in the stereo matching process.

It is a perspective view which shows typically an example of the double arm robot which can apply this invention. It is a block diagram which shows typically the electric constitution which the double arm robot of FIG. 1 comprises. It is a flowchart which shows an example of the flow of the operation | movement performed by three-dimensional recognition. It is a flowchart which shows an example of the flow of the target area | region extraction process in 1st Embodiment. It is the figure which showed typically an example of each image process performed by the target area | region extraction process of FIG. It is a flowchart which shows an example of the flow of the target area | region extraction process in 2nd Embodiment. It is the figure which showed typically an example of each image processing performed by the object area | region extraction process of FIG. It is the figure which showed typically an example of the stereo matching process in 2nd Embodiment.

First Embodiment FIG. 1 is a perspective view schematically showing an example of a double-arm robot to which the present invention is applicable. FIG. 2 is a block diagram schematically showing the electrical configuration of the double-arm robot of FIG. In FIG. 1 and the drawings shown below, xyz orthogonal coordinate axes having the vertical direction as the z-axis direction are shown as appropriate.

  The double-arm robot 1 includes a robot body 2 and a computer 3 that controls the operation of the robot body 2. The robot body 2 has a schematic configuration in which two arms 22 are attached to the body portion 21. Specifically, each arm 22 is attached via a shoulder joint 23 connected to a drive motor M23. The arm 22 can be moved by rotating the shoulder joint 23 with the drive motor M23.

  A hand 25 is attached to the tip of each arm 22 via a wrist joint 24. A drive motor M24 is connected to the wrist joint 24. Therefore, the direction of the hand 25 can be changed by rotating the wrist joint 24 with the drive motor M24. Further, a drive motor M25 is connected to the hand 25, and the hand 25 can be opened and closed by the drive motor M25.

  Then, the double-arm robot 1 controls the drive motors M23 to M25 so that the components 91 (mechanical components and electronic components) arranged on the component tray 9 are grasped by the hand 25 and carried to a predetermined position. Various operations such as changing the posture of the component 91 by rotating it or releasing the component 91 from the hand 25 and placing it at a predetermined position can be executed. That is, the two-arm robot 1 can assemble the component 91 by executing such various operations in combination.

  Further, two imaging cameras C 1 and C 2 are attached to one of the two arms 22. These imaging cameras C1 and C2 image the component 91 as appropriate while moving integrally with one arm 22. As will be described later, the operations of the arm 22 and the hand 25 are controlled based on the result of imaging the component 91 by the imaging cameras C1 and C2, and the assembly of the component 91 is executed.

  Further, the double-arm robot 1 has a neck portion 26 extending upward from the body portion 21 and a head portion 27 attached to the tip of the neck portion 26. The neck portion 26 and the head portion 27 can move integrally in a vertical direction z and a rotation direction Dr with the direction z as a center.

  Such an operation of the robot body 2 is controlled by the computer 3. The computer 3 includes a driver 4 and a three-dimensional recognition unit 5. The driver 4 functions to read the program 7 recorded on the recording medium 6 that can be read by the computer 3. Examples of such a recording medium 6 include various types such as a CD (Compact Disc), a DVD (Digital Versatile Disc), and a USB (Universal Serial Bus) memory. Then, according to the program 7 read out from the recording medium 6 by the driver 4, the three-dimensional recognition unit 5 performs three-dimensional recognition.

  The three-dimensional recognition unit 5 includes a schematic configuration in which the other configuration circuits 51 to 55 are controlled by the control circuit 50, and is realized by a CPU (Central Processing Unit) and a memory provided in the computer 3. Next, the configuration of the three-dimensional recognition unit 5 will be described through the description of the three-dimensional recognition performed by the three-dimensional recognition unit 5.

  FIG. 3 is a flowchart showing an example of a flow of operations executed in the three-dimensional recognition. In the three-dimensional recognition, when the control circuit 50 outputs an imaging command to the imaging cameras C1 and C2, each of the imaging cameras C1 and C2 captures an image of the component 91 and outputs it to the control circuit 50. In this way, two captured images I1 and I2 obtained by capturing the component 91 from different viewpoints are acquired (step S101).

  These two captured images I1 and I2 are subjected to parallelization processing by the parallelization circuit 51. The parallelization circuit 51 performs parallelization processing (image conversion) on the captured images I1 and I2 to make the epipolar lines of the captured images I1 and I2 received from the control circuit 50 parallel (step S102). . Then, the captured images I 1 and I 2 that have undergone the parallelization processing are output to the target region extraction circuit 52.

The target region extraction circuit 52 performs target matching R1, R2 (ROI: Region Of) for performing stereo matching processing from the received captured images I1, I2.
Interest) is extracted (step S103). That is, in the captured images I1 and I2, a plurality of components 91 including the component 91 to be recognized are mixed. Therefore, in this embodiment, in order to appropriately execute the subsequent stereo matching process, regions (target regions R1, R2) where the recognition target component 91 is captured are extracted from the captured images I1, I2. Details of such target area extraction processing executed by the target area extraction circuit 52 are as follows.

  FIG. 4 is a flowchart illustrating an example of a flow of target area extraction processing in the first embodiment. FIG. 5 is a diagram schematically illustrating an example of each image process performed in the target area extraction process of FIG. In the upper left of each column of FIG. 5, the steps in the flowchart of FIG. 4 in which the image processing shown in the column is executed are shown.

  In this target area extraction processing, edge detection is performed on one of the two captured images I1 and I2, and the edges of the components 91a to 91c are obtained (step S201). Based on the edge detection result, the component 91a to be recognized is identified from the components 91a to 91c.

  Specifically, the pattern P of the recognition target component 91 a in two dimensions (xy plane) is stored in advance in the target area extraction circuit 52. Then, two-dimensional pattern matching is performed for comparing the edges of the components 91a to 91c obtained in step S201 with the pattern P (step S202). As a result, it is determined that the parts 91b and 91c whose shape does not match the pattern P are not recognition target parts, while the part 91a whose shape matches the pattern P is specified as the recognition target part (step S203). .

  In the subsequent step S204, the section ΔY in which the recognition target component 91a is captured in the captured image I1 is specified in the Y-axis direction. In step S205, in each of the two captured images I1 and I2, areas that coincide with the section ΔY in the Y-axis direction are extracted as target areas R1 and R2. The above is an example of the target area extraction process (step S103).

  Returning to FIG. 3, the description of the flowchart of FIG. The target areas R1 and R2 extracted by the target area extraction circuit 52 in this way are output to the stereo matching circuit 53. Then, in the stereo matching circuit 53, the stereo matching process is executed only for the target areas R1 and R2 extracted from the captured images I1 and I2 (step S104). In this stereo matching process, points (microregions) having similar features between the target region R1 of the captured image I1 and the target region R2 of the captured image I2 are associated with each other assuming that they correspond to the same part of the component 91.

  More specifically, feature similarity is determined between the target point in the target region R1 and each point in the target region R2 on the epipolar line passing through the target point, and the target point in the target region R1 Are searched for in the target region R2. Here, since the epipolar lines of the captured images I1 and I2 are parallel to each other and the y-coordinates of the corresponding points coincide with each other, each point of the target region R2 having the same y-coordinate as the attention point and the attention point The similarity of features is determined. As a result, the point having the highest similarity among the points in the target region R2 is associated with the point of interest in the target region R1. Further, by repeating the same operation while moving the target point in the target region R1, each point in the target region R1 and each point in the target region R2 are associated with each other.

  Then, the stereo matching circuit 53 calculates the parallax p between the two points thus associated. Specifically, the parallax p between the point in the target region R1 and the corresponding point in the target region R2 is obtained, and the parallax p is given to the point in the target region R1. In this way, the parallax image Is with the parallax p added to each point (x, y) in the target region R1 is generated. In other words, the parallax image Is is information indicating the parallax p (x, y) at each point (x, y) of the target region R1.

  Thus, the parallax image Is generated by the stereo matching circuit 53 is output to the three-dimensional information generation circuit 54. The three-dimensional information generation circuit 54 generates three-dimensional information S from the received parallax image Is (step S106). That is, the information Is indicating the parallax p at the position (x, y) on the xy plane is converted into the three-dimensional information S indicating the z component at the position (x, y). Thus, the three-dimensional information S indicating the position in the three-dimensional coordinate system (x, y, z) is generated.

  Note that when converting the three-dimensional information S, internal parameters such as the focal lengths of the imaging cameras C1 and C2 and a basic matrix F indicating the positional relationship between the imaging cameras C1 and C2 are required. The basic matrix F is obtained when the imaging cameras C1 and C2 are calibrated in advance and is stored in the three-dimensional information generation circuit 54.

  Thus, the obtained three-dimensional information S of the component 91 is output from the three-dimensional information generation circuit 54 to the model matching circuit 55. Then, the model matching circuit 55 performs a model matching process for matching the three-dimensional information indicating the three-dimensional shape of the component 91a with the three-dimensional information S (step S106). The model of the component 91a used at this time is obtained from CAD (computer aided design) data indicating the outer shape of the component 91a and is stored in the model matching circuit 55 in advance.

  The result Rm of the model matching process is output from the model matching circuit 56 to the control circuit 50. Based on the model matching result Rm, the control circuit 50 executes a three-dimensional recognition process for recognizing the position and orientation of the component 91a in three dimensions (step S107). In other words, in step S107, a three-dimensional model indicating the position and orientation of the component 91a in three dimensions is generated based on the model matching result Rm (three-dimensional modeling). Then, the control circuit 50 controls each of the drive motors M23 to M25 based on the three-dimensional model of the component 91a and the coordinate system of the robot body 2, and executes the operation on the component 91a by the double-arm robot 1.

  In the embodiment configured as described above, the target regions R1, R2 in which the recognition target component 91a is captured from the plurality of captured images I1, I2 obtained by capturing the plurality of components 91a to 91c including the recognition target component 91a from different viewpoints. Is extracted. Then, a stereo matching process is performed on the target areas R1 and R2 of the plurality of captured images I1 and I2. That is, in this embodiment, the regions R1 and R2 for performing the stereo matching process are limited to the target region R1 including the recognition target component 91a in the captured images I1 and I2. For this reason, it is possible to execute the stereo matching process by removing the components 91b and 91c other than the recognition target component 91a from the target region R1, and as a result, the correspondence between different objects is performed in the stereo matching process. Generation | occurrence | production can be suppressed.

  In this embodiment, the area where the stereo matching process is executed is limited to the target areas R1 and R2 in which the recognition target component 91a is captured in the captured images I1 and I2. Therefore, an effect that the time required for the stereo matching process can be shortened is also achieved.

  In this embodiment, the image acquisition unit configured by the control circuit 50 and the parallelizing circuit 51 acquires a plurality of captured images I1 and I2 with the epipolar lines in parallel. In such a configuration, the positions in the Y-axis direction orthogonal to the epipolar line are aligned between the plurality of captured images I1 and I2. In other words, in the Y-axis direction, the section ΔY where the recognition target component 91a is present is aligned between the plurality of captured images I1 and I2. Therefore, regarding the section ΔY where the recognition target component 91a exists in the Y-axis direction, if one of the plurality of captured images I1 and I2 is specified, that is, each of the plurality of captured images I1 and I2 is specified. It will be done. Therefore, in this embodiment, in order to simplify the extraction of the regions R1 and R2 where the recognition target component 91a exists, the following configuration is provided.

  That is, in the target area extraction circuit 52, the section ΔY in which the recognition target component 91a is captured in the Y-axis direction is specified from one of the plurality of captured images I1 and I2 (step S204). Then, in each of the plurality of captured images I1 and I2, regions that match the section ΔY and the Y-axis direction are extracted as target regions R1 and R2 (step S205). In this way, the section ΔY in which the recognition target component 91a is captured in the Y-axis direction is identified from one captured image I1, and the areas that match the section ΔY and the Y-axis direction in each of the plurality of captured images I1 and I2 are the target area R1. By extracting as R2, it is possible to easily extract the target areas R1, R2 where the recognition target component 91a exists.

Second Embodiment In the first embodiment, the target regions R1 and R2 are not limited in the x-axis direction parallel to the epipolar line. Therefore, although it is a rare case, when the recognition target component 91a and the other components 91b and 91c are arranged in the x-axis direction, there is a possibility that the above-described erroneous correspondence occurs between them. Therefore, the second embodiment described below includes a configuration that limits the target regions R1 and R2 not only in the y-axis direction but also in the x-axis direction.

  Also in the three-dimensional recognition of the second embodiment, the flow shown in FIG. 3 is basically executed. Therefore, as in the first embodiment, the control circuit 50 uses the imaging cameras C1 and C2 to acquire two captured images I1 and I2 obtained by imaging the component 91 from different viewpoints (step S101). Then, the parallelization circuit 51 executes parallelization processing on these captured images I1 and I2 to make the epipolar lines of the captured images I1 and I2 parallel (step S102). However, the contents of the target area extraction process executed in subsequent step S103 are different from those in the first embodiment.

  FIG. 6 is a flowchart illustrating an example of a flow of target area extraction processing in the second embodiment. FIG. 7 is a diagram schematically illustrating an example of each image processing performed in the target region extraction processing of FIG. In the upper left of each column of FIG. 7, steps in the flowchart of FIG. 6 in which the image processing shown in the column is executed are shown.

  In this target area extraction processing, edge detection is performed on each of the two captured images I1 and I2, and the edges of the components 91a to 91c are obtained (step S301). In subsequent step S302, the same two-dimensional pattern matching as in step S202 in the first embodiment is performed on the captured image I1, and the recognition target component 91a is confirmed in the captured image I1.

  In step S303, the geometric gravity center of the edge of each component 91a-91c is calculated in each of the captured images I1, I2. As a result, three geometric centroids G11, G12, and G13 are obtained in the captured image I1, and three geometric centroids G21, G22, and G23 are obtained in the captured image I2. Then, labels L1, L2,... Are attached to the parts of the captured images I1, I2 according to the positions of these geometric centroids (step S304).

  Specifically, in the example of FIG. 7, labels L1, L2, and L3 are attached in order from the components 91a, 91b, and 91c having the smallest x coordinate of the geometric center of gravity in the captured images I1 and I2, respectively. Although the example of FIG. 7 is not applicable, if there are a plurality of parts having the same geometric center of gravity x-coordinate, these parts may be labeled in order, for example, from the part of the geometric center of gravity having the smallest y-coordinate. . Thus, the labels L1, L2, and L3 are sequentially attached to the components of the captured image I1, and the labels L1, L2, and L3 are sequentially attached to the components of the captured image I2. As a result, the same label is attached to the same component shown in each of the two captured images I1 and I2.

  Based on such a labeling result, the recognition target component 91a in the captured image I2 is identified by finding the component with the same label L1 as the recognition target component 91a in the captured image I1 from the captured image I2. Can do. In this way, the recognition target component 91a can be specified in each of the captured images I1 and I2 (step S305). That is, in this embodiment, the recognition target component 91a in each of the captured images I1 and I2 is based on the correspondence between the two captured images I1 and I2 of the components 91a, 91b and 91c shown in the captured images I1 and I2. Identified.

  In step S306, target regions R1 and R2 in which the recognition target component 91a is captured are extracted from the captured images I1 and I2 (step S306). The target regions R1 and R2 are set so as to completely include the recognition target component 91a therein, and have a contour that matches the contour of the recognition target component 91a or a contour that is slightly larger than the contour of the recognition target component 91a. Have. As a result, the target regions R1, R2 in which the recognition target component 91a is captured are specified in the captured images I1, I2 not only in the y-axis direction orthogonal to the epipolar line but also in the x-axis direction parallel to the epipolar line. In other words, in the captured images I1 and I2, the components 91b and 91c other than the recognition target component 91a are excluded from the target regions R1 and R2 in both the x-axis direction and the y-axis direction.

  Thus, in this embodiment, the x-axis direction and the epipolar line parallel to the epipolar line are based on the correspondence between the two captured images I1 and I2 of the components 91a, 91b and 91c shown in the captured images I1 and I2. In both directions of the y-axis direction orthogonal to the target regions R1 and R2 in which the recognition target component 91a appears in the captured images I1 and I2, these are extracted. The above is an example of the target area extraction process (step S103) in the second embodiment.

  The description of the flow following the target area extraction processing will be continued using FIG. The target areas R1 and R2 extracted by the target area extraction circuit 52 in this way are output to the stereo matching circuit 53. Then, in the stereo matching circuit 53, the stereo matching process is executed only for the target areas R1 and R2 extracted from the captured images I1 and I2 (step S104).

  FIG. 8 is a diagram schematically illustrating an example of a stereo matching process in the second embodiment. The stereo matching process of the second embodiment is common to the stereo matching process of the first embodiment in that a point in the target region R2 corresponding to the target point N in the target region R1 is searched on the epipolar line E. . However, in the second embodiment, the search range Rs for searching for the corresponding points in the stereo matching process is determined based on the extraction results of the target areas R1 and R2.

Specifically, in this stereo matching process, x coordinates α1, β1 at both ends of the target region R1 in the x axis direction and x coordinates α2, β2 at both ends of the target region R2 in the x axis direction are obtained. Then, when the x coordinate of the target point N in the target area R1 is γ, the search distance Rs in the target area R2 is set from the position separated by the distance Δx1 starting from the x coordinate γ to the position separated by the distance Δx2. Is done. Here, the distances Δx1, Δx2
Δx1 = min (α2−α1, β2−β1)
Δx2 = max (α2−α1, β2−β1)
Given in. Also, min (α2-α1, β2-β1) means the minimum value of (α2-α1) and (β2-β1), and max (α2-α1, β2-β1) is (α2-α1 ) And (β2-β1).

  A point corresponding to the target point N in the target region R1 is found from the search range Rs set for the target region R2, and is associated with the target point N. This association is repeatedly executed while moving the attention point N in the target region R1. Thereby, each point of the target region R1 and each point of the target region R2 are associated with each other, and the stereo matching process for the target regions R1 and R2 is executed.

  When this stereo matching process (step S104) is completed, as in the first embodiment, steps S105 to S107 shown in FIG. 3 are executed to recognize the position and orientation of the component 91a in three dimensions. Then, the control circuit 50 controls the drive motors M23 to M25 based on the recognition result, and executes the operation on the component 91a by the double-arm robot 1.

  In the embodiment configured as described above, the target regions R1, R2 in which the recognition target component 91a is captured from the plurality of captured images I1, I2 obtained by capturing the plurality of components 91a to 91c including the recognition target component 91a from different viewpoints. Is extracted. Then, a stereo matching process is performed on the target areas R1 and R2 of the plurality of captured images I1 and I2. That is, in this embodiment, the regions R1 and R2 for performing the stereo matching process are limited to the target region R1 including the recognition target component 91a in the captured images I1 and I2. For this reason, it is possible to execute the stereo matching process by removing the components 91b and 91c other than the recognition target component 91a from the target region R1, and as a result, the correspondence between different objects is performed in the stereo matching process. Generation | occurrence | production can be suppressed.

  In this embodiment, the area where the stereo matching process is executed is limited to the target areas R1 and R2 in which the recognition target component 91a is captured in the captured images I1 and I2. Therefore, an effect that the time required for the stereo matching process can be shortened is also achieved.

  In this embodiment, the x-axis direction parallel to the epipolar line and the y axis orthogonal to the epipolar line are based on the correspondence between the plurality of captured images I1 and I2 of the components 91a to 91c shown in the captured images I1 and I2. In both directions, the target regions R1 and R2 in which the recognition target component 91a appears in the captured images I1 and I2 are identified, and the target regions R1 and R2 are extracted from the plurality of captured images I1 and I2, respectively. That is, the target regions R1 and R2 are specified in both the x-axis direction and the y-axis direction based on the correspondence relationship between the plurality of captured images I1 and I2 of the components 91a to 91c shown in the captured images I1 and I2. Therefore, the components 91b and 91c other than the recognition target component 91a can be removed from the target regions R1 and R2 in each of these directions. As a result, it is possible to more reliably suppress the occurrence of erroneous correspondence such that correspondence is performed between different parts in the stereo matching processing.

  As described above, the stereo matching process is executed by searching for a corresponding point between the plurality of captured images I1 and I2. Accordingly, in this embodiment, the search range Rs for searching for corresponding points between the captured images I1 and I2 in the stereo matching processing for the target regions R1 and R2 of the captured images I1 and I2 is extracted as the extracted target regions R1 and R2. It is set based on the information regarding the position of. In such a configuration, it is possible to appropriately set the search range Rs and reduce the processing time required for searching for corresponding points.

Others As described above, in the above-described embodiment, the stereo imaging system SS (FIG. 2) including the imaging cameras C1 and C2 and the three-dimensional recognition unit 5 corresponds to the “three-dimensional position / posture recognition device” of the present invention. The arm robot 1 corresponds to the “industrial robot” of the present invention, the computer 3 corresponds to the “computer” of the present invention, the recording medium 6 corresponds to the “recording medium” of the present invention, and the program 7 corresponds to the present invention. Corresponds to "Program". The imaging cameras C1, C2, the control circuit 50, and the parallelizing circuit 51 function as the “image acquisition unit” of the present invention, and the target region extraction circuit 52 corresponds to the “target region extraction unit” of the present invention. The circuit 53, the three-dimensional information generation circuit 54, the model matching circuit 55, and the control circuit 50 correspond to the “recognition unit” of the present invention. Further, the arm 22 and the hand 25 correspond to “component moving means” of the present invention, and the control circuit 50 corresponds to “control means” of the present invention. Steps S101 and S102 correspond to the “image acquisition process” of the present invention, step S103 corresponds to the “target region extraction process” of the present invention, and steps S104 to S107 correspond to the “recognition process” of the present invention. To do. The component 91a corresponds to the “recognition target object” of the present invention, the components 91a to 91c correspond to the “object” of the present invention, the captured images I1 and I2 correspond to the “captured image” of the present invention, and the target Regions R1 and R2 correspond to the “target region” of the present invention. The x-axis direction corresponds to the “parallel direction” of the present invention, and the y-axis direction corresponds to the “orthogonal direction” of the present invention.

  The present invention is not limited to the above-described embodiment, and various modifications other than those described above can be made without departing from the spirit of the present invention. For example, in the above embodiment, the recognition target component 91a is confirmed from the captured image I1 by two-dimensional pattern matching (steps S202 and S302). This two-dimensional pattern matching is to perform pattern matching for the edge of the component 91a. However, various modifications can be made to the method for confirming the recognition target component 91a from the captured image I1.

  Specifically, the recognition target component 91a may be confirmed from the components 91a to 91c based on differences in the areas and contour lengths of the components 91a to 91c. Alternatively, when the components 91a to 91c can be identified by the color difference, the recognition target component 91a may be confirmed based on the color in the captured image I1 using a color camera as the imaging cameras C1 and C2. Furthermore, after extracting each of the components 91a to 91c from the captured image I1, for example, the one with the highest matching score may be selected from these extraction results. Alternatively, after sorting these extracted results by sorting in descending order of matching score, etc., and performing 3D recognition for each, select the one with the highest 3D recognition matching rate (recognition rate) You may do it.

  In the second embodiment, the range when searching for a point corresponding to the point of interest N in the target region R1 from the target region R2 is limited to the search range Rs. However, the search range may be searched from the entire target region R2 without limiting the search range.

  Moreover, in the said embodiment, the three-dimensional recognition part 5 was comprised by CPU and memory which the computer 3 built in. However, the three-dimensional recognition unit 5 may be configured by combining a programmable logic device such as an FPGA (Field-Programmable Gate Array) or a discrete circuit element.

  Also, the above-described edge extraction (steps S201 and S301) can be executed by various methods. For example, a method using operators such as Roberts, Sobel, and Prewitt can be employed.

  Further, the number of the imaging cameras C1, C2 and the attachment positions are not limited to those described above, and can be changed as appropriate.

  Moreover, in the said embodiment, the case where the three components 91a-91c were reflected in the captured images I1 and I2 was illustrated. However, the present invention can also be applied to a case where a number of parts other than three are shown in the captured images I1 and I2.

  The method for obtaining the position and orientation of the component 91 in three dimensions is not limited to the above-described model matching, and can be changed as appropriate.

  The industrial robot to which the present invention can be applied is not limited to the above-described double-arm robot 1. Therefore, the present invention can also be applied to robots having other configurations.

  The present invention can be applied to all three-dimensional image recognition techniques, and can be suitably applied particularly to a technique for recognizing the position and orientation of a part in three dimensions in an industrial robot.

DESCRIPTION OF SYMBOLS 1 ... Dual arm robot 2 ... Robot main body 22 ... Arm 25 ... Hand 3 ... Computer 4 ... Driver 5 ... Three-dimensional recognition part 50 ... Control circuit 51 ... Parallelizing circuit 52 ... Target area extraction circuit 53 ... Stereo matching circuit 54 ... Tertiary Original information generation circuit 55 ... Model matching circuit 6 ... Recording medium 7 ... Programs 91, 91a, 91b, 91c ... Parts C1, C2 ... Imaging cameras I1, I2 ... Imaging images R1, R2 ... Target area Is ... Parallax image E ... Edge E
S ... 3D information S
SS ... Stereo imaging system

Claims (9)

An image acquisition unit for acquiring a plurality of captured images obtained by imaging a plurality of objects including a recognition target object from different viewpoints;
A target region extraction unit that extracts a target region in which the recognition target object is captured from each of the plurality of captured images acquired by the image acquisition unit;
And a recognition unit that performs stereo matching processing on the target region of each captured image extracted by the target region extraction unit to recognize the position and orientation of the recognition target in three dimensions. 3D position and orientation recognition device. The image acquisition unit acquires the plurality of captured images in a state where the epipolar lines are parallel to each other,
The target area extraction unit identifies a section in which the recognition target object is captured in an orthogonal direction orthogonal to the epipolar line from one of the plurality of captured images, and each of the plurality of captured images The three-dimensional position / posture recognition apparatus according to claim 1, wherein a region that coincides with a section in the orthogonal direction is extracted as the target region.   The target area extraction unit, based on a correspondence relationship between the plurality of captured images of each object captured in the captured image, in both a parallel direction parallel to an epipolar line and an orthogonal direction orthogonal to the epipolar line, The three-dimensional position / attitude recognition apparatus according to claim 1, wherein the target area is extracted from each of the plurality of captured images by specifying the target area in which the recognition target object is captured in each captured image.   The recognition unit uses information regarding the position of the target region extracted by the target region extraction unit as a search range for searching for a corresponding point between the captured images in the stereo matching process for the target region of each captured image. The three-dimensional position / attitude recognition apparatus according to claim 3, which is set based on the setting.   The three-dimensional position / posture recognition apparatus according to claim 3, wherein the image acquisition unit acquires the plurality of captured images in a state where the epipolar lines are parallel to each other. Parts moving means for picking up and moving parts;
A three-dimensional position / posture recognition apparatus according to any one of claims 1 to 5,
Control means for controlling the part moving means based on the result of recognizing the three-dimensional position and orientation of the part as the recognition object by the three-dimensional position / posture recognition apparatus. Industrial robot. An image acquisition step of acquiring a plurality of captured images obtained by imaging a plurality of objects including a recognition target object from different viewpoints;
From each of the plurality of captured images acquired in the image acquisition step, a target region extraction step for extracting a target region in which the recognition target object is captured;
A recognition step of performing a stereo matching process on the target region of each captured image extracted in the target region extraction step to recognize the position and orientation of the recognition target in three dimensions. 3D position / posture recognition method. An image acquisition function for acquiring a plurality of captured images obtained by imaging a plurality of objects including a recognition object from different viewpoints;
A target area extraction function for extracting a target area in which the recognition target object is captured from each of the plurality of captured images acquired in the image acquisition step;
Performing a stereo matching process on the target region of each captured image extracted in the target region extraction step, and causing a computer to realize a recognition function for recognizing the position and orientation of the recognition target in three dimensions. A program characterized by   A recording medium on which the program according to claim 8 is recorded and can be read by a computer.

Images