JP6052871B2 - Object moving apparatus, method, program, and recording medium - Google Patents

Description

  The present invention recognizes each object and grasps it by a hand when there are irregularly gathered objects of the same shape with a space part in the side shape and divided in outline, such as a coiled spring. The present invention relates to an object moving apparatus, a method, a program, and a recording medium.

  In various industries, robots are used for handling workpieces such as gripping and conveying. Among such robot applications, there is a so-called “unloading” application in which individual workpieces are extracted from workpieces in a messy state in a storage box or on a pallet.

  In bulk pick-up use, first, after determining the approximate position of a work to be picked up based on a two-dimensional image of a wide area imaged by a sensor for whole search using a CCD camera or the like, a three-dimensional visual sensor or the like is used. Accurate measurement of the three-dimensional position and orientation of a specific workpiece based on image data of a narrow area by a precision measurement sensor, and gripping the workpiece by a robot based on the measured three-dimensional position and orientation of the workpiece Is done. As a three-dimensional visual sensor, slit light or spot light is projected onto an object to form a light band or light spot that is brighter than the surroundings on the object surface, and this is observed by an imaging device such as a CCD camera. For example, a device that performs three-dimensional measurement of an object based on the principle of triangulation and a device that measures the position of an object three-dimensionally by stereo image processing using two imaging devices are known.

  For example, Patent Document 1 discloses a composite sensor robot using a combination of a CCD camera for acquiring a two-dimensional image of a wide area and a laser sensor for measuring a three-dimensional position of a work in a narrow area. A system is disclosed.

  Patent Document 2 discloses a technique for determining the priority of picking up a plurality of detected workpieces based on partial characteristics of each workpiece detected by a camera that captures a wide area.

  However, these conventional technologies are applied to workpieces with continuous contours such as bolts, and edge detection is premised, so the outer shape is complicated like a spiral spring, there are gaps, the back is visible, and the flat part is With a small number of objects, there is a problem that there is no clear feature generally used in image processing and it is difficult to apply.

International Publication No. 97/24206 Pamphlet JP 2004-50390 A

  The problem to be solved is that it has been difficult to recognize an object with a complicated outer shape such as a coiled spring, a gap, and a deep view, and to grasp and move it with a hand.

The present invention is a complex contour such as coil spring, there is a gap, in order to make it possible to move gripped by recognizing the object back is visible hand, if there is an end surface and a space portion in the shape contour An object moving device that identifies and moves the object with a hand while the objects of the same shape having irregularly gathered side surfaces are gathered, and illuminates the entire gathered object A highlight extraction processing unit that binarizes each image picked up by two cameras in a lighted state and extracts highlights of each reflection shape for each image, and the extracted highlights A highlight determination processing unit that performs an area threshold process to determine a highlight on a side surface corresponding to the side surface of the target object and a highlight on an end surface corresponding to the end surface of the target object, and a side shape of the target object Multiple to correspond A feature amount extraction processing unit that extracts the feature amount from the highlights on each side surface, a grouping processing unit that determines the highlights on the side surface as a group for each set based on the feature amount, and the grouping processing unit. A correspondence detection processing unit that detects correspondence between the images of the group, and stereo image processing that recognizes the object in the group corresponding to the detection and determines the position and orientation of the object based on the position of each camera The object moving device includes a position / orientation calculation unit that calculates the position of the object and a control unit that controls the operation of the hand based on the calculated position and orientation of the object.

An object of the present invention is to identify and move a target by identifying the target in an irregularly assembled set of objects having an end surface and a side surface in which a space is present and the contour is divided. An object moving method, wherein each image captured by two cameras in a state in which illumination light is applied to the entire aggregated object, and binarization processing is performed for each reflection shape highlight for each image. A highlight extraction step to extract, and an area threshold process is performed on the extracted highlight to discriminate a side highlight corresponding to the side of the object and a highlight on the end corresponding to the end of the object highlight determination step, and the end face highlight detection step of determining the final end face highlights from the highlight of the discriminated end face, a plurality of feature amounts for corresponding to the side shape of the object A feature amount extraction step of extracting from the highlight of each side, grouped determining for each said image a highlight of the side as a group of a plurality set each by the feature amount, a highlight of the group and the end A correspondence detection step for detecting correspondence between the images, and a group and a face highlight corresponding to the detection to recognize the object and determine the position and orientation of the object with reference to the position of each camera. The object moving method includes a position / posture determination step to be determined and an operation step of causing the hand to move according to the position and posture of the object.

The present invention has a function of identifying and moving the object by the hand while the objects having the same shape including the end surface and the side surface where the space portion is divided and irregularly gathered are gathered irregularly. Is an object movement program for causing a computer to perform the binarization process on each image captured by two cameras in a state in which illumination light is applied to the entire assembled object, and highlights of each reflection shape. A highlight extraction function for extracting each of the images, a threshold processing of an area on the extracted highlight, and a highlight on a side surface corresponding to the side surface of the object and an end surface corresponding to the end surface of the object highlight determination function of determining a highlight, a feature extraction function for extracting a plurality of feature amounts for corresponding to the side shape of the object from the highlight of the each side, by the feature quantity And grouping function of determining for each said image a highlight of the serial side as a group of a plurality pair each, and the corresponding detection function for detecting a correspondence between the image of the group, the subject in the group corresponding with the detected A position / posture calculation function for recognizing an object and calculating the position and posture of the target object based on the position of each camera by stereo image processing, and the operation of the hand based on the calculated position and posture of the target object A feature of the object movement program is that an operation control function to be controlled is provided as the function.

  The present invention is characterized in that the object recognition program having the above function is carried.

  Since the object moving device of the present invention has the above-described configuration, the object having the same shape and having the space part present in the shape and divided in outline is imaged by two cameras in an irregular collection. Highlights are grouped into sets of features based on the feature values extracted from the highlights obtained by image processing of each image, and the objects are recognized by the corresponding group between the images, and the objects are based on the position of each camera. The position and orientation of the object can be calculated by stereo image processing, and the object can be grasped and moved by controlling the operation of the hand.

  Since the object recognition method of the present invention has the above-described configuration, the object having the same shape with the space portion existing in the shape and divided in outline is imaged by two cameras in an irregular collection. Highlights are grouped into sets of features based on the feature values extracted from the highlights obtained by image processing of each image, and the objects are recognized by the corresponding group between the images, and the objects are based on the position of each camera. The position and orientation of the object can be determined by stereo image processing, and the object can be grasped and moved by the movement of the hand based on this determination.

  Since the object recognition program according to the present invention has the above-described configuration, the object having the same shape with the space portion present in the shape and divided in outline is irregularly gathered and captured by two cameras. Highlights are grouped into sets of features based on the feature values extracted from the highlights obtained by image processing of each image, and the objects are recognized by the corresponding group between the images, and the objects are based on the position of each camera. The position and orientation of the object can be calculated by stereo image processing, and the function of grasping and moving the object by controlling the operation of the hand can be realized by the computer.

  Since the recording medium of the present invention has the above-described configuration, it can carry an object recognition program for realizing a function of recognizing each object and grasping and moving it by a hand, and can be read by a computer.

It is a block diagram of a target object moving apparatus. Example 1 It is a block diagram of a target object specifying device. Example 1 (A) is image data of the camera, (B) is binarized image data of (A), (C) is image data of side highlight, and (D) is image data of end face highlight. . Example 1 It is explanatory drawing which shows the area of a side highlight, etc. Example 1 It is explanatory drawing which shows the curve of a side surface highlight. Example 1 It is explanatory drawing which shows rotation of a coordinate. Example 1 It is explanatory drawing which shows the curve direction of a side highlight. Example 1 It is explanatory drawing which shows the curve direction of a side highlight. Example 1 It is explanatory drawing which shows the detection of the end surface from side highlight information. Example 1 It is image data which shows the coupling | bonding of an end surface highlight. Example 1 It is image data which shows detection of final end face highlight. Example 1 It is image data which shows grouping of a side surface highlight, and detection of an end surface highlight. Example 1 It is image data which shows grouping of a side surface highlight, and detection of an end surface highlight. Example 1 It is a schematic diagram of a stereo image processing apparatus. Example 1 (A) shows the center of gravity of each side highlight and the center of gravity of the group, and (B) is an explanatory diagram showing the direction of the arrangement of the groups. Example 1 It is explanatory drawing which shows the group of the side highlight which displayed the x coordinate of the group gravity center in connection with the distance between group gravity centers on the right and left. Example 1 It is explanatory drawing which shows the group of the side highlight which displayed the group gravity center in relation to matching of a side highlight on right and left. Example 1 It is explanatory drawing which shows that the group of a left and right side surface highlight is a horizontal state. Example 1 It is explanatory drawing which shows the original image matching of the group of a left and right side surface highlight, and expansion of an area | region. Example 1 It is explanatory drawing which concerns on the definition of the height and attitude | position of a winding spring. Example 1 It is a bird's-eye view of the camera which shows a variable. Example 1 It is explanatory drawing which concerns on the attitude | position evaluation of a winding spring. Example 1 (A) is an explanatory view showing left and right original images, (B) is an explanatory view showing grouping in the left and right images, and (C) is an explanatory view showing correspondence between the left and right groups. Example 1 (A) is an enlarged explanatory view showing grouping in the left and right images, and (B) is an enlarged explanatory view showing correspondence between the left and right groups. Example 1 (A) is an enlarged explanatory view showing grouping in the left and right images, and (B) is an enlarged explanatory view showing correspondence between the left and right groups. Example 1 (A) is an enlarged explanatory view showing grouping in the left and right images, and (B) is an enlarged explanatory view showing correspondence between the left and right groups. Example 1 The picking of the winding spring by the approach from the vertical direction is shown, (A) is θ = 0, (B) is 0 <θ <90 °, and (C) is an explanatory diagram of θ = 90 °. Example 1 The winding spring picking by the approach according to the posture is shown, (A) is θ = 90 °, (B) is 0 ≦ θ <45 °, (C) is 45 ° ≦ θ <90 ° It is. Example 1 It is a flowchart which concerns on picking. Example 1 FIG. 5 is a flowchart illustrating a subroutine for recognizing a winding spring and relating to detection of side highlights and end highlights. Example 1 It is a flowchart which shows the subroutine of a winding spring recognition, and relates to temporary grouping of a side surface highlight. Example 1 FIG. 6 is a flowchart illustrating a grouping of side highlights, showing a subroutine for winding spring recognition. Example 1 FIG. 6 is a flowchart showing a correspondence between side highlight groups, showing a subroutine for detecting correspondence between side highlight groups. FIG. Example 1 FIG. 10 is a flowchart related to detection of correspondence between highlights, showing a subroutine for detection of correspondence between side highlight groups. FIG. Example 1 5 is a flowchart of a subroutine relating to image matching detection, showing a subroutine for detecting correspondence between side highlight groups. Example 1

  In order to recognize an object with a complicated outer shape such as a coil spring, a gap, and a deep view, and to grasp and move the object by a hand, the coil spring 3 in the coiled springs 3. The object moving device 1001 that moves by gripping with the hand 1011 and illuminating the entire winding spring 3... A highlight extraction processing unit 13 that extracts a highlight of each reflection shape for each image by performing a digitization process, and a feature amount extraction processing unit that extracts a feature amount corresponding to the shape of the winding spring 3 from each highlight 17, grouping processing units 19 and 21 for determining a plurality of highlights as a group for each set according to a feature amount for each image, a correspondence detection processing unit 25 for detecting correspondence between images in the group, and corresponding by detection A position / attitude calculation unit 27 for recognizing the winding spring 3 in a group and calculating the position and orientation of the winding spring 3 based on the positions of the cameras 1005 and 1007 by stereo image processing, and the calculated position of the winding spring 3 This is realized by including a robot controller 1009 that controls the operation of the hand 1011 according to the posture.

[Object moving device]
FIG. 1 is a block diagram of the object moving device. In the following description, the left is abbreviated as L and the right is abbreviated as R.

  The object moving device 1001 includes an object specifying device 1003, two left and right cameras (L) 1005, a camera (R) 1007, a robot controller 1009, and a robot / hand 1011.

  The object specifying device 1003 is an apparatus for specifying the object, holding it with a hand, and moving it, while objects of the same shape, which have a space portion in the shape and whose outlines are divided, are irregularly gathered.

  For example, a coil spring is applied as the object. A spiral spring has a complicated outer shape, a gap, and a shape that allows the back to be seen.There is a space portion in the side surface shape, and an object having the same shape that is divided in outline, that is, a ring-shaped end surface and a space portion. It becomes an example of the object of the same shape provided with the side surface which existed and the outline was divided. Each of the spiral springs is stacked in a tray and irregularly gathers, and the object identifying device 1003 recognizes each spiral spring and identifies the position and orientation in each of the spiral springs. The recognition of the winding spring is to detect where the individual winding springs exist from the two camera images. From the result of this recognition, the position and orientation are calculated by stereo image processing, the position of the picking target, Identify posture.

  In other words, the object specifying device 1003 is configured by a computer including a CPU, a ROM, a RAM, and the like, for example, and inputs images captured by the left and right cameras (L) 1005 and the camera (R) 1007, and Recognize and calculate the position and orientation to identify the picking target.

  The robot controller 1009 constitutes a control unit, controls the operation of the hand, that is, the robot / hand 1011 based on the position and orientation of the object calculated as described above, and holds the winding spring with a gripper as will be described later. And move it.

  Therefore, the left and right cameras (L) 1005 and the camera (R) 1007 capture images of the spiral springs stacked in the tray, and the object identifying device 1003 recognizes the spiral springs in the tray from the left and right images. As a result, the position and orientation are estimated, the robot / hand 1011 is controlled by the robot controller 1009, the winding spring is gripped by the gripper, and can be sequentially moved to a predetermined location.

  FIG. 2 is a block diagram of the object identification device.

  As shown in FIG. 2, the object specifying device 1003 specifies each winding spring 3 while the winding springs 3 are stacked in an irregular manner in the tray 5. The identification of the winding spring 3 is to detect where each winding spring 3 exists and in what posture from the camera image as described above.

  This object specifying device 1003 includes an object recognizing unit 1 that recognizes the wound springs 3. The recognition of the winding spring 3 is to detect where the individual winding springs 3 are present from the camera image.

  Illumination light is applied to the coil springs 3... In the tray 5 by, for example, an LED spotlight 7 as an illuminator, and the illumination light is applied to the entire bulk springs 3. Images are captured by the two cameras 1005 and 1007 in the applied state, and image data is input to the object recognition unit 1 and stored and held in the data storage unit 11. For convenience of illustration, two cameras 1005 and 1007 are shown as one block.

The object recognition unit 1 performs processing for each of the left and right cameras 1005 and 1007, and performs grouping processing and edge highlight detection on the left and right images. Here, the outline of the recognition method will be described. The shape of the coil spring is
・ There are few flat parts ・ Similar shapes are continuous ・ There are gaps and the back is visible, so the shape characteristics generally used in image processing are poor. Therefore, when the winding springs are three-dimensionally stacked, recognition by a general-purpose image processing method is difficult.

  On the other hand, in the method of the embodiment of the present invention, recognition is performed by paying attention to highlights (FIG. 3) generated in the winding spring by illumination. Usually, highlights are often avoided in image processing because they hide the original object. However, we succeeded in simplifying the recognition process by utilizing the fact that the winding springs can be regularly highlighted.

  In the recognition process, first, a highlight portion is extracted by binarizing the camera image. Side highlights are extracted when the winding spring is sleeping, and end highlights are mainly extracted when standing. Side highlights and edge highlights are divided according to area (number of pixels). Small areas or half-finished areas are ignored as highlights that are inappropriate for use in recognition (such as noise or a combination of multiple highlights). Subsequently, for the side highlights, those belonging to the same winding spring are grouped. If grouping is possible, it can be said that the spring has been recognized.

Highlights belonging to the same spring are
・ Distance is close. ・ Shapes are similar. ・ Since they are arranged in a straight line at regular intervals, grouping is performed using this. The proximity of the distance checks the distance between the highlight centroids, and the similarity of the shape uses a highlight area (number of pixels), the major axis direction of the image ellipse with respect to the highlight, and a three-dimensional image moment representing curvature information. Check that the lines are aligned at regular intervals by looking at the center of gravity of the highlight. If more than a certain number of highlights can be grouped in the same group, it is considered that one winding spring has been recognized.

  A sleeping spring that can detect only a few side highlights is a situation where a part of the spring is hidden under other winding springs and is not suitable for picking. Therefore, it may not be recognized at this time (recognized as the picking operation proceeds).

  If one end face highlight is detected, it can be said that one winding spring can be recognized. The end face has a substantially circular shape and can be regarded as an ellipse according to the inclination of the winding spring on the image. Therefore, an end face that is successfully fitted to the ellipse is detected. An example of the result of implementing and recognizing this method will be described later.

  Therefore, the object recognition unit 1 includes, in addition to the data storage unit 11, a highlight extraction processing unit 13, a highlight discrimination processing unit 15, a feature amount extraction processing unit 17, a temporary grouping processing unit 19, and a grouping. A processing unit 21, an end face highlight detection unit 22, and a registration unit 23 are provided. Note that the highlight determination processing unit 15 may be omitted if the object is other than the winding spring 3 and has a spherical shape or the like.

  The highlight extraction processing unit 13 binarizes the image to extract highlights of each reflection shape.

  That is, image data from the cameras 1005 and 1007 is read from the data storage unit 11 and binarized to extract highlights.

  The highlight discrimination processing unit 15 performs area threshold processing on the binarized image data from which highlights are extracted, and is a side highlight corresponding to the side of the winding spring 3 (hereinafter referred to as “side highlight”). The highlight of the end surface corresponding to the end surface of the winding spring 3 (hereinafter referred to as “end surface highlight”) is discriminated.

  That is, the binarized image data is read from the data storage unit 11, and the side surface highlight and the end surface highlight are extracted from the region obtained by the labeling based on the area threshold. The extracted side surface highlight and end surface highlight are stored in the data storage unit 11.

  The feature amount extraction processing unit 17 extracts a plurality of feature amounts to correspond to the side surface shape (the shape of the object) of the winding spring 3 from each side surface highlight (highlight).

  That is, the side highlights are sequentially read from the data storage unit 11, and the shape of the highlight, the linear arrangement of the highlights, the interval, and the number are extracted as feature amounts from each side highlight. The shape of the side highlight is, for example, the area of the side highlight, the major axis direction of the side highlight, and the curvature information of the side highlight.

  As described later, the curvature information is the size and direction of bending by fitting an ellipse, and an image moment can be introduced as described later in order to further improve accuracy. In the image moment, as will be described later, the curvature of the side highlight is the third moment, and the direction is the positive or negative of the third moment.

  The temporary grouping processing unit 19 performs a process of making side highlights into a temporary group for each set based on a specific feature amount selected from a plurality of feature amounts.

  That is, side highlights are sequentially read from the data storage unit 11, and processing is performed in which a plurality of side highlights are set as a temporary group for each set according to the proximity of the shape specified as the feature amount. The tentative grouping processing unit 19 performs a process of converting the shape into the area of the side highlight, the major axis direction of the side highlight, and the curvature information of the side highlight.

  In this case, the temporary grouping processing unit 19 selects one of the side highlights and lists nearby side highlights, and then performs the above-described temporary group processing.

  The grouping processing unit 21 performs a process of determining the side highlights set as temporary groups as groups based on other specific feature amounts selected from a plurality of feature amounts.

  That is, a temporary group is sequentially read from the data storage unit 11, and a process of determining the temporary group as a group based on the linear arrangement, interval, and number of side highlights as other feature values selected as the feature value is performed.

  In this case, the grouping processing unit 21 sequentially selects temporary groups and performs a process of determining a group for each selected temporary group.

  The end face highlight detection unit 22 determines the final end face highlight from the determined end face highlight. The end face highlight detection unit 22 according to the present embodiment separates a large highlight by applying a contraction process to a large area highlight excluded as noise and is regarded as a highlight of the end face, thereby separating each end face. The highlight of The final end face highlight is determined from the end face highlight as described later. Details will be described later.

  The determined side highlight group and end face highlight are registered in the registration unit 23.

  The object specifying device 1003 includes a correspondence detection processing unit 25 and a position / attitude calculation unit 27 in addition to the object recognition unit 1.

  The correspondence detection processing unit 25 detects the correspondence between the left and right images of the group, and the position / posture calculation unit 27 recognizes the winding springs 3... The position and orientation of the winding springs 3... With respect to the position are calculated by stereo image processing.

  The correspondence detection processing unit 25 of the present embodiment also detects the correspondence between the left and right images of the end face highlight, and the position / posture calculation unit 27 detects the winding springs 3... By the corresponding group and end face highlight by the detection. The position and orientation of the winding springs 3... With respect to the positions of the cameras 1005 and 1007 are calculated by stereo image processing.

  In this case, the correspondence detection processing unit 25 detects the correspondence of the group by any combination of highlight area, centroid position, alignment direction, distance between group centroids, number, and vertical position of the centroid as described later. .

  The correspondence detection processing unit 25 detects the correspondence between the groups by comparing the images captured by the cameras 1005 and 1007 when the orientation of the side highlight group is considered horizontal or horizontal. That is, returning to each original image captured by the cameras 1005 and 1007, the correspondence is detected by image matching. By using the original image, it is possible to obtain an appropriate correspondence relationship using information other than highlights and information on surrounding springs.

  Therefore, the correspondence detection processing unit 25 sets the search area in the original image captured by the cameras 1005 and 1007 including the group regarded as horizontal or horizontal to include the peripheral information.

In this way, it is possible to recognize each of the stacked springs 3... And to detect the position and orientation by stereo image processing, and to provide automated picking by the robot of the stacked springs 3. it can.
[Recognition of coil spring]
(Highlight extraction and discrimination)
3A is image data of the camera, FIG. 3B is binary image data of FIG. 3A, FIG. 3C is image data of side highlights, and FIG. 3D is image data of edge highlights. It is.

  For highlight extraction and discrimination, attention is paid to highlights that can be generated when the illumination light of the LED spotlight 7 strikes the winding spring 3. When binarization processing is performed on the image data of FIG. 3A captured by the cameras 1005 and 1007, highlights can be extracted as shown in FIG. 3B. The binarized image data of FIG. 3B is subjected to threshold processing with the area of the highlight, noise is removed, and the side surface highlight of FIG. 3C and the end surface highlight of FIG. Determine.

(Grouping by ellipse fitting)
The highlight grouping according to the embodiment of the present invention is based on an image moment described later, and the control flowchart is also a process based on the image moment. However, it does not exclude the grouping by the ellipse fitting, and the procedure is described first as the possibility of grouping although the accuracy is slightly lowered before the grouping by the image moment.

  FIG. 4 is an explanatory diagram showing the area of the side highlight, and FIG. 5 is an explanatory diagram showing the curvature of the side highlight.

  The side highlight is different from the end highlight, and it is difficult to obtain the information of the winding spring 3 from one highlight. Therefore, side highlight grouping is performed on the same winding spring 3.

  In this case, parameters (features) of each side highlight necessary for grouping are obtained. The calculation method will be described below.

-Highlight area, long axis direction, distance between highlights (Figure 4)
The area h _area [pixel] of the highlight and the long axis direction h _laxis [deg] can be calculated from the moment of the image. Also, the barycentric coordinates of the highlight can be calculated from the image moment. We define the distance between the center coordinates of the highlight highlight distance d _r [pixel] and.

・ Curve of highlight (Fig. 5)
To find the curvature of the highlight, extract the highlight skeleton. An ellipse is fitted to the point group constituting the extracted skeleton, and a point on the skeleton that is the shortest distance between the center of the fitted ellipse and the skeleton is obtained. The length of the straight line connecting the fitted ellipse center and the obtained point on the skeleton is defined as the curvature size c _mag [pixel], and the direction is the highlight curvature direction c _dir [deg].

  Next, threshold values are set for the following parameters, and side highlight grouping is performed.

-Highlight area: h _area [pixel]
_-Long axis direction of highlight: h _laxis [deg] (0 _≤ h _laxis <180 °)
・ Highlight curve size: c _mag [pixel]
-Direction of curvature of highlight: c _dir [deg] (0 _≤ c _dir <360 °)
・ Distance between highlights: _dr [pixel]
The procedure for highlight grouping is as follows.

(1) Select one highlight from all detected side highlights (reference highlight)
(A) When the curve of the reference highlight is large
i. Select a highlight with a large curvature other than the standard highlight.
ii. The highlight area h _area , the major axis direction h _laxis , the highlight distance d _r , and the curvature direction c _dir are compared and included in the reference highlight group if the conditions are satisfied.
iii. Perform (i) i and ii for all highlights.
(B) When the curvature of the reference highlight is small
i. Select a highlight with a small curvature other than the standard highlight.
ii. Highlights of the area h _area, longitudinal _h laxis, compares the highlight distance d _r, including satisfies the conditions a group of standards highlights.
iii. Perform (i) and (ii) of (b) for all highlights.
(2) Perform (1) by changing the reference highlight.
(3) When (1) is performed for all highlights, grouping is completed.

The only difference in the grouping procedure depending on the size of the curve is whether or not the parameter includes the curve direction c _dir . This is because highlights with a small curvature have completely different curvature directions even in the same group, which adversely affects grouping. In addition, the magnitude c _{mag of the} curvature is not directly added to the grouping parameter, and is divided into only two types of large and small. This is because the influence of noise is large on the size of the curvature, and if it is added to the grouping parameters, the grouping tends to fail.

  By the procedure so far, it is set as a temporary group.

  However, with the above procedure alone, grouping failure occurs when overlapping groups or winding springs are arranged in parallel.

  Therefore, in the same group, the highlights should be arranged in a straight line, the distance between adjacent highlights should be equal, and when the groups that represent the same winding spring overlap, the number of highlights included in the group should be adopted. The final result of the grouping is corrected by adding the above condition.

(Curve information and grouping by image moment)
FIG. 6 is an explanatory diagram showing the rotation of coordinates, FIG. 7 is an explanatory diagram showing the bending direction of the side highlight, and FIG. 8 is an explanatory diagram showing the bending direction of the side highlight.

  In the above description of FIGS. 4 and 5, an ellipse is applied to the skeleton of each side highlight, and the size and direction of the curve are obtained from the ellipse. However, when the size of the curve is small, the accuracy of the curve information by ellipse fitting is limited.

  On the other hand, accuracy is improved by obtaining curvature information using an image moment.

First, the magnitude of the curvature is set as a third moment μ ′ ₂₁ . If the coordinate obtained by rotating the xy coordinate θ counterclockwise as shown in FIG. 6 is defined as the x′y ′ coordinate, the value of μ ′ ₂₁ can be obtained by Expression (1). However, (theta) is represented by Formula (2).

  Where μ is the (i + j) th moment of the image in the original coordinate system.

  If it remains as it is, it will be affected by the enlargement / reduction of the image, so that it is made dimensionless as shown in equation (3).

  Next, since the shape of the side highlight of the winding spring 3 is usually curved, the direction is discriminated. Since the longitudinal direction of the highlight can be determined as the major axis direction of the image ellipse, there is a problem as to which side of the minor axis direction that is orthogonal thereto is curved. When the coordinate axis is rotated by θ, the major axis direction becomes the x ′ axis and the minor axis direction becomes the y ′ axis.

  Assuming a crescent shape as a typical shape of the side highlight, it is substantially symmetric in the x′-axis direction and is curved in either of the y′-axis directions.

  • Near both ends of the crescent moon, x´ takes both positive and negative values and the absolute value is large. Y ′ takes one of positive and negative values, and this sign corresponds to the direction of curvature.

  • Near the center of the crescent moon, x´ takes both positive and negative values and the absolute value is small. Y ′ takes one of positive and negative values, and has a different sign from the vicinity of both ends.

Here, if μ ^′ ₂₁ corresponding to the integration of x ′ ² y ^′ is calculated, y ′ is integrated with the weight of x ′ ² , so the sign near both ends is heavily weighted, and as a result You can see the direction of the curve. That is,
• When μ ^′ ₂₁ <0: Curved with both ends protruding in the negative direction of the y ′ axis (FIG. 7)
• When μ ^′ ₂₁ > 0: Curved with both ends protruding in the positive direction of the y ′ axis (FIG. 8)
It becomes.

  As described above, the direction of highlight curvature is used as a grouping parameter regardless of the magnitude of highlight curvature. That is, the highlight grouping algorithm is as follows.

(1) Select one highlight from all detected side highlights (reference highlight)
(A) Select a highlight other than the reference highlight.
(B) _Compare the highlight area h _area , the major axis direction h _laxis , the highlight distance d _r , the size of the curve, and the direction of the curve, and if it satisfies the condition, it is included in the reference highlight group.
(C) Perform (a) and (b) for all highlights.
(2) Perform (1) by changing the reference highlight.
(3) When (1) is performed for all highlights, grouping is completed.

  Since the image moment is obtained as a sum of information of the entire image, a robust grouping can be performed.

(Detection of end face from side highlight information)
FIG. 9 is an explanatory diagram showing detection of end faces from side highlight information.

When the coil spring is tilted, the side surface and the end surface can be seen, so that the end surface can be further detected from the side surface information. Side highlights have the characteristic that they do not bend horizontally, but become larger as the tilt approaches vertical. For this reason, there is a possibility that the end face of the grouped spring including the highlight that is greatly curved is visible. The position of the end face is estimated based on the group direction g _dir and the bending direction c _dir , ROI (Region of Interest) is set, and the end face is detected and determined by RANSAC (RANdom SAmple Consensus) [Fischler 1981].

  Assuming that the shape of the end surface is an ellipse, the shape of the end surface is distorted when the winding spring is inclined. Therefore, as shown in FIG. 9, the major axis and minor axis directions of the end face can be estimated to some extent from the direction of the winding spring.

That is, a constraint of ± Δφ can be added to the range of the orientation of the minor axis of the ellipse obtained by RANSAC with respect to g _dir . The end face detection algorithm by RANSAC is shown below.
(1) Binarize and detect edges by the Canny method.
(2) Randomly select five points from the edge and obtain an ellipse formula that passes through the five points.
(3) Check if the direction of the minor axis of the ellipse is within the range of [g _dir −Δφ, g _dir + Δφ].
(4) If (3) is satisfied, the distance to the ellipse obtained for all the points other than the selected five points is obtained, the number of points whose distance is smaller than the set threshold is counted, and the total is obtained as the score of the ellipse. And
(5) Repeat (2) to (4) several times to detect a plurality of ellipses.
(6) The ellipse with the highest score is adopted as the end face shape from the obtained plurality of ellipses.

(Recognition by edge highlight)
Focusing on the vicinity of the edge highlight, set the ROI and detect the edge. When the winding spring 3 is in an upright state, the shape of the end face is not distorted, and thus no restriction is imposed on the detection ellipse.

  Contour extraction is performed in the set ROI, and end faces are detected by performing ellipse fitting. When there is a lot of noise in the image, it is also possible to detect using the same RANSAC as described above.

  In the above, ellipses were derived by selecting the coordinates of five points from the edge. The direction of the group of winding springs and the direction of the short axis of the detection ellipse should be equal, but when the coordinates of 5 points are used, an ellipse with an incorrect orientation is also derived. For this reason, the RANSAC calculation is useless.

Therefore, since the orientation of the ellipse is known from the orientation of the side highlight group, the ellipse may be derived from the coordinates and angles of the four points. That is, the end face detection algorithm by RANSAC is as follows.
(1) Binarize and detect edges by the Canny method.
(2) Four points are selected at random from the edge, and an ellipse formula is obtained that passes through the four points and the direction of the ellipse is the group direction.
(3) The distance to the ellipse obtained for all the points other than the four selected points is obtained, the number of points having a distance smaller than the set threshold is counted, and the total is taken as the score of the ellipse.
(4) Repeat (2) to (3) several times to detect a plurality of ellipses.
(5) The ellipse with the highest score is adopted as the end face shape from the obtained plurality of ellipses.

  As a result, the number of ellipse detections by RANSAC can be increased in the same calculation time, and the success rate of ellipse detection is improved.

(Recognition when upright coil springs are lined up)
FIG. 10 shows image data indicating end face highlight combination, and FIG. 11 shows image data showing final end face highlight detection.

  As shown in FIG. 10, when the upright winding springs are arranged, the area of the end face highlight becomes large, and is regarded as noise in the highlight extraction in the description of [Highlight Extraction and Discrimination] above.

Therefore, when the highlight area is large, the highlight is separated by a shrinking process. The algorithm is shown below.
(1) Apply shrinkage to highlights.
(2) Check if the highlight area falls within the range considered as edge highlight.
(3) Repeat (1) and (2).
(4) If (2) is satisfied, it is considered that the highlight has been cut off.
(5) The separated highlights are used as separate end face highlights.

  In joining edge highlights in FIG. 10, the highlights are connected and the highlight area becomes large, so that noise is generated.

  On the other hand, when the separation is performed by the above algorithm, it can be recognized that the springs are separate even when the highlight area is large as shown in FIG.

(Side highlight grouping and edge highlight detection)
12 and 13 are image data showing grouping of side highlights and detection of end face highlights. FIG. 12 shows image data based on an edge highlight recognition algorithm that does not perform side highlight recognition and highlight separation by ellipse fitting, and FIG. 13 shows side highlight recognition using curvature information based on image moments. It is the image data by the recognition algorithm of the end face highlight which performed highlight separation.

In both cases of FIGS. 12 and 13, the portion surrounded by the square in the image can be recognized. In particular, in the case of FIG. 13, the groupable number is increased as compared with the case of FIG. 12. The success rate of grouping and the success rate of edge detection are slightly improved.
[Estimation of position and orientation of the coil spring]
(Outline of position / posture detection)
After picking up a particular coil spring, it is necessary to know its position and orientation in order to pick it. When a monocular camera is used, it is not easy to know depth information. For this reason, three-dimensional position / posture estimation is performed here using stereo image processing. In general stereo image processing, it is normal to perform corresponding point detection processing on the entire two camera images to acquire three-dimensional information, but there is a problem of miscorrespondence or calculation load. Especially in the case of a coil spring, it is difficult because the shape characteristics are not clear.

  Therefore, in the embodiment of the present invention, the above-described winding spring recognition method is applied to each of the images of the two cameras, and then stereo image processing is performed only for the recognized winding spring.

  This makes it possible to obtain accurate position / posture information with simple processing. In this method, two cameras are arranged in parallel and close to each other in order to increase the overlap of the field of view and facilitate the left and right correspondence processing. Then, the above-described winding spring recognition method is executed in each of the right camera and the left camera, and side highlight groups and end face highlights corresponding to the winding springs (hereinafter also referred to as “group” for convenience). Is detected. The group corresponding to the same coil spring is at the same position in the image of the right camera and the image of the left camera. More specifically, it should be in the same position in the vertical direction and slightly different in the horizontal direction (depending on the depth).

  Therefore, if a group recognized by the image of one camera is searched within a limited range of the image of the other camera, a corresponding group can be found.

  Then, depth information can be obtained from the left and right parallaxes. In the search, a corresponding group is found on the condition that the shapes of the groups are substantially the same and the positional relationship of the groups is appropriate. The shape is based on the average area of highlights (number of pixels), the number of highlights, and the direction of highlight alignment. The positional relationship is such that the group center of gravity position is within a certain distance in the left-right direction, and the vertical direction is substantially the same.

  However, in the above method, if the recognition results of the highlights are different on the left and right (for example, for the same winding spring, the left camera recognized 4 highlights, but the right camera recognized only 3 If you did not).

  Therefore, when the correspondence in the group unit cannot be taken well, the left and right correspondences are examined for each highlight unit in the group, and it is examined whether or not the group having only the highlight whose correspondence has been confirmed can be dealt with again.

  Individual highlights are associated with each other by checking that the position of the center of gravity is within a certain distance in the horizontal direction and that the vertical direction is substantially the same.

  In addition, when the winding spring is completely oriented in the left-right direction, a plurality of candidates for highlights are conceivable, and it cannot be narrowed down to only one from the highlights alone.

  Therefore, in this case, an area larger than the highlight group is considered in the original image on one camera side. Then, matching is performed with the original image on the other camera side, and the corresponding positional relationship is determined using this. By using the original image, it is possible to narrow down the correspondence between the left and right based on the surrounding information as well as the highlight.

  An example of the result of obtaining corresponding winding springs from the left and right camera images using the above method will be described later.

  If the highlight group can be associated between the left and right camera images, the depth of the group centroid of the highlight can be calculated using the parallax of the group centroid. Thus, the position of the winding spring can be obtained. The posture of the winding spring (inclination in the depth direction) can be obtained from the relationship between the interval between highlights and the pitch of the winding spring with respect to the side highlight group. The end face highlight can be obtained from the ratio of the major axis to the minor axis of the fitted ellipse.

  Note that the highlights are not exactly the same in the left and right camera images. This is because the relative positional relationship between the illumination, the camera, and the winding spring is different. For this reason, the accuracy of depth calculation is currently improved by using an empirical formula for correcting parallax. It is also possible to theoretically calculate the correction formula from the shape information of the coil spring and the information of the light source of illumination.

  This will be specifically described below.

(Depth estimation)
The theoretical formula of depth by stereo vision is derived.

  FIG. 14 is a schematic diagram of a stereo image processing apparatus. Symbols are defined as follows:

・ O _{l ,,} Or: Projection center ・ x _l, xr: Origin in x direction of image plane ・ c _x ^left, c _x ^right : Principal point (point where optical axis intersects image plane)
• P: Real world points • x _left, x _right : Horizontal coordinates on the P image plane • f: Focal length • T: Baseline length • d _p : Parallax • Z: Depth • Ps: Pixel size Stereo device is distorted Assuming that the correction is made, the rows are aligned, and the measurement is accurately performed, the depth Z is obtained by Equation (4). However, d _p = x _left −x _right .

Further, the depth resolution ΔZ is obtained by the equation (5) using the parallax error Δd _p .

(Calculation of parallax)
<Parallax calculation position>
In order to calculate the depth by stereo vision, it is necessary to obtain parallax. In the embodiment of the present invention, the parallax calculation position is the highlight centroid of the group of winding springs or the ellipse center obtained by ellipse fitting. Therefore, it is not necessary to detect the end face by RANSAC from the side highlight information when recognizing the winding spring. Winding springs with visible side highlights only perform highlight grouping, and winding springs with only visible end highlights fit an ellipse to the end face highlights.

  Further, in order to obtain the parallax, it is necessary to associate the left and right images. In general stereo vision, the entire image is associated, but the winding spring does not have a clear feature generally used in image processing. Therefore, in this embodiment, association is performed for each group of highlights of the winding spring. In performing the association, it is preferable that the area where the fields of view of the left and right cameras overlap is wider.

  Moreover, it is necessary that the same winding spring looks the same in the left and right cameras. Therefore, it is assumed that the two cameras have short base lengths, the optical axes of the left and right cameras are parallel, and the respective optical axes are perpendicular to the reference plane.

<Parameters of the coil spring group>
A description will be given of a method of calculating parameters used for associating groups of coil springs.

Group centroid, average highlight area, number of highlights, group orientation FIG. 15A shows the centroid of each side highlight and the centroid of the group, and FIG. 15B is an explanatory diagram showing the direction of group alignment. It is.

  In FIG. 15, the center-of-gravity coordinates and area of each highlight are obtained from the image moment. Therefore, the group centroid can be calculated by the average value of the centroid coordinates of the highlights included in the group, and the average highlight area can be calculated by the average value of the areas of the highlights included in the group. The number of highlights included in a group is determined at the time of grouping. The orientation of the group is obtained by fitting a straight line to the center of gravity of each highlight.

・ Distance between group centers of gravity (Fig. 16)
FIG. 16 is an explanatory diagram showing the side highlight group on the left and right, which displays the x coordinate of the group centroid, in relation to the distance between the group centroids.

In FIG. 16, the group centroid x coordinate of the left image is x _lg , and the group centroid x coordinate of the corresponding right image is x _rg . If the left and right images are parallelized, the coordinate value x _tlg in the right image of the group centroid x coordinate of the left image is obtained by Expression (6). If Z is known, x _tlg = x _rg . However, since Z is unknown, using the median of the range of values that Z can take causes a _shift between x _tlg and x _rg . This shift is defined as a distance g _dis between group centroids, and is expressed by Expression (7).

<Method for associating groups of coil springs>
The coil spring group association algorithm is shown below.
(1) The left and right images are parallelized and the y coordinates are aligned.
(2) Group the left and right images.
(3) Select one from the group of winding springs recognized in the left image (left group)
(A) The group barycentric coordinates of the left group are converted into coordinates in the right image.
(B) Select one from the group of winding springs recognized in the right image (right group)
i. Calculate the distance g _dis between the group centroids of the left group and the right group.
(C) Perform (b) on all the groups of the winding springs recognized in the right image.
(D) Check the following conditions for the left group and all right groups.
Condition 1-1 Group average highlight area is almost equal Condition 1-2 Group center-of-gravity distance g _dis <Threshold Condition 1-3 Group direction is almost equal (e) Right group that satisfies all conditions of (d) Candidate for action (candidate right group)
(F) When there is a candidate right group
i. Check the following conditions for the left group and the candidate right group.
Condition 2-1 Group has the same number of highlights Condition 2-2 Group centroid y-coordinate is almost equal
The left group that satisfies all the conditions of ii.i. corresponds to the candidate right group.
(G) If there is no candidate right group, no response is made.
(4) Perform (3) for all the groups of the springs recognized in the left image.

  For winding springs in which only the end face highlight is visible, association is performed only under conditions 1-2 and 2-2.

<Association method for each highlight in the group>
The algorithm described in the method of associating the group of coil springs can be used only when the left and right recognitions are the same. That is, when the left and right recognitions are different, the above conditions 2-1 and 2-2 are not satisfied, so that it is not possible to take a countermeasure.

  Therefore, when the condition 2 is not satisfied, a countermeasure is taken for each highlight as shown in FIG. FIG. 17 is an explanatory diagram showing a group of side highlights displaying a group center of gravity on the left and right sides in association with side highlights.

The correspondence algorithm for each highlight is shown below.
(1) Select one of the highlights in the left group (left highlight).
(A) The highlight barycentric coordinates of the left highlight are converted into coordinates in the right image.
(B) Select one of the highlights included in the candidate right group (right highlight).
i. Calculate the distance between the highlight centroids of the left and right highlights.
(C) Perform (b) on all highlights included in the candidate right group.
(D) Check the following conditions for left highlight and all right highlights.
Condition 3-1 Highlight center-of-gravity distance <threshold Condition 3-2 Highlight center-of-gravity y coordinate is almost equal (e) Left highlight and right highlight that satisfy all the conditions of (d) correspond to each other.
(2) Perform (1) for all highlights included in the left group.
(3) Only the associated highlights are included in the left and right groups.

  The highlight center-of-gravity distance is defined in the same manner as the group center-of-gravity distance described in the parameters of the group of the winding spring. That is, the deviation of the highlight centroid x coordinate of the right image corresponding to the coordinate value in the right image of the highlight centroid x coordinate of the left image.

  FIG. 18 is an explanatory diagram showing that the left and right side highlight groups are in a horizontal state. As shown in FIG. 18, when the winding spring is completely oriented in the left-right direction, a plurality of candidates for highlight can be considered, and it cannot be narrowed down to only one from the highlight alone.

  Therefore, in this case, the original image on one camera side is processed as shown in FIG. FIG. 19 is an explanatory diagram showing original image matching and area enlargement of a group of left and right side highlights. As shown in FIG. 19, in the original image on the right side with respect to the original image on the left side, an area that is slightly larger than the highlight group is considered and the area is enlarged.

  Then, matching is performed between one original image (right-side original image) and the original image of the other camera (left-side original image), and the corresponding positional relationship is determined using this. By using the original image, it is possible to narrow down the correspondence between the left and right based on the surrounding information as well as the highlight.

(Wind spring position / posture estimation)
<Definition of height and posture>
FIG. 20 is an explanatory diagram relating to the definition of the height and posture of the winding spring.

  In FIG. 20, the height Hspr [mm] and the posture (θ [deg], φ [deg]) of the winding spring are defined as follows.

・ H _spr [mm]: Distance between group center of gravity and reference plane ・ θ [deg] (0 _≦ θ _≦ 90 °): Angle formed by a straight line projected from the center line and the center line of the spring to the reference plane ・ φ [deg ] (−90 ° _≦ θ _≦ 90 °): Angle between the straight line projected from the center line of the coil spring and the x axis <Position estimation>
· Placing the distance to the position estimation camera and the reference plane in the vertical direction and L _0. Since the distance from the camera to the subject is Z derived by Equation (4), the distance L ′ between the reference plane and the subject is represented by the following equation.

  This is the theoretical formula for the position in the vertical direction.

-Position estimation in the horizontal direction FIG. 21 is a bird's-eye view of a camera showing variables.

As shown in FIG. 21, the variables to be defined are as follows. The distance from the camera to the subject is L, the width and height of the angle of view are (W, H), the focal length of the lens is f, the CCD size is (W´, V´), and the image resolution is w × h. If it puts, the conversion formula of a real distance and a pixel value can be represented by Formula (10), and the relationship between actual length _Dreal and the pixel value _Dpixel in an image is represented by Formula (11). Here, D _real needs to exist on a plane perpendicular to the optical axis L away from the camera.

Let Ap = (xp, yp) be a point on the image. Since the intersection of the optical axis and the reference plane of the camera as shown in FIG. 21 the origin of the world coordinate? W, consider a coordinate system Shiguma' _p moved the origin of the image coordinate system Σp to the center of the image accordingly. A´ _p which is A _p in this coordinate system is expressed by the following equation.

When obtaining the ^A'' _p projected on the imaging plane (CCD sensor) by the following equation.

  As shown in FIG. 21, a square pyramid with a height f created by a CCD sensor and a square pyramid with a height L created by a plane L away from the camera have a similar relationship, so the following equation is obtained.

This is a theoretical formula for the horizontal position (x _w , y _w coordinates) of the subject.

<Posture estimation>
FIG. 22 is an explanatory diagram relating to the posture evaluation of the winding spring.

The inclination of the winding spring is estimated by using the distance between highlights. When the pitch of the coil spring is set to p and the number of group highlights is set to N _h , the actual distance d _real of the group length D _{plane in} the image is obtained by the following equation.

The relationship between the D _plane and the actual distance d _plane of the group length is expressed by the following equation using the distance Z from the camera to the subject, the focal length f, and the pixel size P _s .

  That is, from FIG. 22, the inclination θ of the winding spring can be expressed by the following equation.

[Recognition / position / posture estimation results]
23A is an explanatory diagram showing left and right original images, FIG. 23B is an explanatory diagram showing grouping in the left and right images, FIG. 23C is an explanatory diagram showing correspondence between the left and right groups, and FIG. Is an enlarged explanatory diagram showing grouping in the left and right images, (B) is an enlarged explanatory diagram showing the correspondence between the left and right groups, FIG. 25 (A) is an enlarged explanatory diagram showing grouping in the left and right images, and (B) is FIG. 26A is an enlarged explanatory view showing grouping in the left and right images, and FIG. 26B is an enlarged explanatory view showing correspondence between the left and right groups.

  FIG. 23 shows the results, and FIGS. 24 to 26 are enlarged with attention to some of the winding springs of FIG. The corresponding springs are surrounded by the same square on the left and right. A point is displayed at the group center of gravity.

As shown in the images of FIGS. 23 and 24, when the left and right recognitions are the same in the group of the end face highlight and the side face highlight, the correct correspondence can be obtained. Further, even when a plurality of groups are detected in the same winding spring or when the left and right recognitions are different, correct correspondence can be obtained as shown in FIGS.
[Winding spring picking]
FIG. 27 shows winding spring picking by approach from the vertical direction, (A) is θ = 0, (B) is 0 <θ <90 °, and (C) is θ = 90 °. 28A and 28B show the picking of the winding spring by an approach according to the posture, where FIG. 28A is θ = 90 ° (B) is 0 ≦ θ <45 °, and (C) is 45 ° ≦ θ <90. FIG.

If the position and posture of each coil spring 3 are obtained, picking can be performed by the robot 1011 (FIG. 1) based on the position and posture. The problem here is
-Which one to pick?-How to pick.

  Regarding the former, it is considered to be advantageous to pick the winding spring 3 that is basically “highest” (“close to the depth direction when viewed from the camera”). In consideration of obstacles, this may not always be the case, but at present, the highest spring spring 3 is picked.

  However, when picking fails, the same winding spring 3 is set not to go picking a certain number of times. This is because if there is something that is difficult to pick for some reason, the work may not proceed at all if you continue to try it.

  Regarding the latter “how to pick”, the outer diameter could be grasped by the hand 1011 using the two claws 1011a and 1011b which are grippers.

In addition to the simple approach approaching from the vertical direction in FIG. 27, it was possible to perform an implementation in which the approach direction was shifted according to the attitude of the winding spring 3 as shown in FIG. 28.
[Object Identification Method, Program, and Recording Medium]
FIG. 29 is a flowchart relating to picking.

  When the flowchart of FIG. 29 is executed by the operation of the object moving device 1001, in steps S1L and S1R (hereinafter, each step S is abbreviated as “S”), “image acquisition (L), (R)” is performed. ”Is executed, the original image data captured by the left and right cameras (L) 1005 and (R) 1007 and stored in the data storage unit 11 is read, and the process proceeds to S2 (L) and S2 (R).

  In S2L and S2R, the processing of “coil spring recognition (L)” and “coil spring recognition (R)”, respectively, identifies side highlights and end surface highlights based on the left and right original images, and groups side highlights. Etc. are performed on each of the left and right sides to recognize the winding spring. More specifically, the recognition result is registered in the registration unit 23 by recognizing the winding spring in each of the left and right images, and the process proceeds to S3.

  In S3, one is selected from the group of winding springs recognized in the left image registered in the registration unit 23 by the process of “select one L spring”, and the process proceeds to S4. This selection of the group is based on either the left or the right, and a setting for selecting the right R group can also be made.

  In S4, it is determined whether the item selected in S3 is a side highlight group or an end face highlight by a determination process of “is it a side highlight group?”. This determination can be made based on whether the highlight is plural or single. When it is determined that the group is a side highlight group (YES), the process proceeds to S5, and when it is determined that the face is highlighted (NO), the process proceeds to S6.

  In S5, the correspondence between the right R side highlight groups with respect to the selected left L side highlight group is detected by the processing of “side highlight group correspondence detection”. Specifically, it is executed by the subroutine of FIG. 33, and the correspondence groups of the left and right images are extracted as in <Method of associating groups of spring springs>, and the process proceeds to S7.

  In S6, the correspondence between the end surface highlights of the left and right images is detected by the processing of “end surface highlight correspondence detection”, and the process proceeds to S7. Edge highlights are contrasted as ellipses and circles.

  In S7, whether or not a corresponding group is found in R for one group selected in L by the detection of correspondence between groups in S5 by the determination process of “corresponding coil spring found in R?”. Is judged. If a corresponding group is found (YES), the process proceeds to S8, and if not found (NO), the process proceeds to S9.

  In S8, when the specific winding spring is recognized by the processing of S5 by the processing of “position / posture calculation”, the position / posture of the winding spring is determined by the above-described method of [estimation of position and orientation of the winding spring]. The calculation result is registered in the registration unit 23, and the process proceeds to S9.

  In S9, a determination process of "Is all the springs of L being checked?" Is performed. If it is determined that the check has not been completed (NO), the process returns to S3, and S3 to S9 are repeated. If it is determined that the examination has been completed (YES), the process proceeds to S10.

  In S10, the process of “determination of the picking target winding spring” determines the winding spring to be picked from the registered position and orientation from the registration unit 23, and the process proceeds to S11.

  In S11, by the “picking” process, the winding springs are sequentially picked by the hand 1011 by the method of “picking a winding spring”, moved to a predetermined location, and the process ends.

(Coil spring recognition)
30 to 32 execute S2L and S2R of FIG. 29, FIG. 30 shows a subroutine for winding spring recognition, a flowchart relating to detection of side highlight and end face highlight, and FIG. FIG. 32 is a flowchart relating to grouping of side highlights. FIG. 32 is a flowchart relating to temporary grouping of side highlights.

  In step S21, binarization processing is executed. In this process, the image data from the cameras 1005 and 1007 is converted into binarized image data by the binarization process, and the process proceeds to S22.

  In S22, side highlights and edge highlights are extracted according to area from the area obtained by labeling in the binarized image data by highlight extraction processing, and the process proceeds to S23 and S24. In highlight extraction, the feature amount is extracted from each side highlight and stored.

  In S23, temporary grouping is performed for all side highlights by the process of “temporary grouping of side highlights”, and the process proceeds to S25. The temporary grouping in S23 is executed by the subroutine of FIG.

  In S24, “end face highlight detection” is performed. The detection of the end face highlight in S24 is as described above.

  In S25, the side highlights are grouped by the “side highlight grouping” process, and the process ends. The grouping in S25 will be described with reference to the subroutine of FIG.

  In the subroutine of FIG. 31 for executing S23 of FIG. 30, one of the side highlights is read out by the process of “select one highlight” in S2301, and the process proceeds to S2302.

  In S2302, the list of neighboring highlights is listed in order according to thresholds such as the distance between highlights and the number of continuations for one side highlight selected in S2301 by the process of “listing neighboring highlights”. The process proceeds to S2303. In this list-up, one side highlight is read as a list for each side highlight.

  In S2303, it is determined based on whether or not the shape of the side highlight is close to the group of side highlights listed by the determination processing of “close shape?”. Whether or not “the shape is close” is determined by whether or not the highlight area (number of pixels), the major axis direction of the highlight, and the curvature information (third image moment) are close. If it is determined that the shape is close (YES), the process proceeds to S2304, and if it is determined that the shape is not close (NO), the process proceeds to S2305.

  In S2304, side highlights having similar shapes are stored as temporary groups by the process of “put into temporary group”, and the process proceeds to S2305.

  In S2305, a determination process of “has all the highlights in the vicinity been checked?” Is performed. If it is determined that the check has not been completed (NO), the process returns to S2302, and S2302, S2303, and S2305 are repeated. . If it is determined that the examination has been completed (YES), the process proceeds to S2306.

  In S2306, a determination process of “has all the highlights been checked?” Is performed. If it is determined that the check has not been completed (NO), the process returns to S2301, and S2301, S2302, S2303, S2305, and S2306 are executed. Repeated. If it is determined that the examination has been completed (YES), the provisional grouping process ends.

  In the subroutine of FIG. 32 for executing S25 of FIG. 30, one of the side highlight temporary groups temporarily grouped in S23 is read out by the process of “select one temporary group” in S2501, and the process proceeds to S2502. To do.

  In S2502, it is confirmed whether or not the configuration highlight is on the same straight line by the side highlights constituting the temporary group selected in S2501 by the process of “delete configuration highlight not aligned in a straight line”. Those not on the board are deleted, and the process proceeds to S2503. The side highlight of the winding spring 3 is determined using the property of being on the substantially same straight line.

  In S2503, a determination process of “There are few configuration highlights?” Is performed. As a result of the deletion in S2502, it is determined whether or not the temporary group is considered to be small in number to constitute the side highlight of the winding spring 3. If it is determined that the configuration highlight is small (YES), the process proceeds to S2504. If it is determined that the configuration highlight is not small (NO), the process proceeds to S2505.

  In S2504, the temporary group that is considered to be small in number to form the side highlight is deleted as data by the process of “delete temporary group”, and the process proceeds to S2505.

  In S2505, a determination process of “has all the temporary groups been checked?” Is performed. If it is determined that the check has not been completed (NO), the process returns to S2501, and S2501 to S2505 are repeated. If it is determined that the examination has been completed (YES), the flow proceeds to S2506.

  In S2506, the processing of “sort in descending order of the number of constituent highlights (in order of increasing highlight area if they are the same)” is performed. By this processing, a predetermined number of temporary group groups arranged on a straight line are arranged in descending order of the number of constituent highlights, and the process proceeds to S2507. However, when the number of configuration highlights is the same, the highlight areas are arranged in descending order, and the process proceeds to S2507.

  In S2507, one of the temporary groups sorted in S2505 is sequentially read out by the process of “select one temporary group in order”, and the process proceeds to S2508.

  In S2508, the configuration highlight that overlaps as data with respect to the side highlight that has already been determined as a group is deleted from the temporary group of the current processing by the process of “erasing configuration highlights that overlap with other registered groups”, and S2509. Move to F.

  In step S2509, a process of “erasing a configuration highlight whose distance between highlights is not constant” is performed. Since the distance between the strands is constant on the side surface of the winding spring 3, highlights whose distance is not constant are removed as noise, and the process proceeds to S2510.

  In S2510, a determination process of “the number of configuration highlights is within the reference range?” Is performed. The number of continuous wires on the side surface of the winding spring 3 is determined, and when the number of side surface highlights constituting the temporary group exceeds or is less than the standard, it is excluded from registration as a group. Therefore, if it is determined in S2510 that the number of configuration highlights is within the reference range (YES), the process proceeds to S2511. If it is determined that the number is not within the reference range (NO), the process proceeds to S2512.

  In step S2511, the processed temporary group is determined as a group by the “register as group” process, is registered in the registration unit 23 (FIG. 2), and the process proceeds to step S2512.

  In S2512, a determination process of “has all the temporary groups been checked?” Is performed. If it is determined that the checking has not been completed (NO), the process returns to S2507 and S2507 to S2512 are repeated. If it is determined that the examination has been completed (YES), the grouping process ends.

(Detection of correspondence between side highlight groups)
FIG. 33, which executes S5 of FIG. 29, shows a subroutine for detecting correspondence between side highlight groups, and is a flowchart relating to correspondence between left and right groups.

In S51, one group of L selected in S3 is registered in the registration unit 23 by the process of “search for the group of R having the same average highlight area and the closest distance between the centers of gravity”. One group is selected from the R groups. In this selection, the average values of the areas of the side highlights of the group are substantially the same, and the distance between the center of gravity g _{dis, which} is the difference between x _tlg and x _rg in FIG. By this selection, the process proceeds to S52 and S53 sequentially, and further narrowing down is performed.

In S <b> 52, a determination process of “the highlights are arranged in substantially the same direction and the distance between the group centroids is equal to or smaller than a threshold value?” Is performed. As shown in FIG. 15, when it is determined that the direction of the side highlight between the LR groups is substantially the same and the distance g _dis between the group centroids is equal to or less than the threshold (YES), the process proceeds to S53. When it is determined that the direction of the side highlight between the LR groups is not the same and the distance g _dis between the group centroids exceeds the threshold value (NO), the process ends, and the process proceeds to S7 in FIG. To do.

  In S53, a determination process of “the number of highlights is equal and the vertical position of the center of gravity is substantially the same?” Is performed. When it is determined that the number of side highlight highlights between the LR groups is equal and the vertical positions of the group centroids are substantially the same (YES), the flow shifts to S54, where the side highlight highlights between the LR groups are high. When it is determined that the number of lights is different and the vertical positions of the group centroids are not the same (NO), the process proceeds to S55.

  In S54, the “recognized as a corresponding group” process registers the corresponding R group as one corresponding to one of the L groups in the registration unit 23, the process ends, and the process proceeds to S7 in FIG.

  In S55, a determination process of “Is the direction of highlight alignment almost horizontal?” Is performed. When it is determined that the direction of the side highlight of the group of LR to be compared is not horizontal as shown in FIG. 18 (NO), the process proceeds to S56, and when it is determined to be horizontal (YES), to S57. Transition.

  In S56, the correspondence of the side highlight between the LR groups is detected by the “detection between highlights” process. Specifically, it is executed by the subroutine of FIG. 34, and correspondence is taken for each side highlight between LR groups as in the above <association method for each highlight in group>. Go to 29 S7.

  In S57, an “image matching correspondence detection” process is performed, and the process proceeds to S7 in FIG. S57 is executed by the subroutine of FIG.

(Detection between highlights)
FIG. 34 shows a subroutine for detecting correspondence between side highlight groups, and is a flowchart relating to detection of correspondence between highlights. If the recognition of the LR group is different, no action can be taken. Therefore, as shown in FIG.

  In S5601, one L side highlight is read from the registration unit 23 by the process of “select one L highlight”, and the process proceeds to S5602.

  In S5602, the process of “search for R highlights having the same vertical position and the closest distance between the centroids” is performed on one side highlight of L as shown in FIG. Lights having the same vertical position and the shortest distance between the centers of gravity are extracted, and the flow shifts to S5603.

  In S5603, a determination process of “Is the distance between the centers of gravity sufficiently close?” Is performed. When the distance between the centers of gravity is close enough to be regarded as the same side highlight (YES), the process proceeds to S5604, and when the distance between the centers of gravity is a size that cannot be regarded as the same side highlight (NO), S5605. Migrate to

  In S5604, the corresponding R side highlight is associated with the L side highlight as shown in S5601 to S5603 by the process of “considering it as a corresponding highlight”. And the process proceeds to S5605.

  In S5605, a determination process of “has all the L highlights been checked?” Is performed. If it is determined that the check has not been completed (NO), the process returns to S5601, and S5601 to S5605 are repeated. If it is determined that the examination has been completed (YES), the flow shifts to S5606.

  In S5606, the R group of only the side highlights whose correspondence has been confirmed is registered in the registration unit 23 by the process of “leaving only the corresponding highlights as a corresponding group”, and the process proceeds to S7 in FIG.

(Image matching support detection)
FIG. 35 shows a subroutine for detecting correspondence between side highlight groups, and is a flowchart of a subroutine relating to image matching detection. As shown in FIG. 18, when the winding spring is completely oriented in the left-right direction, a plurality of highlight correspondence candidates are conceivable, and it is not possible to narrow down to only one from the highlight alone. Therefore, in this case, the original image on one camera side is determined by enlarging the area by considering a region that is slightly larger than the highlight group with respect to the original image as shown in FIG.

  In step S5701, when one of the selected L groups is substantially horizontal by the process of “setting the L original image area including the L group as a template”, the original image area including this group is used as the template. Registration is performed in the registration unit 23, and the process proceeds to S5702.

  In S5702, by setting “a wide search area in the R original image including the R group”, the areas enlarged around the R group are respectively set and registered in the registration unit 23 as shown in FIG. Then, the process proceeds to S5703.

  In S5703, a “matching position is calculated by image matching” process is performed to calculate a matching position by LR image matching from each region information in which R is enlarged, and the process proceeds to S5704.

  In S5704, one L side highlight at the matching position is selected by the process of “select one L highlight”, and the process proceeds to S5705.

  In S5705, a determination process of “Is there a highlight close to the matching area in R?” Is performed. Template matching is performed in the matching region in which R is enlarged by the L side highlight selected in S5704. If there is an R side highlight close to the L side highlight by this matching (YES), the process proceeds to S5706, and if not (NO), the process proceeds to S5707.

  In S5706, the corresponding “side highlight” is associated with the L side highlight by the process of “considering it as a corresponding highlight”, and is registered in the registration unit 23 as the corresponding highlight. Transition.

  In S5707, a determination process of “has all the L highlights been checked?” Is performed. If it is determined that the check has not been completed (NO), the process returns to S5704, and S5704 to S5707 are repeated. If it is determined that the examination has been completed (YES), the flow shifts to S5708.

  In S5708, the R group of only the side highlight that has been confirmed to be supported is registered in the registration unit 23 by the process of “leaving only the corresponding highlight as a corresponding group”, and the process proceeds to S7 in FIG.

  In this way, the object moving program is an object moving program for causing the computer to realize the function of specifying the bulky spring 3 and grasping and moving it by the hand 1011, and the bulky spring 3. Highlight extraction functions S2L and S2R for binarizing each image picked up by two cameras in a state where illumination light is applied to the LED spotlight 7 and extracting highlights of each reflection shape for each image. , S21, S22, and feature quantity extraction functions S2L, S2R, S22 for extracting feature quantities corresponding to the shape of the winding spring 3 from each highlight, and a plurality of highlights as a group for each set based on the feature quantities Grouping functions S2L, S2R, S23, and S25 that are determined every time, a correspondence detection function S5 that detects correspondence between the images of the group, and detection A position / attitude calculation function S8 for recognizing the winding spring 3 in a more corresponding group and calculating the position and orientation of the winding spring 3 based on the position of each camera by stereo image processing, and the calculated position of the winding spring 3; Operation control functions S10 and S11 for controlling the operation of the hand 1011 according to the posture are provided.

  Therefore, it is possible to accurately identify the wound springs 3..., Which are recognized by the calculation of the position, the posture, and to automate the picking of the bulk springs 3.

  The object movement program includes end face highlight detection functions S2L, S2R, and S24 that determine the final end face highlight from the identified end face highlight, and the correspondence detection processing function S5 is an image of the end face highlight. S6, and the position / posture calculation function S8 recognizes the winding spring 3 based on the detected group and the highlight of the end face, and the position of the winding spring 3 with reference to the positions of the cameras 1005 and 1007. The posture is calculated by stereo image processing.

  Therefore, identification by the calculation of recognition, position, and orientation of the winding springs 3.

  The embodiment of the present invention is a computer-readable recording medium carrying an object moving program.

  Therefore, the object moving program can be read by the computer for the automation of picking of the winding springs 3.

  The object moving method of the present invention is an object moving method in which a bulk winding spring 3 is identified and gripped by the hand 1011 and moved. Highlight extraction steps S2L, S2R, S21, and S22 for binarizing each image picked up by two cameras in a state of being applied and extracting each reflection-shaped highlight for each image, and a winding spring Feature amount extraction steps S2L, S2R, and S22 for extracting feature amounts corresponding to the three shapes from the highlights, and a grouping step S2L for determining the highlights for each set as a group for each set based on the feature amounts. , S2R, S23, S25, the correspondence detection step S5 for detecting the correspondence between the images of the group, and the winding spring 3 in the group corresponding to the detection. A position / posture determination step S8 for determining the position and posture of the winding spring 3 based on the position of each camera, and operation steps S10 and S11 for operating the hand 1011 according to the position and posture of the winding spring 3. Prepared.

  Therefore, like the above, it is possible to automate the picking of the winding springs 3.

  The object moving method includes end face highlight detection steps S2L, S2R, and S24 for determining the final end face highlight from the determined end face highlight, and the correspondence detection step S5 is performed between the images of the end face highlights. In step S6, a position / posture determination step S8 detects the position of the spring 3 by recognizing the corresponding group and the highlight of the end face and detecting the position of the spring 3 based on the positions of the cameras 1005 and 1007. To decide.

  Therefore, identification by the calculation of recognition, position, and orientation of the winding springs 3.

[Others]
The present invention is not limited to a wound spring as an object, and in addition, when recognizing each object in a collection of irregularly shaped objects having the same shape and having a space portion, for example, The present invention can also be applied to recognition of an object having the same shape as that of a wound spring but having no spring property, a rectangular body, a sphere, or the like and having a space portion in the shape and the contour is divided.

  In the above-described embodiment, the grouping is performed after the temporary grouping. However, the grouping may be performed directly based on the highlight shape, the linear arrangement of the highlights, the interval, and the number as the feature amount without the temporary grouping. .

1001 Object moving device 1003 Object identifying device 1005, 1007 Camera 1009 Robot controller (control unit)
1011 Robot / Hand 1 Object recognition unit 3 Winding spring (object)
7 LED spot / light 13 Highlight extraction processing unit 15 Highlight discrimination processing unit 17 Feature amount extraction processing unit 19 Temporary grouping processing unit 21 Grouping processing unit 22 End face highlight detection unit 25 Corresponding detection processing unit 27 Position / attitude calculation Part S2L, S2R Highlight extraction function, highlight discrimination function, feature quantity extraction function, highlight extraction step, highlight discrimination step, feature quantity extraction step S5 Correspondence detection function, correspondence detection step S8 Position / attitude calculation function, position / Posture Determination Steps S10, S11 Motion Control Function, Motion Control Step S23 Temporary Grouping Function, Temporary Grouping Step S24 End Surface Highlight Detection Function, End Surface Highlight Detection Step S25 Grouping Function, Grouping Step

Claims (14)

An object moving device that identifies and moves the object with the hand while the object having the same shape having an end face and a side surface in which the shape is present and the contour is divided is irregularly assembled There,
A highlight extraction processing unit that binarizes each image captured by two cameras in a state where illumination light is applied to the entire target object and extracts highlights of each reflection shape for each image. When,
A highlight determination processing unit that performs threshold processing of an area on the extracted highlight and determines a highlight on a side surface corresponding to a side surface of the object and a highlight on an end surface corresponding to an end surface of the object;
A feature amount extraction processing unit for extracting a plurality of feature amounts for corresponding to the side surface shape of the object from highlights of the side surfaces ;
A grouping processing unit that determines, for each image, a plurality of sets of side highlights according to the feature amount;
A correspondence detection processing unit for detecting correspondence between the images of the group;
A position / attitude calculation unit that recognizes the object in a group corresponding to the detection and calculates the position and orientation of the object with reference to the position of each camera by stereo image processing;
A control unit that controls the operation of the hand according to the calculated position and orientation of the object;
An object moving apparatus comprising: An object moving device that identifies and moves the object with the hand while the object having the same shape having an end face and a side surface in which the shape is present and the contour is divided is irregularly assembled There,
A highlight extraction processing unit that binarizes each image captured by two cameras in a state where illumination light is applied to the entire target object and extracts highlights of each reflection shape for each image. When,
A highlight determination processing unit that performs threshold processing of an area on the extracted highlight and determines a highlight on a side surface corresponding to a side surface of the object and a highlight on an end surface corresponding to an end surface of the object;
A feature amount extraction processing unit for extracting a plurality of feature amounts for corresponding to the side surface shape of the object from highlights of the side surfaces ;
A grouping processing unit that determines, for each image, a plurality of sets of side highlights according to the feature amount;
A correspondence detection processing unit for detecting correspondence between the images of the group;
A position / attitude calculation unit that recognizes the object in a group corresponding to the detection and calculates the position and orientation of the object with reference to the position of each camera by stereo image processing;
A control unit that controls the operation of the hand according to the calculated position and orientation of the object;
A temporary grouping processing unit that sets the side highlights as a temporary group for each of a plurality of sets according to a specific feature amount selected from the plurality of feature amounts;
The grouping processing unit determines a highlight of the side surface as the temporary group as a group based on another specific feature amount selected from the plurality of feature amounts.
The object moving apparatus characterized by the above-mentioned. The object moving device according to claim 1 or 2 ,
An end face highlight detection unit for determining a final end face highlight from the determined end face highlight;
The correspondence detection processing unit detects correspondence between the images of the highlight on the end face,
The position / attitude calculation unit recognizes the object by highlighting the corresponding group and end face by the detection, and calculates the position and attitude of the object based on the position of each camera.
The object moving apparatus characterized by the above-mentioned. The object moving device according to any one of claims 1 to 3 ,
The correspondence detection processing unit detects the correspondence of the group by any combination of the area of the highlight, the centroid position, the alignment direction, the distance between group centroids, the number, and the vertical position of the centroid.
The object moving apparatus characterized by the above-mentioned. The object moving device according to any one of claims 1 to 4 ,
The correspondence detection processing unit detects the correspondence of the group by comparing each original image captured by the camera when the orientation of the highlight group is regarded as horizontal or horizontal.
The object moving apparatus characterized by the above-mentioned. The object moving device according to claim 5 ,
The correspondence detection processing unit sets the search area in the original image captured by the camera including the group regarded as horizontal or horizontal to include peripheral information.
The object moving apparatus characterized by the above-mentioned. An object moving device for moving the object by identifying the object and grasping it with a hand while the objects of the same shape, whose space part exists in the shape and whose outline is divided, are irregularly assembled,
A highlight extraction processing unit that binarizes each image captured by two cameras in a state where illumination light is applied to the entire target object and extracts highlights of each reflection shape for each image. When,
A feature amount extraction processing unit that extracts a feature amount corresponding to the shape of the object from each highlight;
A grouping processing unit that determines, for each image, the highlights as a group for each set by the feature amount;
A correspondence detection processing unit for detecting correspondence between the images of the group;
A position / attitude calculation unit that recognizes the object in a group corresponding to the detection and calculates the position and orientation of the object with reference to the position of each camera by stereo image processing;
A controller that controls the operation of the hand according to the calculated position and orientation of the object,
The correspondence detection processing unit detects the correspondence of the group by comparing each original image captured by the camera when the orientation of the highlight group is regarded as horizontal or horizontal.
The object moving apparatus characterized by the above-mentioned. The object moving device according to claim 7 ,
The correspondence detection processing unit sets the search area in the original image captured by the camera including the group regarded as horizontal or horizontal to include peripheral information.
The object moving apparatus characterized by the above-mentioned. The object moving device according to any one of claims 1 to 8,
The object is a coil spring.
The object moving apparatus characterized by the above-mentioned. In the object moving method in which the object of the same shape and a side end face and the space portion is divided the present contour shape moves gripped by a specific hand-said object in the assembled irregularly There,
A highlight extraction step of binarizing each image picked up by two cameras in a state in which illumination light is applied to the entire target object and extracting highlights of each reflection shape for each image; ,
A highlight determination step for performing threshold processing of an area on the extracted highlight to determine a highlight on a side surface corresponding to the side surface of the object and a highlight on an end surface corresponding to the end surface of the object;
An end face highlight detection step for determining a final end face highlight from the determined end face highlight;
A feature amount extracting step of extracting a plurality of feature amounts to correspond to the side surface shape of the object from highlights of the side surfaces ;
A grouping step of determining, for each image, a plurality of sets of side highlights according to the feature amount;
A correspondence detection step for detecting correspondence between the images of the group and end face highlights ;
A position / posture determination step for recognizing the target object by highlighting the corresponding group and end face by the detection and determining the position and posture of the target object based on the position of each camera;
An operation step of causing the hand to move according to the position and orientation of the object;
An object moving method characterized by comprising: Realizing the function of the object of the same shape and a side end face and the space portion is divided the present contour shape moves gripped by a specific hand-said object in the assembled irregularly to the computer An object movement program to cause
A highlight extraction function for binarizing each image picked up by two cameras in a state where illumination light is applied to the entire collected object and extracting highlights of each reflection shape for each image; ,
A highlight determination function for performing threshold processing of an area on the extracted highlight to determine a highlight on a side surface corresponding to a side surface of the object and a highlight on an end surface corresponding to an end surface of the object;
A feature amount extraction function for extracting a plurality of feature amounts to correspond to the side surface shape of the object from the highlights of the side surfaces ;
A grouping function for determining each side image as a group of a plurality of sets of side highlights based on the feature amount;
A correspondence detection function for detecting correspondence between the images of the group;
A position / attitude calculation function for recognizing the object in a group corresponding to the detection and calculating the position and orientation of the object with reference to the position of each camera by stereo image processing;
An operation control function for controlling the operation of the hand according to the calculated position and orientation of the object;
As a function of the object movement program. The computer has a function to identify and hold the object with a hand and move it while the objects of the same shape, which have end faces and side faces that have a space portion and are divided in outline, are irregularly assembled. An object movement program to cause
A highlight extraction function for binarizing each image picked up by two cameras in a state where illumination light is applied to the entire collected object and extracting highlights of each reflection shape for each image; ,
A highlight determination function for performing threshold processing of an area on the extracted highlight to determine a highlight on a side surface corresponding to a side surface of the object and a highlight on an end surface corresponding to an end surface of the object;
A feature amount extraction function for extracting a plurality of feature amounts to correspond to the side surface shape of the object from the highlights of the side surfaces;
A grouping function for determining each side image as a group of a plurality of sets of side highlights based on the feature amount;
A correspondence detection function for detecting correspondence between the images of the group;
A position / attitude calculation function for recognizing the object in a group corresponding to the detection and calculating the position and orientation of the object with reference to the position of each camera by stereo image processing;
An operation control function for controlling the operation of the hand according to the calculated position and orientation of the object;
A provisional grouping function as a temporary group for each one of a plurality of sets of highlights on the side surface according to a specific feature amount selected from the plurality of feature amounts;
The grouping function determines a highlight of the side surface as the temporary group as a group based on another specific feature amount selected from the plurality of feature amounts.
An object moving program characterized by that. Realizing the function of the object of the same shape and a side end face and the space portion is divided the present contour shape moves gripped by a specific hand-said object in the assembled irregularly to the computer An object movement program to cause
A highlight extraction function for binarizing each image picked up by two cameras in a state where illumination light is applied to the entire collected object and extracting highlights of each reflection shape for each image; ,
A highlight determination function for performing threshold processing of an area on the extracted highlight to determine a highlight on a side surface corresponding to a side surface of the object and a highlight on an end surface corresponding to an end surface of the object;
A feature amount extraction function for extracting a plurality of feature amounts to correspond to the side surface shape of the object from the highlights of the side surfaces ;
A grouping function for determining each side image as a group of a plurality of sets of side highlights based on the feature amount;
An end face highlight detection function for determining a final end face highlight from the determined end face highlight;
A correspondence detection function for detecting correspondence between the group and the highlight of the end face between the images;
A position / attitude calculation function for recognizing the object by highlighting the corresponding group and end face by the detection and calculating the position and orientation of the object with reference to the position of each camera by stereo image processing;
An operation control function for controlling the operation of the hand according to the calculated position and orientation of the object;
As a function of the object movement program. A computer-readable recording medium carrying the object moving program according to any one of claims 11 to 13 .