JP7222720B2 - Tip member position estimation method, tip member gripping method, tip member connection method, tip member position estimation system, and tip member gripping system - Google Patents

Description

The present invention relates to a tip member position estimation method, a tip member gripping method, a tip member connection method, a tip member position estimation system, and a tip member gripping system.

Robots that recognize an object with a three-dimensional camera or the like and hold it autonomously are becoming widespread. Regarding gripping a linear object, for example, Japanese Patent Application Laid-Open No. 2014-176917 (Patent Document 1) discloses a robot apparatus for assembling a linear object, in which a fixed end of a linear object is fixed at one end. A device is described in which, after gripping the vicinity, the gripping part is slid along a predetermined trajectory to move to the other end. It is said that this makes it possible to quickly grasp the other end of an electric wire, which is an example of a linear object, which is difficult to accurately estimate due to the peculiarity of the wire.

Japanese Patent Application Laid-Open No. 2016-192138 (Patent Document 2) discloses an invention related to a wire harness manufacturing method and an image processing method, and in the process of manufacturing the wire harness, the three-dimensional shape of the wire assembly is measured. is performed to specify the machining position.

JP 2014-176917 A JP 2016-192138 A

A connector or the like is often connected to the tip of a linear object such as an electric wire. A grasping part of a robot hand, such as a gripper, directly grasps the tip of a linear object or a connector, moves the connector to a predetermined position and fixes it, or attaches the connector to a connector housing and connects the connector. It is assumed that a continuity test is performed on the equipment that has been tested.

Conventionally, the CAD data of the connector is registered in advance, and pattern matching is performed between the measured shape of the connector and the registered data to recognize the position and orientation of the connector, and have the robot grasp and move the connector. rice field. However, connectors have a wide variety of shapes, and there is a problem that it is difficult to register all the shape data in advance. In addition, there is a problem that pattern matching processing between three-dimensional data is heavy load and requires a long calculation time.

Furthermore, when attaching the connector to the connector housing, it is also necessary to grasp the rotational direction of the linear object of the connector gripped by the robot hand about the axis.

When a tip member such as a connector attached to the tip of a linear object is to be attached to a predetermined connecting member, the position of the tip member and, more preferably, the direction of rotation of the tip member should be grasped in advance. It is necessary to keep

An object of the present invention is to solve the above-mentioned problems. A method for estimating the position of a tip member and, more preferably, a rotational direction of the tip member from the shape of the tip portion of a linear object, An object of the present invention is to provide a tip member gripping method, a tip member connecting method, a tip member position estimation system, and a tip member gripping system.

A tip member position estimation method according to this disclosure is a method for estimating a tip member position of a linear object having a tip member at its tip, comprising: a coordinate acquisition step of acquiring coordinates of two or more points on the linear object; a tip member position estimation step of estimating the position of the tip member based on the coordinates of two or more points on the linear object.

In another aspect, the tip member position estimating step includes: calculating an approximate straight line of the linear object based on the coordinates of two or more points on the linear object; and estimating the position of the tip member.

In another aspect, the coordinate obtaining step includes obtaining coordinates of three or more points on the linear object, and the tip end member position estimating step obtains coordinates of three or more points on the linear object. calculating an approximated curve of the linear object based on the approximated curve; and estimating the position of the tip member based on the approximated curve.

In another aspect, the coordinate obtaining step includes obtaining coordinates of the tip of the linear object.

In another form, the coordinates on the linear object are three-dimensional coordinates.
In another aspect, the method further comprises a rotation angle calculation step of acquiring an image of the tip member and calculating a rotation angle of the tip member based on the image.

A method for gripping a tip member according to this disclosure is a method of gripping a tip member provided at the tip of a linear object with a robot hand, wherein the coordinates of two or more points on the linear object are obtained by a measuring device. a tip member gripping position calculation step of calculating a gripping position of the tip member based on coordinates of two or more points on the linear object; and gripping of the tip member at the gripping position by the robot hand. and a step.

In the tip member connection method according to this disclosure, the step of gripping the tip member by the robot hand in the tip member gripping method described above, acquiring an image of the tip member gripped by the robot hand, and obtaining an image of the tip member from the image The method includes a step of calculating a rotation angle of the tip member, and a step of connecting the tip member to the connecting member using the robot hand based on the rotation angle.

In the tip member position estimation system in this disclosure, a measuring device that obtains coordinates of two or more points on a linear object, and a tip of the linear object based on the coordinates of two or more points on the linear object. and a calculation unit for estimating the position of the provided tip member.

In another embodiment, the apparatus further includes image acquisition means for capturing an image of the tip member, and rotation angle calculation means for calculating the rotation angle of the tip member based on the image obtained by the image acquisition means.

A tip member gripping system according to this disclosure is a tip member gripping system for a linear object having a tip member at its tip, comprising: a measuring device for acquiring coordinates of two or more points on the linear object; A computing unit for estimating the position of the tip member provided at the tip of the linear object based on the coordinates of two or more points above, and gripping the tip member based on the estimated position of the tip member. and a gripping portion.

According to the present invention, a tip member position estimation method, a tip member gripping method, a tip member connection method, and a tip member for grasping the position of the tip member from the shape of the linear object and, more preferably, the rotation direction of the target object. It is possible to provide a position estimation system and a tip gripping system.

It is a functional block diagram of a three-dimensional measuring device of related technology. It is a figure for demonstrating the three-dimensional measuring method of related technology. It is a process flow figure of the three-dimensional measuring method of related technology. It is a figure which shows the 1st image imaged with the stereo camera. It is a figure which shows the 2nd image imaged with the stereo camera. It is an operation flow figure of the 1st extraction process of the three-dimensional measuring method of related technology. It is a figure which shows the 1st image from which the 1st line image was extracted. It is a figure which shows the 1st image from which the 2nd line image was extracted. FIG. 10 is a diagram showing a first image in which a point of interest is selected; FIG. 10 is a diagram showing a second image in which intersections between epipolar lines and second line images are obtained; It is a figure which shows an example of a color table. It is a schematic diagram showing a tip member position estimation method of the present embodiment. FIG. 2 is a diagram showing an electrical device in which a connector is connected to the tip of an electric wire; It is a figure which shows the front of a connector. It is a figure which shows the front of a connector housing. 1 is an overall view of a tip member gripping system according to an embodiment; FIG. It is a figure which shows the flow of the automation system of the connection process of this Embodiment. FIG. 3 is a perspective view showing the detailed structure of a gripper employed in the robot hand of this embodiment;

The present embodiment will be described below with reference to the drawings. In the embodiments described below, when referring to the number, amount, etc., the scope of the present invention is not necessarily limited to the number, amount, etc., unless otherwise specified. The same reference numbers are given to the same parts and equivalent parts, and redundant description may not be repeated. It is planned from the beginning to use the configurations in the embodiments in combination as appropriate. In the drawings, the actual dimensional ratios are not shown, but the ratios are partially changed in order to facilitate understanding of the structure.

In the disclosure below, a case where an electric wire is used as an example of a linear object is described, but the wire is not limited to the electric wire. A linear object in this description may be any object having an elongated shape. Examples of linear objects include electric wires, wire harnesses, solder, cords, strings, fibers, glass fibers, optical fibers, tubes, and other linear objects. It is not limited to an electric wire in which thin wires are bundled, but also includes an electric wire or the like composed of a single wire.

(Related technology: three-dimensional measuring method and three-dimensional measuring device for linear objects)
An example of a linear object three-dimensional measurement method and apparatus will be described below as related art with reference to FIGS. 1 to 11 .

Referring to FIG. 1 , three-dimensional measuring apparatus 10 includes stereo camera 11 , arithmetic unit 15 , storage unit 16 , and input/output unit 17 . The calculation unit 15, the storage unit 16, and the input/output unit 17 may be collectively called a control unit.

Stereo camera 11 includes first camera 12 , second camera 13 , and camera control section 14 . The first camera 12 is a color camera that captures a first image that is a two-dimensional color image. The second camera 13 is a color camera that captures a second image, which is a color two-dimensional image, and is fixed in position relative to the first camera 12 .

Camera control unit 14 controls first camera 12 and second camera 13 and communicates with calculation unit 15 . Camera control unit 14 , for example, receives imaging instructions from computing unit 15 , transmits imaging instructions to first camera 12 and second camera 13 , and transfers the first image and second image to computing unit 15 .

In addition to communicating with the camera control unit 14, the calculation unit 15 processes the first image and the second image received from the stereo camera 11 to calculate the three-dimensional position of the linear object. The storage unit 16 stores the first image and the second image captured by the stereo camera 11 and the color table of the object, as well as intermediate data necessary for computation, computation results, and the like. The input/output unit 17 receives commands from the operator and displays measurement results to the operator.

Referring to FIG. 2 , in this measurement method, electric wire 21 , electric wire 22 and electric wire 23 are imaged by first camera 12 and second camera 13 . For a certain point P on the wire 21, if a point Q projected onto the first image 30 by the first camera and a point R projected onto the second image 40 by the second camera are obtained, the known first camera and position information of the second camera, the three-dimensional position of the point P can be calculated. The position information of the first camera and the second camera can be obtained by calibrating the two cameras in advance.

Although FIG. 2 is drawn in black and white, the three electric wires 21, 22 and 23 to be measured are color-coded and have coatings of different colors such as red, blue and yellow. A linear object to be measured is not particularly limited as long as it is a linear object.

FIG. 3 is a flow diagram of the three-dimensional measurement method. Each step will be described below.
Create a color table prior to measurement. The color table is a table that records the color for each type of linear object that can be measured. FIG. 11 shows, as an example, a color table in which the color of each type of electric wire is represented by the brightness of the three primary colors of red, green, and blue (RGB). The color table is stored in the storage unit 16. FIG.

At the time of measurement, the stereo camera 11 images the electric wire 21 , the electric wire 22 and the electric wire 23 . The electric wire 21 , the electric wire 22 and the electric wire 23 are imaged in the first image 30 by the first camera 12 . At the same time, the electric wire 21, the electric wire 22 and the electric wire 23 are imaged by the second camera 13 into a second image 40 from a viewpoint different from that of the first camera. The first image and the second image are transferred to the calculation section 15 and stored in the storage section 16 .

The calculation unit 15 acquires the first image 30 and the second image 40 from the stereo camera 11 . At this time, referring to FIG. 4, images 31, 32 and 33 of the three electric wires 21, 22 and 23 are shown in the first image 30, respectively. Similarly, referring to FIG. 5, second image 40 includes images 41, 42 and 43 of three electric wires 21, 22 and 23, respectively.

The calculation unit 15 extracts the specific electric wire 21 as the first line image on the first image 30 . Referring to FIG. 6, the step of extracting the first line image (first line image extraction step) includes color extraction operation, binarization operation, noise removal operation, and thinning operation.

In the extraction operation by color, the calculation unit 15 acquires the color of the wire 21 to be measured from the color table, and extracts only the image 31 of the linear object 21 of the specific color on the first image 30 as the first line image. Extract as 34.

Specifically, the color of each pixel in the first image is compared with its particular color, and if the two are determined to be the same, the pixel is retained, and if the two are determined to be different, the pixel is retained. erase. Whether the colors are the same or different can be determined by determining whether the difference between the two is equal to or less than a predetermined value.

For example, the RGB value corresponding to the wire 21 is obtained from the color table, and the RGB value of each pixel of the first image 30 is compared with it, and if the difference between the RGB values is less than or equal to a predetermined value, the pixel is It is determined that the color is the same as that of the electric wire 21 . The predetermined value can be determined in consideration of the number of RGB gradations, the degree of color difference between different types of wires, and the like.

Next, the first image 30 is binarized. This is an operation that replaces the value of each pixel with 0 or 1 using an appropriate threshold. The binarization operation facilitates subsequent image processing. The binarization operation may be performed simultaneously with the color extraction operation. Pixels determined to have the same color are set to 1, and pixels determined to have different colors are set to 0, so that binarization can be performed.

Next, a noise removal operation is performed on the first image 30 . Although the first line image 34 has been extracted by the color extraction operation, the first image still has isolated pixels due to camera shot noise or the like. Since the positions of the RGB image pickup elements are actually slightly shifted with respect to one pixel, the color of the image may be changed at portions where the color changes abruptly, such as the contours of the images 31, 32, and 33 of each electric wire. There is a possibility that the image is disturbed or some isolated pixels remain. By removing such pixels, a more accurate first line image 34 is obtained.

Next, the first line image 34 is thinned. This operation narrows the line width to 1 while maintaining the connectivity of the first line image. A known method such as selecting a pixel positioned at the center of the line width can be used for the thinning operation method. This facilitates subsequent image processing, and enables more accurate determination of corresponding points and the like.

FIG. 7 shows the first line image 34 obtained. The first image 30 from which the first line image is extracted is stored in the storage unit 16 .

Returning to FIG. 3, the same operation as for the first image 30 is performed on the second image 40 to extract the second line image 44 (second line image extraction step). The second line image 44 is shown in FIG. The second image from which the second line image is extracted is stored in the storage unit 16 .

Referring to FIG. 9 , the calculation unit selects a point of interest Q on the first line image 34 of the first image 30 . Point Q is the projection point of point P (FIG. 2) of wire 21 onto the first image.

Referring to FIG. 10, the calculation unit obtains epipolar line 45 corresponding to point of interest Q of first image 30 on second image 40 . A point of intersection R between the second line image 44 and the epipolar line 45 is obtained, and this point is set as a point corresponding to the point Q of interest. Point R is the projection of point P (FIG. 2) of wire 21 onto the second image.

Through the above steps, a projection point Q onto the first image 30 and a projection point R onto the second image 40 are obtained for the point P of the electric wire 21 shown in FIG. Calculate the three-dimensional position of

Next, a new point of interest is selected on the first line image 34, and the steps after selection of the point of interest are repeated. As the next point of interest, an adjacent point connected to the previous point of interest can be selected. Three-dimensional measurement of the electric wire 21 is performed by determining the three-dimensional position while shifting the point of interest Q, that is, moving the point P on the electric wire 21 in this way.

When the necessary information about the electric wire 21 is obtained, the above iterative process ends. When performing three-dimensional measurement of another electric wire, for example, the electric wire 22, the color of the electric wire 22 is acquired from the color table, and the original first and second images captured by the first and second cameras are compared. Then, the steps after the first line image extraction step are repeated.

Here, the color table will be explained in more detail.
Although the color table illustrated in FIG. 11 describes one RGB value for each type of linear object, a plurality of RGB values are described for one type of linear object. If it is determined to be the same color as the RGB values of , it may be determined to be the linear object. Colors may be recorded in a color system other than RGB. For example, it may be represented by L*, a*, b* based on the CIELAB color system established by the Commission Internationale de l'Eclairage (CIE). Even if the output from the stereo camera is RGB values, conversion between color systems is easy.

If the difference between the RGB value of the pixel and the RGB value of the color table is equal to or less than a predetermined value, it is determined that the color of the pixel and the color of the table are the same. You can record it in a table. When recording a color range, it is more preferable to express it by L*a*b* values because it is easier to set a robust threshold range against changes in the amount of light. For example, by setting a wide threshold range for the L* value and narrowing the threshold ranges for the a* and b* values, even if the brightness of the linear object changes to a certain extent, it will not be confused with cables of other colors. can be considered to be of the same color.

The color table is preferably created based on the color of the image when the linear object is actually captured in the actual measurement environment. Specifically, a linear object is held by a hand or a robot hand, and captured while moving in various positions and orientations in front of the first or second camera. get. The color of the linear object on the first image and the second image changes depending on various factors such as the type and arrangement of lighting in the measurement environment, the gloss level and orientation of the linear object. By recording in the color table the colors in the range in which the image of the linear object can be captured under the actual measurement conditions, misrecognition when extracting the linear object can be reduced.

For example, as a linear object to be gripped in the three-dimensional measurement method, it is possible to apply not only cables but also wire harnesses and various other linear objects. The steps and operations described above may be rearranged in order of execution or omitted where possible.

The three-dimensional measurement method described above does not exclude the use of known matching methods in the stereo system. When there are a plurality of linear objects of the same color among a large number of linear objects, there is an advantage in using a matching method that focuses on the shape and other features of the measuring tip member.

<First embodiment of tip member position estimation method>
Next, a tip member position estimation method will be described with reference to FIG. 12 . FIG. 12 is a schematic diagram showing the tip member position estimation method of the present embodiment.

The electric wire 21 described above will be described as an example. A connector C<b>1 as a tip member is connected in advance to the tip of the electric wire 21 . The tip member is not limited to the connector, and may be provided at the tip of the linear object and may be different from the linear object in any one of thickness, shape, and color. The color of the electric wire 21 is discriminated based on the three-dimensional measurement method for linear objects. Furthermore, three-dimensional position information (three-dimensional coordinates) of P1, P2, P3, P4, and P5 at the tip portion of the electric wire 21 is obtained as a target point of the electric wire 21. FIG. In the drawing, P1, P2, P3, P4, and P5 are shown separately for the sake of explanation, but the points of interest may actually be close to each other. The length of the tip portion for which the three-dimensional position information should be grasped can be arbitrarily set, and the number of points of interest to be obtained can also be arbitrarily set. It is preferable to obtain three-dimensional coordinates of two or more points, preferably three or more points. One point of interest to be acquired is preferably the tip P21 of the wire 21, which is the point of contact between the wire 21 and the tip member. This is because the position of the tip member can be estimated more accurately by grasping the starting point position of the tip member.

The electric wire 21 is colored in an arbitrary color such as red or covered with an arbitrary color such as red, so that an accurate three-dimensional measurement method can be performed. If the color of the electric wire 21 and the color of the connector C1 are different, it is more preferable because the electric wire 21 and the connector C1 can be easily recognized separately.

Next, the position of connector C1 is estimated. Estimating the position of the connector C1 means estimating the attachment direction of the connector C1 attached to the electric wire 21 . It is assumed that the direction in which the connector C1 is attached coincides with the direction of the extension line of the tip portion of the electric wire 21 . Next, the mounting direction of the connector C1 is estimated from the three-dimensional position information of P1, P2, P3, P4, and P5. When the wire is flexible and the connector C1 is non-flexible, the position and orientation of the connector C1 often follow the shape of the tip of the wire. can do.

Methods for estimating the attachment direction of the connector C1 include the following (i) to (iii).

(i) A method of calculating an approximate straight line based on two or more arbitrary points of interest and estimating that the tip member is positioned on the calculated approximate straight line. Specifically, the direction (x-line) in which an approximate straight line connecting two points of interest P1 and P3 extends is estimated as the mounting direction of the connector C1.

(ii) A method of calculating an approximate curve based on arbitrary three or more points of interest and estimating that the tip member is positioned on the calculated approximate curve. Specifically, an approximate curve of the wire 21 is measured using the points P1 to P5 of interest, and this curve (y curve) is estimated to be the mounting direction of the connector C1 of the wire 21 .

(iii) A method of calculating an approximate curve based on arbitrary three or more points of interest, and estimating that the tip member is positioned on the tangent line of the calculated approximate curve at the tip of the linear object. Specifically, the tangential line (z line) of the curve (y curve) on the point of interest P21 is estimated to be the mounting direction of the tip end connector C1 of the electric wire 21 .

In the method shown in (i), the direction in which the x-ray extends is approximated as the mounting direction of the connector C1. is fast. On the other hand, the method shown in (ii) and (iii) for approximating the direction in which the x-ray extends as the mounting direction of the connector C1 is more accurate than the method shown in (i) because it uses a larger number of points of interest. is considered high.

<Second embodiment of tip member position estimation method>
In the second embodiment of the tip member position estimation method, the stereo camera 11 images a linear object, acquires a first image and a second image, and a first line to be measured on the first image and the second image. The steps up to the step of extracting the image and the second line image are the same as in the first embodiment.

Next, the first line image is extended on the first image, and a first end member estimation point located at a predetermined distance from the end point of the first line image is obtained. Similarly, the second line image is extended on the second image to obtain a second tip end member estimation point at a predetermined distance from the end point of the second line image. Then, it is assumed that the first tip member estimation point and the second tip member estimation point are points where the tip member is positioned on each image.

The calculation unit calculates the three-dimensional position of the connector C1 using the first tip member estimation point and the second tip member estimation point as corresponding points of the two images in the stereo method.

<Third embodiment of tip member position estimation method>
When the linear object is on a predetermined table or along a wall, the position of the tip member can be estimated by acquiring two-dimensional information on the point of interest on the linear object. In this case, a linear object is imaged using a two-dimensional camera, and the attachment direction of the connector C1 is estimated in the same manner as in any of the tip member position estimation method of the first embodiment (i) to (iii). .

<How to grip the tip member>
Using any one of the methods of the first to third embodiments, the attachment direction of the connector C1 is estimated, and the tip member gripping method by the robot hand is determined. The specific gripping position of the robot hand may be the point of interest P21 of the electric wire 21, or a position on the connector C1 located at a predetermined distance (L1) from the point of interest P21 in the direction of attaching the connector estimated by the above-described method. may be determined as the grasping position.

Which one of the first to third embodiments of the tip member position estimation method is used is determined by the wire thickness, the wire material, the wire hardness, the presence or absence of a core wire, the resolution of the camera, and the like. be.

<Tip member position estimation system 200 and tip member gripping system 1000>
Next, a tip member position estimation system 200 using the tip member position estimation method and a tip member gripping system 1000 using a robot hand will be described with reference to FIGS. 13 to 17 .

FIG. 13 shows an electric device 100 in which a connector C1 is connected to the tip of the electric wire 21. As shown in FIG. Examples of electrical equipment 100 include drive motors, alternators, batteries, compressors, automotive electrical components, household appliances, and various other electrical equipment.

In the present embodiment, in order to conduct a continuity test of the electrical device 100, a step ( Tip member connection method) will be described.

FIG. 14 illustrates the connector C1 and FIG. 15 illustrates the connector housing C2. Referring to FIG. 14, connector C1 has a cylindrical body portion C11. It is estimated that the axial direction of the body portion C11 (direction of arrow A in FIG. 13) matches the estimated attachment direction of the connector C1.

A plurality of pins C12 are arranged at predetermined positions inside the body portion C11. A rib C13 extending along the axial direction of the body portion C11 is provided on the outer peripheral surface of the body portion C11 in order to position the connector C1. The state shown in FIG. 14 is assumed to be a state in which the front face of the connector C1 can be recognized. By recognizing the front face of the connector C1, it is possible to recognize the rotation state of the connector C1 (calculation of the rotation angle based on the position of the rib C13).

Referring to FIG. 15, connector housing C2 has a housing C21 provided with pin receivers C22 at positions corresponding to pins C12. The housing C21 is provided with positioning recesses C23 into which the ribs C13 are inserted.

Next, an example of automation of attaching the connector C1 to the connector housing C2 using the robot hand 600 provided on the robot arm 500 will be described with reference to FIGS. 16 and 17. FIG. FIG. 16 is an overall view of the tip member gripping system 1000, and FIG. 17 is a diagram showing the flow of the automation system for the mounting process.

The tip member gripping system 1000 is controlled by the tip member position estimation system 200, which will be described later. This tip member position estimation system 200 is the same as the three-dimensional measuring device 10 described in FIG. In the following description of the tip member position estimation system 200, the same reference numerals as those of the three-dimensional measuring device 10 shown in FIG. 1 may be used.

Referring to FIG. 16, tip member grasping system 1000 has housing 700 . A transparent wall 400 is arranged on the upper part of the housing 700 so that the mounting process inside can be visually observed. A robot 900 and an object placement table 800 are arranged at predetermined positions inside the housing 700 . The robot 900 has a robot arm 500 and a robot hand 600 is attached to the tip of the robot arm 500 . Electrical device 100 having connector C1 shown in FIG. 13 and connector housing C2 are arranged on object placement table 800 .

Further, at predetermined positions inside the housing 700, the tip member position estimation system 200 including the stereo camera 11 and the like, and the area camera 300 are arranged. Although the details will be described later, the area camera 300 functions as front image acquisition means, and is used when measuring the orientation (rotational angle) of the connector C1 by image processing. Front image information obtained from the area camera 300 is input to the three-dimensional measuring device 10 . The three-dimensional measuring device 10 includes rotation angle calculation means for calculating the rotation angle of the connector C1 based on this front image. This rotation angle calculation means may be provided in the calculation unit 15, or a rotation angle calculation unit may be provided.

Although the case where the two-dimensional area camera 300 is employed as the front image acquisition means is described, the stereo camera 11 may also be used as the front image acquisition means.

For the robot arm 500, for example, FANUC Robot LR Mate 200iD/7L (weight capacity 7 kg, reach length 900 mm) is adopted. A pair of grippers 610 are provided at the tip of the robot hand 600, and are opened and closed in parallel by an air cylinder. The gripper 610 has a width of approximately 10 mm and a length of approximately 20 mm. The opening/closing stroke of the gripper 610 is approximately 10 mm.

The tip member position estimation system 200 uses 3D vision for linear object recognition made by Kurabo Industries. The distance between the object and the stereo camera 11 is approximately 500 mm, the field of view is 400 mm×250 mm, and the depth of focus is ±100 mm. As a linear object gripping position recognition function, it is used when outputting the gripping position, etc., and the posture of the robot hand.

An area camera manufactured by Fanuc or an area camera (Dart) manufactured by Balser is used as the area camera 300 . The visual field range is approximately 200 mm×150 mm. It is used to output the rotation angle of the connector as a connector orientation recognition function.

Next, with reference to FIG. 17, a mounting process (wiring automation process) of the connector C1 to the connector housing C2 using the tip member gripping system 1000 will be described. In addition, the following flow is executed by the camera control unit 14 and/or the calculation unit 15 provided in the tip member position estimation system 200 .

Using the tip member position estimation system 200, the tip portion of the electric wire 21 and the connector C1 are 3D scanned (step 1 (referred to as S1, the same applies hereinafter)). The connector C1 is not essential, as long as the tip of the electric wire 21 can be scanned in 3D. Based on the obtained 3D image information, it is determined whether or not the connector C1 is positioned at a position where the connector C1 can be gripped (S2). Specifically, using the three-dimensional measuring device 10, the grasping position of the tip portion is confirmed by exerting the function of recognizing the grasping position of the tip portion. If it is determined that the connector C1 cannot be held, the test is terminated (S3).

When it is determined that the connector C1 can be gripped, it is determined whether to grip a position at a predetermined distance (L1) from the root of the connector C1, or to grip the target point P1 of the electric wire near the root of the connector C1. (gripping position), the near gripping position (predetermined position near the gripping position), and the gripping direction are transmitted to the robot arm 500 and the robot hand 600 (S4).

The gripping position is a position where the robot hand waits or passes before the operation of gripping the tip member, and is a position that does not interfere with the electric wire or the connector. The position before gripping may be, for example, above, below, or to the side of the connector C1, and may be a position separated by a predetermined distance, or may be determined based on the three-dimensional shape of the connector C1.

The gripping direction (gripping posture) in which the robot hand 600 grips the connector C1 is preferably such that the gripping portion of the robot hand and the connector C1 form a substantially right angle. This is because the robot can be easily controlled when inserting the connector C1 into the connector housing C2 after gripping the connector C1. Preferably, the posture of the robot hand 600 is adjusted so that the gripping portion and the connector C1 are perpendicular to each other when the connector C1 is gripped at the near gripping position. After that, the robot hand 600 moves straight from the gripping position toward the gripping position, and grips the connector C1 after reaching the gripping position. This makes it possible to insert the connector C1 into the connector housing C2 in a state where the robot hand 600 is less likely to interfere with the connector housing C2.

Which position to hold should be set in advance. In the following description, it is assumed that the root portion of the connector C1 (near the contact point with the electric wire) is gripped.

Using the tip member position estimation system 200, the robot hand 600 is moved to the base of the connector C1 (S5). The robot hand 600 moves to the vicinity of the root portion of the connector C1 (position before gripping) and assumes a gripping posture (S5). Thereafter, the robot hand advances straight to reach the gripping position (S6), and determines whether or not the root portion of the connector C1 can be gripped (S7). Specifically, it is determined whether or not the open pair of grippers 610 of the robot hand 600 has reached the gripping position.

When the robot hand 600 determines that the root portion of the connector C1 cannot be gripped, the robot arm 500 and the robot hand 600 are stopped (S8), and the test is terminated.

When it is determined that the root of the connector C1 can be gripped by the robot hand 600, the pair of grippers 610 of the robot hand 600 are translated in the closing direction, and the robot hand 600 grips the root of the connector C1. (S9).

Next, the robot arm 500 moves the connector C1 to the connector orientation recognition station (ST) (S10). The connector direction recognition station means controlling the positions of the robot arm 500 and the robot hand 600 so that the front face of the connector C1 (FIG. 14) can be recognized. Specifically, using the area camera 300, the connector orientation recognition function is exhibited to calculate the rotation direction (rotation angle) of the connector (S11). When it is determined that the attachment to the connector housing C2 is impossible, the tip member gripping system 1000 is stopped (S12).

When it is determined that the connector housing C2 can be attached, the robot arm 500 and the robot hand 600 are used to move the connector C1 to the front of the connector housing C2 (S13).

Next, using the robot arm 500 and the robot hand 600, the connector C1 is rotated about the axis of rotation so that the rib C13 of the connector C1 matches the positioning recess C23 of the connector housing C2 (S14).

Next, the robot arm 500 and the robot hand 600 are used to insert the connector C1 into the connector housing C2 (S15). The rotation of the connector C1 (S14) may be performed simultaneously with the movement of the connector C1 to the front of the connector housing (S13). After that, a continuity test of the electric device 100 is performed (S16). With the above, the continuity test of the electrical device 100 is completed.

In the tip member gripping system 1000 described above, the robot hand 600 is used to grip the base of the connector C1. P1 may be grasped. The case where the robot hand 600 grips one portion at the base of the connector C1 has been described, but two or more robot hands 600 may be provided to grip not only the root portion of the connector C1 but also other portions of the electric wire 21. Interference of the electric wire 21 with other equipment may be avoided by holding it with another robot hand 600 .

As described above, according to the tip member position estimation method, the tip member gripping method, the tip member connection method, the tip member position estimation system, and the tip member gripping system of the present embodiment, the orientation of the tip member and, more preferably, the tip member It is possible to provide a linear object tip direction estimation method, a linear object tip direction estimation device, and a mounting automation device for grasping the rotation direction of the in advance.

A detailed structure of gripper 610 employed in robot hand 600 of the present embodiment will be described with reference to FIG. 18 is a perspective view showing the detailed structure of the gripper 610. FIG. A pair of grippers 610 are provided to face each other.

The gripper 610 includes a support portion 610a and arm portions 610b extending in opposite directions (inward) at the lower ends of the support portions 610a. A semi-cylindrical first groove portion 610c and a cylindrical second groove portion 610e communicating with the first groove portion 610c are provided on the distal end surface of the arm portion 610b. Since the diameter of the first groove portion 610c is larger than that of the second groove portion 610e, a locking surface 610d is provided between the first groove portion 610c and the second groove portion 610e.

When the robot hand 600 grips the plug C11, the arm 610b of the gripper 610 abuts against it, thereby holding the connector C1. The plug C11 is held by the first groove 610c, and the electric wire C12 is held by the second groove 610e, or the electric wire C12 is accommodated in the second groove 610e. When inserting the plug C11 into the connector housing C2, one end side of the plug C11 is pressed against the locking surface 610d and inserted. The connector C1 can be prevented from coming off from the gripper 610 by the contact of the plug C11 with the locking surface 610d.

A sheet-like elastic member (non-slip member) may be attached to the first groove portion 610c and/or the second groove portion 610e in order to increase the holding force for the plug C11.

It should be considered that the embodiments disclosed this time are illustrative in all respects and not restrictive. The scope of the present invention is indicated by the scope of the claims rather than the above description, and is intended to include all modifications within the meaning and range of equivalents of the scope of the claims.

10 three-dimensional measuring device, 11 stereo camera, 12 first camera, 13 second camera, 14 camera control unit, 15 calculation unit, 16 storage unit, 17 input/output unit, 21, 22, 23 electric wire (linear object), 30 first image 31 image of electric wire 21 on first image 32 image of electric wire 22 on first image 33 image of electric wire 23 on first image 34 first line image 40 second image 41 Image of wire 21 on second image 42 Image of wire 22 on second image 43 Image of wire 23 on second image 44 Second line image 45 Epipolar line 200 Tip member position estimation system 300 Area camera 400 Transparent wall 500 Robot arm 600 Robot hand 610 Gripper 610a Support part 610b Arm part 610c First groove part 610d Locking surface 610e Second groove part 700 Housing 800 Wiring object arrangement Base, 1000 tip gripping system, C1 connector, C11 body, C12 pin, C13 rib, C2 connector housing, C21 housing, C22 pin receiver, C23 positioning recess, P point on wire 21, Q first image of point P (point of interest), R Projection of point P onto the second image (corresponding point).

Claims (9)

A tip member position estimation method for a linear object having a tip member at its tip,
a coordinate acquisition step of acquiring coordinates of two or more points on the linear object;
a tip member position estimation step of estimating the position of the tip member based on the coordinates of two or more points on the linear object;
with
Further comprising a rotation angle calculation step of obtaining an image of the tip member and calculating a rotation angle of the tip member based on the image,
Tip member position estimation method. In the step of estimating the position of the tip member, a step of calculating an approximate straight line of the linear object based on the coordinates of two or more points on the linear object;
estimating the position of the tip member based on the approximate straight line;
The tip member position estimation method according to claim 1. The coordinate acquisition step includes a step of acquiring coordinates of three or more points on the linear object,
In the step of estimating the position of the tip member, a step of calculating an approximate curve of the linear object based on coordinates of three or more points on the linear object;
estimating the position of the tip member based on the approximated curve;
The tip member position estimation method according to claim 1. The coordinate acquisition step includes a step of acquiring the coordinates of the tip of the linear object.
The tip member position estimation method according to any one of claims 1 to 3. The coordinates on the linear object are three-dimensional coordinates,
The tip member position estimation method according to any one of claims 1 to 4. A method for gripping a tip member provided at the tip of a linear object with a robot hand,
a coordinate acquisition step of acquiring coordinates of two or more points on the linear object with a measuring device;
a tip member gripping position calculation step of calculating a gripping position of the tip member based on the coordinates of two or more points on the linear object;
a gripping step in which the robot hand grips the tip member at the gripping position;
A tip member gripping method comprising: A tip member connection method,
a step of gripping the tip member by the robot hand in the tip member gripping method according to claim 6 ;
acquiring an image of the tip member gripped by the robot hand, and calculating a rotation angle of the tip member from the image;
using the robot hand to connect the tip member to the connection member based on the rotation angle;
A tip member connection method. a measuring device for acquiring coordinates of two or more points on a linear object;
a calculation unit for estimating the position of a tip member provided at the tip of the linear object based on the coordinates of two or more points on the linear object;
with
an image acquisition means for capturing an image of the tip member;
rotation angle calculation means for calculating the rotation angle of the tip member based on the image obtained by the image acquisition means;
A tip member position estimation system , further comprising : A tip member gripping system for a linear object having a tip member at its tip,
a measuring device for acquiring coordinates of two or more points on the linear object;
a calculation unit for estimating the position of the tip member provided at the tip of the linear object based on the coordinates of two or more points on the linear object;
a gripping unit that grips the tip member based on the estimated position of the tip member;
a tip member grasping system.

Images