JP2020112470A - Tip member position estimating method, tip member holding method, tip member connecting method, tip member position estimating system and tip member holding system - Google Patents

Abstract

To provide a tip member position estimating method, a tip member holding method, a tip member connecting method, a tip member position estimating system and a tip member holding system for grasping a position of a tip member and further preferably, a rotation direction of the tip member.SOLUTION: A tip member position estimating method is a method for estimating a position of a tip member of an electric wire 21 having a connector C1 at a tip thereof, and includes a coordinate acquisition step of acquiring two or more coordinates on the electric wire 21 and a tip member position estimating step of estimating a position of the connector C1 on the basis of the two or more coordinates on the electric wire 21.SELECTED DRAWING: Figure 12

Description

The present invention relates to a tip member position estimating method, a tip member gripping method, a tip member connecting method, a tip member position estimating system, and a tip member gripping system.

A robot that recognizes an object with a three-dimensional camera or the like and autonomously grips the object is becoming widespread. Regarding gripping a linear object, for example, in Japanese Patent Application Laid-Open No. 2014-176917 (Patent Document 1), there is a robot device for assembling a linear object, and a fixed end of the linear object whose one end is fixed. There is described a device that grips the vicinity and then slides the grip part along a predetermined locus to move it to the other end. This makes it possible to quickly grasp the other end, which is difficult to accurately estimate due to the habit or the like attached to the electric wire, which is an example of the linear object.

Japanese Patent Laying-Open No. 2016-192138 (Patent Document 2) discloses an invention relating to a method for manufacturing a wire harness and an image processing method, and in the process of manufacturing the wire harness, the three-dimensional shape of the wire assembly is measured. A processing position specifying process for specifying the processing position is performed.

JP, 2014-176917, A JP, 2016-192138, A

A connector or the like is often connected to the tip of a linear object such as an electric wire. The grip of the robot hand, for example, the tip of the linear object or the connector is directly gripped by a gripper, and the connector is moved to a predetermined position and fixed, or the connector is attached to the connector housing and the connector is connected. Conducting a continuity test of the installed equipment is assumed.

Conventionally, by registering CAD data of the connector in advance and pattern matching the measured connector shape with the registered data, the position and orientation of the connector are recognized, and the robot grips and moves the connector. It was However, there are various shapes of connectors, and it is difficult to register all shape data in advance. Further, there is a problem that the pattern matching processing between the three-dimensional data is heavy and takes a long time for calculation.

Further, when the connector is attached to the connector housing, it is necessary to grasp the rotation direction around the axis of the linear object of the connector gripped by the robot hand.

When the tip member such as the connector attached to the tip of the linear object is attached to the predetermined connecting member in this manner, the position of the tip member, and more preferably, the rotation direction of the tip member is grasped in advance. It is necessary to keep it.

The present invention is intended to solve the above problems, the position of the tip member from the shape of the tip of the linear object, and more preferably for grasping the rotation direction of the tip member, a tip member position estimation method, A tip member gripping method, a tip member connecting method, a tip member position estimation system, and a tip member gripping system.

The tip member position estimating method according to the present disclosure is a tip member position estimating method for a linear object having a tip member at the tip, which includes a coordinate acquisition step of acquiring coordinates of two or more points on the linear object, A tip member position estimating step of estimating the position of the tip member based on the coordinates of two or more points on the linear object.

In another aspect, the tip member position estimation step includes a step of calculating an approximate straight line of the linear object based on coordinates of two or more points on the linear object, and a step of calculating the approximate straight line of the linear object. Estimating the position of the tip member.

In another aspect, the coordinate acquisition step includes a step of acquiring coordinates of three or more points on the linear object, and the tip member position estimating step converts coordinates of three or more points on the linear object. Based on the approximate curve, and a step of estimating the position of the tip member based on the approximate curve.

In another aspect, the coordinate acquisition step includes a step of acquiring the coordinates of the tip of the linear object.

In another form, the coordinates on the linear object are three-dimensional coordinates.
In another aspect, a rotation angle calculating step of acquiring an image of the tip member and calculating a rotation angle of the tip member based on the image is further included.

The tip member gripping method according to the present disclosure is a method of gripping a tip member provided at the tip of a linear object with a robot hand, and a coordinate acquisition for acquiring coordinates of two or more points on the linear object by a measuring device. A step, a tip member gripping position calculation step of calculating a gripping position of the tip member based on coordinates of two or more points on the linear object, and a grip of the robot hand gripping the tip member at the grip position. And a process.

In the tip member connecting method according to the present disclosure, the step of the robot hand holding the tip member by the tip member holding method described above, and an image of the tip member held by the robot hand are acquired, and the image is acquired from the image. There is a step of calculating a rotation angle of the tip member, and a step of connecting the tip member to a connecting member based on the rotation angle using the robot hand.

In the tip member position estimation system according to the present disclosure, the tip of the linear object is determined based on a measuring device that acquires coordinates of two or more points on the linear object and coordinates of two or more points on the linear object. And a calculation unit that estimates the position of the provided tip member.

In another aspect, the image pickup device further includes an image acquisition unit that captures an image of the tip member, and a rotation angle calculation unit that calculates a rotation angle of the tip member based on an image obtained by the image acquisition unit.

The tip member gripping system according to the present disclosure is a tip member gripping system for a linear object having a tip member at the tip, and a measuring device that acquires coordinates of two or more points on the linear object, and the linear object. An arithmetic unit that estimates the position of the tip member provided at the tip of the linear object based on the coordinates of two or more points above, and grips the tip member based on the estimated position of the tip member. And a grip portion for

According to the present invention, a tip member position estimating method, a tip member gripping method, a tip member connecting method, and a tip member for grasping the position of the tip member from the shape of a linear object, and more preferably the rotational direction of the object. It is possible to provide a position estimation system and a tip member gripping system.

It is a functional block diagram of the three-dimensional measuring device of related technology. It is a figure for demonstrating the three-dimensional measuring method of related technology. It is a process flow figure of the three-dimensional measuring method of related technology. It is a figure which shows the 1st image imaged with the stereo camera. It is a figure which shows the 2nd image imaged with the stereo camera. It is an operation flow diagram of the 1st extraction process of the three-dimensional measuring method of related technology. It is a figure which shows the 1st image from which the 1st line image was extracted. It is a figure which shows the 1st image from which the 2nd line image was extracted. It is a figure which shows the 1st image in which the point of interest was selected. It is a figure which shows the 2nd image in which the intersection of an epipolar line and a 2nd line image was calculated|required. It is a figure which shows an example of a color table. It is a schematic diagram showing a tip member position presumption method of this embodiment. It is a figure which shows the electric equipment with which the connector was connected to the front-end|tip part of an electric wire. It is a figure which shows the front surface of a connector. It is a figure which shows the front of a connector housing. It is an overall view of the tip member gripping system of the present embodiment. It is a figure which shows the flow of the automation system of the connection process of this Embodiment. It is a perspective view which shows the detailed structure of the gripper employ|adopted for the robot hand of this Embodiment.

The present embodiment will be described below with reference to the drawings. In the embodiments described below, when reference is made to the number, amount, etc., the scope of the present invention is not necessarily limited to the number, amount, etc., unless otherwise specified. The same parts or corresponding parts are designated by the same reference numerals, and duplicate description may not be repeated. It is planned from the beginning that the configurations in the embodiments are appropriately combined and used. In the drawings, the actual dimensional ratios are not shown, but some of the ratios are different for easier understanding of the structure.

In the following disclosure, the case where an electric wire is used as an example of the linear object is described, but the present invention is not limited to the electric wire. The linear object in this description may be any object as long as it has an elongated shape. Examples of linear objects include electric wires, wire harnesses, solders, strings, threads, fibers, glass fibers, optical fibers, tubes, and other linear objects. The electric wire is not limited to an electric wire in which thin wires are bundled, and an electric wire or the like formed of a single wire is also included.

(Related technology: 3D measuring method and 3D measuring device for linear objects)
Hereinafter, an example of a three-dimensional measuring method and apparatus for linear objects will be described as a related technique with reference to FIGS. 1 to 11.

With reference to FIG. 1, the three-dimensional measuring device 10 includes a stereo camera 11, a calculation unit 15, a storage unit 16, and an input/output unit 17. The arithmetic unit 15, the storage unit 16, and the input/output unit 17 may be collectively referred to as a control unit.

The stereo camera 11 includes a first camera 12, a second camera 13, and a camera control unit 14. The first camera 12 is a color camera that captures a first image that is a two-dimensional color image. The second camera 13 is a color camera that captures a second image that is a two-dimensional color image, and its relative position with respect to the first camera 12 is fixed.

The camera control unit 14 controls the first camera 12 and the second camera 13, and communicates with the calculation unit 15. The camera control unit 14 receives, for example, an imaging instruction from the calculation unit 15, transmits the imaging instruction to the first camera 12 and the second camera 13, and transfers the first image and the second image to the calculation unit 15.

The calculation unit 15 calculates the three-dimensional position of the linear object by processing the first image and the second image received from the stereo camera 11, in addition to the communication with the camera control unit 14. The storage unit 16 stores the first image and the second image captured by the stereo camera 11, the color table of the target object, and also stores intermediate data and calculation results necessary for the calculation. The input/output unit 17 receives a command from the worker and displays the measurement result to the worker.

With reference to FIG. 2, in this measuring method, the electric wire 21, the electric wire 22, and the electric wire 23 are imaged by the first camera 12 and the second camera 13. If a projection point Q on the first image 30 by the first camera and a projection point R on the second image 40 by the second camera are obtained with respect to a certain point P on the electric wire 21, the known first camera The three-dimensional position of the point P can be calculated using the position information of the second camera. The position information of the first camera and the second camera can be obtained by calibrating the two cameras in advance.

Although FIG. 2 is drawn in black and white, the three electric wires 21, 22, and 23 to be measured are color-coded and have coatings of different colors, such as red, blue, and yellow. The linear object to be measured is not particularly limited as long as it is a linear object.

FIG. 3 is a flowchart of the three-dimensional measurement method. Hereinafter, each step will be described.
Create a color table prior to measurement. The color table is a table in which the color is recorded for each type of linear object that can be measured. FIG. 11 shows, as an example, a color table in which the color of each type of electric wire is represented by the luminance of the three primary colors of red, green, and blue (RGB). The color table is stored in the storage unit 16.

At the time of measurement, the stereo camera 11 images the electric wire 21, the electric wire 22, and the electric wire 23. The electric wire 21, the electric wire 22, and the electric wire 23 are captured in the first image 30 by the first camera 12. At the same time, the electric wire 21, the electric wire 22, and the electric wire 23 are captured by the second camera 13 in the second image 40 from a viewpoint different from that of the first camera. The first image and the second image are transferred to the calculation unit 15 and stored in the storage unit 16.

The calculation unit 15 acquires the first image 30 and the second image 40 from the stereo camera 11. At this time, referring to FIG. 4, the first image 30 includes images 31, 32 and 33 of the three electric wires 21, 22 and 23, respectively. Similarly, with reference to FIG. 5, the second image 40 includes images 41, 42, and 43 of the three electric wires 21, the electric wires 22, and the electric wires 23, respectively.

The calculation unit 15 extracts the specific electric wire 21 as a first line image on the first image 30. Referring to FIG. 6, the step of extracting the first line image (first line image extracting step) includes a color extracting operation, a binarizing operation, a noise removing operation, and a thinning operation.

In the extraction operation by color, the calculation unit 15 acquires the color of the electric wire 21 to be measured from the color table, and only the image 31 of the linear object 21 of the specific color on the first image 30 is the first line image. Extract as 34.

Specifically, the color of each pixel of the first image is compared with the specific color, and when both are judged to be the same, the pixel is left, and when both are judged to be different, the pixel is Erase. The determination as to whether the colors are the same or different can be made based on whether or not the difference between the two is less than or equal to a predetermined value.

For example, the RGB value corresponding to the electric wire 21 is acquired from the color table, the RGB value of each pixel of the first image 30 is compared with that, and if the difference between the RGB values is less than or equal to a predetermined value, the pixel is It is determined that the color is the same as that of the electric wire 21. The predetermined value can be determined in consideration of the number of RGB gradations, the degree of color difference between different types of electric wires, and the like.

Next, the first image 30 is binarized. This is an operation for replacing the value of each pixel with 0 or 1 by using an appropriate threshold value. The binarization operation facilitates subsequent image processing. The binarization operation may be performed simultaneously with the color extraction operation. Binarization can be performed by setting the pixels determined to have the same color to 1 and the pixels determined to have different colors to 0.

Next, the noise removal operation is performed on the first image 30. Although the first line image 34 is extracted by the above-described color extracting operation, isolated pixels due to shot noise of the camera remain in the first image. Since the position of the RGB image sensor with respect to one pixel is actually slightly deviated, the color of the image is changed at a portion where the color changes sharply, such as the contours of the images 31, 32 and 33 of each wire. May be disturbed or there may remain isolated pixels. A more accurate first line image 34 is obtained by removing such pixels.

Next, the first line image 34 is thinned. This is an operation for reducing the line width to 1 while maintaining the connectivity of the first line images. As a thinning operation method, a known method such as selecting a pixel located at the center of the line width can be used. As a result, subsequent image processing is facilitated, and corresponding points can be obtained more accurately.

FIG. 7 shows the obtained first line image 34. The first image 30 from which the first line image is extracted is stored in the storage unit 16.

Returning to FIG. 3, the same operation as the first image 30 is performed on the second image 40 to extract the second line image 44 (second line image extracting step). The second line image 44 is shown in FIG. The second image from which the second line image is extracted is stored in the storage unit 16.

Referring to FIG. 9, the calculation unit selects the point of interest Q on the first line image 34 of the first image 30. A point Q is a projection point of the point P (FIG. 2) of the electric wire 21 on the first image.

With reference to FIG. 10, the calculation unit obtains the epipolar line 45 corresponding to the point of interest Q of the first image 30 on the second image 40. An intersection R between the second line image 44 and the epipolar line 45 is obtained, and this is set as a point corresponding to the point of interest Q. The point R is a projection point of the point P (FIG. 2) of the electric wire 21 on the second image.

Through the above steps, the projection point Q on the first image 30 and the projection point R on the second image 40 are obtained for the point P of the electric wire 21 shown in FIG. Calculate the three-dimensional position of.

Next, a new point of interest is selected on the first line image 34, and the steps after the point of interest selection are repeated. As the next point of interest, an adjacent point connected to the previous point of interest can be selected. In this way, the three-dimensional measurement of the electric wire 21 is performed by shifting the point of interest Q, that is, by moving the point P on the electric wire 21 to obtain the three-dimensional position.

When the necessary information about the electric wire 21 is obtained, the repeating process is ended. When three-dimensionally measuring another electric wire, for example, the electric wire 22, the color of the electric wire 22 is acquired from the color table, and the first and second images captured by the first camera and the second camera are compared with each other. Then, the steps after the first line image extraction step are repeated.

Here, the color table will be described in more detail.
In the color table illustrated in FIG. 11, one RGB value is described for each type of linear object, but a plurality of RGB values are described for one type of linear object, and If it is determined that the color is the same as the RGB value of, the linear object may be determined. The colors may be recorded in a color system other than RGB. For example, it may be represented by L*, a*, b* based on the CIELAB color system established by the International Commission on Illumination (CIE). Even if the output from the stereo camera is RGB values, conversion between color systems is easy.

If the difference between the RGB value of the pixel and the RGB value of the color table is less than or equal to a predetermined value, it was determined that the color of the pixel and the color of the table were the same, but the range of colors determined to be the same You may record it on the table. When recording the color range, it is more preferable that the value is represented by the value of L*a*b* because it is easy to set a threshold range that is robust against the change in the light amount. For example, by keeping the threshold range of the L* value wide and narrowing the threshold range of the a* value and the b* value, even if the brightness of the linear object changes to a certain extent, it will not be confused with a cable of another color. Can be considered to be the same color.

The color table is preferably created based on the color of the image when the linear object is actually captured in the actual measurement environment. Specifically, by holding the linear object with a hand or a robot hand, the linear object is imaged while moving in various positions and directions in front of the first camera or the second camera, and the color information of the linear object is obtained from the image. get. The color of the linear object on the first image and the second image changes depending on various factors such as the type and arrangement of illumination in the measurement environment, the glossiness and direction of the linear object. By recording in the color table the colors of the range in which the image of the linear object can be taken under the actual measurement conditions, it is possible to reduce erroneous recognition when extracting the linear object.

For example, the linear object to be grasped in the above three-dimensional measuring method can be applied to a wire harness and various other linear objects in addition to the cable. The above steps and operations may be performed in a different order or omitted if possible.

The above-mentioned three-dimensional measurement method does not exclude the combined use with a known matching method in the stereo system. When there are a plurality of linear objects of the same color among a large number of linear objects, there is an advantage in using a matching method that focuses on the shape and other characteristics of the measurement tip member.

<First Embodiment of Tip Member Position Estimation Method>
Next, with reference to FIG. 12, a tip member position estimating method will be described. FIG. 12 is a schematic diagram showing the tip member position estimating method according to the present embodiment.

The above electric wire 21 will be described as an example. A connector C1 is connected in advance to the tip of the electric wire 21 as a tip member. The tip member is not limited to the connector, but may be any member that is provided at the tip of the linear object and has a different thickness, shape, or color from the linear object. The color of the electric wire 21 is determined based on the three-dimensional measuring method of the linear object. Further, as the point of interest of the electric wire 21, three-dimensional position information (three-dimensional coordinates) of P1, P2, P3, P4, and P5 at the tip of the electric wire 21 is obtained. In the figure, P1, P2, P3, P4, and P5 are described apart from each other for the sake of description, but actually, the points of interest may be close to each other. The length of the tip end portion for which the three-dimensional position information is to be grasped can be arbitrarily set, and the number of target points to be acquired can also be arbitrarily set. Three-dimensional coordinates of two or more points, preferably three or more points, may be acquired. One of the points of interest to be acquired is preferably the tip P21 of the electric wire 21, which is a contact point between the electric wire 21 and the tip member. This is because the position of the tip member can be more accurately estimated by grasping the starting point position of the tip member.

The electric wire 21 is colored in any one color such as red or is covered with a cover of any one color such as red, so that an accurate three-dimensional measurement method can be performed. If the color of the electric wire 21 and the color of the connector C1 are different, it is easier to recognize the electric wire 21 and the connector C1 separately, which is more preferable.

Next, the position of the connector C1 is estimated. Estimating the position of the connector C1 means estimating the attachment direction of the connector C1 attached to the electric wire 21. It is assumed that the mounting direction of the connector C1 coincides with the direction of the extension line of the tip portion of the electric wire 21. Next, the mounting direction of the connector C1 is estimated from the three-dimensional position information of P1, P2, P3, P4, and P5. When the electric wire is a flexible object and the connector C1 is a non-flexible object, the position and orientation of the connector C1 often follow the shape of the tip portion of the electric wire, and thus are estimated from the shape and position information of the tip portion of the electric wire. can do.

The following methods (i) to (iii) can be given as methods for estimating the mounting direction of the connector C1.

(I) A method of calculating an approximate straight line based on arbitrary two or more points of interest and estimating that the tip member is located on the calculated approximate straight line. Specifically, the direction (x line) in which the approximate straight line connecting the two points of interest P1 and P3 extends is estimated as the attachment direction of the connector C1.

(Ii) A method of calculating an approximate curve based on arbitrary three or more points of interest and estimating that the tip member is located on the calculated approximate curve. Specifically, the approximated curve of the electric wire 21 is measured using the points of interest P1 to P5, and this curve (y curve) is estimated to be the mounting direction of the connector C1 of the electric wire 21.

(Iii) A method of calculating an approximate curve based on arbitrary three or more points of interest and estimating that the tip member is located on the tangent line of the calculated approximate curve at the tip of the linear object. Specifically, the tangent line (z line) on the point of interest P21 of the curve (y curve) is estimated to be the mounting direction of the distal end connector C1 of the electric wire 21.

In the method of approximating the direction in which the x-ray extends as the mounting direction of the connector C1 by the method shown in (i), the calculation speed in the control device (for example, the calculation unit 15 in FIG. 1) is reduced because the number of points of interest used is small. Is fast. On the other hand, the method of approximating the extending direction of the x-ray as the mounting direction of the connector C1 by the methods shown in (ii) and (iii) is more accurate than the method shown in (i) because of the large number of points of interest to be used. Is considered high.

<Second Embodiment of Tip Member Position Estimation Method>
In the second embodiment of the tip member position estimation method, a linear object is imaged by the stereo camera 11, a first image and a second image are acquired, and a first line to be measured on the first image and the second image. The process up to the step of extracting the image and the second line image is the same as in the first embodiment.

Next, the first line image is extended on the first image, and the first tip member estimation point located at a predetermined distance from the end point of the first line image is obtained. Similarly, the second line image is extended on the second image, and the second tip member estimation point located at a predetermined distance from the end point of the second line image is obtained. Then, it is assumed that the first tip member estimation point and the second tip member estimation point are points where the tip member is located on each image.

The computing unit calculates the three-dimensional position of the connector C1 by using the first tip member estimation point and the second tip member estimation point as corresponding points of the two images in the stereo method.

<Third Embodiment of Tip Member Position Estimation Method>
When the linear object is along a predetermined platform or wall, the position of the tip member can be estimated by acquiring the two-dimensional information of the point of interest on the linear object. In this case, a linear object is imaged using a two-dimensional camera, and the attachment direction of the connector C1 is estimated by the same method as any one of the first embodiment (i) to (iii) of the tip member position estimation method. ..

<Tip member gripping method>
The mounting direction of the connector C1 is estimated by using any one of the methods of the first to third embodiments described above, and the method of gripping the tip member by the robot hand is determined. The specific gripping position by the robot hand may be the target point P21 of the electric wire 21, or a position on the connector C1 located at a predetermined distance (L1) from the target point P21 in the connector mounting direction estimated by the above method. May be determined as the gripping position.

Which of the first to third embodiments of the above-mentioned tip member position estimation method is used is determined by the thickness of the wire, the material of the wire, the hardness of the wire, the presence or absence of the core wire, the resolution of the camera, and the like. It

<Tip member position estimation system 200 and tip member gripping system 1000>
Next, with reference to FIG. 13 to FIG. 17, a tip member position estimating system 200 using the above tip member position estimating method and a tip member gripping system 1000 by a robot hand will be described.

FIG. 13 shows the electric device 100 in which the connector C1 is connected to the tip of the electric wire 21. Examples of the electric device 100 include a drive motor, an alternator, a battery, a compressor, an electric component of an automobile, a home electric appliance, and various other electric devices.

In the present embodiment, a step of automatically connecting the connector C1 connected to the electric device 100 via the electric wire 21 to the connector housing C2 by using a robot hand described later in order to conduct a continuity test of the electric device 100 ( The method of connecting the tip member) will be described.

14 shows a connector C1 and FIG. 15 shows a connector housing C2. Referring to FIG. 14, the connector C1 has a tubular body portion C11. It is estimated that the axial direction of the body portion C11 (the arrow A direction in FIG. 13) matches the estimated mounting direction of the connector C1.

Inside the body portion C11, a plurality of pins C12 are arranged at predetermined positions. A rib C13 extending along the axial direction of the body C11 is provided on the outer peripheral surface of the body C11 for positioning the connector C1. The state shown in FIG. 14 is a state in which the front of the connector C1 can be recognized. Since the front surface of the connector C1 can be recognized, the rotation state of the connector C1 (calculation of the rotation angle based on the position of the rib C13) can be recognized.

Referring to FIG. 15, the connector housing C2 has a housing C21 in which a pin receiver C22 is provided at a position corresponding to the pin C12. The housing C21 is provided with a positioning recess C23 into which the rib C13 is inserted.

Next, with reference to FIGS. 16 and 17, an example of automation of mounting the connector C1 on the connector housing C2 using the robot hand 600 provided on the robot arm 500 will be described. FIG. 16 is an overall view of the tip member gripping system 1000, and FIG. 17 is a view showing the flow of the mounting process automation system.

The tip member gripping system 1000 is controlled by a tip member position estimating system 200 described later. This tip member position estimation system 200 is the same as the three-dimensional measuring apparatus 10 described in FIG. Hereinafter, the description of the tip member position estimation system 200 may be given the same reference numerals as those of the three-dimensional measuring apparatus 10 shown in FIG. 1.

With reference to FIG. 16, the tip member gripping system 1000 has a housing 700. The transparent wall 400 is disposed on the top of the housing 700 so that the mounting process inside can be visually checked. Inside the housing 700, the robot 900 and the object placement table 800 are placed at predetermined positions. The robot 900 includes a robot arm 500, and a robot hand 600 is attached to the tip of the robot arm 500. On the object placement table 800, the electric device 100 having the connector C1 shown in FIG. 13 and the connector housing C2 are placed.

Further, at a predetermined position inside the housing 700, a tip member position estimation system 200 including the stereo camera 11 and the like, and an area camera 300 are arranged. The area camera 300, which will be described in detail later, functions as a front image acquisition unit and is used when measuring the orientation (rotation angle) of the connector C1 by image processing. The front image information obtained from the area camera 300 is input to the three-dimensional measuring device 10. The three-dimensional measuring device 10 includes a rotation angle calculation unit that calculates the rotation angle of the connector C1 based on this front image. This rotation angle calculation means may be provided in the calculation unit 15 or a rotation angle calculation unit.

Although the case where the area camera 300 of the two-dimensional camera is adopted as the front image acquisition unit has been described, the stereo camera 11 may also be used as the front image acquisition unit.

For the robot arm 500, for example, a FANUC Robot LR Mate 200iD/7L (load capacity 7 kg, reach length 900 mm) is adopted. A pair of grippers 610 is provided at the tip of the robot hand 600, and parallel opening and closing is performed by an air cylinder. The gripper 610 has a width of about 10 mm and a length of about 20 mm. The opening/closing stroke of the gripper 610 is about 10 mm.

The tip member position estimation system 200 uses a 3D vision for linear object recognition made by Kurabo Industries. The distance between the object and the stereo camera 11 is about 500 mm, the visual field range is 400 mm×250 mm, and the depth of focus is ±100 mm. As a linear object gripping position recognition function, it is used when outputting the gripping position and the like, and the robot hand posture.

As the area camera 300, a Fanuc area camera or a Balser area camera (Dart) is used. The visual field range is about 200 mm×150 mm. It is used as a connector direction recognition function when outputting the rotation angle of the connector.

Next, with reference to FIG. 17, a mounting process (wiring automation process) of the connector C1 to the connector housing C2 using the tip member gripping system 1000 will be described. Note that the following flow is executed by the camera control unit 14 and/or the calculation unit 15 provided in the tip member position estimation system 200.

Using the tip member position estimation system 200, the tip portion of the electric wire 21 and the connector C1 are 3D-scanned (step 1 (referred to as S1. The same applies hereinafter)). The connector C1 is not essential as long as the tip portion of the electric wire 21 can be 3D-scanned. Based on the obtained 3D image information, it is determined whether or not the connector C1 is located at a position where the connector C1 can be held (S2). Specifically, the three-dimensional measuring device 10 is used to perform the grip position recognizing function of the tip portion to confirm the grip position of the tip portion. When it is determined that the connector C1 cannot be gripped, the test ends (S3).

When it is determined that the connector C1 can be gripped, whether the position of a predetermined distance (L1) from the root of the connector C1 is gripped, or the point of interest P1 of the electric wire near the root of the connector C1 is gripped. Information (grasping position), the gripping front position (predetermined position near the gripping position), and the gripping direction are transmitted to the robot arm 500 and the robot hand 600 (S4).

The gripping front position is a position where the robot hand waits or passes before the operation of gripping the tip member, and is a position where it does not interfere with the electric wire or the connector. The gripping front position may be, for example, above, below, or to the side of the connector C1 and separated by a predetermined distance, or may be determined based on the three-dimensional shape of the connector C1.

In the gripping direction (gripping posture) in which the robot hand 600 grips the connector C1, it is preferable that the robot hand 600 grips the connector C1 at a substantially right angle with the grip portion of the robot hand. This is because the robot can be easily controlled even when the connector C1 is gripped and then inserted into the connector housing C2. Preferably, at the gripping front position, the posture of the robot hand 600 is adjusted so that the gripping portion and the connector C1 form a right angle when gripping the connector C1. After that, the robot hand 600 goes straight from the front gripping position toward the gripping position, and after reaching the gripping position, grips the connector C1. As a result, the connector C1 can be inserted into the connector housing C2 in a state where the robot hand 600 is less likely to interfere with the connector housing C2.

Which position to hold may be set in advance. In the description below, the base of the connector C1 (near the contact point with the electric wire) is gripped.

The robot hand 600 is moved to the base of the connector C1 using the tip member position estimation system 200 (S5). The robot hand 600 moves to the vicinity of the base of the connector C1 (position near the grip) and takes a grip posture (S5). After that, the robot hand goes straight to reach the grip position (S6), and determines whether or not the root portion of the connector C1 can be gripped (S7). Specifically, it is determined whether or not the pair of open grippers 610 of the robot hand 600 has reached the grip position.

When the robot hand 600 determines that the base of the connector C1 cannot be gripped, the operations of the robot arm 500 and the robot hand 600 are stopped (S8), and the test is ended.

When the robot hand 600 determines that the base portion of the connector C1 can be gripped, the pair of grippers 610 of the robot hand 600 are moved in parallel in the closing direction, and the robot hand 600 grips the base portion of the connector C1. (S9).

Next, the robot arm 500 moves the connector C1 to the connector direction recognition station (ST) (S10). The connector direction recognition station means controlling the positions of the robot arm 500 and the robot hand 600 to a position where the front surface (FIG. 14) of the connector C1 can be recognized. Specifically, using the area camera 300, the connector direction recognition function is exerted to calculate the rotation direction (rotation angle) of the connector (S11). When it is determined that the connector housing C2 cannot be mounted, the tip end member gripping system 1000 is stopped (S12).

When it is determined that the connector C1 can be attached to the connector housing C2, the connector C1 is moved to the front of the connector housing C2 using the robot arm 500 and the robot hand 600 (S13).

Next, using the robot arm 500 and the robot hand 600, the rib C13 of the connector C1 is rotated so that the position of the rib C13 matches the positioning recess C23 of the connector housing C2 with the axis of the connector C1 as the rotation center axis (S14).

Next, the connector C1 is inserted into the connector housing C2 using the robot arm 500 and the robot hand 600 (S15). The rotation of the connector C1 (S14) may be performed simultaneously when the connector C1 is moved to the front of the connector housing (S13). Then, the continuity test of the electric device 100 is performed (S16). With the above, the continuity test of the electric device 100 is completed.

In the above-described tip member gripping system 1000, the case where the root part of the connector C1 is gripped using the robot hand 600 has been described, but the focus points of the electric wires near the center part of the connector C1 and the root part of the tip part connector C1. You may hold P1. Although the robot hand 600 has described the case of gripping one part of the base of the connector C1, two or more robot hands 600 are provided to hold not only the base of the connector C1 but also other parts of the electric wire 21. By gripping with another robot hand 600, the interference of the electric wire 21 with other devices may be avoided.

As described above, according to the tip member position estimating method, tip member gripping method, tip member connecting method, tip member position estimating system, and tip member gripping system in the present embodiment, the orientation of the tip member, and more preferably the tip member. It is possible to provide a linear object tip direction estimation method, a linear object tip direction estimation device, and a mounting automation device for grasping the rotation direction of the object in advance.

With reference to FIG. 18, a detailed structure of the gripper 610 adopted in the robot hand 600 of the present embodiment will be described. FIG. 18 is a perspective view showing the detailed structure of the gripper 610. The grippers 610 are provided in a pair so as to face each other.

The gripper 610 includes a support portion 610a and an arm portion 610b extending in the mutually opposing directions (inward) at the lower end of the support portion 610a. A semi-cylindrical first groove portion 610c and a cylindrical second groove portion 610e communicating with the first groove portion 610c are provided on the tip surface of the arm portion 610b. Since the first groove portion 610c has a larger diameter than the second groove portion 610e, a locking surface 610d is provided between the first groove portion 610c and the second groove portion 610e.

When gripping the plug C11 with the robot hand 600, the arm portion 610b of the gripper 610 abuts, so that the connector C1 is held. The plug C11 is held by the first groove 610c and the electric wire C12 is held by the second groove 610e, or the electric wire C12 is settled in the second groove 610e. When the plug C11 is inserted into the connector housing C2, one end of the plug C11 is pressed against the locking surface 610d and inserted. When the plug C11 contacts the locking surface 610d, the connector C1 can be prevented from coming off from the gripper 610.

A sheet-shaped elastic member (slip prevention member) may be attached to the first groove portion 610c and/or the second groove portion 610e in order to increase the holding force with respect to the plug C11.

The embodiments disclosed this time are to be considered as illustrative in all points and not restrictive. The scope of the present invention is shown not by the above description but by the claims, and is intended to include meanings equivalent to the claims and all modifications within the scope.

10 three-dimensional measuring device, 11 stereo camera, 12 first camera, 13 second camera, 14 camera control unit, 15 arithmetic unit, 16 storage unit, 17 input/output unit, 21, 22, 23 electric wire (linear object), 30 first image, 31 image of electric wire 21 on first image, 32 image of electric wire 22 on first image, 33 image of electric wire 23 on first image, 34 first line image, 40 second image, 41 Image of electric wire 21 on second image, 42 Image of electric wire 22 on second image, 43 Image of electric wire 23 on second image, 44 Second line image, 45 Epipolar line, 200 Tip member position estimation system, 300 Area camera, 400 transparent wall, 500 robot arm, 600 robot hand, 610 gripper, 610a support portion, 610b arm portion, 610c first groove portion, 610d locking surface, 610e second groove portion, 700 housing, 800 wiring object placement Platform, 1000 Tip member gripping system, C1 connector, C11 body, C12 pin, C13 rib, C2 connector housing, C21 housing, C22 pin receiver, C23 positioning recess, P Point on electric wire 21, first image of Q point P To the second image (corresponding point).

Claims (11)

A method for estimating the position of the tip member of a linear object having a tip member at the tip,
A coordinate acquisition step of acquiring coordinates of two or more points on the linear object;
A tip member position estimating step of estimating the position of the tip member based on coordinates of two or more points on the linear object;
And a tip member position estimating method. The tip member position estimating step of calculating an approximate straight line of the linear object based on the coordinates of two or more points on the linear object;
Estimating the position of the tip member based on the approximate straight line.
The tip member position estimating method according to claim 1. The coordinate acquisition step includes a step of acquiring coordinates of three or more points on the linear object,
The tip member position estimating step of calculating an approximate curve of the linear object based on coordinates of three or more points on the linear object;
Estimating the position of the tip member based on the approximated curve.
The tip member position estimating method according to claim 1. The coordinate acquisition step includes a step of acquiring the coordinates of the tip of the linear object,
The tip member position estimating method according to any one of claims 1 to 3. The coordinates on the linear object are three-dimensional coordinates,
The tip member position estimation method according to any one of claims 1 to 4. Further comprising a rotation angle calculation step of acquiring an image of the tip member and calculating a rotation angle of the tip member based on the image,
The tip member position estimation method according to any one of claims 1 to 5. A method of gripping a tip member provided at the tip of a linear object with a robot hand,
A coordinate acquisition step of acquiring coordinates of two or more points on the linear object with a measuring device;
A tip member gripping position calculation step of calculating a gripping position of the tip member based on coordinates of two or more points on the linear object;
A gripping step in which the robot hand grips the tip member at the gripping position,
A method for holding a tip member, comprising: A method for connecting a tip member,
A step of gripping the tip member by the robot hand using the tip member gripping method according to claim 7;
Acquiring an image of the tip member gripped by the robot hand, and calculating a rotation angle of the tip member from the image,
Using the robot hand, based on the rotation angle, connecting the tip member to a connecting member,
And a method for connecting a tip member. A measuring device for acquiring coordinates of two or more points on a linear object,
A computing unit that estimates the position of the tip member provided at the tip of the linear object based on the coordinates of two or more points on the linear object;
A tip member position estimation system comprising: An image acquisition unit that images the tip member,
Based on the image obtained by the image acquisition means, rotation angle calculation means for calculating the rotation angle of the tip member,
The tip member position estimation system according to claim 9, further comprising: A tip member gripping system for a linear object having a tip member at the tip,
A measuring device for acquiring coordinates of two or more points on the linear object;
A computing unit that estimates the position of the tip member provided at the tip of the linear object based on the coordinates of two or more points on the linear object;
A gripping portion that grips the tip member based on the estimated position of the tip member;
A tip member gripping system.

Images