JP2021085751A - Method for determining work position, method for controlling robot, work position determination device, and robot control system - Google Patents

Abstract

To provide a method for recognizing a marker on an object and determining a work position.SOLUTION: The method for determining a work position of the present invention includes: an imaging step of taking a first image and a second image of a marker on an object from different viewpoints; a region detection step of detecting the region of the marker in the first image and in the second image; a three-dimensional coordinate acquisition step of acquiring at least one three-dimensional coordinate of the extracted edge; and a work position determination step of determining a work position on the object on the basis of the three-dimensional coordinate.SELECTED DRAWING: Figure 2

Description

The present invention relates to a work position determination method, a robot control method, a work position determination device, and a robot control system.

Robots are used to automate the work of handling flexible planar objects such as cloth. For example, Patent Document 1 discloses a method of acquiring a three-dimensional shape of a piece of cloth such as a towel or sheets based on an image taken by a three-dimensional camera and gripping a corner portion of the piece of cloth. ..

JP-A-2018-114028

With the method described in Patent Document 1, it is difficult to handle an object having a shape other than a rectangle, such as a bag handle.

Therefore, it is an object of the present invention to provide a method of recognizing a marker provided on an object and determining a working position.

The working position determination method according to the present invention is
An imaging step of acquiring a first image and a second image of a marker provided on a flexible object from different viewpoints, and
In each of the first image and the second image, a region detection step of detecting the region of the marker and a region detection step.
A three-dimensional coordinate acquisition step of acquiring at least one three-dimensional coordinate of the detected marker area, and
A work position determination step of determining a work position with respect to the object based on the three-dimensional coordinates,
including.

According to the present invention, it is possible to provide a method of recognizing a marker provided on an object and determining a working position.

It is a block diagram which shows the structure of the work position determination apparatus and the robot control system which concerns on Embodiment 1. FIG. It is the schematic which shows the structure of the robot control system which concerns on Embodiment 1. FIG. It is the schematic of the bag provided with the 3D coordinate acquisition marker which is the object which performs the work by the robot control system which concerns on Embodiment 1. FIG. It is a flowchart of the robot control method which concerns on Embodiment 1. FIG. It is a figure explaining the image processing of the 1st image of the 3D coordinate acquisition marker imaged by the 1st camera. It is a figure explaining the binarization process of the 1st image of the 3D coordinate acquisition marker imaged by the 1st camera. It is a figure explaining the process of extracting an edge in the 1st image of the 3D coordinate acquisition marker imaged by the 1st camera. It is a figure explaining the process of extracting the end point in the 1st image of the 3D coordinate acquisition marker imaged by the 1st camera. It is a figure explaining the process of extracting the end point in the 1st image of the 3D coordinate acquisition marker imaged by the 1st camera. It is a figure which shows the 3D coordinate of the edge of the marker for 3D coordinate acquisition. It is a figure which shows the example of the case where the marker for three-dimensional coordinate acquisition is a tape.

The work position determination method according to the first aspect is
An imaging step of acquiring a first image and a second image of a marker provided on a flexible object from different viewpoints, and
In each of the first image and the second image, a region detection step of detecting the region of the marker and a region detection step.
A three-dimensional coordinate acquisition step of acquiring at least one three-dimensional coordinate of the detected marker area, and
A work position determination step of determining a work position with respect to the object based on the three-dimensional coordinates,
including.

According to this configuration, the working position can be determined by recognizing the marker provided on the object. Further, the working position can be easily determined even for a flexible object.

The work position determination method according to the second aspect is
The three-dimensional coordinate acquisition step includes acquiring three-dimensional coordinates of at least one point of the edge of the marker.

The work position determination method according to the third aspect is
The three-dimensional coordinate acquisition step may include acquiring the three-dimensional coordinates of one end point of the edge.

With these configurations, the working position can be determined more easily. In the case of a flexible object, since the shape is not constant, it is difficult to match the three-dimensional CAD data with the three-dimensional coordinates acquired by the first image and the second image. However, as in the present invention, the working position can be easily determined by acquiring the three-dimensional coordinates of the edge of the marker provided on the object.

The work position determination method according to the fourth aspect is
The edge may include a first edge and a second edge.

According to this configuration, since the three-dimensional coordinates of the two edges of the marker are acquired, the rough surface shape and orientation of the marker can be acquired. Therefore, the working position with respect to the object to which the marker is attached can be determined more accurately.

The work position determination method according to the fifth aspect is
The object has a handle and
The marker may be provided on the handle of the object.

The position-fixing method according to the sixth aspect is
The marker may have edges substantially parallel to the longitudinal direction of the handle of the object.

According to these configurations, since the three-dimensional coordinates of the two edges of the marker provided on the object are acquired, the working position with respect to the marker can be determined more accurately. Further, since the marker is provided on the handle, the handle or the portion close to the handle can be determined as the working position, and the robot can work on the handle or the portion close to the handle.

The robot control method according to the seventh aspect is
An imaging step of acquiring a first image and a second image of a marker provided on a flexible object from different viewpoints, and
An edge extraction step of detecting the marker region in each of the first image and the second image, and
A three-dimensional coordinate acquisition step of acquiring at least one three-dimensional coordinate of the detected marker area, and
A work position determination step of determining a work position with respect to the object based on the three-dimensional coordinates,
The work process of causing the robot to work with respect to the work position,
including.

According to this configuration, the robot can be made to perform the work by determining the work position with respect to the object to which the marker is attached and instructing the robot to perform the work. In particular, it is more effective for flexible objects.

The work position determining device according to the eighth aspect is
A first camera and a second camera that image a marker provided on a flexible object from different viewpoints,
A region detection unit that detects the region of the marker in each of the images captured by the first camera and the second camera, and
A three-dimensional coordinate acquisition unit that acquires three-dimensional coordinates of at least one point in the detected marker area, and
A work position determination unit that determines a work position with respect to the object based on the three-dimensional coordinates.
To be equipped.

According to this configuration, the working position can be determined by recognizing the marker provided on the object. In particular, it is more effective for flexible objects.

The robot control system according to the ninth aspect is
The work position determining device according to the eighth aspect and
A robot having a robot hand that performs work on the work position and
To be equipped.

According to this configuration, the robot can be made to execute the work by determining the work position with respect to the marker provided on the object and instructing the robot on the work position. It is more effective especially for flexible objects.

The bag according to the tenth aspect is
With the bag body
At least one handle provided on the bag body and
The three-dimensional coordinate acquisition marker attached to the handle,
To be equipped.

According to this configuration, the work position with respect to the handle of the bag or the vicinity of the handle can be determined by the three-dimensional coordinate acquisition marker provided on the handle, and the robot can execute the work such as grasping the handle of the bag.

(Embodiment 1)
Hereinafter, the work position determination method, the robot control method, the work position determination device, and the robot control system according to the embodiment will be described with reference to the accompanying drawings. In the drawings, substantially the same members are designated by the same reference numerals.

[overall structure]
FIG. 1 is a block diagram showing a configuration of a work position determining device and a robot control system according to the first embodiment. FIG. 2 is a schematic view showing the configuration of the robot control system according to the first embodiment. FIG. 3 is a schematic view of a bag including a marker for acquiring three-dimensional coordinates to be executed by the robot control system according to the first embodiment.

The work position determining device 2 of the present embodiment includes the first camera 11, the second camera 12, the area detection unit 13, the edge extraction unit 14, the end point extraction unit 15, the three-dimensional coordinate acquisition unit 16, and the work. A positioning unit 17 is provided. Further, as shown in FIG. 1, the robot control system 1 of the present embodiment includes a work position determining device 2, a robot 3, and a robot operation unit 4. In the present embodiment, as shown in FIG. 2, the three-dimensional coordinate acquisition marker 23 provided on the bag 22 placed on the measurement stage 21 is imaged by the cameras 11 and 12 of the working position determining device 2. Based on the images obtained by the cameras 11 and 12, the working position determining device 2 acquires the three-dimensional coordinates of the three-dimensional coordinate acquisition marker 23, and determines the working position of the robot 3. Based on the determined work position, the robot operation unit 4 controls the robot 3 to cause the three-dimensional coordinate acquisition marker 23 to perform the work. The bag 22 corresponds to the "object" of the present invention, and the three-dimensional coordinate acquisition marker corresponds to the "marker" of the present invention.

As shown in FIG. 3, the bag 22 is attached to the bag main body 22a, at least one handle 22b attached to the opening of the bag main body 22a, and the handle 22b for acquiring three-dimensional coordinates having different brightness from the handle 22b. It includes a marker 23. The bag body 22a and the handle 22b are made of a flexible material. Further, the three-dimensional coordinate acquisition marker 23 can be formed of, for example, a rigid material such as synthetic resin, for example, plastic. Alternatively, the three-dimensional coordinate acquisition marker 23 may be formed by wrapping a tape around the handle 22b or coloring or printing a color different from that of the handle 22b. For example, a straight line having a color different from that of the handle 22b can be printed at the center position of the handle 22b in the longitudinal direction, and this straight line can be used as the three-dimensional coordinate acquisition marker 23. In these cases, a marker can be attached to the bag later, so even if the object is a commercially available product, the work position can be easily recognized by attaching a marker for acquiring three-dimensional coordinates. Become. Further, at the time of manufacturing the bag body, the three-dimensional coordinate acquisition marker 23 may be formed by sewing a fabric having a color different from that of the bag or weaving a thread having a color different from that of the bag into the bag.

The position where the three-dimensional coordinate acquisition marker 23 is attached is not limited to the handle 22b of the bag 22. It should be installed in a position that should be recognized for work. As the three-dimensional coordinate acquisition marker 23, a marker having a surface such as a rectangle or a trapezoid, which includes two or more straight lines, can be used. Preferably, if a rectangular parallelepiped having rigidity or low flexibility and being hard to be deformed is used, it is easy to obtain three-dimensional coordinates. When the marker 23 for acquiring the three-dimensional coordinates is wrapped around a flexible tape or the like, the marker is not easily deformed or destroyed even if the marker portion is gripped by the robot hand, which is preferable depending on the work content. Further, the three-dimensional coordinate acquisition marker 23 may have a shape having edges substantially parallel to the longitudinal direction of the handle 22b.

When two or more handles 22b are attached as in the bag 22 shown in FIG. 3, the three-dimensional coordinate acquisition marker 23 may be attached to at least one handle 22b. Further, the three-dimensional coordinate acquisition marker 23 may be attached to all the handles 22b. The robot control system 1 of the present embodiment operates the robot 3 to perform the work of grasping the three-dimensional coordinate acquisition marker 23 attached to the handle 22b of the bag 22. The work of having the robot 3 operate is not limited to this. Further, in the present embodiment, the bag 22 having the handle is the object, but the object is not limited to the bag, and various shapes can be adopted.

<Working position determination device>
The work position determining device 2 acquires the three-dimensional coordinates of the three-dimensional coordinate acquisition marker 23 attached to the handle 22b of the bag 22, and based on the acquired three-dimensional coordinates, the work position Wp of the robot 3 (see FIG. 10). To determine. Here, the three-dimensional coordinates of the three-dimensional coordinate acquisition marker 23 refer to the coordinates of any of the detected coordinates in the area of the three-dimensional coordinate acquisition marker 23. For example, the coordinates of the edge of the three-dimensional coordinate acquisition marker 23, the coordinates of the center of gravity, and the like are included. In the present embodiment, as shown in FIG. 10 described later, the three-dimensional coordinates of the edge E1 and the edge E2 are acquired as the three-dimensional coordinates of the three-dimensional coordinate acquisition marker 23. The edge of the marker for acquiring three-dimensional coordinates is the ridgeline of the shape of the marker for acquiring three-dimensional coordinates and the boundary of colors in the image. Since the handle 22b of the bag 22 is a band-shaped object, the approximate working position Wp by the robot 3 is determined by acquiring the three-dimensional coordinates of the two edges of the three-dimensional coordinate acquisition marker 23 as shown in FIG. can do.

The "work" includes holding work such as gripping, pinching, or suctioning the object, processing work such as painting, marking, welding, cutting, component mounting, sewing, or soldering on the object, or folding work. "Working position" means that a person or robot holds, holds, or sucks an object, paints, marks, welds, cuts, attaches parts, sews, solders, etc., or folds. Refers to the three-dimensional coordinates that are the target position of the work when the work is performed.

Each component constituting the work position determining device 2 will be described below.

<Camera>
The first camera 11 and the second camera 12 take images of the three-dimensional coordinate acquisition marker 23 attached to the handle 22b of the bag 22 placed on the measurement stage 21 from different viewpoints. The three-dimensional coordinates of the three-dimensional coordinate acquisition marker 23 are acquired based on the first image captured by the first camera 11 and the second image captured by the second camera 12. That is, the working position determining device 2 acquires the three-dimensional coordinates of the edges E1 and E2 of the three-dimensional coordinate acquisition marker 23 by the stereo method. When the bag 22 and the handle 22b have flexibility, the effect of the present invention can be more exerted if the three-dimensional coordinate acquisition marker 23 has rigidity or is made of a material having low flexibility and is not easily deformed. Can be done.

<Control unit>
The control unit 18 performs image processing of the first image and the second image. The control unit 18 is, for example, a computer device. As the computer device, a general-purpose device can be used. The control unit 18 includes, for example, a processing unit 18a and a storage unit 18b, as shown in FIG. The control unit 18 may further include a display unit, an input device, a storage device, an interface, and the like.

<Processing unit>
The processing unit 18a may be, for example, a central processing unit (CPU, MPU, etc.), a microcomputer, or a processing device capable of executing instructions that can be executed by the computer.

<Memory>
The storage unit 18b may be, for example, at least one of a ROM, an EEPROM, a RAM, a flash SSD, a hard disk, a USB memory, a magnetic disk, an optical disk, a magneto-optical disk, and the like. The storage unit 18b includes a region detection unit 13, an edge extraction unit 14, an end point extraction unit 15, a three-dimensional coordinate acquisition unit 16, and a work position determination unit 17, which will be described later, as programs that can be executed by the processing unit 18a. It is read from the storage unit 18b and executed by the processing unit 18a.

<Region detector>
The area detection unit 13 detects the area of the three-dimensional coordinate acquisition marker 23 in each of the first image and the second image. For example, the region of the three-dimensional coordinate acquisition marker 23 can be detected by the binarization process. Alternatively, the contour of the region of the three-dimensional coordinate acquisition marker 23 may be detected by raster scan. Alternatively, the region of the three-dimensional coordinate acquisition marker 23 may be detected by another known method.

<Edge extraction section>
The edge extraction unit 14 extracts the edges E1 and the edges E2 of the three-dimensional coordinate acquisition marker 23 from the respective regions of the first image and the second image. The edge E1 and the edge E2 can be extracted by a known method such as a difference calculation method for taking the difference between adjacent pixels.

<End point extractor>
The end point extraction unit 15 extracts one end point of an edge in each of the first image and the second image. As shown in FIG. 9 described later, one end point of the edge is an intersection (end point I1) between the two edges E1 and the edge E2 of the three-dimensional coordinate acquisition marker 23 and the lower end portion of the three-dimensional coordinate acquisition marker 23. , I2). The end points can be extracted by a known method such as a differential calculation method between adjacent pixels.

<Three-dimensional coordinate acquisition unit>
The three-dimensional coordinate acquisition unit 16 acquires the three-dimensional coordinates of at least one point of the edge E1 or the edge E2 based on the end points of the first image and the second image, respectively. In the present embodiment, the three-dimensional coordinates include the three-dimensional coordinates of at least one point of the edge E1 and the three-dimensional coordinates of at least one point of the edge E2. It is preferable to acquire the three-dimensional coordinates of one end point of one or more edges of the edge E1 and the edge E2. The three-dimensional coordinates of the edge can be obtained by comparing the first image and the second image with reference to the end point of the edge.

<Working position determination unit>
The work position determination unit 17 determines the work position Wp with respect to the object to which the three-dimensional coordinate acquisition marker 23 is attached, based on the three-dimensional coordinates of the edge. For example, an arbitrary point on the straight line W at a predetermined distance from the end points I1 and I2 can be determined as the working position Wp (see FIG. 10).

<Robot control system>
The robot control system 1 includes a work position determining device 2, a robot 3, and a robot operating unit 4. The robot 3 is controlled to perform the work with respect to the work position Wp of the three-dimensional coordinate acquisition marker 23 determined by the work position determination device 2 described above. Each component constituting the robot control system will be described below.

<Robot>
The robot 3 performs work on the three-dimensional coordinate acquisition marker 23 based on the work position Wp determined by the work position determination device 2. The robot 3 has, for example, a robot arm 31 and a robot hand 32 having a pair of fingers (grip portions). By driving a plurality of joints 31a provided on the robot arm 31, the robot hand 32 can be moved to perform a gripping operation at the working position Wp of the object to which the three-dimensional coordinate acquisition marker 23 is attached. it can.

<Robot operation unit>
The robot operation unit 4 instructs the robot 3 to work at the work position Wp, and controls the robot 3 to perform the work. The robot operation unit 4 may exist independently as shown in FIG. Alternatively, it may be shared with some of the other components. For example, the control unit 18 of the work position determining device 2 may include the robot operation unit 4. Like the control unit 18, the robot operation unit 4 may include physical components of a normal personal computer, such as a CPU, a memory, a storage unit, a display unit, an input unit, and an interface.

[motion]
FIG. 4 is a flowchart of the robot control method according to the first embodiment. 5 and 7 to 9 are diagrams for explaining image processing of the first image of the three-dimensional coordinate acquisition marker captured by the first camera. FIG. 6 is a diagram illustrating a binarization process of the first image of the three-dimensional coordinate acquisition marker captured by the first camera. FIG. 10 is a diagram showing the three-dimensional coordinates of the three-dimensional coordinate acquisition marker. Although not shown, the second image captured by the second camera is subjected to the same image processing as the first image. The operation of the robot control system 1 will be described with reference to the flowchart of FIG. 4 and FIGS. 5 to 9.

(1) In step S01, the first image L (FIGS. 5 to 9) obtained by capturing the three-dimensional coordinate acquisition marker 23 attached to the handle 22b of the bag 22 from different viewpoints by the first camera 11 and the second camera 12. ) And the second image (not shown) are acquired (imaging step). In order to extract the region and the edge of the three-dimensional coordinate acquisition marker 23 in the image processing described later, the three-dimensional coordinate acquisition marker 23 includes the bag 22 including the bag body 22a and the handle 22b, and the measurement stage 21 and the brightness. It should be different. That is, when the bag 22 is made of, for example, a material having a high lightness color, the three-dimensional coordinate acquisition marker 23 may select a low lightness color. Further, when the bag 22 is made of a material having a low lightness color, a color having a high lightness may be selected for the three-dimensional coordinate acquisition marker 23. In order to detect the three-dimensional coordinate acquisition marker 23, the measurement stage 21 may select a color having a brightness close to that of the bag 22.

(2) In step S02, the region of the three-dimensional coordinate acquisition marker 23 is detected in each of the first image L and the second image (region detection step). FIG. 5 is a first image L acquired by the first camera 11. In the first image L, for example, the area of the three-dimensional coordinate acquisition marker 23 is detected by executing the binarization process. In the present embodiment, the contrast between the first image L and the second image is increased due to the difference in brightness between the three-dimensional coordinate acquisition marker 23, the bag 22, and the measurement stage 21. Therefore, as shown in FIG. 6, the region of the three-dimensional coordinate acquisition marker 23 can be detected by the binarization process. The region detection method is not limited to the binarization process, and a known method can be used. For example, when the contrast between the three-dimensional coordinate acquisition marker 23 and the bag 22 and the measurement stage 21 is small, the region of the three-dimensional coordinate acquisition marker 23 may be detected by raster scan instead of the binarization process. ..

(3) In step S03, the edges E1 and E2 of the three-dimensional coordinate acquisition marker 23 are extracted from the regions detected in each of the first image L and the second image (edge extraction step). The edge E1 corresponds to the "first edge" of the present invention, and the edge E2 corresponds to the "second edge" of the present invention. In the present embodiment, the two edges E1 and the edge E2 of the three-dimensional coordinate acquisition marker 23 are extracted, but the number of edges to be extracted is not limited to this. Edge extraction can be performed using a known method. For example, it can be performed by searching in the horizontal direction in the first image L and detecting the difference in brightness.

Further, the center line C located at the center of the two edges E1 and the edge E2 is calculated. As shown in FIG. 7, for example, when searching in the horizontal direction, the center line C identifies the midpoint C1 between the point E10 indicating the edge E1 and the point E20 indicating the edge E2, and each of them is specified by the horizontal search. It can be calculated by connecting the midpoints. The angle θ between the center line C and the vertical line P in the first image L is calculated. Here, for example, by using an affine transformation or the like, the first image L is rotated by an angle θ so that the center line C is in the vertical direction (see FIG. 8).

(4) In step S04, the end points of the edges E1 and E2 are extracted in each of the first image L and the second image (end point extraction step). In the first image L rotated by an angle θ, the portion sandwiched between the edges E1 and the edge E2 is searched in the vertical direction, and the three-dimensional coordinate acquisition marker 23 and the other boundaries are extracted by the difference in brightness. At this time, each boundary searched in the vertical direction is a set of discontinuous line segments. Each line segment is straightened by, for example, the least squares method to obtain the boundary B (see FIG. 8). As shown in FIG. 9, the intersection of the boundary B and the edge E1 and the intersection of the boundary B and the edge E2 are the end point I1 of the edge E1 and the end point I2 of the edge E2. By extracting the endpoints I1 and I2, the ranges of the edges E1 and E2 are determined. That is, the edge E1 and the edge E2 are determined to be lines starting from the end points I1 and the end points I2, respectively.

(5) In step S05, the three-dimensional coordinates of the edge E1 and the edge E2 are acquired based on the end points I1 and the end points I2 of the first image L and the second image, respectively (three-dimensional coordinate acquisition step). In the present embodiment, a stereo method is adopted in which three-dimensional coordinates are measured by the principle of triangulation using the first image L and the second image captured from different viewpoints. In the stereo method, the corresponding points such as the end points I1 and I2 in the first image L and the second image having different viewpoints are obtained, and the corresponding points in the first image L and the second image and the first camera 11 and the second camera 12 are obtained. As shown in FIG. 10, the three-dimensional coordinates of the edge E1 and the edge E2 can be acquired depending on the positional relationship with. Note that FIG. 10 shows the arrangement of the edges E1 and E2 inside the rectangular parallelepiped shown by the broken line. In the present embodiment, the pixels constituting the edge E1 extending from the end point I1 and the edge E2 extending from the end point I2 in each of the first image L and the second image are made to correspond as corresponding points, and the edges E1 and the edge E2 The three-dimensional coordinates are acquired. In the present embodiment, the three-dimensional coordinates of only the edges E1 and E2 are acquired, not the entire marker 23 for acquiring the three-dimensional coordinates.

(6) In step S06, the work position Wp with respect to the handle 22b or the vicinity of the handle 22b of the bag 22 to which the three-dimensional coordinate acquisition marker 23 is attached is determined based on the three-dimensional coordinates of the edges E1 and E2 (work position determination step). ). As shown in FIG. 10, the working position Wp is, for example, on a straight line W at a distance D from a straight line (boundary B) passing through the end points I1 and I2 extracted in step S04, and the straight line W and the edge E1. It can be determined at the intersection and the midpoint between the straight line W and the edge E2. The distance D may be a predetermined value or may be in a range of a predetermined value. The work position determination device 2 executes steps S01 to S06. The work position Wp can be arbitrarily determined depending on the work content, and even on the three-dimensional coordinate acquisition marker 23, it is on or near the object calculated by relying on the three-dimensional coordinate acquisition marker 23. You may.

(7) In step S07, the working position Wp is instructed to the robot 3 (robot operation step). The robot operation unit 4 instructs the robot 3 to perform the work on the three-dimensional coordinate acquisition marker 23 at the work position Wp determined by the work position determination device 2. In the present embodiment, the robot 3 moves the robot hand 32 in the direction of the arrow in FIG. 10 and grips the three-dimensional coordinate acquisition marker 23. In the present embodiment, the three-dimensional coordinate acquisition marker 23 is attached to the handle 22b of the bag 22, and the handle 22b has flexibility. Therefore, when the three-dimensional coordinates of the edges E1 and E2 of the three-dimensional coordinate acquisition marker 23 and the work position Wp are determined, the robot can perform the work. The work on the three-dimensional coordinate acquisition marker 23 is, for example, grasping, sucking, and hooking the three-dimensional coordinate acquisition marker 23 by the robot hand. In the present embodiment, the robot 3 can be made to perform the work of grasping the work position Wp on the three-dimensional coordinate acquisition marker 23 by the robot hand 32.

[effect]
According to the work position determination method and the work position determination device of the present embodiment, the work position is determined by acquiring the three-dimensional coordinates of the edge of the three-dimensional coordinate acquisition marker 23 attached to the handle 22b of the bag 22. Can be done. When a three-dimensional measuring machine that acquires point cloud data of the three-dimensional coordinates of the entire bag 22 such as a TOF camera is used, edge recognition is difficult and processing is difficult because the amount of data is large. However, by using a stereo camera as in the present embodiment, it is possible to obtain the three-dimensional coordinates of the edge by dropping unnecessary information. Therefore, according to the present embodiment, image processing is easy and three-dimensional coordinates can be stably acquired.

Further, according to the robot control method and the robot control system of the present embodiment, the robot 3 can be instructed to perform the work by instructing the robot 3 on the work position determined by the work position determination method and the work position determination device. Further, since it is sufficient to acquire the three-dimensional coordinates of the edges E1 and E2 instead of the entire three-dimensional coordinate acquisition marker 23, image processing is easy.

Further, in the acquisition of three-dimensional coordinates by the stereo method, the matching process for searching for corresponding points on each image is the process with the highest information processing load and cost, but in the present embodiment, the first image L Since the pixels constituting the edge E1 and the edge E2 may be matched in each of the second image and the second image, the load of the matching process can be reduced.

Further, if the handle 22b is provided with the three-dimensional coordinate acquisition marker 23 as in the bag of the present embodiment, the brightness of the bag body 22a and the handle 22b is different from that of the three-dimensional coordinate acquisition marker 23. In the first image L and the second image, the three-dimensional coordinate acquisition marker 23 can be easily detected and the working position can be determined.

Although the present invention has been described with reference to the above-described embodiments, the embodiments of the present disclosure are not limited thereto.

In the present embodiment, the three-dimensional coordinate acquisition marker 23 is made of a rigid resin. For example, as shown in FIG. 11, the three-dimensional coordinate acquisition marker 23a is attached to the handle 22b of the bag 22. The wrapped tape may be a portion where the handle 22b is colored in a color different from that of the bag 22 or the handle 22b. Any shape that is different in brightness from the bag 22 and can be attached to the handle 22b of the bag 22 can be used as a marker for acquiring three-dimensional coordinates.

It should be noted that the present disclosure includes appropriately combining any of the various embodiments and / or examples described above, and the respective embodiments and / or embodiments. The effects of the examples can be achieved.

Further, in the above-described embodiment, the robot operation unit 4 is provided outside the work position determination device 2, but the robot operation unit 4 may be built in the work position determination device 2. Further, the camera of the work position determining device 2 and the other may be provided as separate devices.

According to the work position determination method, the robot control method, the work position determination device, and the robot control system according to the present invention, the work position is determined for objects having various shapes, and the work position is instructed to the robot. You can let the robot perform the work.

1 Robot control system 2 Working position determination device 3 Robot 4 Robot operation unit 11 1st camera 12 2nd camera 13 Area detection unit 14 Edge extraction unit 15 Edge point extraction unit 16 3D coordinate acquisition unit 17 Work position determination unit 22 Bag 22a Bag Body 22b Handle 23 Marker for acquiring 3D coordinates

Claims (10)

An imaging step of acquiring a first image and a second image of a marker provided on a flexible object from different viewpoints, and
In each of the first image and the second image, a region detection step of detecting the region of the marker and a region detection step.
A three-dimensional coordinate acquisition step of acquiring at least one three-dimensional coordinate of the detected marker area, and
A work position determination step of determining a work position with respect to the object based on the three-dimensional coordinates,
including,
Working position determination method. The three-dimensional coordinate acquisition step includes acquiring three-dimensional coordinates of at least one point of the edge of the marker.
The work position determination method according to claim 1. The three-dimensional coordinate acquisition step includes acquiring the three-dimensional coordinates of one end point of the edge.
The work position determination method according to claim 2. The edge includes a first edge and a second edge.
The work position determination method according to claim 2 or 3. The object has a handle and
The marker is provided on the handle of the object.
The work position determination method according to any one of claims 1 to 4. The marker has an edge substantially parallel to the longitudinal direction of the handle of the object.
The work position determination method according to claim 5. An imaging step of acquiring a first image and a second image of a marker provided on a flexible object from different viewpoints, and
In each of the first image and the second image, a region detection step of detecting the region of the marker and a region detection step.
A three-dimensional coordinate acquisition step of acquiring at least one three-dimensional coordinate of the detected marker area, and
A work position determination step of determining a work position with respect to the object based on the three-dimensional coordinates,
The work process of causing the robot to work with respect to the work position,
including,
Robot control method. A first camera and a second camera that image a marker provided on a flexible object from different viewpoints,
A region detection unit that detects the region of the marker in each of the images captured by the first camera and the second camera, and
A three-dimensional coordinate acquisition unit that acquires three-dimensional coordinates of at least one point in the detected marker area, and
A work position determination unit that determines a work position with respect to the object based on the three-dimensional coordinates.
To prepare
Working position determination device. The work position determining device according to claim 8 and
A robot having a robot hand that performs work on the work position and
To prepare
Robot control system. With the bag body
At least one handle provided on the bag body and
The three-dimensional coordinate acquisition marker attached to the handle,
To prepare
bag.

Images