JP2011083882A - Robot system - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To provide a robot system for performing bin picking without reducing the detection accuracy while preventing a camera's view from being shielded by a returnable box and preventing the camera from being intervened by the returnable box. <P>SOLUTION: An image processing device 906 includes: a two-dimensional model 109 of a workpiece; a two-dimensional workpiece position detecting unit 108; an area model 106 for dividing the returnable box into a plurality of areas and defining an area-based visual line as to each area; an area determining unit 103 for outputting the defined area-based visual line to an area to which the workpiece to be picked belongs; a three-dimensional model 105 of the workpiece; a three-dimensional workpiece position detecting unit 102 for detecting a three-dimensional position posture from an image capturing the workpiece to be picked; a visual line model 107 for defining a visual direction of the camera from a plurality of directions relative to the workpiece; and a visual line calculating unit 104 for calculating a visual line of the camera in approaching the workpiece to be picked. <P>COPYRIGHT: (C)2011,JPO&INPIT

Description

The present invention relates to a robot system that detects a workpiece stacked in a returnable box or the like with a visual sensor such as a camera and takes out the detected workpiece.

The operation of detecting the position of a workpiece with a visual sensor such as a camera from a state in which a plurality of workpieces are stacked in a return box or the like and picking up the detected workpiece with a robot is known as bin picking. When bin picking is performed by mounting the visual sensor on the hand of the robot, it is necessary to arrange the visual sensor so that it does not interfere with the return box, or to place the visual sensor so that the visual field of view is not blocked by the return box.
As a technique related to the arrangement of the visual sensor, Patent Document 1 is disclosed. FIG. 8 shows a schematic configuration example of the picking apparatus in Patent Document 1. In FIG. 8, the target object 14 is imaged by the television cameras 11a to 11c fixed at three different positions. The obtained image is processed by the three-dimensional measuring unit 12, and the position / posture of the grasped object is obtained by the grasped object determining unit 13. Based on the result, the gripping means 16 is moved by the moving means 15 to pick up the target object 14. Here, the three-dimensional measuring unit 12 and the grasped object determining unit 13 are configured by a microcomputer. The gripping means 16 is a robot hand, and the moving means 15 is a 6-axis articulated robot.

Next, an arrangement method of the television cameras 11a to 11c for the purpose of improving measurement accuracy will be described. The images suitable for measurement are images obtained by capturing the surface 41 registered as the object model shown in FIG. 9 from the front, and it is desirable to arrange the television cameras 11a to 11c at positions where such images can be obtained. . On the other hand, as shown in FIG. 10, when the target object 14 overlaps, the surface 41 registered as an object model faces in various directions as indicated by arrows. However, if the thickness of the object is small, such as the target object 14, the range in which the orientation can be obtained can be predicted.
Therefore, in the conventional technique, when the direction in which the surface 41 can be oriented is a conical shape with an apex angle 2θ as shown in FIG. 11, the TV cameras 11a to 11c are arranged at positions 61a to 61c shown in FIG. The positions 61a to 61c are arranged at an inclination of θ / 2 every 120 degrees with the vertex of the cone as the center. Thereby, the surface 41 can be imaged as an image suitable for measurement as much as possible.

JP 2000-94374 A (pages 4 and 6, FIG. 1, FIG. 4-6)

Here, the case where the technique described in Patent Document 1 is applied to bin picking is considered.
1A and 1B are diagrams illustrating a configuration of a bin picking system, in which FIG. 1A is a top view and FIG. 1B is a side view. The camera 902 and the hand 903 are mounted on the hand of the articulated robot 901 and can be moved within the movable range of the robot 901. The camera 902 acquires an image of the work 905 randomly placed in the return box 904 and outputs it to the image processing apparatus 906. The image processing apparatus 906 calculates the position of the workpiece 905 and outputs the position to the robot controller 907. The robot controller 907 moves the hand 903 to the calculated position, and the hand 903 grips or sucks the work 905 and removes it from the return box.
When bin picking is performed, the camera 902 acquires an image of the workpiece 905 in the passing box 904, calculates the position of the workpiece (target workpiece) to be picked from the image acquired by the image processing device 906, and the robot 901 It moves to the position of the target workpiece calculated 903 and picks up the workpiece.

However, applying the technique described in Patent Document 1 to the bin picking system as shown in FIG. 1 has the following problems.
FIG. 12 is an explanatory diagram showing a state in which a certain target workpiece A in the return box 904 is detected. In FIG. 12, the target workpiece A 908 positioned in front of the robot among the workpieces in the return box is to be detected by the camera 902. According to the prior art, “the image is a front image of a surface registered as an object model, so it is desirable to place the camera 902 at a position where such an image can be obtained”. In such a case, if the camera 902 is arranged at such a position, the field of view of the camera 902 passes through and is blocked by the wall of the box 904, so that there is a problem that an image of the target workpiece A 908 cannot be acquired.
With respect to such a problem, in the conventional technology, as shown in FIG. 11, the camera 902 is allowed to be tilted by θ / 2 every 120 degrees. In order to avoid this, if θ is set large, there will be a problem that the detection accuracy is remarkably lowered or the camera interferes with the wall of the return box 904.

FIG. 13 is an explanatory diagram showing a state in which a certain target work B in the returnable box 904 is detected. In FIG. 13, the target workpiece B 909 located at the bottom of the return box and before the robot is detected by the camera 902. According to the prior art, “It is desirable to place the camera 902 at a position where such an image can be obtained because it is an image obtained by imaging the surface registered as the object model from the front”. When trying to arrange, the problem that the camera 902 interferes with the wall of the passing box 904 occurs.
As described above, in the conventional technique, the camera 902 is arranged at the same place as when the model is registered for the workpiece 905, so that the field of view of the camera 902 is blocked by the wall of the box 904, and the target There were a problem that the image of the workpiece A 908 could not be acquired and a problem that the camera 902 passed through and interfered with the wall of the box 904. On the other hand, if the tolerance of the arrangement of the camera 902 is increased, there is a problem that the detection accuracy is remarkably lowered.
The present invention has been made in view of such problems, and does not reduce the detection accuracy, the camera's field of view is not shielded by the passing box, and the camera performs bin picking without interfering with the passing box. An object of the present invention is to provide a robot system that can be used.

In order to solve the above problem, the present invention is configured as follows.
According to the first aspect of the present invention, there is provided a robot that picks up a work stacked in a box that is passed by a hand attached to the tip, a camera that is mounted on the robot and picks up the work, and is picked up by the camera. An image processing apparatus that inputs an image of the workpiece, calculates and outputs the position and orientation of the workpiece, and moves the robot to the position of the workpiece based on the output of the image processing apparatus, and moves the workpiece by the hand. In the robot system including the robot controller for picking, the image processing apparatus uses the two-dimensional model of the workpiece and the two-dimensional position of the workpiece to be picked from an image obtained by imaging the plurality of workpieces. A two-dimensional work position detection unit for detecting the posture and the return box are divided into a plurality of areas. The area model in which the line of sight for each area is defined and the position of the picking target work detected by the two-dimensional work position detection unit belong to which area of the area model, and the picking target work belongs An area determination unit that outputs the line of sight defined for each area defined by the area, a three-dimensional model of the workpiece, and an image obtained by imaging the picking workpiece using the three-dimensional model. Detected by a three-dimensional workpiece position detection unit that detects a three-dimensional position and orientation, a line-of-sight model that defines the camera gaze direction from a plurality of directions with respect to the workpiece, the line-of-sight model, and the three-dimensional workpiece position detection unit The picking pair is determined based on the posture of the picked workpiece and the area-specific gaze obtained by the area determination unit. A line-of-sight calculation unit that calculates the line of sight of the camera when approaching a workpiece, and the two-dimensional workpiece position detection unit outputs the detected two-dimensional position and orientation of the picking target workpiece to the robot controller, The line-of-sight calculation unit outputs the line of sight of the camera when approaching the calculated picking target work to the robot controller.

According to a second aspect of the present invention, the robot controller operates the robot on the basis of the two-dimensional position and orientation of the workpiece to be picked detected by the two-dimensional workpiece position detection unit, and the workpiece to pick the camera. The robot is operated based on the line of sight of the camera calculated by the line-of-sight calculation unit, and the camera is approached to the workpiece to be picked.

According to a third aspect of the present invention, the three-dimensional model stores positions of four or more characteristic portions provided on the workpiece, and the three-dimensional workpiece position detection unit images the workpiece. The three-dimensional position / orientation of the workpiece is detected by extracting the feature from an image.

According to a fourth aspect of the present invention, when the robot performs picking on a work existing in the area, the visual field of the camera is not shielded by the return box, and the camera is It represents the line-of-sight direction of the camera when the camera is arranged so as not to interfere with the return box.

The invention according to claim 5 is characterized in that the line of sight by area is defined as a vector based on a robot coordinate system.

According to a sixth aspect of the present invention, the line-of-sight calculation unit calculates the inner product of the line-of-sight direction of the camera defined by the line-of-sight model and the line-of-sight of each area set in the area. The camera's line of sight suitable for picking is determined.

The invention according to claim 7 is characterized in that the two-dimensional workpiece position detecting unit detects a two-dimensional position and orientation of the workpiece by template matching between an image captured by the camera and the two-dimensional model. Is.

According to an eighth aspect of the present invention, the two-dimensional workpiece position detection unit has the highest degree of coincidence with the two-dimensional model when a plurality of workpieces are detected from an image captured by the camera by template matching. It is characterized by selecting a high one.

The invention according to claim 9 is characterized in that the robot is a vertical articulated robot.

According to the present invention, the area determination unit determines which area the work belongs to, outputs the determination result and the line-of-sight direction of the camera defined for each area, and the line-of-sight calculation unit outputs the line of sight to the work. Since the gaze direction closest to the camera gaze direction obtained by the area determination unit is selected from the gaze model from the gaze model that defines the direction and the camera gaze direction obtained by the area determination unit, the camera view is reduced. There is an effect that the camera is not shielded by the return box and the camera does not interfere with the return box.
In addition, since there is a three-dimensional model for each of the gaze directions calculated and output by the gaze computation unit, the camera can be arranged at a position close to the camera arrangement for the workpiece when the three-dimensional model is registered. Thus, there is an effect that the detection accuracy is not lowered.
According to the third aspect of the invention, since the three-dimensional position and orientation of the workpiece can be calculated only from the image acquired by one camera, it is necessary to provide a plurality of cameras, laser light sources, and the like at the tip of the robot. The robot tip can be made compact, and there is an effect that the camera does not interfere with the box.
According to the invention described in claims 4 to 6, the visual field of the camera is shielded by the viewing line by obtaining the visual line model closest to the visual line for each area defined in the area where the workpiece exists, among the visual line models. In other words, it is possible to determine the direction of the line of sight of the camera without causing the camera to interfere with the box.
According to invention of Claim 7, 8, the thing which is easy to pick can be selected from the some workpiece | work loaded in the return box.
According to the ninth aspect of the present invention, since the posture of the camera can be freely changed, the actual posture of the camera can be changed in accordance with the line of sight of the camera obtained by the line-of-sight calculation unit, and the field of view of the camera passes. It is possible to realize an arrangement of the camera that is not shielded by the box and does not interfere with the camera.

It is a figure which shows the structure of the robot system which performs bin picking. It is a figure which shows the flow of bin picking by a robot system. It is a block diagram of the image processing apparatus of the robot system of this invention. It is a conceptual diagram of the area model of this invention. It is a conceptual diagram of the gaze model of this invention. It is operation | movement explanatory drawing of the gaze calculating part of this invention. It is explanatory drawing showing arrangement | positioning of a workpiece | work and a camera. It is a schematic block diagram of the picking apparatus in a prior art. It is explanatory drawing which shows the object model in a prior art. It is explanatory drawing which shows the attitude | position of the measurement target object in a prior art. It is explanatory drawing which shows the position of the several viewpoint in a prior art. It is explanatory drawing which shows a mode that the object workpiece | work A is detected. It is explanatory drawing which shows a mode that the target workpiece | work B is detected.

Hereinafter, embodiments of the present invention will be described with reference to the drawings.

First, the necessity for the proper camera arrangement with respect to the workpiece described in the background art will be described in detail. In order to obtain the 3D position and orientation of the work to be worked from the image captured by the camera, stereo vision using a plurality of cameras, or triangulation that irradiates the work with light such as laser and captures the state with the camera. In recent years, a method for calculating the three-dimensional position and orientation of a workpiece from only one image acquired by one camera without using a plurality of cameras or lasers is known. (For example, technical report Yaskawa Electric, Vol. 71, No. 3, No. 276 (2007) P144-148).
By using this method, it is not necessary to provide a plurality of cameras, laser light sources, etc. at the tip of the robot, and the robot tip can be made compact. Making the tip of the robot compact is particularly useful in applications such as bin picking.
A specific method for calculating the three-dimensional position and orientation of a workpiece from only an image acquired by a single camera is not the essence of the present invention, and a detailed description thereof will be omitted. However, in order to use this method, a target is used. It is necessary to set four or more feature points on the workpiece, and further, it is necessary to satisfy the condition that these feature points do not exist on the same straight line when captured by the camera.

For example, consider a case where an image is acquired by the camera 902 for the work 501 as shown in FIG. In the case of FIG. 7, the work 501 has four holes 502. An image of the workpiece 501 is acquired by the camera 902 disposed above the workpiece 501, and four holes are detected by the image processing device 906, and the positional relationship is obtained using these four holes as feature points. Prior to bin picking, the positions of these four holes are registered as object models. This object model is stored in the image processing apparatus as a three-dimensional model to be described later.
When the bin picking shown in FIG. 1 is performed, an image of the work 905 is acquired by the camera 902 and four holes are detected by the image processing device 906, as in the case of registering the object model. By comparing the positions of the four holes detected here with the positions of the four holes of the object model, the three-dimensional position and orientation of the workpiece 905 can be calculated from an image acquired from one camera 902.
The detection accuracy is higher when the arrangement of the camera 902 when detecting the workpiece 905 when performing bin picking is the same as the arrangement when the object model is registered.
If the arrangement of the workpiece with respect to the camera is different between when the object model is registered and when bin picking is performed, the shape of the hole 502 as viewed from the camera 902 changes, so that the detection accuracy of the hole 502 decreases, resulting in four This is because the accuracy of the three-dimensional position and orientation of the workpiece obtained from the positional relationship of the holes is lowered.

In this embodiment, description will be made based on the robot system that performs bin picking shown in FIG. In FIG. 1, the robot 901 uses a vertical articulated robot so that the position and orientation of the camera 902 and the hand 903 can be freely changed according to the posture of the work 905 in the return box 904. The camera 902 is attached to the tip of the robot in parallel with the hand 903 so that the line of sight coincides with the long axis direction of the hand 903.

First, a rough procedure of bin picking will be described with reference to FIG. FIG. 2 is a diagram showing a flow of bin picking work (one time) by the robot system of the present invention.
First, the robot is operated so that the entire return box 904 can be captured in the field of view of the camera 902, and the camera 902 is placed at an appropriate position away from the box 904 to some extent. This position can be taught in advance and stored in the robot controller 907 to ensure reproducibility (step 1). Further, the posture of the camera at this time is set to the vertical direction as shown in FIG.
When the movement of the camera is completed, an image in the passing box 904 is acquired by the camera 902, and the image is output to the image processing device 906 (step 2).
The image processing apparatus 906 detects the approximate position of the work (target work) to be picked in the return box by performing a two-dimensional detection process, and outputs the detected position to the robot controller (step 3). While maintaining the posture of the camera in the state shown in FIG. 1, the robot controller moves the robot in the horizontal direction based on the detected position of the work to bring the camera 902 closer to the target work so that a detailed image of the work can be obtained. (Step 4).
Then, the image of the target workpiece is acquired again by the camera 902, and the image is output to the image processing device 906 (step 5).
The image processing apparatus 906 performs a three-dimensional detection process this time, detects the detailed three-dimensional position and orientation of the target workpiece, and outputs the camera position when the object model is registered to the robot controller (step 6). The robot controller operates the robot again to place the camera at an appropriate position (step 7), brings the hand 903 closer to the target work from the approach direction, holds the work with the hand 903, and removes it from the return box (step 8). ).
As described above, the hand 903 is taken out from the returnable box by gripping or adsorbing the workpiece 905. When gripping, a gripper having an opening / closing mechanism as a hand is provided at the tip of the robot. In the case of suction, a suction cup or the like is provided as a hand.

FIG. 3 is a diagram showing main elements of the image processing apparatus 906 constituting the robot system of the present invention. In the following, the procedure described in FIG. 2 will be described in more detail while describing each element of the image processing apparatus 906 shown in FIG.
In FIG. 3, the image input unit 101 receives an image acquired by the camera 902 and outputs the image to the two-dimensional work position detection unit 108 and the three-dimensional work position detection unit 102 as appropriate.
First, when an image in the return box 904 showing a plurality of workpieces is acquired in step 2 of FIG. 2, the images are output to the two-dimensional workpiece position detection unit 108.
The two-dimensional workpiece position detection unit 108 uses a two-dimensional model 109 relating to the workpiece 501 registered in advance, and uses a two-dimensional position and orientation (two-dimensional position and position) of the workpiece 905 on the camera imaging surface from the input image in the return box 904. Rotation angle) is detected and output to the area determination unit 103 and the robot controller 907. The process of outputting the two-dimensional position and orientation of the work 905 to be picked from the two-dimensional work position detection unit 108 to the robot controller 907 corresponds to step 3 in FIG.

The detection here uses, for example, template matching to find out where the work exists by calculating the degree of coincidence between a part of the images in the returnable box and a model image (template image) registered in advance. In this case, the two-dimensional model 109 becomes a template image related to the work 501, and the two-dimensional work position detection unit 108 detects the position and orientation of the work 905 from the input image (scene image) in the return box 904. When a plurality of workpieces 905 are stacked in the return box 904, a plurality of workpieces are detected. However, in template matching, the degree of matching (matching degree) with each two-dimensional model for each detected workpiece. Therefore, it is sufficient to extract one with the highest score as a detection result and set it as a work (target work) to be picked. A workpiece having a high score has an inclination with respect to the camera 902 that is close to that when the two-dimensional model is registered.
In general, since a two-dimensional model is registered in a state where a workpiece placed on a plane is viewed from directly above, a workpiece close to the state of facing the camera's line of sight is a workpiece having a high score.
When a plurality of workpieces are actually detected, the one with the highest matching score (coincidence) with the two-dimensional model is selected from the top ten, for example, and the positions thereof are stored, and the highest matching score is obtained. By picking in order, the steps 1 and 2 in FIG. 2 can be avoided when determining the workpiece to be picked next.

In addition, when the bin picking operation is repeated and the number of the workpieces 905 in the return box decreases, the height of the work located on the uppermost surface from the bottom of the return box also decreases. When performing template matching in step 3 of FIG. 2, it is desirable that the distance between the camera 902 and the uppermost workpiece is constant.
Therefore, a distance sensor (not shown) is provided in the vicinity of the camera at the tip of the robot, and each time Step 1 is executed, the distance to the work in the box is measured with the distance sensor, and the camera height is set to a predetermined value. The size of the work on the camera imaging surface is stored in a memory (not shown) in the image processing apparatus when template matching is performed, and the camera passes through each time step 1 is executed. The distance between the work in the return box and the camera can be kept appropriate by taking an image of the work in the box and adjusting the camera height so that it is equivalent to the size of the stored work. May be.
Further, matching processing corresponding to a scale change in which the size of the template image is changed at the time of template matching without adjusting the height of the camera may be performed.

The area determination unit 103 uses the area model 106 obtained by dividing the returnable box 904 viewed from above into a plurality of areas, and to which area the position of the target workpiece detected by the two-dimensional workpiece position detection unit 108 belongs. And the direction of the line of sight (line of sight for each area) of the camera 902 defined for the area is output.
FIG. 4 shows a conceptual diagram of the area model. FIG. 4 shows the return box 904 as viewed from above, and the inside of the return box 904 is divided into a plurality of areas 201. The area is divided by dotted lines. In FIG. 4, the area is divided into eight areas from area 1 to area 8, but this is only an example, and the number of areas and the way of division are not limited to the form shown in FIG. 4.
In the area model 106, the line-of-sight direction of the camera 902 (area-specific line of sight 202) is also defined for each area 201. These line-of-sight directions indicate that when the robot picks a work in the area, the camera 902 is not shielded by the box 904 through which the camera 902 passes and the camera 902 does not interfere with the box 904 through which the camera 902 passes. It represents the line-of-sight direction of the camera 902 when placed.
As a specific method of expressing the line of sight for each area, for example, a vector based on the robot coordinate system as shown in FIG. 4 can be used. For example, the line of sight defined by area 1 can be expressed as (1, -1), and the line of sight defined by area 2 can be expressed as (0, -1).

On the other hand, the robot controller 907 that has received the two-dimensional position of the target work on the camera imaging surface output from the two-dimensional work position detection unit 108 converts the position into the coordinate system position of the robot. Then, the robot is moved toward a position based on the converted robot coordinate system, and the camera 902 is brought close to the target work (corresponding to step 4 in FIG. 2).
It is assumed that calibration has been completed in advance for the positional relationship between the coordinate system of the camera imaging surface and the robot coordinate system. Further, the camera is moved while maintaining the posture when the image in the returnable box 904 is acquired.
As the robot moves, the camera 902 is positioned almost directly above the target workpiece.

When the movement of the robot is completed, the robot controller 907 notifies the image processing apparatus 906.
Upon receiving this notification, the image input unit 101 inputs the image acquired by the camera 902 again and outputs the image to the three-dimensional work position detection unit 102 (corresponding to step 5 in FIG. 2).
The three-dimensional workpiece position detection unit 102 detects the three-dimensional position and orientation of the target workpiece from the image in the return box 904 including the target workpiece using the three-dimensional model 105 related to the workpiece 501 registered in advance, and sends it to the line-of-sight calculation unit 104. Output.
As described above, the detection here detects four feature points (four holes in the case of the workpiece 501) from the image of one camera, and calculates the three-dimensional position and orientation of the workpiece from these positions. Use the technique. In this case, the three-dimensional model 105 is a positional relationship between the four feature points, and the three-dimensional workpiece position detection unit 102 detects four feature points (four holes) from the image of the target workpiece, By comparing, the three-dimensional position and orientation of the target workpiece are detected.

The line-of-sight calculation unit 104 uses the line-of-sight model 107 in which the line-of-sight direction is defined with respect to the work 501, and the posture of the work 905 detected by the three-dimensional work position detection unit 102 and the line-of-sight by area 202 obtained by the area determination unit 103. From this information, an appropriate camera gaze for the work to be picked is calculated and output.
FIG. 5 shows a conceptual diagram of the line-of-sight model. FIG. 5A is a view (top view) of the work 501 viewed from directly above, and FIG. 5B is a view (perspective view) of the work 501 viewed obliquely. In the case of FIG. 5A, the line-of-sight model 107 defines eight line-of-sight directions (line-of-sight directions 1 to 8) with respect to the workpiece 501.
The line-of-sight direction corresponds to the line of sight of the camera 902, and when viewed three-dimensionally, is as shown in FIG. In FIG. 5B, only the line-of-sight directions 1, 5, and 7 are displayed to make the drawing easier to see. The line-of-sight calculation unit 104 rotates the line-of-sight model 107 in accordance with the posture of the target workpiece detected by the three-dimensional workpiece position detection unit 102 and compares it with the area-specific line of sight 202 obtained by the area determination unit 103.
In FIG. 5, the number of the line-of-sight directions defined in advance is eight, but the number of the line-of-sight directions is not limited to this and can be arbitrarily set according to the shape of the workpiece.

FIG. 6 is a diagram for explaining the operation of the line-of-sight calculation unit. FIG. 6 is an enlarged view of the area 3 corresponding to the upper right part of the returnable box in FIG. Now, it is assumed that the area determination unit 103 determines that the work (target work) to be picked belongs to the area 3, and the area-specific gaze 202 of the area 3 is obtained as shown in FIG. Also, assume that the line-of-sight model 107 is rotated using the three-dimensional position and orientation of the workpiece 905 detected by the three-dimensional workpiece position detection unit 102, and the result is as shown in FIG.
Here, among the line-of-sight directions 1 to 8, the line-of-sight direction closest to the area-specific line-of-sight 202 set in the area 3 is specified.
Among the eight line-of-sight directions set in the line-of-sight model, those closest to the area-specific line-of-sight 202 are, for example, the eight line-of-sight directions projected on the XY plane as shown in FIG. Can be obtained by performing an inner product operation.
If the directions of the two vectors are close, the inner product increases. An inner product is obtained between each of the eight line-of-sight directions in FIG. 6 and the line-of-sight by area 202, and the line having the largest result is the line-of-sight direction close to the line-of-sight by area. In the case of FIG. 6, it can be seen that the line-of-sight direction 3 is applicable.
The line-of-sight calculation unit 104 then outputs the line-of-sight direction number closest to the area-specific line-of-sight and the vector of the line-of-sight direction 3 based on the orthogonal coordinate system of FIG. Such a process corresponds to Step 6 in FIG.

The output line of sight thus selected is the closest to the line of sight in which the field of view of the camera 902 is not shielded by the box 904 and the camera 902 does not interfere with the box 904 among the eight lines of sight in FIG. It has become.

In the robot controller 907, the posture and arrangement of the camera 902 corresponding to each line of sight (line-of-sight directions 1 to 8) of the line-of-sight model shown in FIG.
By adjusting the teaching data in accordance with the output line of sight of FIG. 6, the field of view of the camera 902 is not shielded by the box 904 with respect to the work arranged as shown in FIG. A line-of-sight direction that does not interfere can be obtained.
The robot controller 907 obtains teaching data of the corresponding camera 902 from the line-of-sight direction number output from the line-of-sight calculation unit 104, and coordinates-converts the teaching data using the vector of the output line of sight 401, according to the state of FIG. Find the posture and placement of the camera. After that, the robot is actually operated to place the camera in an appropriate position and posture. This operation corresponds to step 7 in FIG.
By this operation, the posture of the camera that has been maintained in the vertical direction as shown in FIG. 1 changes to the position and posture of the camera when the object model is registered.

After the camera is appropriately arranged by the series of processes described above, the target work is imaged by the camera as in the conventional technique, and the hand is moved based on the processing result of the image processing apparatus 906 to hold the target work. It is adsorbed and removed from the returnable box (step 8 in FIG. 2).

As described above, according to the present invention, the area determination unit 103 determines which area of the return box the work to be picked belongs to, and defines the determination result and the area to which the work belongs. The line-of-sight direction of the camera (line of sight by area) is output, and the line-of-sight calculation unit 104 defines the line-of-sight direction for the workpiece, and the line-of-sight direction of the camera obtained by the area determination unit 103 (line of sight by area) From the line-of-sight model 107, the line-of-sight direction closest to the line-of-sight obtained by the area determination unit 103 is selected and the direction of the camera is adjusted to the line-of-sight direction. In addition, the camera 902 is arranged so that the camera 902 does not interfere with the passing box 904.

DESCRIPTION OF SYMBOLS 101 Image input part 102 Three-dimensional work position detection part 103 Area determination part 104 Gaze calculation part 105 Three-dimensional model 106 Area model 107 Gaze model 108 Two-dimensional work position detection part 109 Two-dimensional model 201 Area 202 Area-specific gaze 301 Gaze direction 401 Output line of sight 501 Work 502 Hole 901 Robot 902 Camera 903 Hand 904 Passing box 905 Work 906 Image processing device 907 Robot controller 908 Target work A
909 Target work B
11a, 11b, 11c Television camera 12 Three-dimensional measuring means 13 Grasping object determining means 14 Object 15 Moving means 16 Grasping means

Claims (9)

A robot that picks up the work stacked in the return box with the hand attached to the tip, and
A camera mounted on the robot for imaging the workpiece;
An image processing device that inputs an image of the workpiece imaged by the camera, calculates and outputs the position and orientation of the workpiece, and
A robot controller that moves the robot to the position of the workpiece based on the output of the image processing apparatus and picks the workpiece by the hand;
In a robot system equipped with
The image processing apparatus includes:
A two-dimensional model of the workpiece;
A two-dimensional workpiece position detection unit that detects a two-dimensional position and orientation of a workpiece to be picked from an image obtained by imaging the plurality of workpieces using the two-dimensional model;
An area model that divides the returnable box into a plurality of areas and defines a line of sight for each area;
It is determined which area of the area model the position of the picking target work detected by the two-dimensional work position detection unit, and the line of sight defined by the area defined by the area to which the picking target work belongs An area determination unit that outputs
A three-dimensional model of the workpiece;
A three-dimensional workpiece position detection unit that detects a three-dimensional position and orientation of the picking target workpiece from an image obtained by imaging the picking target workpiece using the three-dimensional model;
A line-of-sight model defining the line-of-sight direction of the camera from a plurality of directions with respect to the workpiece;
The camera's line of sight when approaching the picking target work is calculated from the line-of-sight model, the posture of the picking target work detected by the three-dimensional work position detection unit, and the area-specific visual line obtained by the area determination unit. A line-of-sight calculation unit to
With
The two-dimensional workpiece position detection unit outputs the detected two-dimensional position and orientation of the picking target workpiece to the robot controller;
The line-of-sight calculation unit outputs the calculated line of sight of the camera when approaching the picking target work to the robot controller.   The robot controller operates the robot based on the two-dimensional position / posture of the workpiece to be picked detected by the two-dimensional work position detector, causes the camera to approach the picking workpiece, and The robot system according to claim 1, wherein the robot is operated based on the calculated line of sight of the camera, and the camera approaches the workpiece to be picked. The three-dimensional model stores positions of four or more feature portions provided on the workpiece, and the three-dimensional workpiece position detection unit extracts the feature portions from an image obtained by imaging the workpiece. The robot system according to claim 1, wherein a three-dimensional position / orientation of the workpiece is detected by the method. The line of sight for each area is such that when the robot picks a work existing in the area, the field of view of the camera is not shielded by the return box and the camera does not interfere with the return box. The robot system according to claim 1, wherein the robot system represents a line-of-sight direction of the camera when the camera is arranged. The robot system according to claim 1, wherein the line of sight for each area is defined as a vector based on a robot coordinate system. The line-of-sight calculation unit calculates the inner product of the line-of-sight direction of the camera defined by the line-of-sight model and the line-of-sight of each area set in the area, so that the line-of-sight of the camera suitable for picking the workpiece to be picked is calculated. The robot system according to claim 1, wherein: The robot system according to claim 1, wherein the two-dimensional workpiece position detection unit detects a two-dimensional position and orientation of the workpiece by template matching between an image captured by the camera and the two-dimensional model. The two-dimensional workpiece position detecting unit selects the one having the highest degree of coincidence with the two-dimensional model when a plurality of workpieces are detected from images captured by the camera by template matching. The robot system according to claim 7.   The robot system according to claim 1, wherein the robot is a vertical articulated robot.

Images