JP6288586B2 - Imaging direction changing device for robot - Google Patents

Description

  The present invention relates to an apparatus for changing the imaging direction of a camera that is used in a robot that has a camera on its head, hand, etc. and recognizes the position and orientation of a work object from an image captured by the camera. .

  In an articulated robot including an image recognition device including a camera and an image processing device, image information that can be acquired depends on the mounting position and posture of the camera. Therefore, when it is necessary to capture an image from another direction in order to acquire necessary image information, it is necessary to change the camera mounting position or move the camera by the operation of the robot.

  When changing the mounting position of the camera, it is necessary to perform a calibration operation for precisely matching the positional relationship between the robot and the camera every time the mounting position is changed. Therefore, every time the camera mounting position is changed, adjustment time for mounting work and calibration work is generated. Further, when the mounting position is not changed, an extra operation for moving the camera to the imaging position is required, and the tact time for each process is increased.

  The technique described in Patent Document 1 automatically performs calibration from an image obtained by a 2D or 3D camera provided at the tip of a robot arm. Therefore, it is not necessary for an operator to enter the safety fence, and it is not necessary to perform full calibration if the camera coordinate deviation is within an allowable range.

  In the technique described in Patent Document 2, a concave reflecting mirror is configured by a plurality of plane mirrors, and light from a large number of light emitting elements arranged in a long and narrow identification area adjacent to the working area is used as a camera. The light from each light emitting element reflected by the reflecting mirror is imaged by a camera, and an intruder entering the work area is monitored based on whether or not the light is blocked. Therefore, it is possible to monitor intrusions into the work area over a wide area.

Japanese Patent No. 5523392 JP2013-052485A

  However, in the technique described in Patent Document 1, usable cameras are limited, and when changing the mounting position of the camera, it is necessary to perform normal calibration, and the adjustment time cannot be shortened. Further, in the technique described in Patent Document 2, there is a limitation on the installation position of the reflecting mirror, and a necessary image may not be acquired unless one or more cameras are occupied for intrusion monitoring, resulting in an increase in equipment cost. Invite.

  Therefore, an object of the present invention is to provide an imaging direction changing device that can be easily attached to a robot, a camera, or a robot periphery, and can acquire an image from a direction different from the normal imaging direction with the same camera. It is said.

The present invention advantageously solves the above problems, and an imaging direction changing device for a robot according to the present invention includes:
A reflecting mirror that occupies a part of the imaging range of the camera provided by the robot and changes the imaging direction of the camera from the optical axis direction of the camera itself;
Reflecting mirror arrangement means for arranging the reflecting mirror at a predetermined position within the imaging range of the camera during predetermined work of the robot;
It is characterized by comprising.

In the robot imaging direction changing device according to the present invention, the reflecting mirror occupies a part of the imaging range of the camera included in the robot and changes the imaging direction of the camera from the optical axis direction of the camera itself. Arranging means arranges the reflecting mirror at a predetermined position within the imaging range of the camera during a predetermined operation of the robot.

  Therefore, according to the imaging direction changing apparatus for a robot of the present invention, the adjustment time required for camera mounting and calibration can be shortened. Also, extra operations other than work operations such as moving the imaging target to another location and imaging with the camera of the robot, or moving to the imaging range of the camera for imaging with another camera This eliminates the need to reduce work tact time.

Furthermore , in the imaging direction changing device for a robot according to the present invention, since the reflecting mirror occupies a part of the imaging range of the camera included in the robot, the camera itself is in an imaging range not occupied by the reflecting mirror. The image in the optical axis direction can be taken simultaneously with the image in the reflecting mirror.

In particular , in the imaging direction changing device for a robot according to the present invention, the robot is equipped with a hand for gripping a work object at the wrist at the tip of its arm, and the camera is directed to the optical axis direction of the camera itself. The reflector is fixed to the front of the hand, and the reflector arrangement means fixes and arranges the reflector on the hand within the imaging range of the camera and holds the image as an image in the reflector by the hand. The fixed work object is fixed so that it can be imaged .

Therefore, according to the robot imaging direction changing apparatus of the present invention, the hand is held by the camera fixed to the hand attached to the wrist at the tip of the robot arm with the optical axis direction forward of the hand. In addition to being able to directly image the work object going to the camera, the work object gripped by the hand in the gripping operation can be imaged via the reflecting mirror, so when imaging the work object with the camera fixed to the hand The extra movement of the hand can be eliminated.

(A) And (b) shows the product carrying-out system by the multi-joint type double-arm robot which respectively comprises 1st Embodiment of the imaging direction changing apparatus for robots of this invention, and 1st and 2nd reference form. It is a front view and a left side view. (A) And (b) is the top view which shows the product carrying-out system by the said articulated type | mold double-arm robot, and the perspective view seen from the upper left front diagonally. (A) And (b) is the right view which shows the product carrying-out system by the said articulated type | mold double arm robot, and the perspective view seen from the upper right front. (A), (b), (c), (d), and (e) are the objects to be worked on with the gripper of the hand that is attached to each wrist part of the two arms of the multi-joint type dual arm robot. It is the front view, side view, back view, top view, and bottom view which are shown with the imaging direction change apparatus for robots of the said 1st Embodiment in the state which clamped the thing. (A), (b), (c), and (d) are perspective views of the hand together with the imaging direction changing device for the robot of the first embodiment as seen from the upper right front, and the left front left. It is the perspective view seen from the upper part, the perspective view seen from the upper right diagonal rear, and the perspective view seen from the upper left diagonal rear. The articulated double-arm in order to explain the movable shaft of the wrist portion of the arm of the robot an enlarged part of the left arm shown in FIG. 3 (b) in the first embodiment and the first referential embodiment of the robot It is a perspective view shown with the imaging direction change apparatus for images. It is sectional drawing which shows the imaging direction change apparatus for robots of the said 1st Embodiment provided with the camera in the said hand. It is explanatory drawing which shows the state which is imaging the product which is going to be clamped with the said hand with the said camera, without going through the imaging direction change apparatus for robots of the said 1st Embodiment. It is explanatory drawing which shows the state which is imaging the label of the product clamped with the said hand with the said camera via the imaging direction change apparatus for robots of the said 1st Embodiment. It is sectional drawing which shows one reference example which deform | transformed the imaging direction change apparatus for robots of the said 1st Embodiment provided in the hand with which the wrist part of one arm of the said multi-joint type | mold double arm is equipped.

Hereinafter, embodiments of the present invention will be described in detail with reference to the drawings. Here, FIGS. 1A and 1B are a multi-joint type double arm comprising the first embodiment and the first and second reference forms of the imaging direction changing device for a robot of the present invention, respectively. A front view and a left side view showing a product delivery system by a robot, FIGS. 2 (a) and 2 (b) are a plan view showing the product delivery system by the multi-joint type double-arm robot and an oblique view from the upper left side. FIGS. 3 (a) and 3 (b) are a right side view and a perspective view of the product carrying system by the multi-joint type double-arm robot as seen from above and obliquely forward to the right.

  The product carry-out system using the multi-joint type double-arm robot shown in FIGS. 1 to 3 includes a work table T, for example, a multi-joint type double-arm robot 1 as the multi-joint type robot arranged toward the work table T, An articulated double arm comprising a belt conveyor C disposed between the articulated double arm robot 1 and the work table T and a product transport box B storing a plurality of products G to be worked in a random orientation. The robot 1 has a humanoid-type upper body 3 mounted on a hand-held or self-propelled carriage 2, and the upper body 3 has a torso 4, a head 5, a left arm 6, and a right arm 7. Examples of the multi-joint type double-arm robot 1 include those disclosed in Japanese Patent Application Laid-Open No. 2010-064198 previously disclosed by the applicant of the present application and double arms of the product name “NEXTTAG (registered trademark)” marketed by the applicant. B Tsu door can be used.

  The articulated double-arm robot 1 has two cameras 8 on its head 5 and two hands 10 having the same configuration on the wrist 9 of the left arm 6 and the right arm 7, for example. Each camera 11 is attached. The product G has a bottle shape with a cap, for example, and a product label L is attached to the front side of the body of the bottle, or a product name is printed.

However, in this product carry-out system, the articulated double-arm robot 1 performs image processing on the images from the two cameras 8 of the head 5 based on the principle of triangulation, for example, within the working area of the robot. The position and orientation of the belt conveyor C and the product transport box B that are positioned in the three-dimensional coordinate system of the robot are three-dimensionally recognized, and the position of each product G in the product transport box B in the three-dimensional coordinate system of the robot The hand G of the wrist arm 9 of the left arm 6 and the right arm 7 is alternately held using the image from the camera 11 and the products G in the product transport box B are sandwiched one by one as will be described later. The product is taken out and transferred onto the belt conveyor C with the front side to which the product label L is attached facing upward. And the belt conveyor C carries out those products G to the following process which is not shown in figure located in the right side in Fig.1 (a). In order to assist this unloading work, this product unloading system includes the robot imaging direction changing device 20 of the first embodiment, the robot imaging direction changing device 30 of the first reference embodiment, and the second embodiment . The robot imaging direction changing device 40 of the reference form is provided.

4 (a), FIG. 4 (b), FIG. 4 (c), FIG. 4 (d) and FIG. 4 (e) show the wrist portions of the two arms 6 and 7 of the multi-joint type double arm robot 1, respectively. 9 is a front view and a side view showing the hand 10 mounted on the robot 9 together with the robot imaging direction changing device 20 of the first embodiment in a state where the product G as a work object is sandwiched by the gripper 10e of the hand 10. FIG. 5 (a), FIG. 5 (b), FIG. 5 (c) and FIG. 5 (d) show the hand 10 as the robot of the first embodiment. The perspective view seen from the upper right oblique front, the perspective seen from the upper left oblique front, the perspective seen from the upper right oblique rear, and the oblique perspective seen from the upper left oblique rear. FIG. FIG. 6 is an enlarged view of a part of the left arm 6 shown in FIG. 3B for explaining the movable axis of the wrist portion 9 of each arm 6, 7 of the multi-joint type double arm robot 1. It is a perspective view shown with the imaging direction change apparatuses 20 and 30 for robots of this embodiment and 1st reference form.

The wrist portions 9 of the arms 6 and 7 of the multi-joint type double-arm robot 1 are arranged around the rolling axis C1 extending in the extending direction of the forearm, as representatively shown in FIG. 6 for the left arm 6 in FIG. Rolling with respect to the forearm, pitching with respect to the forearm around the pitching axis C2 extending in the vertical direction in FIG. 6 but normally extending in the horizontal direction, and extending in the horizontal direction in FIG. And has three movable axes, yawing to the forearm around the yawing axis C3.

  The hand 10 is detachably attached to the wrist 9 and has a coupling portion 10a for positioning and fixing the wrist 9 and connecting an electric circuit and a pneumatic circuit, and a flange 10b between the coupling portion 10a. A base portion 10c that is mounted with an air cylinder and is fixedly supported by the base portion 10c, and two fingers 10d are indicated by arrows in FIG. 6 by the forward and backward movement of the piston rod of the air cylinder in the base portion 10c. A normal gripper 10e composed of, for example, a parallel opening and closing chuck is driven, and the gripper 10e closes its two fingers 10d, and as shown in FIGS. Cap CA attached to BL can be clamped from the side, and the cap that has been clamped can be opened by opening the two fingers 10d. It is possible to release the CA.

  FIG. 7 is a sectional view showing the robot imaging direction changing device 20 of the first embodiment provided in the hand 10 together with the camera 11, and FIG. 8 is a robot imaging direction of the first embodiment. FIG. 9 is a diagram illustrating a state in which the product G to be clamped with the hand 10 is captured by the camera 11 without using the changing device 20, and FIG. 9 is a diagram illustrating a change in the imaging direction for the robot according to the first embodiment. It is explanatory drawing which shows the state which is imaging the label L of the product G currently clamped with the said hand 10 with the said camera 11 via the apparatus 20. FIG.

  The camera 11 has a hooded bracket 11a solidified on the flange 10b of the hand 10 and a lens optical axis LC fixed to the hooded bracket 11a so as to be substantially parallel to the extending direction of the two fingers 10d of the gripper 10e. The robot imaging direction changing device 20 according to the first embodiment includes a generally U-shaped stay 21 as a reflector arrangement unit, and a stay 21 of the stay 21. A flat mirror 22 as a reflecting mirror attached to the central portion, and loosely inserted into through holes (not shown) formed on both sides of the stay 21 and screwed into a female screw hole (not shown) of the hooded bracket 11a. As a result, the plane mirror 22 is fixed to the hooded bracket 11a so as to be adjustable in angle, and this also has a retaining screw 23 as a reflector arrangement means.

  Here, the camera body 11b is an image pickup range on the upper side in FIG. 7 with respect to the lens optical axis LC, and is reflected by the plane mirror 22 of the robot image pickup direction changing device 20 of the first embodiment. At the same time as imaging the imaging range R1 including the front or back (depending on the orientation of the product G gripped by the gripper 10e), the imaging range R2 is lower in FIG. 7 than the lens optical axis LC. As shown, the front direction of the camera body 11b is also imaged.

  As a result, the multi-joint type double-arm robot 1 first recognizes the position and posture of each product G with the camera 8 of the head 5, and then takes out the product G to be next taken out from the plurality of products G in the product transport box B. And the hand 10 is brought close to the product G with the arms 6 and 7, and the product G is imaged in the imaging range R2 of the camera 11 of the hand 10 and the positional relationship with the hand 10 is adjusted as shown in FIG. While gripping the side of the cap CA with the gripper 10e, the product G is gripped.

  Next, the articulated double-arm robot 1 pulls up the hand 10 as shown with respect to the right arm 7 in FIGS. 1 to 3 and positions the camera 11 on the upper side of the hand 10 as shown with respect to the left arm 6 in FIGS. In addition, the hand 10 is arranged so that the front or back surface of the bottle BL of the product G faces the camera 11 side, that is, the upper side, and as shown in FIG. In the imaging range R1 reflected by the flat mirror 22, the upward surface of the bottle BL of the product G is imaged, and the image L of the upward surface captured by the camera 11 is subjected to image processing so that the label L is present on the upward surface. If the front of the bottle BL is facing upward, and the label L is not present, it is determined that the front of the bottle BL is facing downward.

  Next, the multi-joint type double-arm robot 1 keeps the front of the bottle BL (label L side) facing upward, and keeps the orientation of the arms 6 and 7 as shown in FIG. By operation, the hand 10 is moved together with the product G above the belt conveyor C in operation, and the two fingers 10d are opened by the gripper 10e of the hand 10 so that the side portion of the cap CA that has been sandwiched is removed. The product G is released and transferred onto the belt conveyor C.

  On the other hand, the articulated double-arm robot 1 rotates the wrist 9 around the yawing axis C3 when the front of the bottle BL (label L side) faces downward, and the front (label) of the bottle G of the product G. The hand 10 is turned upside down so that the L side is facing upward, and then the arms 10 and 7 are operated to move the hand 10 together with the product G above the belt conveyor C in operation. By opening the two fingers 10d with the ten grippers 10e, the side portion of the cap CA that has been held is released, and the product G is transferred onto the belt conveyor C.

  Therefore, according to the robot imaging direction changing device 20 of the first embodiment, the camera 11 can be attached and calibrated only once at the time of the first attachment, so that the adjustment time required for them can be shortened. Further, the product G to be imaged is moved to another location and imaged by the camera 11 of the articulated double-arm robot 1 or moved to the imaging range of the camera for imaging by another camera. Since no extra operation is required, the tact time of the work can be shortened.

The robot imaging direction changing device 30 according to the first embodiment shown in FIGS. 1 to 3 is shown in FIG. 6 in an enlarged manner with respect to the left arm 6, and the yaw axis C <b> 3 is attached to the wrist portion 9 of each arm 6, 7. It has a bracket 31 fixed and supported so as not to rotate around, and a plane mirror 32 as a reflecting mirror fixed to the bracket 31 substantially in parallel with the extending direction of the yawing axis C3. The arm robot 1 recognizes the position and posture of each product G in the product transport box B in the three-dimensional coordinate system of the robot as described above by the camera 8 of the head 5 during the work of carrying out the product G. In addition, the presence or absence of the product G on the belt conveyor C reflected in the plane mirror 32 on the right arm 7 side, for example, included in the imaging range of the camera 8 at the time of recognition is imaged, and the product G on the belt conveyor C is imaged. If is left For example, it informs the surrounding workers with a lamp or the like (not shown) and stops taking out the product G from the product transport box B. The camera 8 of the head 5 uses the camera 8 at the time of recognition. For example, when the situation of the side of the product carry-out system reflected by the plane mirror 32 on the left arm 6 side included in the imaging range is imaged and it is recognized that a worker or the like is present near the side of the product carry-out system. For example, this is notified to surrounding workers by a lamp or the like (not shown) and removal of the product G from the product transport box B is suspended.

Furthermore, the robot imaging direction changing device 40 of the second reference form shown in FIGS. 1 to 3 includes a bracket 41 erected on a work table on which a product transport box B on the work table T is mounted, and the bracket. 41 has a concave mirror 42 as a reflecting mirror fixed substantially parallel to the extending direction of the yawing axis C3 or a concave mirror-like combination of a plurality of plane mirrors. During the unloading operation, the camera 8 of the head 5 recognizes the position and posture of each product G in the product transport box B in the three-dimensional coordinate system of the robot as described above. The situation behind the multi-joint type double-arm robot 1 reflected by the concave mirror 42 included in the imaging range of the camera 8 is imaged, and it is confirmed that an operator or the like exists in the vicinity of the rear of the product carry-out system. Once pauses the removal of product G from the product conveying box B together inform other workers around such for example a lamp or the like (not shown) to that effect.

Therefore, according to the robot imaging direction changing device 30 of the first reference form and the robot imaging direction changing device 40 of the second reference form, the position of the head 5 is changed from the position of the head 5 during normal carrying-out work. Since it is possible to check the presence or absence of the product G on the belt conveyor C and the intruders and intruders in the product carry-out system with or without the movement of the movable shaft, the original work Since no extra operation other than the operation for the camera is required, the tact time of the work can be shortened, and the camera 8 needs to be mounted and calibrated only once at the time of the first mounting, so that the adjustment necessary for them can be performed. Time can be shortened.

FIG. 10 is a cross-sectional view showing a reference example in which the robot imaging direction changing device according to the first embodiment provided in a hand attached to one wrist of a double arm of the multi-joint type double arm robot is modified. As in the first embodiment, the hand 10 includes a coupling portion 10a and a base portion 10c that is attached to the coupling portion 10a with a flange 10b interposed between the base portion 10c and a built-in air cylinder. Although not shown here, the base 10c is fixedly supported, and the two fingers 10d are driven to open and close as indicated by arrows in FIG. 6 by the forward and backward movement of the piston rod of the air cylinder in the base 10c. It has a normal gripper 10e made of an open / close chuck, and is attached to the bottle BL of the product G by this gripper 10e as shown in FIGS. Can hold the cap CA from the side, also, it is possible to release the cap CA that has been pinched.

  Further, the camera 11 provided in the hand 10 includes a bracket 11a with a hood solidified on the flange 10b of the hand 10, and a direction in which the two fingers 10d of the gripper 10e extend the lens optical axis LC to the bracket 11a with the hood. The robot imaging direction changing device 20 of this modified example is attached to a generally U-shaped stay 21 and a central portion of the stay 21 in a front view. The flat mirror 22 as a reflecting mirror and the through holes (not shown) formed on both sides of the stay 21 are loosely inserted into the female screw holes (not shown) of the hooded bracket 11a. It has a retaining screw 23 that is fixed to the bracket 11a with a hood so that the angle can be adjusted.

However, in this reference example , for example, by shifting the fixing position of the stay 21 to the bracket 11a with the hood from the position in the previous example, the camera body 11b of the camera 11 performs all imaging above and below the lens optical axis LC. In the range, the arrangement of the plane mirror 22 with respect to the camera body 11b is set so that the imaging range R3 reflected by the plane mirror 22 is imaged. If the robot imaging direction changing device 20 of this reference example is used for the camera 11 of the hand 10 of the double arms 6 and 7, for example, the gripper of the hand 10 while recognizing the position and posture of the product G with the camera 8 of the head 5, for example. When the product G is taken out from the product transport box B by holding the cap CA of the product G in 10d and the label L of the product G is imaged by the camera 11 of the hand 10, the plane mirror 22 is compared with the plane mirror 22 of FIG. Since the captured image range is wide, the label L can be enlarged in the camera image, so that the recognition accuracy of the label L can be improved. For example, there are different labels L on both sides of the bottle BL but there are different labels L on the surface. The high recognition accuracy can be used when identifying the.

  Although the present invention has been described based on the illustrated examples, the present invention is not limited to the above-described examples, and can be appropriately changed within the scope of the claims. For example, the articulated double-arm robot 1 Instead of this, the robot imaging direction changing apparatus of the present invention may be applied to a system that uses a conventional articulated work robot and performs an operation while imaging with a camera provided in the robot hand or the like.

Further, the shape of the reflecting mirror is not limited to the plane mirror or the concave mirror in the above embodiment.

  Thus, according to the imaging direction changing device for a robot of the present invention, the adjustment time required for camera mounting and calibration can be shortened. Also, extra operations other than work operations such as moving the imaging target to another location and imaging with the camera of the robot, or moving to the imaging range of the camera for imaging with another camera This eliminates the need to reduce work tact time.

DESCRIPTION OF SYMBOLS 1 Articulated type double arm robot 2 Bogie 3 Upper body 4 Torso 5 Head 6 Left arm 7 Right arm 8 Camera 9 Wrist 10 Hand 10a Coupling part 10b Flange 10c Base 10d Finger 10e Gripper 11 Camera 11a Bracket with hood 11b Camera body 20 Imaging direction changing device for robot according to the first embodiment 21 Stay 22 Plane mirror 23 Fastening screw 30 Imaging direction changing device for robot according to the second embodiment 31 Bracket 32 Plane mirror 40 Imaging direction changing device for the robot according to the third embodiment 41 Bracket 42 Concave mirror B Product transport box BL Bottle C Belt conveyor CA Cap G Product T Work table

Claims (1)

A hand for gripping the workpiece is attached to a wrist portion of the arm end of the robot, and the optical axis direction occupies a part of the imaging range of a camera which is fixed toward the front of the hand, imaging of the camera A reflector that changes the direction from the optical axis direction of the camera itself,
The fixed object is fixed to the hand and the reflecting mirror is fixedly arranged at a predetermined position within the imaging range of the camera during the predetermined work of the robot, and the work object gripped by the hand is imaged as an image in the reflecting mirror. An imaging direction changing device for a robot, comprising: a reflecting mirror arrangement means for enabling the robot.

Images