JP2023017385A - Robot control method, robot control system, control device and control program - Google Patents

Abstract

To provide a robot control method, a robot control system, a control device and a control program which enable even a mobile-type robot to efficiently perform operation.SOLUTION: A robot control method is executed in a system which is provided with a first photographing part arranged above a work-piece, a mobile-type robot which performs predetermined operation to the work-piece using a movable arm in which a second photographing part is arranged and a control device that controls the robot on the basis of images photographed by the first photographing part and the second photographing part respectively. The control device executes: a first step of storing correspondence information concerning a predetermined correspondence relation; a second step of storing identification information for identifying a position of the work-piece and a direction of the work-piece, on the basis of an image of the work-piece photographed by the first photographing part; and a third step of storing posture information concerning a posture of the arm at the time when the arm of the robot performs the predetermined operation to the work-piece, when the robot is at a reference position.SELECTED DRAWING: Figure 1

Description

The present invention relates to a robot control method, a robot control system, a control device, and a control program.

2. Description of the Related Art Conventionally, there have been robots that grip workpieces. The technology described in Patent Literature 1 includes a wide-range sensor and a narrow-range sensor. The wide-range sensor can image a wide range including the entire storage box in which the works are stored, and detect the rough position of each work included in the set of works. The narrow-range sensor takes a high-resolution image of a narrow range including a specific work and accurately measures the three-dimensional position and orientation of the work. The technique described in Patent Document 1 selects a specific workpiece to be gripped by a robot based on the rough position of the workpiece detected by a wide-area sensor, and selects the workpiece using a narrow-area sensor. A specific workpiece is precisely measured, and the robot is controlled based on the measurement results to grip and transport the workpiece. The technique described in Patent Literature 1 repeats the above processing until there are no more workpieces stored in the storage box.

JP 2008-087074 A

In general, many of the robots placed in factories and the like are of the stationary type. In other words, most of the robots placed in factories or the like are immovable stationary robots, and the arms move even if movable members such as arms are placed. Considering the manufacturing cost of the product, the manufacturing process of the product, and the like, it is sometimes desired to make the robot itself mobile.
However, if the robot itself is of a mobile type, for example, a robot that grips and transports a workpiece as described in the above-mentioned Patent Document 1, each time the robot is moved, the positional relationship between the workpiece and the robot and the sensor The positional relationship between the robot and the robot changes, and it takes time and effort to calibrate the above-described positional relationship before starting the robot operation.

SUMMARY OF THE INVENTION An object of the present invention is to provide a robot control method, a robot control system, a control device, and a control program that enable even a mobile robot to work efficiently.

According to one aspect of the robot control method, a movable arm having a first imaging unit arranged above a workpiece and a second imaging unit arranged thereon is provided, and a movable arm performs a predetermined operation on the workpiece with the arm. and a control device for controlling the robot based on the image captured by the first imaging unit and the image captured by the second imaging unit, the method being executed in a robot control system comprising: a first step in which a control device having a storage unit stores correspondence information relating to a predetermined correspondence relationship in the storage unit; a second step of storing identification information for identification in a storage unit; 3 steps, and in the first step, as correspondence information, a first correspondence relationship that associates the position of the first imaging unit with the position of the workpiece imaged by the first imaging unit, and the first imaging unit and a reference position of the robot, and a third correspondence relationship between the position of the second imaging unit and the position of the robot is stored in the storage unit. .

A robot control system according to one aspect includes a first imaging unit arranged above a work and a movable arm on which a second imaging unit is arranged. and a control device for controlling the robot based on the image captured by the first imaging unit and the image captured by the second imaging unit, wherein the control device controls a predetermined Correspondence information related to the correspondence relationship, identification information identifying the position and orientation of the work, and posture information related to the posture of the arm when performing a predetermined operation on the work with the robot arm when the robot is at the reference position. a storage unit for storing data; a first control unit for controlling the robot to move to a position where the predetermined operation is performed on the work when the robot actually performs the predetermined operation on the work; The robot arm takes a posture for imaging the work imaged by the first imaging unit with the second imaging unit based on the image of the work that is stored in the storage unit, the correspondence information, the identification information, and the orientation information stored in the storage unit. and a third control unit for controlling the arm of the robot to perform a predetermined operation on the workpiece based on the image of the workpiece captured by the second imaging unit, and storing The unit includes, as correspondence information, a first correspondence that associates the position of the first imaging unit with the position of the workpiece imaged by the first imaging unit, the position of the first imaging unit, and the reference position of the robot. and a third correspondence relationship between the position of the second imaging unit and the position of the robot.

A control device of one aspect includes a first imaging unit arranged above a work and a movable arm on which a second imaging unit is arranged. A robot control device for a system comprising a robot, wherein correspondence information relating to a predetermined correspondence relationship, identification information for identifying the position and orientation of the work, and an arm of the robot when the robot is at a reference position and a storage unit for storing posture information relating to the posture of the arm when a predetermined operation is performed on a workpiece by the robot. Based on the first control unit that controls the robot, the image of the work imaged by the first imaging unit, and the correspondence information, identification information, and orientation information stored in the storage unit, the image is captured by the first imaging unit. A second control unit that controls the arm of the robot so that it assumes a posture in which the work is imaged by the second imaging unit, and a predetermined operation is performed on the work based on the image of the work that is imaged by the second imaging unit. a third control unit configured to control the arm of the robot such that the storage unit associates the position of the first imaging unit with the position of the workpiece imaged by the first imaging unit as correspondence information; A correspondence relationship, a second correspondence relationship that associates the position of the first imaging unit with the reference position of the robot, and a third correspondence relationship that associates the position of the second imaging unit with the position of the robot. store the information it contains.

One aspect of the control program includes a movable arm having a first imaging unit arranged above the work and a second imaging unit arranged thereon, and performing a predetermined operation on the work with the arm. A robot control program for a system comprising a robot and a control device for controlling the robot, wherein the control device for controlling the robot is provided with correspondence information relating to a predetermined correspondence relationship, identification for identifying the position and orientation of the work. and a memory function for storing posture information relating to the posture of the robot arm when performing a predetermined operation on a work piece when the robot is in a reference position, and a memory function for storing the posture information regarding the posture of the arm when the robot arm performs a predetermined operation on the work piece when the robot is at the reference position. , a first control function for controlling the robot to move to a position for performing a predetermined operation on the workpiece, an image of the workpiece captured by the first imaging unit, correspondence information stored in the storage function, and identification information and posture information, a second control function for controlling the arm of the robot to take a posture in which the workpiece imaged by the first imaging unit is imaged by the second imaging unit, and imaged by the second imaging unit a third control function for controlling the arm of the robot to perform a predetermined operation on the work based on the image of the work; A first correspondence relationship that associates the position of the workpiece imaged by one imaging unit, a second correspondence relationship that associates the position of the first imaging unit with the reference position of the robot, and the position of the second imaging unit , and a third correspondence that associates the positions of the robots with each other.

According to one aspect of the robot control method, a movable arm having a first imaging unit arranged above a workpiece and a second imaging unit arranged thereon is provided, and a movable arm performs a predetermined operation on the workpiece with the arm. and a control device for controlling the robot based on the image captured by the first imaging unit and the image captured by the second imaging unit, the method being executed in a robot control system comprising: , correspondence information relating to a predetermined correspondence relationship, identification information for identifying the position and orientation of the workpiece based on the image of the workpiece captured by the first imaging unit, and the position of the robot when the robot is at the reference position. Since posture information relating to the posture of the arm when performing a predetermined operation on the workpiece by the arm is stored in the storage unit, even a mobile robot can efficiently perform the work based on the plurality of pieces of information described above. can be done.
The robot control system, the control device, and the control program of one aspect can produce the same effects as the robot control method of one aspect described above.

1 is a diagram for explaining a robot control system according to one embodiment; FIG. 1 is a block diagram for explaining a robot control system according to one embodiment; FIG. FIG. 4 is a diagram for explaining an example of a work; FIG. FIG. 4 is a diagram for explaining an example of gripping positions of a workpiece; 4 is a first flowchart for explaining a robot control method according to one embodiment; 7 is a second flowchart for explaining the robot control method according to one embodiment;

An embodiment of the present invention will be described below.

[Schematic configuration of robot control system 1]
FIG. 1 is a diagram for explaining a robot control system 1 according to one embodiment.

The robot control system 1 controls the robots 200 in various facilities where the robots 200 are used, such as factories, warehouses and stores that store goods, and distribution warehouses that are used when goods are transported. used for As illustrated in FIG. 1, the robot control system 1 controls a robot 200 that moves a workpiece 400 such as an article (or a component that constitutes the article) in an article manufacturing facility such as a factory.

The robot control system 1 includes a first imaging section 100 , a robot 200 and a control device 300 .

The first imaging unit 100 is, for example, a camera device that images the workpiece 400 and generates image information (first image information).

The robot 200 can move, for example, in a planar direction, and can relatively approach or move away from the workpiece 400 . The robot 200 has, for example, an arm 220 and a gripper 230 that grips the workpiece 400 at the tip of the arm 220 . The robot 200 also includes a second imaging section 210 at the tip of the arm 220 (a position relatively close to the tip (the gripping section 230)). The second imaging unit 210 is, for example, a camera device that images the workpiece 400 and generates image information (second image information).

For example, the control device 300 controls the robot 200 based on image information (first image information) generated by the first imaging unit 100 and image information (second image information) generated by the second imaging unit 210. to control. For example, the control device 300 controls the robot 200 positioned relatively close to the workpiece 400 to grip the workpiece 400 based on the first image information and the second image information. Note that the control device 300 may be mounted on the robot 200 , or may be arranged independently from the robot 200 without being mounted on the robot 200 .

[Detailed Configuration of Robot Control System 1]
Next, the robot control system 1 will be described in detail.
FIG. 2 is a block diagram for explaining the robot control system 1 according to one embodiment.

[First imaging unit 100]
FIG. 3 is a diagram for explaining an example of the workpiece 400. As shown in FIG.

For example, the first imaging unit 100 is fixedly arranged with respect to the arrangement place of the workpiece 400 . As an example, the first imaging unit 100 is arranged above the workpiece 400 . For example, when one or a plurality of workpieces 400 are stored in a storage container 401 such as a pallet (see FIG. 3), the first imaging unit 100 performs It may be arranged above. The "above" may include concepts such as "substantially above" and "diagonally above". That is, the first imaging unit 100 is arranged at a position where it can image the work 400 , as a specific example, at a position where it can image the work 400 accommodated in the container 401 .

The first imaging unit 100 may have, for example, an angle of view capable of imaging the entire container 401 in the planar direction.
The first imaging unit 100 images the workpiece 400 as a still image or moving image. When capturing a still image, the first imaging unit 100 may capture the still image continuously in terms of time (for example, at each predetermined timing). For example, the first imaging unit 100 generates first image information and transmits the first image information to the control device 300 when performing an imaging operation.

[Robot 200]
The robot 200 may be movable by having a traveling member such as wheels 201, for example. The robot 200 may be self-propelled, for example, by providing a drive unit (not shown) that rotates the wheels 201 and the like. Note that the robot 200 may run independently (automatically). Various known methods, for example, can be used for the automatic operation method of the robot 200 .

The robot 200 has a movable arm 220 on which a second imaging section 210 is arranged. As a specific example, the robot 200 may include an articulated arm 220 . The robot 200 has an operation section at the tip of the arm 220 for performing a predetermined operation on the workpiece 400 . The predetermined operation may be various operations such as gripping the workpiece 400, for example. In other words, the robot 200 may have a grip part 230 (manipulation part) at the tip of the arm 220, for example. As an example, the grip part 230 may be of a type that has two fingers and grips the workpiece 400 with those fingers.

For example, the second imaging unit 210 is arranged near the tip of the arm 220 (near the tip), that is, near the grip 230 . The second imaging unit 210 can capture an image of the work 400 to be subjected to a predetermined operation by the operation unit, that is, the work 400 to be gripped by the gripping unit 230 . The second imaging unit 210 images the workpiece 400 as a still image or moving image. When capturing still images, the second imaging unit 210 may capture still images continuously in terms of time (for example, at predetermined timings). For example, the second imaging section 210 may generate second image information and transmit the second image information to the control device 300 when performing an imaging operation.

[Control device 300]
The control device 300 controls the robot 200 based on an image (first image information) captured by the first imaging section 100 and an image (second image information) captured by the second imaging section 210 . As a specific example, the control device 300 determines a workpiece to be subjected to a predetermined operation by the robot 200, and when the robot 200 moves to a position for performing a predetermined operation on the workpiece, the second imaging unit The position and orientation of the arm 220 (second imaging unit 210) may be instructed to the robot 200 so that the workpiece can be imaged by the robot 210. FIG. The control device 300 includes a communication section 320 , a storage section 330 , a first control section 311 , a second control section 312 and a third control section 313 in order to control the robot 200 . The first control unit 311 , the second control unit 312 and the third control unit 313 may be implemented as one function of the control unit 310 (for example, an arithmetic processing unit or the like) of the control device 300 .

The communication unit 320 communicates with the first imaging unit 100, the second imaging unit 210, and the robot 200, respectively. That is, the communication section 320 may receive the first image information transmitted from the first imaging section 100 and the second image information transmitted from the second imaging section 210, for example. The communication unit 320 may transmit control information for controlling the robot 200 to the robot 200 under the control of the control unit 310, for example.

The storage unit 330 stores various information and programs, for example. The storage unit 330 stores, for example, correspondence information, identification information, and posture information.

Correspondence information is information about a predetermined correspondence relationship. That is, the correspondence information includes, for example, information on first correspondence, second correspondence, and third correspondence.

The first correspondence is a correspondence that associates the position of the first imaging unit 100 with the position of the work 400 imaged by the first imaging unit 100 . The first correspondence is, for example, the positional relationship in the planar direction between the first imaging unit 100 and the position of the work 400, that is, the relationship between the position of the first imaging unit 100 and the position of the work 400 when viewed from above. There may be.

The position of the workpiece 400 is not limited to one specific point, for example, and may be an area extending in the plane direction. The position of the work 400 may be a predetermined position where the work 400 is placed. That is, the position of the workpiece 400 may be a position (for example, a table or the like) where the container 401 that accommodates the workpiece 400 is placed. In this case, the first correspondence may be generated based on the predetermined positional relationship between the first imaging section 100 and the workpiece 400 and then stored in the storage section 330 . That is, for example, when the first correspondence is generated, the control unit 310 or the like includes the first correspondence in the correspondence information, or stores the information of the first correspondence as the correspondence information in the storage unit 330. may

Alternatively, the first correspondence may be set each time the workpiece 400 is placed in the imaging range of the first imaging unit 100 (for example, below the first imaging unit 100). That is, for example, when the work 400 is placed, the first imaging unit 100 may image the work 400 . In this case, based on the image (image information) of the workpiece 400 imaged by the first imaging unit 100, the control unit 310 and the like associate the position of the first imaging unit 100 with the position of the workpiece 400 as a first image. Correspondence may be generated. As an example, since the first imaging unit 100 is fixed, the control unit 310 can pre-set the positional relationship with the first imaging unit 100 in the image imaged by the first imaging unit 100, thereby enabling the work to be performed. 400 is placed and the work 400 is imaged by the first imaging unit 100, the positional relationship between the first imaging unit 100 and the work 400 (for example, the positional relationship in the planar direction) is acquired to perform the first correspondence. It is possible to create relationships. After generating the first correspondence, the control unit 310 can include the first correspondence in the correspondence information or store the information of the first correspondence as the correspondence information in the storage unit 330 .
Note that the first correspondence relationship is not limited to being set as in the above example, and may be set by various methods.

The second correspondence is a correspondence that associates the position of the first imaging unit 100 with the reference position of the robot 200 (the positional relationship between the first imaging unit 100 and the robot 200). The reference position of the robot 200 is, for example, when the movable robot 200 moves relatively close to the workpiece 400 (first imaging unit 100) and performs a predetermined operation (eg, gripping, etc.) on the workpiece 400. It may be a position where it is possible to perform That is, the reference position of the robot 200 may be, for example, a position to be a destination (arrival point) of the robot 200 that performs a predetermined operation, or may be a preset position. The reference position, for example, is not limited to one specific point, and may be an area extending in the plane direction. In this case, the second correspondence is, for example, the positional relationship in the plane direction between the position of the first imaging unit 100 and the reference position, that is, the positional relationship between the first imaging unit 100 and the reference position in plan view. Alternatively, it may be generated based on the positional relationship being set in advance, and then stored in the storage unit 330 . That is, for example, when the second correspondence is generated, the control unit 310 or the like includes the second correspondence in the correspondence information, or stores the information of the second correspondence as the correspondence information in the storage unit 330. may
Note that the second correspondence relationship is not limited to being set as in the above example, and may be set by various methods.

The third correspondence is a correspondence that associates the position of the second imaging unit 210 with the position of the robot 200 . That is, the third correspondence is, for example, the positional relationship in the planar direction between the position of the second imaging unit 210 and the position of the robot 200, that is, the position of the second imaging unit 210 and the position of the robot 200 when viewed from above. It can be a relationship. The second imaging unit 210 is installed on the arm 220 of the robot 200 . Therefore, even when the arm 220 operates in various patterns, the relationship between the second imaging unit 210 and the position of the robot 200 can be acquired in advance. The third correspondence relationship may be, for example, a relationship in which the position of the second imaging unit 210 and the position of the robot 200 are associated in advance according to various operation patterns of the arm 220, and the relationship is set in advance. It may be generated based on the above, and then stored in the storage unit 330 . That is, for example, when the third correspondence is generated, the control unit 310 or the like includes the third correspondence in the correspondence information, or stores the information of the third correspondence as the correspondence information in the storage unit 330. may
Note that the third correspondence relationship is not limited to being set as in the above example, and may be set by various methods.

The identification information is information that identifies the position of the work 400 and the orientation of the work 400 . Work 400 has various shapes. As a specific example, when there is a ring-shaped workpiece 400 or the like, the workpiece 400 may appear to have various shapes depending on its orientation. Also, when a plurality of works 400 overlap, the works 400 may appear to have various shapes depending on how the works 400 overlap. The shape (visible shape) of these workpieces 400 can be specified in advance. That is, the control unit 310 and the like may store the identification information in the storage unit 330, for example, when generating the identification information.
Note that the identification information is not limited to being set as in the above example, and may be set by various methods.

The identification information is information that specifies the orientation of the workpiece 400 according to the shape of the workpiece 400 recorded in the first image information obtained by imaging the workpiece 400 by the first imaging unit 100. good too. Further, when the plurality of works 400 are imaged by the first imaging unit 100, the identification information is determined according to the shape of the plurality of works 400 recorded in the first image information obtained by the imaging. (For example, when there are a plurality of works 400, the work 400 located on the uppermost surface, the work 400 located below, etc.) may be specified.

The posture information is information about the posture of the arm 220 when the arm 220 of the robot 200 performs a predetermined operation on the workpiece 400 when the robot 200 is at the reference position. Posture information is, for example, information about the posture of the arm 220 when the arm 220 operates in various patterns. As an example, the posture information includes the posture of the arm 220 when the articulated arm 220 is extended, and the posture of the arm 220 when the articulated arm 220 is bent. This is information about the various poses of the arm 220, which are shown in FIG. That is, the posture information is, for example, various postures of the arm 220 when performing a predetermined operation on the work 400 when the robot 200 is at the reference position. the posture of the arm 220, the posture when the second imaging unit 210 can image the target work 400, the posture of the arm 220 when the work 400 is conveyed, etc.). The postures of these arms 220 can be specified in advance. Therefore, posture information may be generated based on presetting various postures of arm 220 and then stored in storage unit 330 . That is, the control unit 310 and the like may store the posture information in the storage unit 330, for example, after generating the posture information.
Note that the posture information is not limited to being set as in the above example, and may be set by various methods.

The correspondence information, the identification information, and the orientation information described above are not limited to the examples generated by the control unit 310 and the like as described above, and may be generated outside the control device 300 (external device). In this case, for example, the control device 300 may acquire the correspondence information, the identification information, and the orientation information via the communication unit 320 and then store them in the storage unit 330 .

When storing the correspondence information, the identification information, and the orientation information in the storage unit 330 as described above, the control unit 310 controls the robot 200 as follows so that the robot 200 performs a predetermined operation on the workpiece 400. do.
In this case, the control unit 310 selects from among the plurality of works 400 arranged in an overlapping manner,
A workpiece 400 on which a predetermined operation is likely to be performed may be identified from the image captured by the first imaging unit 100, and the workpiece 400 to be subjected to the predetermined operation may be identified.

The first control unit 311 controls the robot 200 to move to a position where the predetermined operation is performed on the work 400 when the robot 200 actually performs the predetermined operation on the work 400 . That is, the first control unit 311 controls the robot 200 to move to a position relatively close to the work 400 so that the robot 200 can perform a predetermined operation on the work 400 . The position at which the predetermined operation is performed (the position relatively close to the workpiece 400) may be, for example, the reference position described above. The first control unit 311 moves the robot 200 by rotating traveling members such as the wheels 201 based on, for example, controlling the driving unit of the robot 200 . In this case, for example, when the robot 200 receives an instruction to move to the reference position based on the control signal transmitted from the first control unit 311, the robot 200 can move to the reference position independently (automatic operation). good.

The control signal may be, for example, a signal transmitted to the robot 200 via the communication unit 320 in order to control the robot 200 by the control unit 310, and includes various control contents for controlling the robot 200. It may be

Based on the image of the work 400 captured by the first imaging unit 100 and the correspondence information, identification information, and posture information stored in the storage unit 330, the second control unit 312 controls the image captured by the first imaging unit 100. The arm 220 of the robot 200 is controlled so that the workpiece 400 is imaged by the second imaging unit 210 .
For example, when the second control unit 312 acquires an image (first image information) captured by the first imaging unit 100 at a predetermined timing, the second control unit 312 performs image recognition based on the first image information, The work 400 to be recorded is recognized. The second control unit 312 can use various known methods for image recognition, for example.
The second control unit 312 takes the posture of the arm 220 for performing a predetermined operation on the workpiece 400 based on the first image information (result of image recognition), the correspondence information, the identification information, and the posture information, for example. A control signal is sent to the robot 200 to That is, since the second control unit 312 can specify the position of the work 400 based on the first image information and the correspondence information, for example, the work 400 can be determined based on the specified position, identification information, and posture information of the work 400 A control signal is transmitted to the robot 200 so that the arm 220 takes a posture for performing a predetermined operation on the robot 200 . In this case, the second control unit 312 may control the arm 220 to take the above-described posture so that the workpiece 400 can be imaged by the second imaging unit 210 .

Further, the second control unit 312 specifies the operation position when a predetermined operation is performed on the workpiece 400, for example, when the predetermined operation is to grip the workpiece 400, the content of specifying the location where the workpiece 400 is gripped. A control signal may be sent to the robot 200 that includes: Note that the second control unit 312 controls the direction of gripping by the gripping unit 230 (the direction of the dashed line illustrated in FIG. 4), the position in the image captured by the second imaging unit 210 (the posture of the arm 220 and the position of the second imaging unit 210). ) is recorded in advance in posture information, etc., a control signal including content specifying a position to grip the work 400 is transmitted to the robot 200 based on the posture information. may That is, the orientation information includes, for example, the above-described gripping position of the workpiece 400 and the position in the image captured by the second imaging unit 210 (such as the attitude of the arm 220 and the direction in which the workpiece 400 is captured by the second imaging unit 210). may include information on

The third control unit 313 controls the arm 220 of the robot 200 to perform a predetermined operation on the work 400 based on the image of the work 400 captured by the second imaging unit 210 .
As described above, when the second control unit 312 controls the robot 200, the second imaging unit 210 can image a specific work 400 (for example, the work 400 to be gripped). That is, for example, when the third control unit 313 acquires an image (second image information) captured by the second imaging unit 210, the third control unit 313 performs image recognition based on the second image information, and the image is recorded in the second image information. The workpiece 400 that is to be moved is recognized. The third control unit 313 can use various known methods for image recognition, for example.
However, since the robot 200 is movable, there is a possibility that the robot 200 is not positioned exactly at the reference position. (The positional relationship between the workpiece 400 and the second imaging unit 210) is shifted compared to when the robot 200 is accurately at the reference position.
Therefore, the third control unit 313 adjusts the posture of the arm 220 based on the image (second image information) of the work 400 captured by the second imaging unit 210 so that a predetermined operation can be performed on the work 400 . is transmitted to the robot 200 (for example, a signal including the three-dimensional coordinates of the second imaging unit 210 and the rotation (adjustment) directions of the three axes of the arm 220). That is, the third control unit 313 controls the posture of the arm 220 based on the second image information and the posture information described above, for example, so that a predetermined operation can be performed at a predetermined location on the workpiece 400 . In other words, the third control unit 313 controls the posture of the arm 220 so that the content of the image of the work 400 (the position of the work 400 in the image) is obtained based on the posture information described above.

FIG. 4 is a diagram for explaining an example of the gripping position of the workpiece 400. As shown in FIG.

The third control unit 313 may control the arm 220 to grip the workpiece 400 with the gripping unit 230 arranged at the tip of the arm 220 as the control of the arm 220 for performing a predetermined operation on the workpiece 400 . . For example, the third control unit 313 controls the posture of the arm 220 as described above so that the position and posture of the work 400 imaged by the second imaging unit 210 change the image content of the work 400 based on the above-described posture information. , a control signal is transmitted to the robot 200 so as to perform a predetermined operation (for example, gripping) on the workpiece 400 . For example, by controlling the arm 220 as described above, the third control unit 313 can obtain an angle at which the workpiece 400 can be gripped by the gripping unit 230. Since the arm 220 is adjusted so that the workpiece 400 can be gripped from the direction of , the workpiece 400 can be gripped by the gripper 230 by transmitting a control signal for gripping the workpiece 400 to the robot 200 .
This enables the robot 200 to perform a predetermined operation (for example, gripping) on the workpiece 400 . Further, for example, when the robot 200 grips the work 400, it becomes possible to transport the work 400 to another place.

[Robot control method]
Next, a robot control method according to one embodiment will be described.

First, a preparation method for controlling the robot 200, which is a robot control method according to an embodiment, will be described.
FIG. 5 is a first flowchart for explaining a robot control method according to one embodiment.

In step ST101, storage section 330 stores correspondence information. Correspondence information includes, for example, information on a first correspondence, a second correspondence, and a third correspondence. The first correspondence is a correspondence that associates the position of the first imaging unit 100 with the position of the work 400 imaged by the first imaging unit 100 . The second correspondence is a correspondence that associates the position of the first imaging unit 100 with the reference position of the robot 200 . The third correspondence is a correspondence that associates the position of the second imaging unit 210 with the position of the robot 200 .

In step ST102, storage section 330 stores the identification information. The identification information is information that identifies the position of the work 400 and the orientation of the work 400 .

In step ST103, storage section 330 stores posture information. The posture information is information about the posture of the arm 220 when the arm 220 of the robot 200 performs a predetermined operation on the workpiece 400 when the robot 200 is at the reference position. Further, the posture information includes, for example, the direction in which the gripping unit 230 grips the work 400 and the position in the image captured by the second imaging unit 210 (the posture of the arm 220 and the direction in which the second imaging unit 210 captures the work 400). etc.).

Next, a method for controlling a robot to perform a predetermined operation, which is a method for controlling a robot according to an embodiment, will be described.
FIG. 6 is a second flowchart for explaining the robot control method according to one embodiment.

In step ST201, the control section 310 controls the robot 200 to enter the premises. That is, when the robot 200 actually performs a predetermined operation on the workpiece 400, the first control unit 311 controls the robot 200 to move to a position (for example, a reference position) where the predetermined operation is performed on the workpiece 400. .

In step ST202, the second control section 312 operates based on the image of the workpiece 400 captured by the first imaging section 100 and the correspondence information, identification information, and posture information stored in steps ST101 to ST103 shown in FIG. , the arm 220 of the robot 200 is controlled so that the workpiece 400 imaged by the first imaging unit 100 is imaged by the second imaging unit 210 .

In step ST<b>203 , the third control section 313 controls the arm 220 of the robot 200 to perform a predetermined operation on the work 400 based on the image of the work 400 captured by the second imaging section 210 . That is, the third control unit 313 controls the movement of the arm 220 so that a predetermined operation can be performed on the work 400 based on the image (second image information) of the work 400 captured by the second imaging unit 210, for example. Control to fine-tune posture.

In step ST204, the third control unit 313 controls the arm 220 to perform a predetermined operation on the workpiece 400 so that the workpiece 400 is gripped by the gripping unit 230 disposed at the tip of the arm 220 (robot control unit 313). 200).

Each part of the control device 300 described above may be implemented as a function of an arithmetic processing device of a computer. That is, the first control unit 311, the second control unit 312, and the third control unit 313 (the control unit 310) of the control device 300 provide the first control function, the second control function, and the third control by an arithmetic processing unit of a computer or the like. Each may be implemented as a function (control function).
The control program can cause the computer to implement each function described above. The control program may be recorded in a computer-readable non-temporary recording medium such as an external memory or an optical disc.
Further, as described above, each part of the control device 300 may be realized by an arithmetic processing device of a computer or the like. The arithmetic processing unit or the like is configured by an integrated circuit or the like, for example. Therefore, each part of the control device 300 may be implemented as a circuit that constitutes an arithmetic processing device or the like. That is, the first control unit 311, the second control unit 312 and the third control unit 313 (the control unit 310) of the control device 300 are the first control circuit, the second control circuit and the 3 control circuit (control circuit).
Also, the communication unit 320 and the storage unit 330 of the control device 300 may be implemented as a communication function and a storage function including functions such as an arithmetic processing unit, for example. Also, the communication unit 320 and the storage unit 330 of the control device 300 may be realized as a communication circuit and a storage circuit by being configured by an integrated circuit or the like, for example. Also, the communication unit 320 and the storage unit 330 of the control device 300 may be configured as a communication device and a storage device by being configured by a plurality of devices, for example.

Although the term "information" is used in this specification, the term "information" can be interchanged with "data" and the term "data" can be interchanged with "information."

[Aspects and effects of the present embodiment]
Aspects of the present embodiment and effects of each aspect will be described below.

(Aspect 1)
According to one aspect of the robot control method, a movable arm having a first imaging unit arranged above a workpiece and a second imaging unit arranged thereon is provided, and a movable arm performs a predetermined operation on the workpiece with the arm. and a control device for controlling the robot based on the image captured by the first imaging unit and the image captured by the second imaging unit, the method being executed in a robot control system comprising: a first step in which a control device having a storage unit stores correspondence information relating to a predetermined correspondence relationship in the storage unit; a second step of storing identification information for identification in a storage unit; 3 steps, and in the first step, as correspondence information, a first correspondence relationship that associates the position of the first imaging unit with the position of the workpiece imaged by the first imaging unit, and the first imaging unit and a reference position of the robot, and a third correspondence relationship between the position of the second imaging unit and the position of the robot is stored in the storage unit. .
Thus, the robot control method can appropriately control a robot that performs a predetermined operation and efficiently perform the predetermined operation even if the robot is mobile. In addition, the robot control method can store various information necessary for the robot to perform a predetermined operation in the storage section, and can control the robot using the information.
In addition, since the robot control method can control the robot so as to perform predetermined operations a plurality of times using information stored in the storage unit, calibration (positioning of the imaging unit, workpiece, and robot) is performed each time a predetermined operation is performed. Since there is no need to perform a task of sharing coordinates (coordinate system), the time required to start a predetermined operation can be shortened.
In addition, since the robot control method uses the fixed first imaging unit (fixed camera), it is possible to reliably image the workpiece, thereby reliably controlling the robot to perform a predetermined operation. .
In addition, since the robot control method utilizes the second imaging unit (on-hand camera) arranged on the arm, even if the robot is mobile, the image within the angle of view of the second imaging unit can be detected based on the first image information and the like. The robot can be controlled to reliably perform a predetermined operation.

(Aspect 2)
In one aspect of the robot control method, when the robot actually performs a predetermined operation on the work, the control device controls the robot to move to a position for performing the predetermined operation on the work; Based on the image of the work imaged by the imaging unit and the correspondence information, the identification information, and the orientation information stored in the storage unit, the posture of the work imaged by the first imaging unit is determined by the second imaging unit. and a sixth step of controlling the robot arm to perform a predetermined operation on the workpiece based on the image of the workpiece captured by the second imaging unit. may be executed.
Thus, the robot control method can appropriately control a robot that performs a predetermined operation and efficiently perform the predetermined operation even if the robot is mobile.

(Aspect 3)
In one aspect of the robot control method, the control device controls the arm for performing a predetermined operation on the work, which is executed in the sixth step. may be controlled.
In other words, the robot control method can control the robot to reliably grip the workpiece even if the robot is mobile.

(Aspect 4)
A robot control system according to one aspect includes a first imaging unit arranged above a work and a movable arm on which a second imaging unit is arranged. and a control device for controlling the robot based on the image captured by the first imaging unit and the image captured by the second imaging unit, wherein the control device controls a predetermined Correspondence information related to the correspondence relationship, identification information identifying the position and orientation of the work, and posture information related to the posture of the arm when performing a predetermined operation on the work with the robot arm when the robot is at the reference position. a storage unit for storing data; a first control unit for controlling the robot to move to a position where the predetermined operation is performed on the work when the robot actually performs the predetermined operation on the work; The robot arm takes a posture for imaging the work imaged by the first imaging unit with the second imaging unit based on the image of the work that is stored in the storage unit, the correspondence information, the identification information, and the orientation information stored in the storage unit. and a third control unit for controlling the arm of the robot to perform a predetermined operation on the workpiece based on the image of the workpiece captured by the second imaging unit, and storing The unit includes, as correspondence information, a first correspondence that associates the position of the first imaging unit with the position of the workpiece imaged by the first imaging unit, the position of the first imaging unit, and the reference position of the robot. and a third correspondence relationship between the position of the second imaging unit and the position of the robot.
Thereby, the robot control system can achieve the same effects as the robot control method of one aspect described above.

(Aspect 5)
A control device of one aspect includes a first imaging unit arranged above a work and a movable arm on which a second imaging unit is arranged. A robot control device for a system comprising a robot, wherein correspondence information relating to a predetermined correspondence relationship, identification information for identifying the position and orientation of the work, and an arm of the robot when the robot is at a reference position and a storage unit for storing posture information relating to the posture of the arm when a predetermined operation is performed on a workpiece by the robot. Based on the first control unit that controls the robot, the image of the work imaged by the first imaging unit, and the correspondence information, identification information, and orientation information stored in the storage unit, the image is captured by the first imaging unit. A second control unit that controls the arm of the robot so that it assumes a posture in which the work is imaged by the second imaging unit, and a predetermined operation is performed on the work based on the image of the work that is imaged by the second imaging unit. a third control unit configured to control the arm of the robot such that the storage unit associates the position of the first imaging unit with the position of the workpiece imaged by the first imaging unit as correspondence information; A correspondence relationship, a second correspondence relationship that associates the position of the first imaging unit with the reference position of the robot, and a third correspondence relationship that associates the position of the second imaging unit with the position of the robot. store the information it contains.
Accordingly, the control device can achieve the same effect as the robot control method of one aspect described above.

(Aspect 6)
One aspect of the control program includes a movable arm having a first imaging unit arranged above the work and a second imaging unit arranged thereon, and performing a predetermined operation on the work with the arm. A robot control program for a system comprising a robot and a control device for controlling the robot, wherein the control device for controlling the robot is provided with correspondence information relating to a predetermined correspondence relationship, identification for identifying the position and orientation of the work. and a memory function for storing posture information relating to the posture of the robot arm when performing a predetermined operation on a work piece when the robot is in a reference position, and a memory function for storing the posture information regarding the posture of the arm when the robot arm performs a predetermined operation on the work piece when the robot is at the reference position. , a first control function for controlling the robot to move to a position for performing a predetermined operation on the workpiece, an image of the workpiece captured by the first imaging unit, correspondence information stored in the storage function, and identification information and posture information, a second control function for controlling the arm of the robot to take a posture in which the workpiece imaged by the first imaging unit is imaged by the second imaging unit, and imaged by the second imaging unit a third control function for controlling the arm of the robot to perform a predetermined operation on the work based on the image of the work; A first correspondence relationship that associates the position of the workpiece imaged by one imaging unit, a second correspondence relationship that associates the position of the first imaging unit with the reference position of the robot, and the position of the second imaging unit , and a third correspondence that associates the positions of the robots with each other.
Thereby, the robot control program can achieve the same effects as the robot control method of one aspect described above.

1 robot control system 100 first imaging unit 200 robot 210 second imaging unit 220 arm 230 gripping unit 300 control device 310 control unit 311 first imaging unit 312 second imaging unit 313 third imaging unit 320 communication unit 330 storage unit 400 work

Claims (6)

a first imaging unit arranged above the workpiece;
a movable robot having a movable arm on which a second imaging unit is arranged, and performing a predetermined operation on a workpiece using the arm;
A robot control method executed in a robot control system including a control device that controls the robot based on an image captured by the first imaging section and an image captured by the second imaging section. hand,
The control device comprising a storage unit,
a first step of storing correspondence information relating to a predetermined correspondence relationship in the storage unit;
a second step of storing, in the storage unit, identification information for identifying the position and orientation of the work based on the image of the work captured by the first imaging unit;
a third step of storing, in the storage unit, posture information relating to the posture of the arm when the arm of the robot performs a predetermined operation on a workpiece when the robot is at a reference position;
In the first step, as correspondence information,
a first correspondence that associates the position of the first imaging unit with the position of the workpiece imaged by the first imaging unit;
a second correspondence relationship that associates the position of the first imaging unit with the reference position of the robot;
A robot control method for storing, in the storage unit, information including a third correspondence relationship in which the position of the second imaging unit and the position of the robot are associated with each other. The control device
a fourth step of controlling the robot to move to a position where the predetermined operation is performed on the work when the robot actually performs the predetermined operation on the work;
The workpiece imaged by the first imaging unit is captured by the second imaging unit based on the image of the workpiece captured by the first imaging unit and the correspondence information, the identification information, and the posture information stored in the storage unit. a fifth step of controlling the arm of the robot to take an imaging posture by means of a unit;
a sixth step of controlling the arm of the robot to perform a predetermined operation on the workpiece based on the image of the workpiece captured by the second imaging unit;
The robot control method according to claim 1, wherein The control device controls the arm so as to grip the workpiece with a gripping portion disposed at the tip of the arm as the control of the arm for performing a predetermined operation on the workpiece, which is executed in the sixth step. The robot control method according to claim 2. a first imaging unit arranged above the workpiece;
a movable robot having a movable arm on which a second imaging unit is arranged, and performing a predetermined operation on a workpiece using the arm;
A robot control system comprising: a control device that controls the robot based on an image captured by the first imaging unit and an image captured by the second imaging unit,
The control device is
Correspondence information related to a predetermined correspondence relationship, identification information for identifying the position and orientation of the work, and the arm when performing a predetermined operation on the work by the arm of the robot when the robot is at a reference position a storage unit that stores posture information about the posture of the
a first control unit for controlling the robot to move to a position for performing a predetermined operation on the work when the robot actually performs a predetermined operation on the work;
The workpiece imaged by the first imaging unit is captured by the second imaging unit based on the image of the workpiece captured by the first imaging unit and the correspondence information, the identification information, and the posture information stored in the storage unit. a second control unit that controls the arm of the robot to take an imaging posture by a unit;
a third control unit that controls the arm of the robot to perform a predetermined operation on the work based on the image of the work imaged by the second imaging unit;
The storage unit stores, as correspondence information,
a first correspondence that associates the position of the first imaging unit with the position of the workpiece imaged by the first imaging unit;
a second correspondence relationship that associates the position of the first imaging unit with the reference position of the robot;
A robot control system that stores information including a third correspondence that associates the position of the second imaging unit with the position of the robot. a first imaging unit arranged above the workpiece;
a movable robot having a movable arm on which a second imaging unit is arranged, and performing a predetermined operation on a workpiece using the arm;
A control device for the robot of a system comprising
Correspondence information related to a predetermined correspondence relationship, identification information for identifying the position and orientation of the work, and the arm when performing a predetermined operation on the work by the arm of the robot when the robot is at a reference position a storage unit that stores posture information about the posture of the
a first control unit for controlling the robot to move to a position for performing a predetermined operation on the work when the robot actually performs a predetermined operation on the work;
The workpiece imaged by the first imaging unit is captured by the second imaging unit based on the image of the workpiece captured by the first imaging unit and the correspondence information, the identification information, and the posture information stored in the storage unit. a second control unit that controls the arm of the robot to take an imaging posture by a unit;
a third control unit that controls the arm of the robot to perform a predetermined operation on the work based on the image of the work imaged by the second imaging unit;
The storage unit stores, as correspondence information,
a first correspondence that associates the position of the first imaging unit with the position of the workpiece imaged by the first imaging unit;
a second correspondence relationship that associates the position of the first imaging unit with the reference position of the robot;
A control device that stores information including a third correspondence that associates the position of the second imaging unit with the position of the robot. a first imaging unit arranged above the workpiece;
a movable robot having a movable arm on which a second imaging unit is arranged, and performing a predetermined operation on a workpiece using the arm;
a control device that controls the robot;
A control program for the robot of a system comprising
In the control device that controls the robot,
Correspondence information related to a predetermined correspondence relationship, identification information for identifying the position and orientation of the work, and the arm when performing a predetermined operation on the work by the arm of the robot when the robot is at a reference position a memory function for storing posture information related to the posture of the
a first control function for controlling the robot to move to a position for performing a predetermined operation on the work when the robot actually performs the predetermined operation on the work;
The workpiece imaged by the first imaging unit is captured by the second imaging unit based on the image of the workpiece imaged by the first imaging unit and the correspondence information, the identification information, and the posture information stored in the storage function. a second control function for controlling the arm of the robot to assume an imaging posture by a unit;
realizing a third control function of controlling the arm of the robot to perform a predetermined operation on the workpiece based on the image of the workpiece captured by the second imaging unit;
The storage function stores, as correspondence information,
a first correspondence that associates the position of the first imaging unit with the position of the workpiece imaged by the first imaging unit;
a second correspondence relationship that associates the position of the first imaging unit with the reference position of the robot;
A control program that stores information including a third correspondence that associates the position of the second imaging unit with the position of the robot.

Images