JP2014144522A - Control apparatus, control method, robot, and robot system - Google Patents

Abstract

PROBLEM TO BE SOLVED: To ensure control of an operation to deform an object by an action of applying load to the object.SOLUTION: A storage section stores in advance an image of an object. A deformation calculation section calculates a degree of deformation of the object on the basis of an image of the object taken by an imaging section and the image of the object stored in the storage section. An action control section controls load applied to the object by a robot, according to the degree of deformation calculated by the deformation calculation section. This ensures control of an operation to deform the object.

Description

  The present invention relates to a control device, a control method, a robot, and a robot system.

2. Description of the Related Art A robot that includes a manipulator, a hand, and a camera and performs an operation for manipulating an object is known. As such a robot, for example, there is a robot that performs an assembling work to be assembled to another object by controlling a force applied to the object.
For example, Patent Document 1 describes a cooperative robot system that includes at least two robots and performs a fitting operation by a robot control unit including a force control unit so as to be force-controlled. In this cooperative robot system, the time after the pressing force becomes equal to or greater than a preset threshold is determined as the control effective time of the force control means. Thereby, the time during which the force generated by pressing the object is equal to or greater than the threshold is monitored, and it is detected that the work has been completed.

  Japanese Patent Application Laid-Open No. 2005-228561 describes a component assembly apparatus that inserts a lens into a lens housing hole provided in a lens barrel and assembles the lens barrel and a lens assembly including the lens. In this component assembling apparatus, the lens in the lens barrel is imaged by the camera, and the lens insertion state is confirmed by the captured image. Here, it is determined whether or not the inner diameter of the insertion opening falls within the allowable range of the value stored in advance from the captured image.

JP 2008-307665 A JP 2012-228747 A

For example, consider an operation of inserting one object into another object using the cooperative robot system described in Patent Document 1. When the postures of the objects are adjusted with high accuracy in advance before the work, the drag generated after the insertion is larger than the frictional force acting between the objects. For this reason, the fact that the generated force exceeds the threshold value is associated with the end of the work. On the other hand, when the postures of the objects do not match each other, the frictional force acting between the objects is larger than the drag generated after being inserted. For this reason, it may not be possible to make the generated force exceed the threshold and correspond to the end of the work.
Moreover, in the component assembly apparatus described in Patent Document 2, only the position where the lens inserted into the lens barrel is inserted is determined, and the operation may be terminated in a desired state when the object is deformed. could not.

  Therefore, the present invention has been made in view of the above problems, and a control device and a control method that can control (for example, end) an operation of deforming an object by an operation of applying a load to the object. An object is to provide a robot and a robot system.

(1) According to one aspect of the present invention, a storage unit that stores an image of an object in advance, an image of the object captured by the imaging unit, and a deformation of the object based on the image of the object stored in the storage unit A control device comprising: a deformation degree calculation unit that calculates a degree; and an operation control unit that controls a load applied by the robot to the object according to the deformation degree calculated by the deformation degree calculation unit. is there.
According to this configuration, when the robot performs an operation of applying a load to the object, the operation can be controlled based on the degree of deformation of the object calculated based on the captured image. Here, the degree of deformation is, for example, the degree of deformation of the whole or a part of the object with reference to the shape of the object before the operation is performed.

(2) In the control device described above, according to one aspect of the present invention, the deformation degree calculation unit is a target indicated by an image captured by the imaging unit from a shape of an object indicated by an image stored in the storage unit. A degree of deformation indicating a degree of deformation of the object is calculated, and the operation control unit terminates the operation of applying a load to the object when the degree of deformation reaches a predetermined threshold. .
According to this configuration, when the robot performs an operation of applying a load to the object, the operation can be terminated when the degree of deformation of the object calculated based on the captured image reaches a predetermined threshold. .

(3) In the control device described above, according to one aspect of the present invention, the motion control unit is configured such that when the degree of deformation calculated by the degree of deformation calculation unit is smaller than the threshold, the robot moves the object. The operation of applying a load to the is terminated.
According to this configuration, for example, when a load is applied, the degree of deformation initially increases, and an object whose degree of deformation decreases at a predetermined time, the robot operates when the degree of deformation decreases to a predetermined threshold. Can be terminated.

(4) In the control device described above, according to an aspect of the present invention, the motion control unit may be configured such that when the degree of deformation calculated by the degree of deformation calculation unit is greater than the threshold value continues for a predetermined time. Terminates the operation of applying a load to the object.
According to this configuration, for example, when a load is applied, the degree of deformation initially increases, and for an object whose increase in degree of deformation stops at a predetermined time, a state in which the degree of deformation is greater than a set threshold value continues for a predetermined time. After that, the robot operation can be terminated. Even if the degree of deformation temporarily increases above the threshold value, the operation is not immediately terminated, so that erroneous determination due to noise or a calculation error of the degree of deformation can be avoided.

(5) In the control device described above, according to an aspect of the present invention, the deformation degree calculation unit stores the image of the object captured by the imaging unit and the storage unit as the deformation degree of the object. It is characterized by calculating a normalized cross-correlation with the image of the object.
According to this configuration, the similarity between the shape of the object and the shape of the object set in advance can be quantified as an index value for the degree of deformation.

(6) In the control device described above, according to one aspect of the present invention, the operation control unit controls the operation of the robot so that the magnitude of a load applied to the object becomes a predetermined target value. The target value is controlled based on a temporal change in the deformation degree calculated by the deformation degree calculation unit.
According to this configuration, it is possible to adaptively control the load applied to the object by the robot in accordance with the temporal change in the degree of deformation of the object.

(7) In the control device described above, one aspect of the present invention is characterized in that the image captured by the imaging unit is an image related to deformation of at least a part of the object.
According to this configuration, it is possible to observe the deformation state of at least a part of the object based on the image captured by the imaging unit.

(8) In the control device described above, according to an aspect of the present invention, the image captured by the imaging unit is stored in the storage unit before the robot starts an operation of applying a load to the object. It is characterized by.
According to this configuration, in an environment where the robot performs an operation of applying a load to the object, an image of the object is acquired before the operation is performed. Therefore, the object corresponding to the environment is acquired based on the acquired image. The amount of change can be acquired.

(9) In the control device described above, according to one aspect of the present invention, an image acquisition unit that generates an image of the object based on the design data of the object and stores the generated image in the storage unit is provided. It is characterized by providing.
According to this configuration, since an image of the object can be generated based on the design data, the current degree of deformation of the object can be calculated based on the shape of the object at the time of design.

(10) In the control device described above, one embodiment of the present invention is characterized in that the storage unit is an auxiliary storage device that holds information even when power supply is stopped.
According to this configuration, since the image data indicating the image of the object is retained even when the supply of power is stopped, the image data can be used even when the supply of power is interrupted.

(11) In the control device described above, one aspect of the present invention is characterized in that at least one of the deformation degree calculation unit and the operation control unit is configured by an integrated circuit.
According to this configuration, the processing dedicated to each part is performed in the integrated circuit, so that the processing speed and size can be reduced.

(12) In the control device described above, according to one aspect of the present invention, the integrated circuit includes the deformation degree calculation unit and the operation control unit.
According to this configuration, the processing can be further reduced in size.

(13) One aspect of the present invention is a control method in a control device that controls the operation of a robot, wherein the target is based on an image of an object captured by an imaging unit and an image of the object stored in advance in a storage unit. A degree of deformation calculation process for calculating the degree of deformation of an object, and an operation control process for controlling a load applied to the object by the robot according to the degree of deformation calculated in the degree of deformation calculation process;
It is the control method characterized by having.
According to this configuration, when the robot performs an operation of applying a load to the object, the operation can be controlled based on the degree of deformation of the object calculated based on the captured image.

(14) According to one aspect of the present invention, a storage unit that stores an image of an object in advance, an image of the object captured by the imaging unit, and a deformation of the object based on the image of the object stored in the storage unit The robot includes: a deformation degree calculation unit that calculates a degree; and an operation control unit that controls a load applied to the object according to the deformation degree calculated by the deformation degree calculation unit.
According to this configuration, when the robot performs an operation of applying a load to the object, the operation can be controlled based on the degree of deformation of the object calculated based on the captured image.

(15) One aspect of the present invention is a robot system including a robot and a control device, wherein the control device stores a storage unit that stores an image of the object in advance, and an image of the object captured by the imaging unit. And a deformation degree calculating unit that calculates the degree of deformation of the object based on the image of the object stored in the storage unit, and the robot adds to the object according to the degree of deformation calculated by the degree of deformation calculating unit. An operation control unit that controls a load.
According to this configuration, when the robot performs an operation of applying a load to the object, the operation can be controlled based on the degree of deformation of the object calculated based on the captured image.

(16) According to one aspect of the present invention, when the degree of deformation indicating the degree of deformation of the object from the shape of the object indicated by the image stored in the storage unit reaches a predetermined threshold, a load is applied to the object. The robot is characterized in that the adding operation is terminated.
According to this configuration, when the robot performs an operation of applying a load to the object, the operation can be controlled based on the degree of deformation of the object calculated based on the captured image.

(17) According to one aspect of the present invention, when the degree of deformation indicating the degree of deformation of the target object from the shape of the target object indicated by the image stored in the storage unit reaches a predetermined threshold, the robot is notified of the target object. The control device is characterized by terminating the operation of applying a load.
According to this configuration, when the robot performs an operation of applying a load to the object, the operation can be controlled based on the degree of deformation of the object calculated based on the captured image.

(18) One embodiment of the present invention is a control device that terminates an operation in which a robot applies a load to an object based on an image of the object and information on a load that the robot applies to the object.
According to this configuration, when the robot performs an operation of applying a load to the object, the operation can be terminated depending on the degree of deformation of the object calculated based on the captured image.

1 is a schematic perspective view of a robot system according to a first embodiment. It is a figure for demonstrating the coordinate system in 1st Embodiment. It is a schematic block diagram of the control apparatus in 1st Embodiment. It is a schematic side view which shows an example of the component which concerns on an assembly | attachment operation | work, and a to-be-assembled component. It is a schematic block diagram of the operation control part in 1st Embodiment. It is a schematic block diagram of the load displacement conversion part in 1st Embodiment. It is a figure for demonstrating a sensor coordinate system and an object coordinate system. It is a schematic block diagram of the robot control part in 1st Embodiment. It is a flowchart which shows an example of the flow of a process of the control apparatus in 1st Embodiment. It is a conceptual diagram which shows an example of the deformation | transformation state of the to-be-assembled part accompanying the assembly | attachment operation | work by 1st Embodiment. It is a figure which shows an example of the normalization cross correlation accompanying the assembly | attachment operation | work by 1st Embodiment. It is a conceptual diagram which shows the other example of the deformation | transformation state of the to-be-assembled part accompanying the assembly | attachment operation | work by 1st Embodiment. It is a conceptual diagram which shows the other example of the normalized cross correlation accompanying the assembly | attachment operation | work by 1st Embodiment. It is a schematic block diagram of the control apparatus in 2nd Embodiment. It is a schematic block diagram of the operation control part in 2nd Embodiment. It is a schematic block diagram of the load displacement conversion part in 2nd Embodiment. It is a flowchart which shows an example of the flow of a process of the control apparatus in 2nd Embodiment.

Hereinafter, embodiments of the present invention will be described in detail with reference to the drawings.
[First Embodiment]
FIG. 1 is a schematic perspective view of a robot system 1 according to the first embodiment.
As shown in FIG. 1, the robot system 1 includes a control device 10, an articulated robot (hereinafter also referred to as a robot) 20, and an imaging device 30. Note that the scales of the components, structures, and the like in FIG. 1 are different from actual ones for the sake of clarity.

The control device 10 controls the operation of the robot 20. Details of the control by the control device 10 will be described later.
The robot 20 includes a support base 20a fixed to the ground, a manipulator unit (manipulator) 20b rotatably connected to the support base 20a, a gripping part 20c connected to the manipulator part 20b, and a force sensor 20d. Moreover, the holding part 20c has an opening / closing function. Thereby, the holding part 20c can hold | grip the target object (henceforth a target object). In the present embodiment, description will be given using the component 200 as an example of the object.

  The manipulator unit 20b is, for example, a six-axis vertical articulated manipulator, and can freely change the position and orientation of the component 200 gripped by the gripping unit 20c. The robot 20 operates one or both of the manipulator unit 20b and the gripping unit 20c under the control of the control device 10.

The force sensor 20d detects the force and moment applied to the grip 20c at predetermined time intervals (1 kHz as an example in the present embodiment). The force sensor 20d is, for example, a 6-axis or 7-axis force sensor. The force sensor 20 d outputs grip portion load information indicating the detected force and moment to the control device 10.
Note that the degree of freedom of the robot 20 is not limited to six axes. Moreover, you may install the support stand 20a in the place fixed with respect to the grounds, such as a floor, a wall, and a ceiling.
The robot 20 includes an encoder 21 (not shown). The encoder 21 detects the angle of each joint of the robot 20 and transmits the detected angle of each joint to the control device 10.

The part (object) 200 is a part to be assembled to the part to be assembled 210, for example, a connector (insertion tool). The component 200 has an insertion portion 201 inserted into the assembly component 210 at one end thereof.
The part to be assembled 210 is a part to be assembled to the part 200, for example, a connector (receiver). The assembly part 210 has a receiving part 211 (described later) into which an insertion part 201 (described later) of the part 200 is inserted at one end thereof. The other end of the assembly part 210 may be fixed with a jig.

  The imaging device 30 is installed at a position where, for example, the state in which the insertion portion 201 of the component 200 and the receiving portion 211 of the assembly component 210 are fitted together can be optically observed. The imaging device 30 is, for example, a CCD (Charge Coupled Device) camera. The imaging device 30 has an imaging region (ROI: Region of) for both the insertion unit 201 of the component 200 and the reception unit 211 of the assembly component 210 at a predetermined frame rate (in this embodiment, 30 Hz as an example). (Interest, region of interest, region of interest). The imaging device 30 outputs image data indicating a captured image (hereinafter also referred to as a camera image) obtained by imaging to the control device 10.

FIG. 2 is a diagram for explaining a coordinate system according to the first embodiment.
In the figure, ΣR is a robot coordinate system (XYZ coordinate system) based on the manipulator 20b, ΣS is a sensor coordinate system (x′y′z ′ coordinate system) based on the force sensor 20d, and ΣO is an object (FIG. In the example shown in FIG. 2, an object coordinate system (xyz coordinate system) based on the component 200) and ΣC is a camera coordinate system (uvw coordinate system) based on the imaging device 30.

FIG. 3 is a schematic block diagram of the control device 10 according to the first embodiment.
The control device 10 includes a gripping instruction unit 11, a start position movement instruction unit 12, a storage unit 15, an operation control unit 16, an image acquisition unit 17, a similarity calculation unit (deformation degree calculation unit) 18, a determination A unit 19 and a load calculation unit 40 are provided.

  The control device 10 according to the first embodiment does not destroy both the component 200 and the component to be assembled 210 by impedance control, and puts them in a state of being assembled with each other. The control device 10 detects the current position and orientation of the component 200 using the image data captured by the imaging device 30. Based on the detected position and orientation of the component 200, the control device 10 determines an axis that defines the operation when the robot 20 is moved by impedance control.

The gripping instruction unit 11 transmits to the robot 20 a gripping instruction signal instructing to grip the component 200 with the gripping unit 20c. As a result, the robot 20 grips the component 200 with the grip portion 20c.
The start position movement instruction unit 12 controls the manipulator unit 20b to output a work start instruction signal indicating that the component 200 is moved to a predetermined assembly work start position to the operation control unit 16. The assembly work start position is a position in the vicinity of the part 210 to be assembled, for example, a position where the center of gravity of the insertion part 201 is separated from the center of gravity of the reception part 211 in a vertical direction.

  The load calculation unit 40 calculates a load (for example, force and moment) applied to the component 200 from the grip unit load information input from the force sensor 20d, and sends the load information indicating the calculated load to the operation control unit 16. Output.

The motion control unit 16 controls the manipulator unit 20b of the robot 20 based on the work start instruction signal input from the start position movement instruction unit 12, and moves the component 200 to a predetermined assembly work start position.
For example, the operation control unit 16 detects the position of the grip 20c of the robot 20, and controls the position and posture of the manipulator unit 20b so that the position of the component 200 becomes the assembly work start position. When the position of the component 200 reaches the assembly work start position, the motion control unit 16 stops the change in the position and posture of the manipulator unit 20b.
Thereafter, the operation control unit 16 outputs an initial imaging instruction signal instructing to capture an image indicating the initial state of the component 200 and the assembly component 210 to the image acquisition unit 17.
Further, the operation control unit 16 sets target load information indicating a load necessary for the assembling work. The target load information indicates, for example, a target value of force applied to the component 200 when the component 200 is assembled to the component 210 to be assembled. This target value is, for example, a predetermined constant value.

  The motion control unit 16 moves the position of the grip 20c of the robot 20 based on the force applied to the component 200 indicated by the load information input from the load calculation unit 40. Thereby, the component 200 moves according to the position of the grip part 20c of the robot. Details of the process in which the operation control unit 16 moves the position of the grip 20c will be described later. The motion control unit 16 moves the position of the grip 20c of the robot 20 and then outputs a movement controlled signal indicating that the movement control has been performed to the image acquisition unit 17.

  That is, the motion control unit 16 performs movement control of the robot based on the force information, that is, movement control of the robot by impedance control. Then, the operation control unit 16 continues to control the robot 20 until the determination unit 19 determines that the distortion of the assembly target component 210 has reached a predetermined state and receives an end signal from the determination unit 19. Thereafter, the operation control unit 16 may repeat the operation of moving the component 200 to a predetermined assembly work start position.

When the initial imaging instruction signal is input from the operation control unit 16, the image acquisition unit 17 causes the imaging device 30 to image the component 200 and the assembly component 210. Then, the image acquisition unit 17 acquires image data obtained by imaging from the imaging device 30. The image acquisition unit 17 stores the acquired image data in the storage unit 15 as initial state image data.
When receiving the movement controlled signal from the operation control unit 16, the image acquisition unit 17 causes the imaging device 30 to image the component 200 and the assembly component 210. Then, the image acquisition unit 17 acquires image data obtained by imaging from the imaging device 30 and outputs the acquired image data to the similarity calculation unit 18.

The similarity calculation unit 18 reads the initial state image data from the storage unit 15. The similarity calculation unit 18 calculates the similarity between the image data input from the image acquisition unit 17 and the initial state image data. The similarity calculation unit 18 outputs the calculated similarity to the determination unit 19.
Specifically, the similarity calculation unit 18 may calculate a normalized cross-correlation function (NCC: Normalized Cross-Correlation) as the similarity, for example. The higher the similarity, the closer the current position and shape are to the initial position and shape. That is, the deformation amount from the initial shape is small with respect to the shape of the part 210 to be assembled to the part 200. On the other hand, a lower similarity indicates that the current position and shape do not approximate the initial position and shape. That is, it indicates that the deformation amount from the initial shape is large with respect to the shape of the part 210 to be assembled to the part 200.

The determination unit 19 determines whether or not the deformation of the part to be assembled 210 to be assembled to the part 200 is in a predetermined state based on the similarity input from the similarity calculation unit 18. The predetermined state means that the degree of deformation has been reached in a state where the assembly work has been completed. The criteria for determining that the predetermined state has been reached vary depending on the shape and material of the part to be assembled 210, the initial state of assembly, the direction of the force with which the part 200 to be assembled is assembled to the part to be assembled 210, etc. When 200 is inserted into the assembly part 210 and the insertion amount initially increases, the degree of deformation of the assembly part increases monotonously, and when the insertion amount exceeds a predetermined insertion amount threshold, Consider a model in which the degree of deformation is reduced and work is completed. In the following description, this model may be referred to as model 1.

In the model 1, the determination unit 19 determines whether or not the calculated similarity is greater than a predetermined threshold 2 after the calculated similarity is smaller than the predetermined threshold 1. When NCC is used as the similarity, each of the threshold values 1 and 2 is a real number smaller than 1 and larger than 0, and the threshold value 2 is larger than or equal to the threshold value 1. The threshold 2 is not necessarily equal to the similarity in the initial state (for example, NCC is 1).
In order to determine the end of work, the determination unit 19 may determine whether or not the calculated amount of change in similarity exceeds a predetermined threshold 3 from a negative value. When NCC is used as the similarity, the threshold value 3 is a real number larger than 0. The amount of change in similarity is, for example, a difference in similarity calculated before (previous) from recently calculated similarity.

  In addition, in the assembling work, as the part 200 is inserted into the part to be assembled 210 and the amount of insertion increases, the degree of deformation of the part to be assembled monotonously increases, and the amount of insertion exceeds the predetermined insertion amount threshold. Then, consider a model that terminates the operation when the amount of change in the degree of deformation approximates zero. In the following description, this model may be referred to as model 2.

In the model 2, the determination unit 19 determines that the calculated similarity is smaller than a predetermined threshold 4 and is longer than a predetermined time (for example, 0.1 second) for a change amount of the similarity. It is determined whether or not the absolute value is smaller than a predetermined threshold value 5. When NCC is used as the similarity, the threshold 4 is a real number smaller than 1 and larger than 0. The threshold value 5 is a real number larger than 0.
In order to determine the end of the work, the determination unit 19 determines the amount of change in the similarity for a time longer than a predetermined time after passing through a state in which the amount of change in the similarity is smaller than the predetermined threshold 6. It may be determined whether or not the absolute value of is less than a predetermined threshold value 5. Here, the threshold 6 is a real number smaller than 0.

Note that the determination unit 19 may perform smoothing by performing, for example, a moving average process on the similarity input before calculating the above-described similarity change amount. As a result, it is possible to perform an accurate determination by eliminating the influence of the noise having the temporarily generated similarity.
When the determination unit 19 determines that the deformation of the part to be assembled 210 is in a predetermined state, the determination unit 19 generates an end signal indicating that the work is completed, and outputs the generated end signal to the operation control unit 16.

FIG. 4 is a schematic side view showing an example of the part 200 and the part to be assembled 210 related to the assembling work.
FIG. 4 shows that the component 200 is arranged at the assembly work start position. In the example shown in FIG. 4, the component 200, the lower surface, and the lower surface of the assembly component 210 are arranged on the same plane. In this example, the assembly work start position is a position away from the center of the assembly target component 210 by a predetermined distance toward the left side of the drawing. This distance corresponds to the distance between point b and point d in FIG. 4 or the distance between point a and point e.

In FIG. 4, a broken-line rectangle surrounding the reception unit 211 indicates the imaging region r1. The imaging region r1 indicates the range of the subject represented in the image data acquired by the imaging device 30. The imaging region r <b> 1 includes the entire reception unit 211 and a region where the insertion unit 201 contacts the reception unit 211.
Note that the imaging region r1 may include at least a part of the part 200 and other parts of the part to be assembled 210 as long as the entire reception unit 211 is included. .
Further, even if only a part of the reception unit 211 is included in the imaging region r1, if the part is a part that is deformed by the contact of the part 200, at least a part of the part 200 or the assembled part Other parts of the component 210 may be included or may not be included.

An insertion part 201 having a tapered shape is formed at the right end of the component 200. The plug-in portion 201 is a portion surrounded by a line segment that connects the points a, b, and c. Point a is the vertex at the upper right end of the component 200, and point b is the other end of the line segment that is inclined to the lower right with point a as one end. The point c is the other end of the line segment perpendicular to the bottom of the component 200 with the point b as one end.
The surface of the insertion part 201 is inclined downward toward the right end. The inclination from point a to point b is substantially constant. On the left side of the point c, the surface direction of the component 200 is parallel to the bottom surface direction. Therefore, the vertical width of the insertion portion 201 increases from the point a toward the point b, and the width becomes the maximum at the point b. On the left side of the point c, the vertical width of the component 200 is substantially constant. This width is smaller than the maximum value of the vertical width of the insertion part 201.

On the other hand, the part to be assembled 210 is made of an elastic body, and a receiving portion 211 having a claw-like shape (a bowl shape) is formed at the left end. The accepting unit 211 is a part surrounded by a line segment connecting the d point, the e point, and the f point. The point d is the vertex at the lower left end of the part to be assembled 210, and the point e is the other end of the line segment that is inclined to the lower right with the point d as one end. The point f is the other end of a line segment perpendicular to the bottom of the part 210 to be assembled with the point e as one end.
The bottom surface of the reception unit 211 is inclined downward toward the right end. The slope from point d to point e is substantially constant and is equal to the slope from point b to point a. On the right side of the point e, the direction of the bottom surface of the component 210 to be assembled is parallel to the direction of the bottom surface of the component 200. Accordingly, in a state where the component 200 is not in contact with the assembly component 210, the vertical width of the receiving portion 211 increases from the point d to the point e, and the width becomes maximum at the point e. On the right side of the point f, the vertical width of the part to be assembled 210 is the narrowest. The part to be assembled 210 has a region where the vertical width is the narrowest from point f to point g. The horizontal width of the region, that is, the distance between the points f and g is at least larger than the distance between the points c and a.

  Therefore, when the insertion part 201 is moved toward the right side by the gripping part 20c of the robot 20, the surface of the insertion part 201 passing through the point b and the point a, the point d, and the point e are obtained. The bottom surface of the passing reception unit 211 comes into contact. When a force is further applied to the insertion part 201 toward the right side, the insertion part 201 is inserted into the part 210 to be assembled, and the receiving part 211 is deformed so as to open outside the part 200, that is, upward. When the insertion unit 201 is completely inserted into the reception unit 211, the reception unit 211 suddenly changes from a state of opening upward to a closed state before contacting the insertion unit 201. The fully inserted state is a state where points a, b, and c are on the right side of points e, d, and f, respectively. When this state is reached, the robot 20 stops the operation related to the assembling work.

  In the image represented in the imaging region r1, the deformation state of the part 210 to be assembled is mainly captured by the assembling work. Therefore, in the determination unit 19, in order to determine the end of the work of assembling the component 200 shown in FIG.

FIG. 5 is a schematic block diagram of the operation control unit 16 in the first embodiment.
The operation control unit 16 includes a compliant motion control unit 150.
The compliant motion control unit 150 controls the manipulator unit 20 b of the robot 20 based on the first control signal generated by the visual control unit 110 and the load information indicating the force acting on the robot 20. As an example, the compliant motion control unit 150 in the present embodiment controls the manipulator unit 20b of the robot 20 using an impedance control technique. Here, the compliant motion control unit 150 includes a load displacement conversion unit 120 and a robot control unit 140.

The load displacement conversion unit 120 acquires load information (for example, force applied to the component 200) input from the load calculation unit 40 at a predetermined sampling rate (1 kHz as an example in the present embodiment). Then, the load displacement conversion unit 120 generates a control signal indicating a relative position to which the robot 20 is moved based on the load information acquired from the load calculation unit 40. Then, the load displacement conversion unit 120 outputs the generated control signal to the robot control unit 140 at a predetermined rate (1 kHz as an example in the present embodiment).
In the present embodiment, as an example, the control signal is a signal indicating a relative position to which the robot 20 is moved. However, the control signal is not limited thereto, and may be a speed at which the robot 20 is moved.
Further, when an end signal is input from the determination unit 19, the load displacement conversion unit 120 stops outputting the control signal to the robot control unit 140. Thereby, the operation of the robot 20 is stopped.

The robot controller 140 detects the current position of the grip 20c of the robot 20 at a predetermined rate (in this embodiment, 1 kHz as an example).
Each time the control signal is input from the load displacement conversion unit 120, the robot control unit 140 adds the control signal and the detected current position signal X_c indicating the detected current position of the gripping unit 20c, and moves the robot 20 to the target. A position instruction signal X_d indicating the position is generated. The robot control unit 140 moves the position of the grip unit 20c of the robot 20 to the position indicated by the position instruction signal X_d. The robot control unit 140 performs the above process a predetermined number of times to move the position of the grip unit 20c of the robot 20. The number of times is a value (33 times in this embodiment) obtained by dividing the generation rate of the second control signal (1 kHz in this embodiment) by the generation rate of the first control signal (30 Hz in this embodiment). . Thereafter, the robot control unit 140 outputs a movement control completed signal indicating that movement control has been performed to the image acquisition unit 17.
When the control signal is a speed, the robot control unit 140 controls the robot 20 at the speed indicated by the control signal.

FIG. 6 is a schematic block diagram of the load displacement converter 120 in the first embodiment.
The load displacement conversion unit 120 includes a force target value setting unit 121 and a transfer function multiplication unit 122.
The force target value setting unit 121 sets a load indicated by the load information input from the load calculating unit 40, that is, a target value vector F_d indicating a target value of the force vector F_s detected by the force sensor 20d. Here, the force vector F_s and the target value vector F_d each include a force and a moment. The force target value setting unit 121 outputs, for example, a target vector signal indicating a target value vector F_d indicating a predetermined force target value to the transfer function multiplying unit 122. Further, the force target value setting unit 121 outputs a sensor output vector signal indicating the force vector F_s to the transfer function multiplication unit 122.

  An end signal may be input from the determination unit 19 to the force target value setting unit 121. When the end signal is input, the force target value setting unit 121 sets a zero vector as the force target value, and outputs a target vector signal indicating the set zero vector to the transfer function multiplication unit 122. By setting the force target value to 0 in this way, the load applied to the object can be set to 0 and the operation of the robot 20 can be terminated.

  The transfer function multiplying unit 122 transmits a predetermined transfer from the force vector F_s indicated by the sensor output vector signal input from the force target value setting unit 121 to the difference F_s−F_d between the target value vector F_d indicated by the target vector signal. Multiply function. Six elements of force and moment included in the force vector F_s, six elements of force and moment included in the target value vector F_d, and six elements of position and orientation included in Δx (→) are respectively represented by F_S [i]. , F_D [i] and Δx [i], the transfer function determined in advance is expressed by Expression (1).

  Equation (1) represents a transfer function used in impedance control. Here, a sign in which an arrow → is added on Δx is represented as Δx (→) in the text, and Δx (→) represents a vector after being multiplied by a transfer function. Further, F and [i], and a symbol consisting of S or D between them, are the force element F_S [i] included in the force vector F_s and the force included in the target value vector F_d, respectively. Element F_D [i]. In addition, m [i], d [i], and k [i] are expressed as Δx [→] each element Δx [ i] is a variable that is set so as to have a mechanical impedance characterized by an inertial mass m [i], a damper coefficient d [i], and a spring constant k [i], and is set appropriately for each element. The transfer function multiplication unit 122 outputs a post-multiplication signal indicating the vector Δx (→) after the transfer function multiplication to the robot control unit 140 as a control signal.

FIG. 7 is a diagram for explaining the sensor coordinate system and the object coordinate system.
In the drawing, a state where the gripping portion 20c grips the component 200 is shown. Moreover, in the same figure, the force sensor 20d and the assembly | attachment component 210 are shown. In addition, a sensor coordinate system 81 (x′y′z ′ coordinate system) having the origin of the center of the force sensor 20d is shown. Further, an object coordinate system 82 (xyz coordinate system) having one end of the component 200 as an origin is shown.

FIG. 8 is a schematic block diagram of the robot control unit 140 in the first embodiment.
The robot control unit 140 includes an angle position conversion unit 141, an addition unit 142, a position angle conversion unit 143, a subtraction unit 144, and an amplification unit 145.
The angle position conversion unit 141 acquires the angle θc of each joint of the robot 20 from the encoder 21 held by the robot 20. Then, the angle position conversion unit 141 calculates the current position of the grasping unit 20c from the acquired angle θc of each joint, and outputs a current position signal X_c indicating the calculated current position to the addition unit 142.
The adder 142 adds the current position signal X_c input from the angular position converter 141 and the control signal input from the load displacement converter 120 to generate a position instruction signal X_d. The addition unit 142 outputs the generated position instruction signal X_d to the position angle conversion unit 143.

  The position angle conversion unit 143, for example, from the target position to which the robot 20 indicated by the position instruction signal X_d input from the addition unit 142 is moved, the target position of the robot 20 is realized by a known inverse kinematic process. Calculate the target angle for each joint. Then, the position angle conversion unit 143 outputs a target angle signal indicating the calculated target angle to the subtraction unit 144.

  The subtraction unit 144 subtracts the current joint angle signal indicating the current joint angle θc input from the encoder 21 from the target angle signal input from the position angle conversion unit 143. The subtraction unit 144 outputs the difference signal obtained by the subtraction to the amplification unit 145.

The amplifying unit 145 outputs to the robot 20 a current signal obtained by amplifying the difference signal input from the subtracting unit 144 by multiplying by a predetermined gain _Kθ . Here, the current signal is, for example, a 6-axis vector indicating the current applied to the motor of each joint of the robot 20. Accordingly, the robot 20 drives the motor of each joint based on the current signal input from the amplifying unit 145, thereby moving the position of the gripping unit 20c to the target position indicated by the position instruction signal X_d.

FIG. 9 is a flowchart illustrating an example of a processing flow of the control device according to the first embodiment.
(Step S101) The start position movement instructing unit 12 controls the position of the manipulator unit 20b to move the gripping unit 20c to a position where the component 200 can be gripped. Thereafter, the process proceeds to step S102.
(Step S102) The gripping instruction unit 11 controls the gripping unit 20c to grip the component 200. Thereafter, the process proceeds to step S103.
(Step S103) The start position movement instructing unit 12 causes the operation control unit 16 to control the position of the manipulator unit 20b to move the component 200 to the assembly work start position near the component 210 to be assembled. Thereafter, the process proceeds to step S104.

(Step S <b> 104) The operation control unit 16 sets a target value of force necessary for a predetermined assembly work. Thereafter, the process proceeds to step S105.
(Step S <b> 105) The image acquisition unit 17 acquires image data indicating an image captured by the imaging device 30, and stores the acquired image data in the storage unit 15. The stored image data is initial state image data indicating the positional relationship between the component 200 and the assembly component 210. Thereafter, the process proceeds to step S106.

(Step S106) The operation control unit 16 controls the operation of the manipulator unit 20b so that the force detected by the force sensor 20d becomes the set target value as the load information input from the load calculation unit 40. Here, a force corresponding to the target value is applied to the component 200 and is assembled to the component 210 to be assembled. Thereafter, the process proceeds to step S107.
(Step S <b> 107) The image acquisition unit 17 acquires image data indicating an image captured by the imaging device 30, and outputs the acquired image data to the similarity calculation unit 18. The similarity calculation unit 18 calculates the degree of deformation (distortion) of the part to be assembled 210 based on the image data input from the image acquisition unit 17 and the initial state image data stored in the storage unit 15. The similarity calculation unit 18 calculates, for example, NCC as a deformation index. Thereafter, the process proceeds to step S108.

(Step S108) The determination unit 19 determines whether or not the calculated degree of deformation (distortion) has reached a predetermined state. As a criterion for determining whether or not a predetermined state has been reached, for example, the criterion according to the model 1 is used for the component 200 and the assembled component 210 illustrated in FIG. Here, the determination unit 19 determines whether or not the NCC has become larger than the threshold value 2 after the NCC has become smaller than the threshold value 1 described above. Or the determination part 19 determines whether the variation | change_quantity of NCC became larger than the above-mentioned threshold value 3 from the negative value.
If it is determined that the degree of deformation has reached a predetermined state (YES in step S108), the process proceeds to step S109. If it is determined that the degree of deformation is not in a predetermined state (NO in step S108), the process proceeds to step S106.
(Step S109) The motion control unit 16 sets the target value of force to zero. Thereby, the force applied to the component 200 becomes zero, and the operation of the manipulator unit 20b is stopped. And the control apparatus 10 complete | finishes the process of this flowchart.

Next, an example of a deformed state of the part to be assembled 210 associated with the assembling work according to the present embodiment will be described.
FIG. 10 is a conceptual diagram illustrating an example of a deformed state of the part 210 to be assembled accompanying the assembling work according to the first embodiment. In the example shown in FIG. 10, when a load is applied to the part 210 to be assembled, the shape of the receiving unit 211 that is a part thereof is elastically deformed.
FIGS. 10A to 10F show the positional relationship between the part 200 and the part to be assembled 210 at time frames 500, 1000, 1500, 2000, 2500, and 2950, respectively. This time is the time when the time when the initial state image data is acquired is set to 0, and thereafter, the number of frames of the acquired image data is indicated as a unit. FIG. 4 shows the positional relationship between the component 200 and the assembly component 210 in the time 0 frame.

  Here, a right-pointing arrow shown on the left side of the component 200 indicates the direction of the force applied to the component 200 by the grip 20 c of the robot 20. The part 200 moves to the right as time passes. FIG. 10A shows that the component 200 is not yet in contact with the assembly target component 210 at the time 500 frame. According to FIG. 10B, at the time 1000 frame, the component 200 starts to contact the assembly component 210, and the receiving portion 211 of the assembly component 210 begins to deform. 10 (b)-(e), the degree that the insertion part 201 of the component 200 is assembled to the receiving part 211 increases with time, and the degree of deformation of the receiving part 211 upwards increases. Is shown. According to FIG. 10 (f), when the part where the vertical width of the insertion part 201 is suddenly narrowed reaches just below the part where the vertical width of the receiving part 211 is maximized, the upper part is deformed upward. It is shown that the shape of the receiving unit 211 that has been abruptly approximates the shape in the initial state.

FIG. 11 is a diagram illustrating an example of a normalized cross-correlation (NCC) associated with the assembly work according to the first embodiment.
FIG. 11 shows a time change of NCC in the imaging region r1 shown in FIG. In FIG. 11, the horizontal axis represents time (frame), and the vertical axis represents NCC. In this example, one frame corresponds to 1/30 second. (A)-(f) shown at the starting point of each upward or downward arrow indicates the time when the images shown in FIGS. 10 (a)-(f) are acquired. According to FIG. 11, NCC is maximized at time 0 frame. The NCC gradually decreases until the vicinity of 1000 frames when the component 200 starts to contact the assembly target component 210. Thereafter, until the vicinity of 2600 frames, as the degree of deformation of the reception unit 211 increases, NCC decreases more significantly. Near 2600 frames, the NCC increases rapidly and approximates the initial NCC. At this time, the shape of the reception unit 211 rapidly approximates the shape in the initial state. After that, NCC gradually decreases.
Therefore, in this example, the end of the assembly work is accurately determined by determining that the state in which the NCC decreases rapidly from the state in which the NCC decreases using the determination criterion based on the model 1 has reached the predetermined threshold value. Can do. This avoids the risk that the unnecessary operation continues and the component 200 and the assembled component 210 are damaged.

FIG. 12 is a conceptual diagram showing another example of the deformation state of the part to be assembled 310 associated with the assembling work according to the first embodiment. The deformed state shown in FIG. 12 shows a state in which the placement of the fixture 311 that is a part thereof changes when a load is applied to the part 310 to be assembled. In the present embodiment and the embodiments described later, the deformation includes, for example, a change in the arrangement of a part or the whole of the target object in addition to the elastic deformation illustrated in FIG.
The component 300 is a component that is gripped by the gripping portion 20 c instead of the component 200 and is assembled to the assembly target component 310. The component 300 is, for example, an electronic circuit board. The component 300 is configured to be assembled with the component to be assembled 310 at the lower end when a force is applied from above. The right side surface of the component 300 has a notch 301. The assembly component 310 is a component to which the assembly component 300 is assembled, for example, a connector. The position of the part 310 to be assembled is fixed. The downward arrow above the component 300 indicates the direction of the force applied to the component 300 by the grip portion 20c.

The assembled component 310 has a groove that is assembled to the lower end of the component 300. At the right end of the part to be assembled 310, there is a fixture 311 whose lower end rotates counterclockwise when the part 300 is pushed down. When the component 300 is completely assembled in the groove of the component 310 to be assembled, the protruding end of the fixing tool 311 is fitted into the notch 301 on the right side surface of the component 300, and the component 300 is fixed to the component 310 to be assembled.
A broken-line rectangle around the fixture 311 indicates the imaging region r2. The imaging region r2 includes substantially the entire fixture 311 and the lower right end of the component 300.

12A to 12C show the positional relationship between the component 300 and the assembly component 310 at different times. Each time is delayed in the order of (a)-(c).
FIG. 12A shows that the part 300 is not yet in contact with the part 310 to be assembled. At this time, the fixture 311 is arranged from the lower end toward the upper right.
FIG. 12B shows that the lower end of the part 300 is in contact with the part to be assembled 310 and is being assembled. At this time, the angle formed by the direction of the fixture 311 and the right side of the component 300 is smaller than the angle shown in FIG.
FIG. 12 (c) shows that the lower end of the component 300 is completely assembled to the assembly component 310. At this time, the direction of the fixture 311 is substantially equal to the direction of the right side of the component 300, and the fixture 311 contacts the notch 301 of the component 300. In this state, even if a downward force is applied to the component 300, the arrangement of the fixture 311 does not change.

FIG. 13 is a conceptual diagram illustrating another example of normalized cross-correlation (NCC) associated with assembly work according to the first embodiment.
FIG. 13 shows the NCC time change for the imaging region r2 shown in FIG. In FIG. 13, the horizontal axis indicates time, and the vertical axis indicates NCC. In FIG. 13, specific values of time and NCC are omitted. (A)-(c) shown at the starting point of each upward or downward arrow indicates the time according to FIGS. 12 (a)-(c), respectively. According to FIG. 13, NCC is maximized in the initial state. NCC gradually decreases in the section until the part 300 starts to contact the part 310 to be assembled. This section includes the time according to FIG. Thereafter, in a section in the middle of the part 300 being incorporated in the part 310 to be assembled, the degree of deformation of the fixture 311, that is, the direction thereof changes, and NCC is more significantly reduced. This section includes the time according to FIG. Further, thereafter, in the section where the component 300 is completely assembled into the assembly component 310, the orientation of the fixture 311 does not change, and the NCC converges to a substantially constant value. This section includes the time according to FIG.
Therefore, in this example, it is determined that a state in which the amount of change in NCC is close to 0 for a time longer than a predetermined time has reached a predetermined threshold from a state in which NCC decreases using a determination criterion based on model 2. Thus, it is possible to accurately determine the end of the assembly work. This avoids the risk that the unnecessary operation continues and the component 300 and the assembled component 310 are damaged.

As described above, in this embodiment, an image (for example, initial state image data) of an object (for example, the assembled parts 210 and 310) is stored in the storage unit 15 in advance. In the present embodiment, the degree of deformation (for example, NCC) of the object is calculated based on the image of the object captured by the imaging unit (for example, the imaging device 30) and the image of the object stored in the storage unit 15. In this embodiment, the load applied to the object by the robot is controlled based on the degree of deformation.
Thus, the operation can be controlled by the degree of deformation calculated for the work involving deformation of the object. For example, when the degree of deformation reaches a preset threshold value (for example, the threshold value related to models 1 and 2), the robot can finish the operation accurately by ending the operation of applying a load to the object. It becomes.

[Second Embodiment]
Subsequently, a second embodiment of the present invention will be described. Elements common to those in FIG. 1 are denoted by the same reference numerals, and detailed description thereof is omitted. The external configuration of the robot system 1b in the second embodiment is the same as the external configuration of the robot system 1 (FIG. 1) in the first embodiment. However, the configuration of the robot system 1b is such that the control device 10 is changed to the control device 10b with respect to the configuration of the robot system 1 of FIG. In the control device 10b according to the second embodiment, the end determination process is different from that of the control device 10 according to the first embodiment.

FIG. 14 is a schematic block diagram of the control device 10b according to the second embodiment.
The control device 10b is obtained by changing the operation control unit 16 to the operation control unit 16b with respect to the control device 10 of FIG.

The operation control unit 16b has the same function as the operation control unit 16 in the first embodiment, but differs from the operation control unit 16 in the following points.
The motion control unit 16b controls the target value of the force required for the assembling work based on the temporal change in the similarity input from the similarity calculation unit 18. Thereby, the progress of the assembling work is adjusted according to the time change of the deformation degree of the object.
Further, when the operation control unit 16b determines that the set target value is equal to or larger than the upper limit value of the predetermined target value, the operation control unit 16b sets an abnormal end flag, and sets the set abnormal end flag to the robot 20. Output. Thereby, the operation of the manipulator unit 20b of the robot 20 is stopped (abnormal end). Here, the operation control unit 16b may set the target value to 0 and stop the operation of the manipulator unit 20b.
As a result, the assembly work is finished before the force applied to the parts 200 and 300 becomes excessive, and the risk of damage to the parts 200 and 300 and the parts to be assembled 210 and 310 can be avoided.

FIG. 15 is a schematic block diagram of the operation control unit 16b in the second embodiment.
In the operation control unit 16b, the load displacement conversion unit 120 is changed to a load displacement conversion unit 120b with respect to the operation control unit 16 of FIG.
FIG. 16 is a schematic block diagram of the load displacement conversion unit 120b in the second embodiment.
In the load displacement conversion unit 120b, the force target value setting unit 121 is changed to a force target value setting unit 121b with respect to the load displacement conversion unit 120 of FIG. The force target value setting unit 121b has the same function as the force target value setting unit 121 in the first embodiment, but differs from the force target value setting unit 121 in the following points.

The force target value setting unit 121b determines whether or not the change amount of the degree of deformation input from the similarity calculation unit 18 is larger than a predetermined change amount threshold value. Here, the force target value setting unit 121b calculates, as the amount of change, the difference in the amount of change calculated before (previous time) from the recently calculated degree of deformation. Thereby, when the force whose magnitude is the target value is applied to the parts 200 and 300, it is determined whether or not the degree of deformation of the parts to be assembled 210 and 310 is changing smoothly.
The force target value setting unit 121b increases and sets the already set force target value when it is determined that the amount of change in the degree of deformation is not larger than a predetermined change amount threshold. Here, the force target value setting unit 121b adds, for example, a predetermined increase amount of the force target value to the already set force target value, and determines the added value as the force target value. As a result, when the degree of deformation of the parts to be assembled 210 and 310 does not change smoothly, the force target value is increased and the change of the degree of deformation is promoted.

  The force target value setting unit 121b may smooth the input degree of deformation before calculating the difference in change amount. In the smoothing, the force target value setting unit 121b may adopt, for example, a value obtained by performing a moving average based on the degree of deformation from the past to the present for a predetermined time as the current degree of deformation.

  The force target value setting unit 121b determines whether or not the set force target value is equal to or greater than the upper limit value of the predetermined force target value. The operation control unit 16b sets the target value to 0 when it is determined that the set target value is equal to or greater than the upper limit value of the predetermined target value. Thereby, since the force applied to the object is controlled to 0, the operation of the manipulator unit 20b of the robot 20 is stopped.

FIG. 17 is a flowchart illustrating an example of a processing flow of the control device 10b according to the second embodiment.
The processing in steps S101 to S109 is the same as the processing in steps S101 to S109 in FIG.
However, if it is determined in step S108 that the degree of deformation of the part to be assembled is not in a predetermined state (NO in step S108), the process proceeds to step S209 without proceeding to step S106.

(Step S209) The operation control unit 16b determines whether or not the calculated change amount of deformation (distortion) is larger than a predetermined change amount threshold value. If it is determined that the change amount is greater than the predetermined change amount threshold (YES in step S209), the process proceeds to step S106. If it is determined that the change amount is not larger than the predetermined change amount threshold (NO in step S209), the process proceeds to step S210.

(Step S210) The operation control unit 16b increases and sets the target value of the force required for the assembly work. Then, it progresses to step S211.
(Step S211) The operation control unit 16b determines whether or not the set target value is smaller than the upper limit value of the predetermined target value. When it is determined that the target value is smaller than the upper limit value (YES in step S210), the process proceeds to step S106. When it is determined that the target value is not smaller than the upper limit value (NO in step S210), the process proceeds to step S212.
(Step S212) The operation control unit 16b sets an abnormal end flag and stops the operation of the manipulator unit 20b. And the control apparatus 10b complete | finishes the process of this flowchart. Thereafter, the control device 10b may repeat the processing of this flowchart.

  As described above, in the present embodiment, in addition to the configuration of the first embodiment, the operation of the robot 20 is controlled so that the load applied to the object becomes the target value, and the calculated deformation time The target value is controlled based on the change. Thereby, according to the progress of the operation | movement (for example, assembly | attachment operation | work) of the robot which the time change of a deformation degree shows, the operation | movement of the robot 20 can be accelerated | stimulated or suppressed. For example, by increasing the target value when the time change of the degree of deformation is small, the operation of the robot 20 can be promoted when the progress of the assembly work is slow. When the target value exceeds a predetermined upper limit value, the operation of the robot 20 is stopped, and the risk of damage or accidents of the parts 200 and 300 and the parts to be assembled 210 and 310 due to excessive force applied. Can be avoided.

  In each embodiment described above, the similarity calculation unit 18 calculates the similarity, and the determination units 19 and 19b determine whether or not the similarity has reached a predetermined state. However, the present invention is not limited to this. For example, the similarity calculation unit 18 may calculate the degree of difference, and the determination units 19 and 19b may determine whether or not a predetermined state has been reached based on the degree of difference. At this time, for example, the similarity calculation unit 18 may calculate SSD (Sum of Squared Differences) or SAD (Sum of Absolute Differences) as the degree of difference. For example, as the degree of difference is lower, the current shape of the assembled parts 210 and 310 is closer to the shape of the assembled parts 210 and 310 in the initial state.

Moreover, in each embodiment mentioned above, the system provided with a some apparatus may distribute and process each process of the control apparatuses 10 and 10b of each embodiment with those some apparatus.
The control devices 10 and 10b may be configured as a robot integrated with the robot 20 described above.
In the above description, the case where a part of the object (for example, the parts to be assembled 210 and 310) is deformed has been described as an example. However, in each of the above-described embodiments, the whole object may be deformed. Therefore, the imaging device 30 may capture an image showing the entire object, and the control devices 10 and 10b may use the image.

In addition, the control devices 10 and 10b according to each embodiment include a storage unit 15 that stores an image of an object (for example, the assembled parts 210 and 310) and an imaging unit (for example, the imaging device 30). A deformation degree calculation unit (similarity calculation unit 18) that calculates a deformation degree (for example, NCC) of the target object based on the captured image of the target object and an image of the target object (for example, initial state image data) stored in the storage unit 15. ) And an operation control unit 16 that controls the load applied to the object by the robot 20 according to the degree of deformation calculated by the degree of deformation calculation unit.
As described above, the degree of deformation indicates the degree of deformation of the object indicated by the image captured by the imaging unit from the shape of the object indicated by the image stored in the storage unit 15. Further, the robot 20 ends the operation of applying a load to the object when the calculated degree of deformation reaches a predetermined threshold value. Further, the control devices 10 and 10b end the operation of applying a load to the object by the robot 20 when the calculated degree of deformation reaches a predetermined threshold value.

With such a configuration of the control devices 10 and 10b, when the image of the first object whose degree of deformation does not reach the preset threshold is acquired, the manipulator unit 20b applies a load to the first object. Observe that the manipulator unit 20b ends the operation of applying a load to the first object when the image of the first object whose degree of deformation has reached a preset threshold value is acquired by continuing the operation. Can be confirmed. Even if the robot 20 is integrated with the control devices 10 and 10b, the same can be confirmed.
The preset threshold value is, for example, a threshold value related to model 1 and model 2 described above. Therefore, the image signal related to the image of the first object deformed with the degree of deformation in the state where the work has been completed, and the first object deformed with the degree of deformation which has not reached the degree of deformation in the state where the work has been completed. At least one of the image signals related to the image is prepared in advance. Then, each of the prepared image signals may be supplied to the image acquisition unit 17 and the operation of the robot 20 in an operating state may be observed.

In addition, the control devices 10 and 10b according to the embodiments end the operation in which the robot 20 applies a load to the object based on the image of the object and information on the load that the robot 20 applies to the object.
The configuration of the control devices 10 and 10b is such that when the load information obtained from the load calculation unit 40 is input to the operation control unit 16 of the control devices 10 and 10b when a load is applied to the first object, It can be confirmed by observing the following phenomenon.
Here, when the image of the first object whose degree of deformation does not reach the preset threshold is input to the image acquisition unit 17, the manipulator unit 20b applies a load corresponding to the load information to the first object. The same operation as when added is performed.

  When the image of the first object whose degree of deformation has reached a preset threshold value is input to the image acquisition unit 17, the manipulator unit 20b stops the operation of applying a load to the first object. The operation of applying a load according to the load information is, for example, increasing the force applied to the second object when the force indicated by the load information is smaller than a target value. Further, when the force indicated by the load information is larger than the target value, the force applied to the second object is reduced.

Therefore, the following set of information is prepared in advance; (1) An image signal related to the image of the first object deformed with the degree of deformation indicating the state of completion of the work and a force larger than the target value A set of load information, (2) a set of load information indicating an image signal related to the image of the first object deformed with a degree of deformation indicating a state where the work is completed, and a force smaller than the target value, and (3) work A set of image information relating to the image of the first object deformed with a degree of deformation indicating a state in which the operation is not completed and load information indicating a force greater than the target value, (4) a state in which the operation has not been completed The set of the load information which shows the force smaller than the image signal and target value which concern on the image of the 1st target object deform | transformed with the deformation | transformation degree shown. Here, a set of load information corresponding to all of (1) to (4) may be prepared, or a set of load information corresponding to at least (1) and (2) may be prepared. good.
Then, the image signal prepared for each set may be supplied to the image acquisition unit 17, the load information may be supplied to the operation control unit 16, and the operation of the robot 20 in the operating state may be observed.

The storage unit 15 is a storage device (auxiliary storage device) that is provided separately from the main storage device, such as a hard disk drive or an IC (Integrated Circuit) memory, and keeps information even when power supply is stopped. May be.
In addition, the operation control units 16 and 16b, the similarity calculation unit 18, and the determination unit 19 may each be configured by a dedicated integrated circuit, or all of the operation control unit 16, the similarity calculation unit 18, and the determination unit 19 may be included. Or these arbitrary combinations may be comprised by the integrated circuit respectively integrated.

Further, the image acquisition unit 17 is configured based on the design data indicating the dimensions of the components constituting the components 200 and 300 and the components to be assembled 210 and 310, and the position of a predetermined viewpoint, for example, CG technology or CAD (Computer Aided). Design) technology may be used to generate initial state image data. Further, the image acquisition unit 17 may input initial state image data generated based on the design data from an external device. The storage unit 15 stores the generated initial state image data. The control devices 10 and 10b may include an image acquisition unit that generates initial state image data using the CG technique or the CAD technique based on the design data and the position of the viewpoint.
In that case, the process of acquiring the initial state image data according to step S105 (FIGS. 9 and 17) may be omitted.

  Also, a program for executing each process of the control devices 10 and 10b of each embodiment is recorded on a computer-readable recording medium, and the program recorded on the recording medium is read into a computer system and executed. Thus, the various processes described above related to the control devices 10 and 10b may be performed.

  Here, the “computer system” may include an OS (Operating System) and hardware such as peripheral devices. Further, the “computer system” includes a homepage providing environment (or display environment) if a WWW system is used. The “computer-readable recording medium” refers to a portable medium such as a flexible disk, a magneto-optical disk, a ROM (Read Only Memory), a writable nonvolatile memory such as a flash memory, and a CD (Compact Disc) -ROM. A storage device such as a hard disk built in a computer system.

  Further, the “computer-readable recording medium” refers to a volatile memory (for example, DRAM (Dynamic) in a computer system serving as a server or a client when a program is transmitted via a network such as the Internet or a communication line such as a telephone line. Random Access Memory)) that holds a program for a certain period of time is also included. The program may be transmitted from a computer system storing the program in a storage device or the like to another computer system via a transmission medium or by a transmission wave in the transmission medium. Here, the “transmission medium” for transmitting the program refers to a medium having a function of transmitting information, such as a network (communication network) such as the Internet or a communication line (communication line) such as a telephone line. The program may be for realizing a part of the functions described above. Furthermore, what can implement | achieve the function mentioned above in combination with the program already recorded on the computer system, what is called a difference file (difference program) may be sufficient.

  As mentioned above, although embodiment of this invention was explained in full detail with reference to drawings, the concrete structure is not restricted to this embodiment, The design etc. of the range which does not deviate from the summary of this invention are included.

DESCRIPTION OF SYMBOLS 1, 1b Robot system 10, 10b Control apparatus 11 Grasp instruction | indication part 12 Start position movement instruction | indication part 15 Memory | storage part 16, 16b Operation control part 17 Image acquisition part 18 Similarity calculation part 19 Judgment part 20 Robot 20a Support stand 20b Manipulator part 20c Gripping unit 20d Force sensor 30 Imaging device 40 Load calculation unit 110 Visual control unit 120, 120b Load displacement conversion unit 121, 121b Force target value setting unit 122 Transfer function multiplication unit 140 Robot control unit 141 Angular position conversion unit 142 Addition unit 143 Position Angle conversion unit 144 Subtraction unit 145 Amplification unit 150 Compliant motion control unit

Claims (18)

A storage unit for storing in advance an image of the object;
A deformation degree calculation unit that calculates the deformation degree of the object based on the image of the object imaged by the imaging unit and the image of the object stored in the storage unit;
An operation control unit that controls a load applied by the robot to the object according to the degree of deformation calculated by the degree of deformation calculation unit;
A control device comprising: The deformation degree calculation unit calculates a deformation degree indicating a degree of deformation of the object indicated by the image captured by the imaging unit from a shape of the object indicated by the image stored in the storage unit,
The control device according to claim 1, wherein when the degree of deformation reaches a predetermined threshold, the operation control unit ends an operation in which the robot applies a load to the object.   3. The motion control unit according to claim 2, wherein when the degree of deformation calculated by the degree-of-deformation calculation unit becomes smaller than the threshold, the robot ends an operation of applying a load to the object. Control device.   The operation control unit terminates an operation in which the robot applies a load to the object when a state in which the deformation degree calculated by the deformation degree calculation unit is greater than the threshold value continues for a predetermined time. Item 3. The control device according to Item 2.   The deformation degree calculation unit calculates a normalized cross-correlation between the image of the object captured by the imaging unit and the image of the object stored in the storage unit as the deformation degree of the object. The control device according to claim 1. The motion control unit controls the motion of the robot so that the magnitude of the load applied to the object becomes a predetermined target value;
The control device according to claim 1, wherein the target value is controlled based on a temporal change in the degree of deformation calculated by the degree of deformation calculation unit.   The control apparatus according to claim 1, wherein the image captured by the imaging unit is an image related to deformation of at least a part of the object.   The control according to any one of claims 1 to 7, wherein an image captured by the imaging unit is stored in the storage unit before the robot starts an operation of applying a load to the object. apparatus. An image acquisition unit that generates an image of the target based on the design data of the target and stores the generated image in the storage;
The control apparatus according to claim 1, further comprising:   The control device according to claim 1, wherein the storage unit is an auxiliary storage device that retains information even when power supply is stopped.   The control apparatus according to claim 1, wherein at least one of the deformation degree calculation unit and the operation control unit is configured by an integrated circuit.   The control device according to claim 11, wherein the integrated circuit includes the deformation degree calculation unit and the operation control unit. A control method in a control device for controlling the operation of a robot,
Deformation degree calculation process for calculating the degree of deformation of the object based on the image of the object imaged by the imaging unit and the image of the object previously stored in the storage unit;
An operation control process for controlling a load applied to the object by the robot in accordance with the deformation degree calculated in the deformation degree calculation process;
A control method characterized by comprising: A storage unit for storing in advance an image of the object;
A deformation degree calculation unit that calculates the deformation degree of the object based on the image of the object imaged by the imaging unit and the image of the object stored in the storage unit;
An operation control unit that controls a load applied to the object according to the degree of deformation calculated by the degree of deformation calculation unit;
A robot characterized by comprising: A robot system comprising a robot and a control device,
The control device includes:
A storage unit for storing in advance an image of the object;
A deformation degree calculation unit that calculates the deformation degree of the object based on the image of the object imaged by the imaging unit and the image of the object stored in the storage unit;
An operation control unit that controls a load applied to the object by the robot according to the degree of deformation calculated by the degree-of-deformation calculation unit;
A robot system comprising:   When the degree of deformation indicating the degree of deformation of the object from the shape of the object indicated by the image stored in the storage unit reaches a predetermined threshold, the operation of applying a load to the object is terminated. robot.   When the degree of deformation indicating the degree of deformation of the object from the shape of the object indicated by the image stored in the storage unit reaches a predetermined threshold, the operation of applying a load to the object is terminated. Control device.   A control device that terminates an operation in which the robot applies a load to the object based on an image of the object and information on a load applied by the robot to the object.

Images