CN118288288A

CN118288288A - Workpiece assembly method, device and system, electronic equipment and storage medium

Info

Publication number: CN118288288A
Application number: CN202410535873.6A
Authority: CN
Inventors: 王彪; 陈祖文
Original assignee: Hangzhou Hikrobot Co Ltd
Current assignee: Hangzhou Hikrobot Co Ltd
Priority date: 2024-04-29
Filing date: 2024-04-29
Publication date: 2024-07-05

Abstract

The embodiment of the application provides a workpiece assembly method, a device, a system, electronic equipment and a storage medium, wherein the method comprises the following steps: controlling a mechanical arm clamping a target workpiece to move to an image acquisition area, and acquiring a target image acquired by image acquisition equipment; determining the current pose of the clamp flange, the current pose of the target workpiece and the pose of the object to be assembled based on the target image and parameters of the image acquisition equipment calibrated in advance; calculating the final pose of the clamp flange according to the current pose of the clamp flange, the current pose of the target workpiece, the pose of the object to be assembled and the preset assembly pose relation; the preset assembly pose relationship is determined based on the pose relationship when the teaching workpiece corresponding to the target workpiece and the teaching object corresponding to the object to be assembled are correctly assembled; and controlling the mechanical arm to assemble the target workpiece to the object to be assembled based on the current pose of the clamp flange and the final pose of the clamp flange. And workpiece assembly is performed based on a preset assembly pose relationship, so that the workpiece assembly precision is improved.

Description

Workpiece assembly method, device and system, electronic equipment and storage medium

Technical Field

The present application relates to the field of industrial automation technology, and in particular, to a method, an apparatus, a system, an electronic device, and a storage medium for assembling a workpiece.

Background

With the continuous development of industrial automation technology, in the industries of automobile manufacturing and the like, the mechanical arm is widely applied to a workpiece assembly link, so that the workpiece assembly efficiency is improved. Generally, when a workpiece is assembled by using a mechanical arm, a carrier such as an AGV (Automated Guided Vehicl, automatic guided vehicle) transports an object to be assembled to a preset position, the object to be assembled is fixed to the preset position according to a teaching pose by means of a physical clamping position or the like, and then the mechanical arm holding a target workpiece is moved to a fixed position according to a teaching mode, and the target workpiece is assembled with the object to be assembled. However, in the actual assembly process, since there is a case where the object to be assembled cannot be fixed at a preset position accurately in accordance with the teaching pose, in this case, the robot arm cannot complete the assembly of the workpiece in accordance with the teaching manner.

In order to overcome the above problems, in an assembling method, the pose relationship between the fixture flange of the mechanical arm and the object to be assembled during the correct assembly can be obtained through teaching in advance. In the actual assembly process, the flange pose of the clamp of the mechanical arm during the correct assembly can be calculated according to the pose of the object to be assembled and the pose relation, and the workpiece is assembled according to the flange pose of the clamp.

However, in the above-mentioned assembly method, after the target workpiece is clamped by the clamp flange due to the unreasonable design of the clamp, the relative pose of the target workpiece and the clamp flange deviates from the teaching relative pose, so that the target workpiece and the object to be assembled cannot be assembled accurately according to the calculated clamp flange pose of the mechanical arm when assembled correctly, and the workpiece assembly precision is still not high.

Disclosure of Invention

The embodiment of the application aims to provide a workpiece assembly method, a device, a system, electronic equipment and a storage medium, so as to improve workpiece assembly precision. The specific technical scheme is as follows:

In a first aspect, an embodiment of the present application provides a method for assembling a workpiece, the method including:

Controlling a mechanical arm clamping a target workpiece to move to an image acquisition area of image acquisition equipment, and acquiring a target image acquired by the image acquisition equipment, wherein a pre-placed object to be assembled is positioned in the image acquisition area;

Determining the current pose of a clamp flange of the mechanical arm, the current pose of a target workpiece and the pose of an object to be assembled based on the target image and the parameters of the image acquisition equipment calibrated in advance;

calculating the final pose of the clamp flange according to the current pose of the clamp flange, the current pose of the target workpiece, the pose of the object to be assembled and the preset assembly pose relation; the preset assembly pose relationship is determined based on the pose relationship when the teaching workpiece corresponding to the target workpiece and the teaching object corresponding to the object to be assembled are correctly assembled in the teaching process;

and controlling the mechanical arm to assemble the target workpiece to the object to be assembled based on the current pose of the clamp flange and the final pose of the clamp flange.

Optionally, the step of calculating the final pose of the fixture flange according to the current pose of the fixture flange, the current pose of the target workpiece, the pose of the object to be assembled and the preset assembly pose relationship includes:

And calculating the final pose of the clamp flange when the target workpiece and the object to be assembled are correctly assembled based on the current pose relationship, the object pose to be assembled and the preset assembly pose relationship, wherein the current pose relationship is the pose relationship between the clamp flange of the mechanical arm and the target workpiece, which is characterized by the current pose of the clamp flange and the current pose of the target workpiece.

Optionally, the step of calculating the final pose of the fixture flange when the target workpiece and the object to be assembled are correctly assembled based on the current pose relationship, the object to be assembled pose and the preset assembly pose relationship includes:

The final pose of the fixture flange is calculated according to the following formula:

T′2R＝obj₂2R×(obj₂2obj₁)^-1×(obj₁2R)^-1×T2R

Wherein T'2R is the final pose of the clamp flange under the mechanical arm coordinate system; obj ₂ R is the pose of the object to be assembled under the coordinate system of the mechanical arm; the obj ₂2obj₁ is a preset assembly pose relation matrix; obj ₁ R is the current pose of the target workpiece in the mechanical arm coordinate system, and T2R is the current pose of the fixture flange in the mechanical arm coordinate system.

Optionally, the method for constructing the preset assembly pose relationship includes:

the teaching mechanical arm which is used for clamping the teaching workpiece is controlled to move to a teaching image acquisition area of a teaching image acquisition device, and a first teaching image acquired by the teaching image acquisition device is acquired, wherein a teaching object placed in advance is positioned in the teaching image acquisition area;

determining the current pose of a teaching fixture flange, the current pose of a teaching workpiece and the current pose of a teaching object of the teaching mechanical arm based on the first teaching image and the parameters of the pre-calibrated teaching image acquisition equipment;

Controlling the teaching mechanical arm to assemble the teaching workpiece to the teaching object, and acquiring a second teaching image acquired by the teaching image acquisition device;

Determining the final pose of the teaching fixture flange of the teaching mechanical arm when the teaching mechanical arm is correctly assembled according to the second teaching image and the parameters of the pre-calibrated teaching image acquisition equipment;

And calculating the preset assembly pose relation between the teaching workpiece and the teaching object when the teaching fixture is correctly assembled based on the current pose of the teaching fixture flange, the current pose of the teaching workpiece, the pose of the teaching object and the final pose of the teaching fixture flange.

Optionally, the step of calculating the preset assembly pose relationship between the teaching workpiece and the teaching object when the teaching workpiece is correctly assembled based on the current pose of the teaching fixture flange, the current pose of the teaching workpiece, the pose of the teaching object and the final pose of the teaching fixture flange includes:

And calculating the preset assembly pose relationship of the teaching workpiece and the teaching object when the teaching workpiece is correctly assembled based on the teaching current pose relationship, the teaching object pose and the final pose of the teaching fixture flange, wherein the teaching current pose relationship is the teaching pose relationship between the teaching fixture flange and the teaching workpiece, which is characterized by the teaching fixture flange current pose and the teaching workpiece current pose.

Optionally, the step of calculating the preset assembly pose relationship between the teaching workpiece and the teaching object when the teaching workpiece is assembled correctly based on the teaching current pose relationship, the teaching object pose and the final pose of the teaching fixture flange includes:

The preset assembly pose relationship is calculated according to the following formula:

obj₂2obj₁＝(obj_teach12R_teach)^-1×T_t′_each2R_teach×(T_teach2R_teach)^-1

×obj_teach22R_teach

wherein obj _teach12R_teach is the current pose of the teaching workpiece under the coordinate system of the teaching mechanical arm; t' _teach2R_teach is the current pose of the teaching fixture flange under the coordinate system of the teaching mechanical arm; t _teach2R_teach is the final pose of the teaching fixture flange under the coordinate system of the teaching mechanical arm; obj _teach22R_teach is the pose of the teaching object under the coordinate system of the teaching mechanical arm.

Calculating the pose relation between the teaching workpiece and the teaching object as a candidate assembly pose relation when the teaching workpiece is assembled correctly based on the current pose of the teaching fixture flange, the current pose of the teaching workpiece, the pose of the teaching object and the final pose of the teaching fixture flange;

Calculating a difference value between the candidate assembly pose relationship and a preset standard assembly pose relationship, and taking the candidate assembly pose relationship as the preset assembly pose relationship when the difference value is not larger than a preset threshold value, wherein the preset standard assembly pose relationship is pre-constructed, and is determined based on the pose relationship when a standard workpiece corresponding to the teaching workpiece and the teaching object are correctly assembled, the teaching workpiece is one of the workpieces in the same batch, and the standard workpiece is a production template workpiece corresponding to the workpiece in the batch.

Optionally, the step of controlling the mechanical arm to assemble the target workpiece to the object to be assembled based on the current pose of the fixture flange and the final pose of the fixture flange includes:

Planning a moving path of the mechanical arm based on the current pose of the clamp flange and the final pose of the clamp flange;

And controlling the mechanical arm to move according to the moving path, and controlling the clamp flange of the mechanical arm to assemble the target workpiece to the object to be assembled in the final pose of the clamp flange.

In a second aspect, embodiments of the present application provide a workpiece assembly device, the device comprising:

The image acquisition module is used for controlling the mechanical arm clamping the target workpiece to move to an image acquisition area of the image acquisition equipment and acquiring a target image acquired by the image acquisition equipment, wherein a pre-placed object to be assembled is positioned in the image acquisition area;

The first pose determining module is used for determining the current pose of the clamp flange of the mechanical arm, the current pose of the target workpiece and the pose of the object to be assembled based on the target image and the parameters of the image acquisition equipment calibrated in advance;

The second pose determining module is used for calculating the final pose of the clamp flange according to the current pose of the clamp flange, the current pose of the target workpiece, the pose of the object to be assembled and the preset assembling pose relation; the preset assembly pose relationship is determined based on the pose relationship when the teaching workpiece corresponding to the target workpiece and the teaching object corresponding to the object to be assembled are correctly assembled in the teaching process;

and the assembly module is used for controlling the mechanical arm to assemble the target workpiece to the object to be assembled based on the current pose of the clamp flange and the final pose of the clamp flange.

Optionally, the second pose determining module includes:

the first pose determining unit is used for calculating the final pose of the clamp flange when the target workpiece and the object to be assembled are correctly assembled based on the current pose relationship, the object pose to be assembled and the preset assembly pose relationship, wherein the current pose relationship is the pose relationship between the clamp flange of the mechanical arm and the target workpiece, which is characterized by the current pose of the clamp flange and the current pose of the target workpiece.

Optionally, the first pose determining unit includes:

a first calculating subunit, configured to calculate a final pose of the fixture flange according to the following formula:

T′2R＝obj₂2R×(obj₂2obj₁)^-1×(obj₁2R)^-1×T2R

Optionally, the device further includes a preset assembly pose relationship building module, where the preset assembly pose relationship building module includes:

The first image acquisition unit is used for controlling the teaching mechanical arm holding the teaching workpiece to move to a teaching image acquisition area of the teaching image acquisition equipment and acquiring a first teaching image acquired by the teaching image acquisition equipment, wherein a pre-placed teaching object is positioned in the teaching image acquisition area;

the second pose determining unit is used for determining the current pose of the teaching fixture flange, the current pose of the teaching workpiece and the current pose of the teaching object of the teaching mechanical arm based on the first teaching image and the parameters of the pre-calibrated teaching image acquisition equipment;

The second image acquisition unit is used for controlling the teaching mechanical arm to assemble the teaching workpiece to the teaching object and acquiring a second teaching image acquired by the teaching image acquisition device;

The third pose determining unit is used for determining the final pose of the teaching fixture flange of the teaching mechanical arm when the teaching mechanical arm is correctly assembled according to the second teaching image and the parameters of the pre-calibrated teaching image acquisition equipment;

And the fourth pose determining unit is used for calculating the preset assembly pose relationship between the teaching workpiece and the teaching object when the teaching workpiece is correctly assembled based on the current pose of the teaching fixture flange, the current pose of the teaching workpiece, the pose of the teaching object and the final pose of the teaching fixture flange.

Optionally, the fourth pose determining unit includes:

and the second calculating subunit is used for calculating the preset assembly pose relationship between the teaching workpiece and the teaching object when the teaching workpiece is correctly assembled based on the teaching current pose relationship, the teaching object pose and the final pose of the teaching fixture flange, wherein the teaching current pose relationship is the teaching pose relationship between the teaching fixture flange and the teaching workpiece, which is characterized by the teaching fixture flange current pose and the teaching workpiece current pose.

Optionally, the second computing subunit is specifically configured to:

obj₂2obj₁＝(obj_teach12R_teach)^-1×T′_teach2R_teach×(T_teach2R_teach)^-1

×obj_teach22R_teach

Optionally, the fourth pose determining unit includes:

A third calculation subunit, configured to calculate, based on the current pose of the teaching fixture flange, the current pose of the teaching workpiece, the pose of the teaching object, and the final pose of the teaching fixture flange, a pose relationship between the teaching workpiece and the teaching object when correctly assembled, as a candidate assembly pose relationship;

the difference value calculating subunit is configured to calculate a difference value between the candidate assembly pose relationship and a preset standard assembly pose relationship, and when the difference value is not greater than a preset threshold, use the candidate assembly pose relationship as the preset assembly pose relationship, where the preset standard assembly pose relationship is pre-constructed, and is determined based on the pose relationship when the standard workpiece corresponding to the teaching workpiece and the teaching object are correctly assembled, where the teaching workpiece is one of the workpieces in the same batch, and where the standard workpiece is a workpiece corresponding to a production template of the workpiece in the batch.

Optionally, the assembly module includes:

the path planning unit is used for planning the moving path of the mechanical arm based on the current pose of the clamp flange and the final pose of the clamp flange;

And the control unit is used for controlling the mechanical arm to move according to the moving path and controlling the clamp flange of the mechanical arm to assemble the target workpiece to the object to be assembled in the final pose of the clamp flange.

In a third aspect, embodiments of the present application provide a workpiece assembly system, the system comprising an image acquisition device and a processor:

The image acquisition equipment is used for acquiring images of all objects in the image acquisition area;

the processor is configured to perform the method steps described in the first aspect.

In a fourth aspect, an embodiment of the present application provides an electronic device, including:

A memory for storing a computer program;

And a processor, configured to implement the method steps described in the first aspect when executing the program stored in the memory.

In a fifth aspect, an embodiment of the present application provides a computer readable storage medium, in which a computer program is stored, which computer program, when being executed by a processor, implements the method steps described in the first aspect.

In a sixth aspect, embodiments of the present application also provide a computer program product comprising instructions which, when run on a computer, cause the computer to perform the method steps of the first aspect described above.

The embodiment of the application has the beneficial effects that:

In the scheme provided by the embodiment of the application, the electronic equipment can control the mechanical arm clamping the target workpiece to move to the image acquisition area of the image acquisition equipment and acquire the target image acquired by the image acquisition equipment, wherein the pre-placed object to be assembled is positioned in the image acquisition area; determining the current pose of a clamp flange of the mechanical arm, the current pose of a target workpiece and the pose of an object to be assembled based on the target image and parameters of the image acquisition equipment calibrated in advance; calculating the final pose of the clamp flange according to the current pose of the clamp flange, the current pose of the target workpiece, the pose of the object to be assembled and the preset assembly pose relation; the preset assembly pose relation is determined based on the pose relation when the teaching workpiece corresponding to the target workpiece and the teaching object corresponding to the object to be assembled are correctly assembled in the teaching process; and then based on the current pose of the clamp flange and the final pose of the clamp flange, controlling the mechanical arm to assemble the target workpiece to the object to be assembled.

Aiming at the target workpiece and the object to be assembled corresponding to the target workpiece, the pose relationship of the target workpiece and the object to be assembled should meet the preset assembly pose relationship when the target workpiece and the object to be assembled are correctly assembled. Because the preset assembly pose relationship is the pose relationship between the target workpiece and the object to be assembled and does not change along with the change of the pose relationship between the target workpiece and the clamp flange of the mechanical arm, the workpiece assembly is carried out on the target workpiece and the object to be assembled by utilizing the preset assembly pose relationship, the problem that the target workpiece and the clamp flange pose are offset due to unreasonable design of the clamp flange, and the assembly precision is improved. And because the preset assembly pose relationship is the pose relationship between the target workpiece and the object to be assembled when the object to be assembled is correctly assembled, the pose relationship is irrelevant to the current pose of the object to be assembled, the method can be suitable for randomly placing the object to be assembled, and the object to be assembled does not need to be fixed at the same position. The pose of the object to be assembled and the pose relation between the target workpiece and the clamp flange can be corrected and assembled by detecting the pose of the object to be assembled and the pose relation between the target workpiece and the clamp flange in real time, so that the assembly cost can be reduced. Of course, it is not necessary for any one product or method of practicing the application to achieve all of the advantages set forth above at the same time.

Drawings

In order to more clearly illustrate the embodiments of the application or the technical solutions in the prior art, the drawings used in the embodiments or the description of the prior art will be briefly described below, and it is obvious that the drawings in the following description are only some embodiments of the application, and other embodiments may be obtained according to these drawings to those skilled in the art.

FIG. 1 is a flow chart of a method for assembling a workpiece according to an embodiment of the application;

FIG. 2 is a schematic view of a pose of a robot arm, a target workpiece, and an object to be assembled according to an embodiment of the present application;

FIG. 3 is a schematic flow chart of a preset assembly pose relationship construction process according to an embodiment of the present application;

FIG. 4 (a) is a schematic view of a pose of the robot arm, the target workpiece, and the object to be assembled during teaching;

FIG. 4 (b) is a schematic view of another pose of the robot arm, the target workpiece, and the object to be assembled during teaching;

FIG. 5 is a flowchart showing a step S305 in the embodiment shown in FIG. 3;

FIG. 6 is a specific flowchart of step S104 in the embodiment shown in FIG. 1;

FIG. 7 is a schematic diagram of a workpiece assembly device according to an embodiment of the present application;

FIG. 8 is a schematic diagram of a workpiece assembly system according to an embodiment of the present application;

fig. 9 is a schematic structural diagram of an electronic device according to an embodiment of the present application.

Detailed Description

The following description of the embodiments of the present application will be made clearly and completely with reference to the accompanying drawings, in which it is apparent that the embodiments described are only some embodiments of the present application, but not all embodiments. Based on the embodiments of the present application, all other embodiments obtained by the person skilled in the art based on the present application are included in the scope of protection of the present application.

The following describes terms related to the embodiments of the present application:

The workpiece is a product component in an industrial manufacturing process;

the assembly is a process of assembling the workpieces according to the specified technical requirements, and debugging and checking the workpieces to form qualified products;

The flange or flange plate is a symmetrical disk-shaped structure used for connecting a pipeline, a container or a fixed shaft type mechanical part, and is usually connected with a workpiece through a bolt and a thread structure;

An AGV is a transport that is characterized by wheel-type movement, is controlled by a computer, is equipped with a power (or power conversion) device and an electromagnetic (or optical, etc.) automatic guide device, and is capable of automatically traveling along a prescribed guide path;

the registration is to obtain the pose relation of the object A and the object B in a teaching mode;

pose is the position and pose of an object.

In order to improve workpiece assembly accuracy, embodiments of the present application provide a workpiece assembly method, apparatus, system, electronic device, computer readable storage medium, and computer program product, and a workpiece assembly method provided by the embodiments of the present application is first described below.

The workpiece assembly method provided by the embodiment of the application can be applied to any electronic equipment which can control the mechanical arm and has an image processing function. For example, the robot arm may be controlled and an image pickup device equipped with an image processing module, a robot arm capable of communicating with the image pickup device and equipped with an image processing module, a server communicating with the image pickup device and serving the robot arm, or the like, and is not particularly limited herein. For clarity of description, hereinafter, referred to as an electronic device.

As shown in fig. 1, a method of assembling a workpiece, the method comprising:

S101: controlling a mechanical arm clamping a target workpiece to move to an image acquisition area of image acquisition equipment, and acquiring a target image acquired by the image acquisition equipment;

Wherein, the pre-placed object to be assembled is positioned in the image acquisition area;

S102: determining the current pose of a clamp flange of the mechanical arm, the current pose of a target workpiece and the pose of an object to be assembled based on the target image and the parameters of the image acquisition equipment calibrated in advance;

S103: calculating the final pose of the clamp flange according to the current pose of the clamp flange, the current pose of the target workpiece, the pose of the object to be assembled and the preset assembly pose relation;

The preset assembly pose relationship is determined based on the pose relationship when the teaching workpiece corresponding to the target workpiece and the teaching object corresponding to the object to be assembled are correctly assembled in the teaching process;

S104: and controlling the mechanical arm to assemble the target workpiece to the object to be assembled based on the current pose of the clamp flange and the final pose of the clamp flange.

Aiming at the target workpiece and the object to be assembled corresponding to the target workpiece, the pose relationship of the target workpiece and the object to be assembled should meet the preset assembly pose relationship when the target workpiece and the object to be assembled are correctly assembled. Because the preset assembly pose relationship is the pose relationship between the target workpiece and the object to be assembled and does not change along with the change of the pose relationship between the target workpiece and the clamp flange of the mechanical arm, the workpiece assembly is carried out on the target workpiece and the object to be assembled by utilizing the preset assembly pose relationship, the problem that the target workpiece and the clamp flange pose are offset due to unreasonable design of the clamp flange, and the assembly precision is improved. And because the preset assembly pose relationship is the pose relationship between the target workpiece and the object to be assembled when the object to be assembled is correctly assembled, the pose relationship is irrelevant to the current pose of the object to be assembled, the method can be suitable for randomly placing the object to be assembled, and the object to be assembled does not need to be fixed at the same position. The pose of the object to be assembled and the pose relation between the target workpiece and the clamp flange can be corrected and assembled by detecting the pose of the object to be assembled and the pose relation between the target workpiece and the clamp flange in real time, so that the assembly cost can be reduced.

In the industries such as automobile manufacturing, in order to meet the assembly accuracy requirement and the assembly efficiency requirement, a mechanical arm is generally used for workpiece assembly, for example, automobile door panel assembly, bearing assembly and the like are performed by using the mechanical arm. For each product, in order to facilitate workpiece assembly work by the mechanical arm in the production process of the product, the workpiece mounting process of the product may be taught in advance. Therefore, in actual production, the mechanical arm can be controlled to assemble the workpiece according to the teaching process.

Wherein the assembly process of the product may include an assembly process of a plurality of workpieces for each product, and the assembly process of the workpieces may be taught separately for each workpiece.

In a more complex workpiece assembly scene, in order to avoid workpiece assembly failure caused by the situations that the actual position of an object to be assembled is offset compared with the teaching position, the position relation of a target workpiece and a clamp flange is offset compared with the teaching position relation due to unreasonable clamp design, and the like, the workpiece assembly device is suitable for the precision requirement of workpiece assembly, and a 3D (three-dimensional) vision guiding mechanical arm can be utilized for workpiece assembly, namely, the 3D vision positioning guiding mechanical arm is utilized for correctly assembling the workpiece and the object to be assembled.

In order to perform workpiece assembly using the 3D vision-guided robot arm, an image pickup device may be installed in the workpiece assembly area in advance. The image capturing apparatus may be a monocular camera, a binocular camera, or the like, and is not particularly limited herein; when the image acquisition device is installed, the image acquisition area of the image acquisition device is covered on the workpiece assembly area, namely the image acquisition device can clearly and completely acquire images of the mechanical arm and the object to be assembled, which are positioned in the workpiece assembly area.

After the image acquisition equipment is installed, parameter adjustment and parameter calibration can be performed on the image acquisition equipment, namely, the conversion relation between the camera coordinates of the image acquisition equipment and an image coordinate system and the conversion relation between the camera coordinates of the image acquisition equipment and a mechanical arm coordinate system can be calibrated, so that the subsequent calculation of each pose relation is facilitated. The parameter calibration method may be Zhang Zhengyou calibration method, nine-point calibration method, etc., and is not specifically limited herein.

In the assembly operation, when the workpiece assembly of the object to be assembled is desired, the object to be assembled may be placed in the image acquisition region in advance. For example, the AGV carrier may be controlled to transport the object to be assembled into the image acquisition area, etc.

Further, in the above step S101, the electronic apparatus controls the robot arm holding the target workpiece to move to the image pickup area of the image pickup apparatus, and acquires the target image picked up by the image pickup apparatus.

When the manipulator is expected to be used for assembling the target workpiece for the object to be assembled in the image acquisition area, the electronic equipment can control the manipulator to move the position of the target workpiece and control the clamp flange of the manipulator to clamp the target workpiece. And then the electronic equipment can control the mechanical arm holding the target workpiece to move to the image acquisition area and acquire the target image acquired by the image acquisition equipment.

As one embodiment, the electronic device controls the image capturing device to capture the target image when controlling the robot arm holding the target workpiece to move to the image capturing area of the image capturing device. The image acquisition device acquires a target image and sends the target image to the electronic device so that the electronic device can acquire the target image.

As an embodiment, the image capturing apparatus performs image capturing on the image capturing area according to a preset frequency, where the preset frequency may be set according to actual needs, for example, 10 times/second, 50 times/second, etc., and is not specifically limited herein. When the electronic equipment controls the mechanical arm holding the target workpiece to move to the image acquisition area of the image acquisition equipment, an image acquisition request carrying a time stamp representing the moment when the mechanical arm moves to the image acquisition area is sent to the image acquisition equipment, and a target image sent by the image acquisition equipment based on the time stamp is received.

Because the mechanical arm clamping the target workpiece and the object to be assembled are both positioned in the image acquisition area, the target image acquired by the electronic equipment is the image comprising the mechanical arm, the target workpiece and the object to be assembled.

For example, after the electronic device controls the robot arm holding the target workpiece to move to the image capturing area of the image capturing device, the respective positions of the image capturing device 210, the robot arm 220, the target workpiece 230, and the object to be assembled 240 are as shown in fig. 2, the image capturing device 210 may capture clear images of the robot arm 220, the target workpiece 230, and the object to be assembled 240, so that the electronic device may capture the target image including the robot arm 220, the target workpiece 230, and the object to be assembled 240 captured by the image capturing device 210.

Further, in the step S102, the electronic device determines the current pose of the fixture flange of the mechanical arm, the current pose of the target workpiece and the pose of the object to be assembled based on the target image and the parameters of the image acquisition device calibrated in advance.

Because the target image comprises the mechanical arm, the target workpiece and the object to be assembled, after the electronic equipment acquires the target image, the current pose of the clamp flange of the mechanical arm, the current pose of the target workpiece and the pose of the object to be assembled can be determined based on the target image and the parameters of the image acquisition equipment calibrated in advance.

The current pose of the fixture flange of the mechanical arm may be the current pose of the fixture flange under the coordinate system of the base of the mechanical arm, or the pose of the fixture flange under the coordinate system of other reference objects in the image acquisition area, which is not particularly limited herein. Correspondingly, the current pose of the target workpiece can be the current pose of the target workpiece under the coordinate system of the mechanical arm base, and can also be the pose of the target workpiece under the coordinate system of other reference objects in the image acquisition area; the pose of the object to be assembled can be the current pose of the object to be assembled under the coordinate system of the mechanical arm base, or the pose of the object to be assembled under the coordinate system of other reference objects in the image acquisition area. For the convenience of calculation, the current pose of the clamp flange of the mechanical arm, the current pose of the target workpiece and the pose of the object to be assembled are usually determined to be the poses under the same coordinate system.

When the method runs each time, the pose of the object to be assembled is determined by utilizing the target image, so that the object to be assembled can be subjected to secondary deviation correcting and positioning, and the problems that the object to be assembled cannot be assembled or is assembled in deviation and the like due to the fact that the position of the object to be assembled is low in the traditional assembly scheme are solved.

For example, as shown in fig. 2, the electronic device may determine, in the robot base coordinate system, a current pose T2R of the fixture flange of the robot 220, a current pose ibj ₁ R of the target workpiece 230, and a pose obj ₂ R of the object to be assembled 240 based on the target image and parameters of the image capturing device calibrated in advance.

For various workpieces, the teaching workpiece and the teaching object corresponding to the workpieces can be utilized in advance to teach the assembly process of the workpiece. In the teaching process, the electronic device can determine a preset assembly pose relationship based on the pose relationship when the teaching workpiece and the teaching object are assembled correctly. Correspondingly, for the target workpiece, the workpiece assembly process can be taught by utilizing the teaching workpiece corresponding to the target workpiece and the teaching object corresponding to the object to be assembled in advance, and in the teaching process, the electronic equipment can determine the preset assembly pose relationship based on the pose relationship when the teaching workpiece and the teaching object are assembled correctly, namely, the preset assembly pose relationship is the pose relationship between the target workpiece and the object to be assembled when the teaching workpiece and the teaching object are assembled correctly.

When the preset assembly pose matrix is used for representing the preset assembly pose relation, the process of obtaining the preset assembly pose matrix of the target workpiece and the image to be assembled in a correct assembly mode can be called assembly matrix registration.

For clarity of the line, a detailed description will be given below of the manner in which the preset assembly pose relationship is established.

Further, in the step S103, the electronic device may calculate the final pose of the fixture flange according to the current pose of the fixture flange, the current pose of the target workpiece, the pose of the object to be assembled, and the preset assembly pose relationship.

For any fixture flange, after the fixture flange clamps a target workpiece, the pose relation between the fixture flange and the target workpiece is not changed in the assembly operation process, and the position relation between the target workpiece and an object to be assembled is required to meet the preset assembly pose relation calibrated in advance when the assembly is successful no matter whether the object to be assembled is placed at the teaching position or not.

In addition, in the assembly operation process, the electronic equipment assembles the target workpiece and the object to be assembled by controlling the mechanical arm to move, so that the electronic equipment can move through the clamp flange of the mechanical arm under the condition that the position and the posture of the object to be assembled are fixed, and the electronic equipment can adjust the position and the posture of the target workpiece by controlling the clamp flange to move under the condition that the position and the posture of the object to be assembled, namely, the position and the posture of the clamp flange and the posture of the target workpiece are kept unchanged.

After determining the current pose of the fixture flange, the current pose of the target workpiece and the pose of the object to be assembled, the electronic equipment can calculate the final pose of the fixture flange according to the current pose of the fixture flange, the current pose of the target workpiece, the pose of the object to be assembled and the preset assembly pose relation. The final position of the fixture flange is the position of the fixture flange when the fixture flange is correctly assembled, namely, when the fixture flange has the final position of the fixture flange, the target workpiece can be correctly assembled with an object to be assembled.

As an implementation manner, the electronic device may calculate the pose relationship between the target workpiece and the fixture flange based on the current pose of the fixture flange and the current pose of the target workpiece, and calculate the final pose of the target workpiece when correctly assembled based on the pose relationship between the object to be assembled and the preset assembly pose, and further the electronic device may calculate the final pose of the fixture flange based on the pose relationship between the target workpiece and the fixture flange and the final pose of the target workpiece.

As an implementation manner, the electronic device may calculate the pose relationship between the target workpiece and the fixture flange based on the current pose of the fixture flange and the current pose of the target workpiece, and calculate the pose relationship between the fixture flange and the object to be assembled when correctly assembled based on the pose relationship between the target workpiece and the fixture flange and the preset assembly pose relationship, and further the electronic device may calculate the final pose of the fixture flange based on the pose relationship between the fixture flange and the object to be assembled when correctly assembled and the pose of the object to be assembled.

Further, in the step S104, the electronic device may control the robot arm to mount the target workpiece to the object to be mounted based on the current pose of the fixture flange and the final pose of the fixture flange. After the final pose of the clamp flange is determined, the electronic equipment can determine a control strategy of the mechanical arm according to the current pose of the clamp flange and the final pose of the clamp flange, and then control the mechanical arm to move according to the determined control strategy so as to enable the mechanical arm to assemble the target workpiece to the object to be assembled. And when the target workpiece is assembled to the object to be assembled, the pose of the clamp flange of the mechanical arm is the final pose of the clamp flange.

As an implementation manner of the embodiment of the present application, the step of calculating the final pose of the fixture flange according to the current pose of the fixture flange, the current pose of the target workpiece, the pose of the object to be assembled, and the preset assembly pose relationship may include:

After the fixture flange clamps the target workpiece, the pose relation between the fixture flange and the target workpiece is protected from being changed in the assembly operation process, so that after the current pose of the fixture flange, the current pose of the target workpiece and the pose of the object to be assembled are determined, the electronic equipment can determine the pose relation between the fixture flange of the mechanical arm and the target workpiece based on the current pose of the fixture flange and the current pose of the target workpiece, and the current pose relation represented by the current pose of the fixture flange and the current pose of the target workpiece is obtained. By carrying out secondary deviation rectifying and positioning on the pose relation between the target element and the clamp flange, the problem that the target element and the clamp flange cannot be assembled due to pose deviation caused by unreasonable clamp design in the traditional assembly scheme is solved.

And then, the electronic equipment can calculate the final position of the fixture flange when the target workpiece and the object to be assembled are correctly assembled according to the current position relation, the object to be assembled and the preset assembly position relation.

As an embodiment, the electronic device may calculate the final pose of the target workpiece when correctly assembled based on the pose of the object to be assembled and the preset assembly pose relationship, and calculate the final pose of the fixture flange based on the current pose relationship and the final pose of the target workpiece.

As an implementation manner, the electronic device may calculate the pose relationship between the fixture flange and the object to be assembled when correctly assembled based on the current pose relationship and the preset assembly pose relationship, and calculate the final pose of the fixture flange based on the pose relationship between the fixture flange and the object to be assembled when correctly assembled, and the pose of the object to be assembled.

In this embodiment, after determining the current pose relationship between the fixture flange and the target workpiece, the electronic device may calculate the final pose of the fixture flange when the target workpiece and the object to be assembled are correctly assembled by using the current pose relationship, the object pose to be assembled, and the preset assembly pose relationship. The current position and posture relation between the fixture flange and the target workpiece is calculated in real time, namely, the current position and posture relation between the fixture flange and the target workpiece is required to be recalculated after the fixture flange clamps the target workpiece for various fixture flanges, so that the workpiece assembly failure caused by the situation that the position and posture relation between the target workpiece and the fixture flange deviates from the teaching position and posture relation due to unreasonable fixture design and the like can be avoided, and the workpiece configuration precision is improved.

As an implementation manner of the embodiment of the present application, the step of calculating the final pose of the fixture flange when the target workpiece and the object to be assembled are correctly assembled based on the current pose relationship, the object to be assembled pose and the preset assembly pose relationship may include:

T′2R＝obj₂2R×(obj₂2obj₁)^-1×(obj₁2R)^-1×T2R

Based on a pre-calibrated preset assembly pose relation matrix, when the electronic equipment calculates the final pose of the clamp flange according to the formula, the pose of the object to be assembled under the coordinate system of the mechanical arm can be kept unchanged; the current pose of the target workpiece under the mechanical arm coordinate system and the current pose of the clamp flange under the mechanical arm coordinate system can be the same, and the represented current pose relationship between the target workpiece and the clamp flange can also be kept unchanged. Therefore, according to the current pose relationship, the pose of the object to be assembled and the preset assembly pose relationship, the electronic equipment can calculate the final pose of the clamp flange when being assembled correctly.

In this embodiment, the electronic apparatus calculates the final pose of the jig flange when the target workpiece is correctly assembled with the object to be assembled by the above formula. Therefore, workpiece assembly is performed more accurately and rapidly, and workpiece assembly accuracy is improved.

As shown in fig. 3, the method for constructing the preset assembly pose relationship according to the embodiment of the present application may include:

S301: the teaching mechanical arm which is clamped with the teaching workpiece is controlled to move to a teaching image acquisition area of a teaching image acquisition device, and a first teaching image acquired by the teaching image acquisition device is acquired;

wherein, the teaching object placed in advance is positioned in the teaching image acquisition area;

S302: determining the current pose of a teaching fixture flange, the current pose of a teaching workpiece and the current pose of a teaching object of the teaching mechanical arm based on the first teaching image and the parameters of the pre-calibrated teaching image acquisition equipment;

s303: controlling the teaching mechanical arm to assemble the teaching workpiece to the teaching object, and acquiring a second teaching image acquired by the teaching image acquisition device;

S304: determining the final pose of the teaching fixture flange of the teaching mechanical arm when the teaching mechanical arm is correctly assembled according to the second teaching image and the parameters of the pre-calibrated teaching image acquisition equipment;

S305: and calculating the preset assembly pose relation between the teaching workpiece and the teaching object when the teaching fixture is correctly assembled based on the current pose of the teaching fixture flange, the current pose of the teaching workpiece, the pose of the teaching object and the final pose of the teaching fixture flange.

For each workpiece assembly process, the workpiece assembly process may be taught in advance using the teaching workpiece and the teaching object. In the teaching process, the teaching image acquisition apparatus may be installed in the teaching assembly area in advance. When the teaching image acquisition device is installed, the teaching image acquisition area of the teaching image acquisition device is enabled to cover the teaching workpiece assembly area, namely, the teaching image acquisition device can clearly and completely acquire images of the teaching mechanical arm and the teaching object in the teaching workpiece assembly area.

After the teaching image acquisition equipment is installed, parameter adjustment and parameter calibration can be performed on the teaching image acquisition equipment, namely, the conversion relation between the camera coordinates of the teaching image acquisition equipment and an image coordinate system and the conversion relation between the camera coordinates of the teaching image acquisition equipment and a teaching mechanical arm coordinate system can be calibrated, so that the subsequent calculation of each pose relation is facilitated.

In the assembly operation process, when workpiece assembly is desired to be performed on the teaching object, the teaching object may be placed in advance in the teaching image acquisition area.

The electronic device can control the teaching mechanical arm to move to the teaching workpiece and clamp the teaching workpiece, then control the teaching mechanical arm clamped with the teaching workpiece to move to a teaching image acquisition area of the teaching image acquisition device, and acquire a first teaching image acquired by the teaching image acquisition device.

Then, the electronic device may determine the current pose of the teaching fixture flange, the current pose of the teaching workpiece and the current pose of the teaching object of the teaching mechanical arm based on the first teaching image and the parameters of the pre-calibrated teaching image acquisition device.

And then controlling the teaching mechanical arm to assemble the teaching workpiece to the teaching object, and acquiring a second teaching image acquired by the teaching image acquisition device. The electronic equipment can determine the final pose of the teaching fixture flange of the teaching mechanical arm when the electronic equipment is correctly assembled according to the second teaching image and the parameters of the pre-calibrated teaching image acquisition equipment. And further, based on the current pose of the teaching fixture flange, the current pose of the teaching workpiece, the pose of the teaching object and the final pose of the teaching fixture flange, calculating the preset assembly pose relationship between the teaching workpiece and the teaching object when the teaching fixture flange is correctly assembled.

Thus, through the teaching process, the preset assembly pose relationship determined based on the pose relationship when the teaching workpiece and the teaching object are assembled correctly can be obtained.

In the subsequent assembly operation process, the workpiece can be assembled according to the preset assembly pose relationship determined by the teaching workpiece and the teaching object aiming at the workpiece corresponding to the teaching workpiece.

For example, after the electronic device controls the teaching robot arm holding the teaching workpiece to move to the teaching image collection area of the teaching image collection device, the respective positions of the teaching image collection device 410, the teaching robot arm 420, the teaching workpiece 430, and the teaching object 440 are as shown in fig. 4 (a), the teaching image collection device 410 may collect clear images of the teaching robot arm 420, the teaching workpiece 430, and the teaching object 440, so that the electronic device may acquire a first teaching image including the teaching robot arm 420, the teaching workpiece 430, and the teaching object 440 collected by the teaching image collection device 410, and determine the teaching jig flange current pose T' _teach2R_teach of the teaching robot arm 420, the teaching workpiece current pose obj _teach12R_teach of the teaching workpiece 430, and the teaching object current pose obj _teach22R_teach of the teaching object 440 using the first teaching image and parameters of the teaching image collection device calibrated in advance.

Thereafter, the electronic device may continue to control the teaching robot 420 to assemble the teaching workpiece 430 to the teaching object 440, and upon proper assembly, the respective positions of the teaching image acquisition device 410, the teaching robot 420, the teaching workpiece 430, and the teaching object 440 are as shown in fig. 4 (b). The electronic device obtains the second teaching image collected by the teaching image collection device 410, and then determines the final pose T _teach2R_teach of the teaching fixture flange of the teaching mechanical arm 420 when the teaching mechanical arm is correctly assembled according to the second teaching image and the parameters of the teaching image collection device calibrated in advance.

And the electronic equipment can calculate the preset assembly pose relation obj ₂2obj₁ between the teaching workpiece and the teaching object when the electronic equipment is correctly assembled based on the current pose of the teaching fixture flange, the current pose of the teaching workpiece, the pose of the teaching object and the final pose of the teaching fixture flange.

As an embodiment, two image acquisition devices may be installed in the teaching assembly area, and when the electronic device controls the teaching mechanical arm to move into the image acquisition area, the point cloud images acquired by the two image acquisition devices are acquired, and then the point cloud images are processed to extract point cloud images respectively including only the target workpiece and only the object to be assembled. And then the electronic equipment calculates the assembly angles and positions of the two point cloud pictures by using GASD algorithm to obtain the preset assembly pose relation of the target workpiece and the object to be assembled when the two point cloud pictures are assembled correctly.

In this embodiment, because the pose relation matrix of the teaching workpiece and the teaching object is irrelevant to various factors such as design, quality, batch and the like of the fixture flange, even if the fixture is replaced in the subsequent assembly operation process, the preset assembly pose relation is not required to be calculated again, and the assembly can be smoothly performed by directly using the preset assembly pose relation.

As one implementation manner of the embodiment of the present application, the step S305, when calculating the correct assembly based on the current pose of the teaching jig flange, the current pose of the teaching workpiece, the pose of the teaching object, and the final pose of the teaching jig flange, may include:

After the fixture flange clamps the target workpiece, the pose relation between the fixture flange and the target workpiece is protected from being changed in the assembly operation process, so that after the current pose of the teaching fixture flange, the current pose of the teaching workpiece and the pose of the teaching object are determined, the electronic equipment can determine the pose relation between the fixture flange of the teaching mechanical arm and the teaching workpiece based on the current pose of the teaching fixture flange and the current pose of the teaching workpiece, and the teaching current pose relation represented by the current pose of the teaching fixture flange and the current pose of the teaching workpiece is obtained.

And then, the electronic equipment can calculate the preset assembly pose relationship between the teaching workpiece and the teaching object when the electronic equipment is correctly assembled according to the teaching current pose relationship, the teaching object pose and the final pose of the teaching fixture flange.

As an embodiment, the electronic device may calculate the final pose of the target workpiece when correctly assembled based on the current pose relation of teaching and the final pose of the flange of the teaching fixture, and calculate the preset assembly pose relation of the teaching workpiece and the teaching object when correctly assembled based on the final pose of the target workpiece and the pose of the teaching object.

As an embodiment, the electronic device may calculate, based on the pose of the teaching object and the final pose of the teaching jig flange, the pose relationship between the teaching object and the teaching jig flange when correctly assembled, and calculate, based on the pose relationship between the teaching object and the teaching jig flange, the preset assembly pose relationship between the teaching workpiece and the teaching object when correctly assembled, according to the teaching current pose relationship.

In this embodiment, after determining the current pose relationship of teaching between the teaching fixture flange and the teaching workpiece, the electronic device may calculate the preset assembly pose relationship between the teaching workpiece and the teaching object when correctly assembled by using the current pose relationship of teaching, the pose of the teaching object, and the final pose of the teaching fixture flange. Therefore, the obtained preset assembling pose relation is irrelevant to the clamp, the workpiece assembling failure caused by the situation that the pose relation of the target workpiece and the clamp flange deviates from the teaching pose relation due to unreasonable clamp design and the like can be avoided, and the workpiece arranging precision is improved.

As an implementation manner of the embodiment of the present application, the step of calculating, when the teaching workpiece is correctly assembled, a preset assembly pose relationship between the teaching workpiece and the teaching object based on the teaching current pose relationship, the teaching object pose, and the final pose of the teaching fixture flange may include:

×obj_teach22R_teach

In the teaching process, the pose of the teaching object under the coordinate system of the teaching mechanical arm can be kept unchanged; the relation between the current position and the pose of the teaching workpiece and the teaching fixture flange can be kept unchanged. Therefore, according to the teaching current pose relationship, the teaching object pose to be assembled and the final pose of the teaching fixture flange, the electronic equipment can calculate and obtain the preset assembly pose relationship of the teaching workpiece and the teaching object during correct assembly.

In this embodiment, after the electronic device calculates the preset assembly pose relationship between the teaching workpiece and the teaching object according to the above formula, the workpiece can be assembled more accurately and rapidly by using the preset assembly pose relationship, so as to improve the workpiece assembly precision.

As shown in fig. 5, the step S305 of calculating, based on the current pose of the teaching jig flange, the current pose of the teaching workpiece, the pose of the teaching object, and the final pose of the teaching jig flange, a preset assembly pose relationship between the teaching workpiece and the teaching object when correctly assembled may include:

S501: calculating the pose relation between the teaching workpiece and the teaching object as a candidate assembly pose relation when the teaching workpiece is assembled correctly based on the current pose of the teaching fixture flange, the current pose of the teaching workpiece, the pose of the teaching object and the final pose of the teaching fixture flange;

S502: calculating a difference value between the candidate assembly pose relationship and a preset standard assembly pose relationship, and taking the candidate assembly pose relationship as the preset assembly pose relationship when the difference value is not greater than a preset threshold value;

The preset standard assembly pose relationship is pre-constructed, and is determined based on the pose relationship when the standard workpiece corresponding to the teaching workpiece and the teaching object are assembled correctly, wherein the teaching workpiece is one of the workpieces in the same batch, and the standard workpiece is a production template workpiece corresponding to the workpiece in the batch.

In the production of individual workpieces, a standard workpiece of the workpiece is usually first produced according to a design drawing, as a production template workpiece for the workpiece. After the standard workpiece is manufactured, the standard workpiece and the teaching object can be used for teaching, namely, the steps S301-S305 are executed, and when the standard workpiece is correctly assembled, the preset standard assembly pose relation of the standard workpiece and the teaching object is obtained. The electronic device stores the preset standard assembly pose relationship.

For each batch of workpieces, one workpiece can be selected as a teaching workpiece in the batch of workpieces, and teaching can be performed by the teaching object. In the teaching process, when determining the current pose of the teaching fixture flange, the current pose of the teaching workpiece, the pose of the teaching object and the final pose of the teaching fixture flange, the electronic equipment can calculate the pose relationship between the teaching workpiece and the teaching object when correctly assembling, and serve as candidate assembling pose relationship.

Then, the electronic device can calculate the difference value between the candidate assembly pose relationship and the preset standard assembly pose relationship stored by the electronic device, and determine the quantity relationship between the difference value and the preset threshold value. The preset threshold may be set according to actual needs, for example, an angle threshold of 0.4 degrees, a coordinate threshold of 3, and the like, which are not specifically limited herein.

When the difference value is not greater than a preset threshold value, the candidate assembly pose relationship is used as the correct assembly, and the preset assembly pose relationship between the workpiece and the teaching object is taught; when the difference value is larger than a preset threshold value, outputting a notification message carrying the candidate assembly pose relation so as to facilitate a worker to execute corresponding checking measures based on the notification message, for example, checking a teaching process to determine whether the teaching process has an error to cause the error of the candidate assembly pose relation to be overlarge; and checking the batch of workpieces to determine whether the batch of workpieces are unqualified or not, and the like.

In this embodiment, for each batch of workpieces, when the electronic device uses the preset standard assembly pose relationship to correct assembly calculated in the teaching process, the candidate assembly pose relationship between the teaching workpiece and the teaching object is verified, so that whether the teaching process is correct and/or whether the batch of workpieces is qualified can be detected. Therefore, the problem that a large error exists in the preset assembly pose relation due to the error in the teaching process and/or the qualification of the batch of workpieces is avoided, and the assembly precision of the workpieces is improved.

As an implementation manner of the embodiment of the present application, as shown in fig. 6, the step S104, where the step of controlling the mechanical arm to assemble the target workpiece to the object to be assembled based on the current pose of the fixture flange and the final pose of the fixture flange may include:

S601: planning a moving path of the mechanical arm based on the current pose of the clamp flange and the final pose of the clamp flange;

S602: and controlling the mechanical arm to move according to the moving path, and controlling the clamp flange of the mechanical arm to assemble the target workpiece to the object to be assembled in the final pose of the clamp flange.

After determining the current pose of the clamp flange and the final pose of the clamp flange, the electronic equipment can plan the moving path of the mechanical arm based on the current pose of the clamp flange and the final pose of the clamp flange.

And then the electronic equipment can control the mechanical arm to move according to the planned moving path, and control the clamp flange of the mechanical arm to assemble the target workpiece to the object to be assembled according to the final pose of the clamp flange, so that the assembly of the target workpiece and the object to be assembled is completed.

After the target workpiece is assembled to the object to be assembled, the assembly result may be used as a new object to be assembled, and the workpiece corresponding to the new object to be assembled may be used as a new target workpiece, and then, when the new target workpiece and the new object to be assembled are correctly assembled, the steps S101 to S104 are continuously performed, and the new target workpiece is assembled to the new object to be assembled, using the preset assembly pose relationship of the new target workpiece and the new object to be assembled, which is determined through the teaching process. And updating the target object and the object to be assembled until the workpiece assembly of the product is completed.

In this embodiment, the electronic device may determine the movement path of the mechanical arm according to the current pose of the fixture flange and the final pose of the fixture flange, and then control the mechanical arm according to the movement path, so as to accurately assemble the target workpiece to the object to be assembled, thereby improving the assembly precision.

It should be noted that, in the technical solution provided in the present application, the mechanical arm includes a clamp flange, and the workpiece is assembled by clamping the target workpiece and the object to be assembled by using the clamp flange, but in the actual production process, the clamp of the mechanical arm may also be other types of clamps, for example, a gripping clamp, a vacuum adsorption clamp, etc., so that the type of the clamp is not limited, and any solution for assembling the workpiece by using the workpiece assembling method provided in the embodiment of the present application is within the protection scope of the present application.

Corresponding to the workpiece assembly method, the embodiment of the application also provides a workpiece assembly device, and the workpiece assembly device provided by the embodiment of the application is described below.

As shown in fig. 7, a workpiece assembling apparatus includes:

An image acquisition module 701, configured to control a mechanical arm holding a target workpiece to move to an image acquisition area of an image acquisition device, and acquire a target image acquired by the image acquisition device, where a pre-placed object to be assembled is located in the image acquisition area;

A first pose determining module 702, configured to determine a current pose of a fixture flange of the mechanical arm, a current pose of a target workpiece, and a pose of a target object to be assembled based on the target image and parameters of the image capturing device calibrated in advance;

A second pose determining module 703, configured to calculate a final pose of the fixture flange according to the current pose of the fixture flange, the current pose of the target workpiece, the pose of the object to be assembled, and a preset assembly pose relationship; the preset assembly pose relationship is determined based on the pose relationship when the teaching workpiece corresponding to the target workpiece and the teaching object corresponding to the object to be assembled are correctly assembled in the teaching process;

And an assembling module 704, configured to control the mechanical arm to assemble the target workpiece to the object to be assembled based on the current pose of the fixture flange and the final pose of the fixture flange.

As an implementation of the embodiment of the present application, the second pose determining module 703 includes:

As one implementation of the embodiment of the present application, the first pose determining unit includes:

T′2R＝obj₂2R×(obj₂2obj₁)^-1×(obj₁2R)^-1×T2R

As an implementation manner of the embodiment of the present application, the apparatus further includes a preset assembly pose relationship building module, where the preset assembly pose relationship building module includes:

As one implementation of the embodiment of the present application, the fourth pose determining unit includes:

As an implementation manner of the embodiment of the present application, the second computing subunit is specifically configured to:

×obj_teach22R_teach

As an implementation of the embodiment of the present application, the assembling module 704 includes:

Corresponding to the workpiece assembly method, the embodiment of the application also provides a workpiece assembly system, and the workpiece assembly system provided by the embodiment of the application is described below.

As shown in fig. 8, a workpiece assembly system includes an image acquisition device 801 and a processor 802.

The image acquisition device 801 is configured to acquire images of objects located in an image acquisition area;

The processor 802 is configured to perform any of the steps of the workpiece assembly method provided in the embodiments of the present application.

The embodiment of the application also provides an electronic device, as shown in fig. 9, including:

A memory 901 for storing a computer program;

A processor 902, configured to implement the method steps described in any of the foregoing embodiments when executing a program stored in the memory 901.

And the electronic device may further include a communication bus and/or a communication interface, where the processor 902, the communication interface, and the memory 901 perform communication with each other via the communication bus.

The communication bus mentioned above for the electronic device may be a peripheral component interconnect standard (PERIPHERAL COMPONENT INTERCONNECT, PCI) bus or an extended industry standard architecture (Extended Industry Standard Architecture, EISA) bus, etc. The communication bus may be classified as an address bus, a data bus, a control bus, or the like. For ease of illustration, the figures are shown with only one bold line, but not with only one bus or one type of bus.

The communication interface is used for communication between the electronic device and other devices.

The Memory may include random access Memory (Random Access Memory, RAM) or may include Non-Volatile Memory (NVM), such as at least one disk Memory. Optionally, the memory may also be at least one memory device located remotely from the aforementioned processor.

The processor may be a general-purpose processor, including a central processing unit (Central Processing Unit, CPU), a network processor (Network Processor, NP), etc.; but may also be a digital signal Processor (DIGITAL SIGNAL Processor, DSP), application SPECIFIC INTEGRATED Circuit (ASIC), field-Programmable gate array (Field-Programmable GATE ARRAY, FPGA) or other Programmable logic device, discrete gate or transistor logic device, discrete hardware components.

In yet another embodiment of the present application, a computer readable storage medium is provided, in which a computer program is stored, which when executed by a processor, implements the steps of any of the above-described workpiece assembly methods.

In yet another embodiment of the present application, there is also provided a computer program product containing instructions that, when run on a computer, cause the computer to perform any of the workpiece assembly methods of the above embodiments.

In the above embodiments, it may be implemented in whole or in part by software, hardware, firmware, or any combination thereof. When implemented in software, may be implemented in whole or in part in the form of a computer program product. The computer program product includes one or more computer instructions. When loaded and executed on a computer, produces a flow or function in accordance with embodiments of the present application, in whole or in part. The computer may be a general purpose computer, a special purpose computer, a computer network, or other programmable apparatus. The computer instructions may be stored in or transmitted from one computer-readable storage medium to another, for example, by wired (e.g., coaxial cable, optical fiber, digital Subscriber Line (DSL)), or wireless (e.g., infrared, wireless, microwave, etc.). The computer readable storage medium may be any available medium that can be accessed by a computer or a data storage device such as a server, data center, etc. that contains an integration of one or more available media. The usable medium may be a magnetic medium (e.g., floppy disk, hard disk, tape), an optical medium (e.g., DVD), or a Solid state disk (Solid STATE DISK, SSD), etc.

It is noted that relational terms such as first and second, and the like are used solely to distinguish one entity or action from another entity or action without necessarily requiring or implying any actual such relationship or order between such entities or actions. Moreover, the terms "comprises," "comprising," or any other variation thereof, are intended to cover a non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements does not include only those elements but may include other elements not expressly listed or inherent to such process, method, article, or apparatus. Without further limitation, an element defined by the phrase "comprising one … …" does not exclude the presence of other like elements in a process, method, article, or apparatus that comprises the element.

In this specification, each embodiment is described in a related manner, and identical and similar parts of each embodiment are all referred to each other, and each embodiment mainly describes differences from other embodiments. In particular, for apparatus, systems, electronic devices, computer readable storage media, and computer program product embodiments, the description is relatively simple as it is substantially similar to method embodiments, as relevant points are found in the partial description of method embodiments.

The foregoing description is only of the preferred embodiments of the present application and is not intended to limit the scope of the present application. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present application are included in the protection scope of the present application.

Claims

1. A method of assembling a workpiece, the method comprising:

2. The method of claim 1, wherein the step of calculating a final fixture flange pose based on the fixture flange current pose, the target workpiece current pose, the object pose to be assembled, and a preset assembly pose relationship comprises:

3. The method according to claim 2, wherein the step of calculating a final position of the jig flange when the target workpiece is correctly assembled with the object to be assembled based on the current position relationship, the object to be assembled position relationship, and a preset assembly position relationship, comprises:

T′2R＝obj₂2R×(obj₂2obj₁)^-1×(obj₁2R)^-1×T2R

4. The method according to claim 1, wherein the construction method of the preset assembly pose relationship comprises:

5. The method of claim 4, wherein the step of calculating a preset assembly pose relationship of the teaching workpiece and the teaching object when properly assembled based on the teaching fixture flange current pose, the teaching workpiece current pose, the teaching object pose, and the teaching fixture flange final pose comprises:

6. The method according to claim 5, wherein the step of calculating a preset fitting pose relationship of the teaching workpiece and the teaching object when properly assembled based on the teaching current pose relationship, the teaching object pose, and the teaching jig flange final pose comprises:

obj₂2obj₁＝(v_teach12R_teach)^-1×T′_teach2R_teach×(T_teach2R_teach)^-1×obj_teach22R_teach

7. The method according to any one of claims 4 to 6, wherein the step of calculating a preset assembly pose relationship of the teaching workpiece and the teaching object when properly assembled based on the teaching jig flange current pose, the teaching workpiece current pose, the teaching object pose, and the teaching jig flange final pose comprises:

8. The method according to any one of claims 1 to 6, wherein the step of controlling the robot arm to mount the target workpiece to the object to be mounted based on the current pose of the jig flange and the final pose of the jig flange includes:

9. A workpiece assembly device, said device comprising:

10. The apparatus of claim 9, wherein the second pose determination module comprises:

The first pose determining unit is used for calculating the final pose of the clamp flange when the target workpiece and the object to be assembled are correctly assembled based on the current pose relationship, the object pose to be assembled and the preset assembly pose relationship, wherein the current pose relationship is the pose relationship between the clamp flange of the mechanical arm and the target workpiece, which is characterized by the current pose of the clamp flange and the current pose of the target workpiece;

The first pose determination unit includes:

T′2R＝obj₂2R×(obj₂2obj₁)^-1×(obj₁2R)^-1×T2R

Wherein T'2R is the final pose of the clamp flange under the mechanical arm coordinate system; obj ₂ R is the pose of the object to be assembled under the coordinate system of the mechanical arm; the obj ₂2obj₁ is a preset assembly pose relation matrix; obj ₁ R is the current pose of the target workpiece under the mechanical arm coordinate system, and T2R is the current pose of the fixture flange under the mechanical arm coordinate system;

the device further comprises a preset assembly pose relationship construction module, wherein the preset assembly pose relationship construction module comprises:

A fourth pose determining unit, configured to calculate a preset assembly pose relationship between the teaching workpiece and the teaching object when correctly assembled, based on the current pose of the teaching fixture flange, the current pose of the teaching workpiece, the pose of the teaching object, and the final pose of the teaching fixture flange;

The fourth pose determination unit includes:

The second calculating subunit is used for calculating the preset assembly pose relationship between the teaching workpiece and the teaching object when the teaching workpiece is assembled correctly based on the teaching current pose relationship, the teaching object pose and the final pose of the teaching fixture flange, wherein the teaching current pose relationship is the teaching pose relationship between the teaching fixture flange and the teaching workpiece, which is characterized by the teaching fixture flange current pose and the teaching workpiece current pose;

The second computing subunit is specifically configured to:

obj₂2obj₁＝(obj_teach12R_teach)^-1×T′_teach2R_teach×(T_teach2R_teach)^-1×obj_teach22R_teach

Wherein obj _teach12R_teach is the current pose of the teaching workpiece under the coordinate system of the teaching mechanical arm; t' _teach2R_teach is the current pose of the teaching fixture flange under the coordinate system of the teaching mechanical arm; t _teach2R_teach is the final pose of the teaching fixture flange under the coordinate system of the teaching mechanical arm; obj _teach22R_teach is the pose of the teaching object under the coordinate system of the teaching mechanical arm;

The fourth pose determination unit includes:

The difference value calculating subunit is used for calculating a difference value between the candidate assembly pose relation and a preset standard assembly pose relation, and taking the candidate assembly pose relation as the preset assembly pose relation when the difference value is not larger than a preset threshold value, wherein the preset standard assembly pose relation is pre-constructed and is determined based on the pose relation when the standard workpiece corresponding to the teaching workpiece and the teaching object are correctly assembled, the teaching workpiece is one of the workpieces in the same batch, and the standard workpiece is a production template workpiece corresponding to the workpiece in the batch;

The assembly module includes:

11. A workpiece assembly system, the system comprising an image acquisition device and a processor:

The processor being configured to perform the method of any of claims 1-8.

12. An electronic device, comprising:

A memory for storing a computer program;

A processor for implementing the method of any of claims 1-8 when executing a program stored on a memory.

13. A computer readable storage medium, characterized in that the computer readable storage medium has stored therein a computer program which, when executed by a processor, implements the method of any of claims 1-8.