CN116641532A

CN116641532A - Intelligent control method and device for mechanical arm based on dynamic vision and brick laying robot

Info

Publication number: CN116641532A
Application number: CN202211520027.4A
Authority: CN
Inventors: 匡忠; 李飞骈; 刘大伟; 刘海芹
Original assignee: Guangdong Bozhilin Robot Co Ltd
Current assignee: Pai Turner Foshan Robot Technology Co ltd
Priority date: 2022-11-30
Filing date: 2022-11-30
Publication date: 2023-08-25

Abstract

The invention discloses a dynamic vision-based intelligent control method and device for a mechanical arm and a brick laying robot, wherein the mechanical arm is arranged on the brick laying robot, and a camera is arranged on the brick laying robot, and the method comprises the following steps: determining target compensation information of the brick laying robot based on attitude information, acquired by a camera, of the brick laying robot in the process of moving the brick laying robot to the first brick laying point; performing correction operation on the first brick paving point according to the target compensation information to obtain a second brick paving point, and controlling the brick paving robot to move to the second brick paving point; when the brick laying robot reaches the second brick laying point, the movement control parameters of the mechanical arm are determined based on the image information of the brick laying environment to be acquired by the camera, and the mechanical arm is controlled to execute the operation matched with the movement control parameters. Therefore, the intelligent movement control of the mechanical arm of the brick laying robot can be realized based on dynamic vision, and the efficiency and the accuracy of brick laying operation of the brick laying robot are improved.

Description

Intelligent control method and device for mechanical arm based on dynamic vision and brick laying robot

Technical Field

The invention relates to the technical field of intelligent control, in particular to a mechanical arm intelligent control method and device based on dynamic vision and a brick laying robot.

Background

Along with development of science and technology, automatic paving can be realized by a paving robot instead of the traditional manual paving in the building direction, so that the paving efficiency is greatly improved. Most of the existing paving robots adopt visual detection, and the mechanical arm of the paving robot is controlled based on the visual detection result to realize the paving operation, so that the paving robot has low efficiency in the paving process. It is important to provide a novel intelligent control method for paving bricks by using a mechanical arm to improve the paving efficiency of a paving robot.

Disclosure of Invention

The invention aims to solve the technical problem of providing a dynamic vision-based intelligent control method and device for a mechanical arm and a brick laying robot, which can realize intelligent movement control of the mechanical arm of the brick laying robot based on dynamic vision, and are beneficial to improving the efficiency of brick laying operation of the brick laying robot and the accuracy of brick laying operation of the brick laying robot.

In order to solve the technical problem, a first aspect of the invention discloses an intelligent control method of a mechanical arm based on dynamic vision, wherein the mechanical arm is arranged on a brick laying robot, a camera is arranged on the brick laying robot, and the method comprises the following steps:

determining target compensation information of the brick laying robot based on attitude information, acquired by the camera, of the brick laying robot in the process that the brick laying robot moves to a first brick laying point; the attitude information of the brick laying robot comprises brick grabbing attitude information of the brick laying robot and/or chassis attitude information of the brick laying robot, and the first brick laying point is a brick laying point corresponding to a target moving track generated by a track simulation model;

performing correction operation on the first brick paving point according to the target compensation information to obtain a second brick paving point, and controlling the brick paving robot to move to the second brick paving point;

when the brick paving robot reaches the second brick paving point, determining movement control parameters of the mechanical arm based on the image information of the brick paving environment to be acquired by the camera, and controlling the mechanical arm to execute the operation matched with the movement control parameters; the environment image information to be paved comprises real-time image information of the target object.

As an alternative embodiment, in the first aspect of the present invention, before the tile robot moves to the first tile point, the method further comprises:

acquiring target environment information corresponding to a position of a brick to be paved, and inputting the target environment information and structural information of the brick paving robot into a preset track simulation model to obtain a target moving track of the brick paving robot, wherein the target moving track comprises a target brick taking point and a first brick paving point;

controlling the brick laying robot to move to the target brick taking point based on the target moving track, and controlling the brick laying robot to execute brick grabbing operation when the brick laying robot reaches the target brick taking point;

judging whether the brick laying robot finishes brick grabbing operation or not;

when the brick laying robot is judged to finish the brick grabbing operation, controlling the brick laying robot to move to the first brick laying point;

and when judging that the brick-grabbing operation is not completed by the brick-laying robot, re-triggering and executing the steps of controlling the brick-laying robot to execute the brick-grabbing operation and judging whether the brick-grabbing operation is completed by the brick-laying robot.

As an optional implementation manner, in the first aspect of the present invention, when the pose information of the brick laying robot includes the brick grabbing pose information of the brick laying robot, the determining, based on the pose information of the brick laying robot acquired by the camera, target compensation information of the brick laying robot includes:

based on the brick grabbing posture information of the brick paving robot acquired by the camera, judging whether the brick grabbing posture information meets preset brick grabbing posture conditions or not;

when the brick grabbing posture information is judged to not meet the preset brick grabbing posture condition, acquiring first environment information of the first brick paving point;

determining first error information of the brick laying robot according to the brick grabbing posture information and the first environment information, and generating first compensation information based on the first error information, wherein the first error information is error information generated by the brick laying robot in the process of executing the brick grabbing operation;

and determining target compensation information of the brick laying robot according to the first compensation information.

As an optional implementation manner, in the first aspect of the present invention, when the pose information of the brick laying robot includes chassis pose information of the brick laying robot, the determining, based on the pose information of the brick laying robot acquired by the camera, target compensation information of the brick laying robot includes:

Judging whether the chassis posture information meets preset chassis posture conditions or not based on the chassis posture information, which is acquired by the camera and is aimed at the brick laying robot;

when the chassis posture information is judged to not meet the preset chassis posture condition, determining a chassis inclination coefficient of the brick laying robot according to the chassis posture information of the brick laying robot;

determining second error information of the brick laying robot based on the chassis inclination coefficient of the brick laying robot, and generating second compensation information based on the second error information, wherein the second error information is the chassis inclination error information of the brick laying robot;

and determining target compensation information of the brick laying robot according to the second compensation information.

In an optional implementation manner, in a first aspect of the present invention, the determining, based on the image information of the environment to be tiled acquired by the camera, a control parameter of the mechanical arm includes:

calculating a brick joint distance between a first brick site corresponding to a brick to be paved and a predetermined reference object based on real-time image information of a target object acquired by the camera, and judging whether the brick joint distance meets a preset brick joint condition;

When the brick joint distance is judged to not meet the preset brick joint condition, determining a dynamic tracking parameter of the mechanical arm according to the brick joint distance and the first brick joint point;

generating movement control parameters of the mechanical arm according to the dynamic tracking parameters; the dynamic tracking parameters are used for representing the dynamic relative position relationship between the mechanical arm and the first brick site;

and controlling the mechanical arm to execute the operation matched with the movement control parameter, including:

and controlling the mechanical arm to move based on the movement control parameters so that the brick joint distance between the first brick site corresponding to the brick to be paved and the predetermined reference object meets the preset brick joint condition.

As an optional implementation manner, in the first aspect of the present invention, after the tiling robot reaches the second tiling point, before the determining the control parameter of the mechanical arm based on the to-be-tiled environment image information acquired by the camera, the method further includes:

controlling the camera to acquire environmental image information of the brick to be paved, determining a brick paving position relation between the mechanical arm and the ground of the brick to be paved based on a visual servo technology and the environmental image information of the brick to be paved, and judging whether the brick paving position relation meets a preset position condition; the brick laying positional relationship is used for representing the relative positional relationship between the mechanical arm and bricks included in the ground to be tiled;

When the brick laying position relation is judged to not meet the preset position condition, triggering and executing the operation of determining the control parameters of the mechanical arm based on the image information of the brick laying environment to be laid acquired by the camera;

and when judging that the brick laying position relation meets the preset position condition, controlling the brick laying robot to execute brick laying operation.

As an optional implementation manner, in the first aspect of the present invention, a sucker is provided on the mechanical arm, and the sucker is used for grabbing the brick;

the judging whether the brick laying robot finishes brick grabbing operation or not comprises the following steps:

acquiring a vacuum value of the sucker, and judging whether the vacuum value is larger than a preset vacuum threshold;

when the vacuum value is judged to be larger than the preset vacuum threshold value, determining that the brick laying robot has completed brick grabbing operation;

and when the vacuum value is not larger than the preset vacuum threshold value, determining that the brick-laying robot does not complete the brick-grabbing operation.

The invention discloses a mechanical arm intelligent control device based on dynamic vision, which is applied to a brick laying robot, wherein the mechanical arm is arranged on the brick laying robot, a camera is arranged on the brick laying robot, and the device comprises:

The determining module is used for determining target compensation information of the brick laying robot based on the posture information of the brick laying robot, which is acquired by the camera, in the process that the brick laying robot moves to the first brick laying point; the attitude information of the brick laying robot comprises brick grabbing attitude information of the brick laying robot and/or chassis attitude information of the brick laying robot, and the first brick laying point is a brick laying point corresponding to a target moving track generated by a track simulation model;

the correction module is used for executing correction operation on the first brick paving point according to the target compensation information to obtain a second brick paving point;

the control module is used for controlling the brick laying robot to move to the second brick laying point;

the determining module is further used for determining movement control parameters of the mechanical arm based on the image information of the environment to be paved, which is acquired by the camera, when the brick paving robot reaches the second brick paving point;

the control module is also used for controlling the mechanical arm to execute the operation matched with the movement control parameter; the environment image information to be paved comprises real-time image information of the target object.

As an alternative embodiment, in the second aspect of the present invention, the apparatus further includes:

the acquisition module is used for acquiring target environment information corresponding to the position of the brick to be paved before the brick paving robot moves to the first brick paving point;

the input module is used for inputting the target environment information and the structural information of the brick laying robot into a preset track simulation model to obtain a target moving track of the brick laying robot, wherein the target moving track comprises a target brick taking point and a first brick laying point;

the control module is further used for controlling the brick laying robot to move to the target brick taking point based on the target moving track, and controlling the brick laying robot to execute brick grabbing operation when the brick laying robot reaches the target brick taking point;

the judging module is used for judging whether the brick laying robot finishes brick grabbing operation or not; when the brick-laying robot is judged to not finish the brick-grabbing operation, the control module is triggered again to execute the operation of controlling the brick-laying robot to execute the brick-grabbing operation and judge whether the brick-laying robot has completed the brick-grabbing operation;

and the control module is also used for controlling the brick laying robot to move to the first brick laying point when the judging module judges that the brick grabbing operation of the brick laying robot is finished.

In a second aspect of the present invention, as an optional implementation manner, the determining module determines, based on the pose information for the tiling robot acquired by the camera, specific modes of target compensation information of the tiling robot include:

when the attitude information of the brick laying robot comprises brick grabbing attitude information of the brick laying robot, judging whether the brick grabbing attitude information meets preset brick grabbing attitude conditions or not based on the brick grabbing attitude information of the brick laying robot acquired by the camera;

When the posture information of the brick laying robot comprises chassis posture information of the brick laying robot, judging whether the chassis posture information meets preset chassis posture conditions or not based on the chassis posture information of the brick laying robot acquired by the camera;

In a second aspect of the present invention, as an optional implementation manner, the determining module determines, based on the environmental image information to be tiled acquired by the camera, a specific manner of a control parameter of the mechanical arm includes:

and the specific mode of controlling the mechanical arm to execute the operation matched with the movement control parameter by the control module comprises the following steps:

As an optional implementation manner, in the second aspect of the present invention, the control module is further configured to, after the tiling robot reaches the second tiling point, control the camera to acquire the environmental image information to be tiled before the determining module determines the control parameters of the mechanical arm based on the environmental image information to be tiled acquired by the camera;

The determining module is further used for determining the brick paving position relation between the mechanical arm and the brick paving ground based on a visual servo technology and the image information of the environment to be paved;

the judging module is further used for judging whether the brick laying position relation meets a preset position condition; the brick laying positional relationship is used for representing the relative positional relationship between the mechanical arm and bricks included in the ground to be tiled; when the brick laying position relation is judged to not meet the preset position condition, triggering the determining module to execute the operation of determining the control parameters of the mechanical arm based on the image information of the brick laying environment to be laid acquired by the camera;

and the control module is also used for controlling the brick laying robot to execute brick laying operation when the judging module judges that the brick laying position relation meets the preset position condition.

As an alternative embodiment, in the second aspect of the present invention, a sucker is provided on the mechanical arm, and the sucker is used for grabbing the brick;

the specific mode for judging whether the brick laying robot finishes brick grabbing operation or not by the judging module comprises the following steps:

The third aspect of the invention discloses another intelligent control device of a mechanical arm based on dynamic vision, which is applied to a brick laying robot and comprises the following components:

a memory storing executable program code;

a processor coupled to the memory;

the processor calls the executable program codes stored in the memory to execute the intelligent control method of the mechanical arm based on dynamic vision disclosed in the first aspect of the invention.

The fourth aspect of the invention discloses a brick laying robot, and the computer storage medium stores computer instructions which are used for executing the intelligent control method of the mechanical arm based on dynamic vision disclosed in the first aspect of the invention when the computer instructions are called.

In a fifth aspect, the present invention discloses a computer storage medium storing computer instructions for executing part or all of the steps of any of the depth camera-based intelligent tile implementation methods disclosed in the first aspect of the present invention when the computer instructions are invoked.

Compared with the prior art, the embodiment of the invention has the following beneficial effects:

in the embodiment of the invention, in the process of moving the brick laying robot to a first brick laying point, the target compensation information of the brick laying robot is determined based on the posture information of the brick laying robot acquired by a camera; performing correction operation on the first brick paving point according to the target compensation information to obtain a second brick paving point, and controlling the brick paving robot to move to the second brick paving point; when the brick laying robot reaches the second brick laying point, the movement control parameters of the mechanical arm are determined based on the image information of the brick laying environment to be acquired by the camera, and the mechanical arm is controlled to execute the operation matched with the movement control parameters. Therefore, the intelligent movement control of the mechanical arm of the brick laying robot can be realized based on dynamic vision, and the efficiency and the accuracy of brick laying operation of the brick laying robot are improved.

Drawings

In order to more clearly illustrate the technical solutions of the embodiments of the present invention, the drawings required for the description of the embodiments will be briefly described below, and it is apparent that the drawings in the following description are only some embodiments of the present invention, and other drawings may be obtained according to these drawings without inventive effort for a person skilled in the art.

Fig. 1 is a schematic flow chart of a method for controlling an intelligent mechanical arm based on dynamic vision according to an embodiment of the present invention;

FIG. 2 is a schematic flow chart of another intelligent control method for a mechanical arm based on dynamic vision according to the embodiment of the invention;

fig. 3 is a schematic structural diagram of an intelligent control device for a mechanical arm based on dynamic vision according to an embodiment of the present invention;

FIG. 4 is a schematic structural diagram of another intelligent control device for a mechanical arm based on dynamic vision according to an embodiment of the present invention;

fig. 5 is a schematic structural diagram of another intelligent control device for a mechanical arm based on dynamic vision according to an embodiment of the present invention.

Detailed Description

In order that those skilled in the art will better understand the present invention, a technical solution in the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings in which it is apparent that the described embodiments are only some embodiments of the present invention, not all embodiments. All other embodiments, which can be made by those skilled in the art based on the embodiments of the invention without making any inventive effort, are intended to be within the scope of the invention.

The terms first, second and the like in the description and in the claims and in the above-described figures are used for distinguishing between different objects and not necessarily for describing a sequential or chronological order. Furthermore, the terms "comprise" and "have," as well as any variations thereof, are intended to cover a non-exclusive inclusion. For example, a process, method, apparatus, article, or article that comprises a list of steps or elements is not limited to only those listed but may optionally include other steps or elements not listed or inherent to such process, method, article, or article.

Reference herein to "an embodiment" means that a particular feature, structure, or characteristic described in connection with the embodiment may be included in at least one embodiment of the invention. The appearances of such phrases in various places in the specification are not necessarily all referring to the same embodiment, nor are separate or alternative embodiments mutually exclusive of other embodiments. Those of skill in the art will explicitly and implicitly appreciate that the embodiments described herein may be combined with other embodiments.

The invention discloses a dynamic vision-based intelligent control method and device for a mechanical arm and a brick laying robot, which can realize intelligent movement control of the mechanical arm of the brick laying robot based on dynamic vision, and are beneficial to improving the efficiency and the accuracy of brick laying operation of the brick laying robot.

Example 1

Referring to fig. 1, fig. 1 is a schematic flow chart of a method for controlling an intelligent mechanical arm based on dynamic vision according to an embodiment of the present invention. The intelligent control method of the mechanical arm based on dynamic vision described in fig. 1 can be applied to a brick laying robot, can be applied to an intelligent control device of the mechanical arm based on dynamic vision, and can also be applied to a local server or a cloud server of the intelligent control of the mechanical arm based on dynamic vision. As shown in fig. 1, the intelligent control method for the mechanical arm based on dynamic vision can include the following operations:

101. and determining target compensation information of the brick laying robot based on the posture information of the brick laying robot acquired by the camera in the process that the brick laying robot moves to the first brick laying point.

In the embodiment of the invention, the posture information of the brick laying robot comprises brick grabbing posture information of the brick laying robot and/or chassis posture information of the brick laying robot, and the first brick laying point is a brick laying point corresponding to a target moving track generated by a track simulation model.

In the embodiment of the invention, optionally, the camera installed on the robot may be a three-dimensional depth camera, or may be one of a depth camera, a 3D camera and a binocular camera.

In the embodiment of the invention, optionally, the brick grabbing posture information of the brick paving robot is posture information between the brick and the mechanical arm when the mechanical arm of the brick paving robot grabs the brick; the chassis posture information of the brick laying robot is chassis inclination angle information and/or chassis offset information of the brick laying robot; the chassis inclination angle information is used for representing the inclination angle between the chassis of the brick laying robot and the horizontal plane.

102. And performing correction operation on the first brick paving point according to the target compensation information to obtain a second brick paving point, and controlling the brick paving robot to move to the second brick paving point.

In an embodiment of the present invention, optionally, before performing a correction operation on the first tile point according to the target compensation information to obtain the second tile point, the method further includes:

judging whether the target compensation information is used for indicating that the first brick paving point needs to be corrected;

when the target compensation information is judged to be used for indicating that the first brick paving point needs to be corrected, triggering and executing the operation of correcting the first brick paving point according to the target compensation information to obtain a second brick paving point;

when the target compensation information is judged to be used for indicating that the first brick paving point does not need to be corrected, the brick paving robot is controlled to move to the first brick paving point, and when the brick paving robot moves to the first brick paving point, the movement control parameters of the mechanical arm are determined based on the image information of the environment to be paved, acquired by the camera, of the brick to be paved, and the mechanical arm is controlled to execute the operation matched with the movement control parameters.

Therefore, when the target compensation information does not need to be corrected on the first brick paving point, the brick paving robot is directly controlled to move to the first brick paving point, the efficiency of the control mechanical arm can be improved, and further the efficiency of the brick paving robot for executing brick paving operation can be improved.

103. When the brick laying robot reaches the second brick laying point, the movement control parameters of the mechanical arm are determined based on the image information of the brick laying environment to be acquired by the camera, and the mechanical arm is controlled to execute the operation matched with the movement control parameters.

In the embodiment of the invention, the image information of the environment to be tiled comprises the real-time image information of the target object. Wherein the target object comprises one or more of a floor to be constructed, a brick face of a brick to be tiled. Optionally, when the target object includes the ground to be constructed, the real-time image information of the target object includes ground real-time image information of the ground to be constructed; when the target object comprises a brick face of a brick to be tiled, the real-time image information of the target object comprises real-time image information of the brick face of the brick to be tiled. Further, the real-time image information of the ground to be constructed comprises one or more of real-time image information of paved bricks of the ground to be paved and real-time image information of corners corresponding to the ground to be paved.

In an embodiment of the present invention, optionally, the movement control parameters of the mechanical arm include a parallel movement parameter and/or a vertical movement parameter; the parallel movement parameter is used for representing a control parameter for controlling the left-right movement of the mechanical arm parallel to the ground; the vertical movement parameter is used to represent a control parameter for the up and down movement of the robotic arm parallel to the ground.

Therefore, implementing the intelligent control method of the mechanical arm based on dynamic vision described in fig. 1 can determine the target compensation information of the tile paving robot based on the posture information of the tile paving robot acquired by the camera in the process that the tile paving robot moves to the first tile paving point, can adjust the tile grabbing posture of the mechanical arm in the tile paving robot according to the target compensation information, can compensate errors generated when the tile paving robot manually carries out tile grabbing operation, can improve the accuracy of the tile paving operation of the tile paving robot, can carry out correction operation on the first tile paving point according to the target compensation information, obtain the second tile paving point, and control the tile paving robot to move to the second tile paving point, can dynamically adjust the motion end point of the tile paving robot, can improve the accuracy and reliability of determining the motion end point corresponding to the tile paving robot, and can improve the efficiency of determining the motion end point corresponding to the tile paving robot, thereby being beneficial to improving the efficiency of the subsequent tile paving operation of the robot.

Example two

Referring to fig. 2, fig. 2 is a schematic flow chart of a method for controlling an intelligent mechanical arm based on dynamic vision according to an embodiment of the present invention. The method for controlling the mechanical arm intelligently based on the dynamic vision described in fig. 2 can be applied to the mechanical arm intelligent control device based on the dynamic vision, and can also be applied to a local server or a cloud server for controlling the mechanical arm intelligently based on the dynamic vision.

As shown in fig. 2, the intelligent control method for the mechanical arm based on dynamic vision can include the following operations:

201. and acquiring target environment information corresponding to the position of the brick to be paved, and inputting the target environment information and the structural information of the brick paving robot into a preset track simulation model to obtain a target moving track of the brick paving robot.

In the embodiment of the invention, the target moving track comprises a target brick taking point and a first brick paving point.

In an embodiment of the present invention, optionally, the target movement track further includes a movement track of the brick laying robot from the current position to the target brick taking point and a movement track of the brick laying robot from the target brick taking point to the first brick laying point.

In the embodiment of the invention, optionally, the target moving track may be one of a shortest path moving track, a shortest moving track with a shortest moving duration, and a moving track with a minimum number of obstacle avoidance objects.

In the embodiment of the present invention, optionally, the target environment information corresponding to the position of the tile to be paved includes one or more of obstacle position information in the target environment, obstacle volume information in the target environment, and obstacle type information in the target environment. Optionally, the structural information of the brick laying robot includes appearance model information of the brick laying robot, wherein the appearance model information of the brick laying robot includes one or more of mechanical arm information of the brick laying robot, end carrier information of the brick laying robot and load model information of the brick laying robot.

In the embodiment of the invention, optionally, the target brick taking point is a corresponding position when the brick laying robot executes the brick taking operation for the time; the first brick paving points are corresponding positions when the corresponding brick paving robots execute the brick paving operation according to the target moving track generated by the track simulation model.

202. And controlling the brick laying robot to move to a target brick taking point based on the target moving track, and controlling the brick laying robot to execute brick grabbing operation when the brick laying robot reaches the target brick taking point.

In an embodiment of the present invention, optionally, controlling the tile-laying robot to perform a tile-grabbing operation includes:

And controlling the suction disc in the mechanical arm arranged on the brick laying robot to be opened, and executing suction operation on bricks based on the suction disc in the mechanical arm.

In the embodiment of the invention, optionally, the moving distance of the brick laying robot is calculated by collecting point laser data; when the calculated moving distance of the brick laying robot meets a preset distance condition, determining that the brick laying robot reaches a target brick taking point.

203. And judging whether the brick laying robot finishes brick grabbing operation.

In the embodiment of the invention, when the brick laying robot is judged to have completed brick grabbing operation, the execution step 204 is triggered; when it is determined that the brick laying robot does not complete the brick grabbing operation, the step of controlling the brick laying robot to execute the brick grabbing operation in step 202 and the step of determining whether the brick laying robot has completed the brick grabbing operation in step 203 are triggered to execute.

204. The tiling robot is controlled to move to a first tiling point.

In an embodiment of the present invention, optionally, controlling the tile robot to move to the first tile point includes:

and controlling the brick laying robot to execute a moving operation matched with the target moving track according to the target moving track so as to enable the brick laying robot to move to the first brick laying point.

The control of the moving track of the target can control the moving of the paving robot to the first paving point, the moving accuracy of the paving robot to the first paving point can be improved, and the moving reliability of the paving robot to the first paving point can be improved.

205. And determining target compensation information of the brick laying robot based on the posture information of the brick laying robot acquired by the camera in the process that the brick laying robot moves to the first brick laying point.

206. And performing correction operation on the first brick paving point according to the target compensation information to obtain a second brick paving point, and controlling the brick paving robot to move to the second brick paving point.

207. When the brick laying robot reaches the second brick laying point, the movement control parameters of the mechanical arm are determined based on the image information of the brick laying environment to be acquired by the camera, and the mechanical arm is controlled to execute the operation matched with the movement control parameters.

In the embodiment of the present invention, for other descriptions of step 205 to step 207, please refer to the detailed descriptions of step 101 to step 103 in the first embodiment, and the description of the embodiment of the present invention is omitted.

Therefore, the implementation of the intelligent control method of the mechanical arm based on dynamic vision described in fig. 2 can acquire the target environment information corresponding to the position of the brick to be paved, input the target environment information and the structural information of the brick paving robot into a preset track simulation model to obtain the target moving track of the brick paving robot, generate the target moving track of the brick paving robot based on the track simulation model, improve the intelligence of generating the target moving track, and improve the accuracy and reliability of generating the target moving track; the brick laying robot is controlled to move to a target brick taking point based on the target moving track and is controlled to execute brick grabbing operation, so that the accuracy of brick laying operation executed by the brick laying robot can be improved; and just control the tiling robot to remove to first tiling point when judging that the tiling robot has accomplished the operation of grabbing the brick, can improve the tiling robot and carry out the precision and the reliability of grabbing the brick operation and removing the operation, reduce the condition that the tiling robot grabs the brick and fails, and then be favorable to improving the efficiency and the accuracy that the tiling robot carried out the tiling operation.

In an alternative embodiment, when the pose information of the brick laying robot includes the brick grabbing pose information of the brick laying robot, determining the target compensation information of the brick laying robot based on the pose information of the brick laying robot acquired by the camera includes:

based on the brick grabbing posture information of the brick paving robot, which is acquired by the camera, judging whether the brick grabbing posture information meets preset brick grabbing posture conditions;

when the brick grabbing posture information is judged to not meet the preset brick grabbing posture condition, acquiring first environment information of a first brick paving point;

determining first error information of the brick laying robot according to the brick grabbing posture information and the first environment information, and generating first compensation information based on the first error information, wherein the first error information is error information generated by the brick laying robot in the brick grabbing operation process;

In this alternative embodiment, optionally, the cameras disposed on the tile robot may be disposed at the chassis of the tile robot, and the number of cameras may be 4, and the specific number of cameras is not limited in the embodiment of the present invention.

In this optional embodiment, optionally, determining whether the tile grabbing posture information meets a preset tile grabbing posture condition includes:

according to the brick grabbing attitude information, brick inclination angle information and brick horizontal difference information of bricks grabbed by the brick paving robot are determined, wherein the brick inclination angle information is used for representing inclination angles between the bricks and a horizontal line, and the brick horizontal difference information is used for representing difference distances between the bricks and standard grabbing points preset by the brick paving robot;

judging whether brick inclination angle information of bricks grabbed by the brick paving robot meets preset brick inclination conditions or not, and judging whether brick level difference information of the bricks grabbed by the brick paving robot meets preset brick difference distance conditions or not;

when the brick inclination angle information grasped by the brick paving robot is judged to meet the preset brick inclination condition and the brick level difference information of the brick grasped by the brick paving robot is judged to meet the preset brick difference distance condition, determining that the brick grasping posture information meets the preset brick grasping posture condition;

when the brick inclination angle information grabbed by the brick paving robot is judged to not meet the preset brick inclination condition and/or the brick level difference information of the brick grabbed by the brick paving robot is judged to not meet the preset brick difference distance condition, determining that the brick grabbing posture information does not meet the preset brick grabbing posture condition.

In this optional embodiment, optionally, determining target compensation information of the tile robot according to the first compensation information includes:

the first compensation information is determined as target compensation information for the tiling robot.

In this alternative embodiment, optionally, the first error information comprises tilt error information and/or horizontal distance error information generated by the tiling robot during the execution of the tile grabbing operation.

In this alternative embodiment, optionally, the first environmental information of the first brick laying point includes ground information corresponding to the first brick laying point, wherein the ground information corresponding to the first brick laying point includes ground laid brick information.

In this optional embodiment, optionally, when it is determined that the tile grabbing posture information satisfies a preset tile grabbing posture condition, the present process may be ended.

It can be seen that, implementing this optional embodiment can judge whether the brick-grabbing posture information of the brick-laying robot that is collected based on the camera meets the brick-grabbing posture condition that is preset, if yes, obtain the first environmental information of first brick-laying point, and confirm the first error information of brick-laying robot according to brick-grabbing posture information and first environmental information, and generate first compensation information and then confirm the target compensation information of brick-laying robot according to first error information, can confirm the first error information of robot based on the brick-grabbing posture information of brick-laying robot that is collected by the camera and first environmental information, can improve the accuracy and the reliability of confirming the first error information, and can improve the intelligence of confirming the first error information, thereby be favorable to improving the accuracy and the reliability of producing the first compensation information, be favorable to improving the accuracy and the reliability of confirming the target compensation information of brick-laying robot, can realize dynamic compensation to the intelligent control of robot based on dynamic vision, and then be favorable to improving the accuracy and the reliability of controlling the brick-laying robot.

In another alternative embodiment, when the pose information of the brick laying robot includes chassis pose information of the brick laying robot, determining target compensation information of the brick laying robot based on the pose information for the brick laying robot acquired by the camera includes:

based on chassis posture information, acquired by a camera, of the brick laying robot, judging whether the chassis posture information meets preset chassis posture conditions or not;

In this optional embodiment, optionally, determining target compensation information of the tile robot according to the second compensation information includes:

and determining the second compensation information as target compensation information of the brick laying robot.

In this optional embodiment, optionally, when the posture information of the brick laying robot includes chassis posture information of the brick laying robot and brick grabbing posture information of the brick laying robot, determining target compensation information of the brick laying robot based on the posture information for the brick laying robot acquired by the camera includes:

And determining target compensation information of the brick laying robot according to the first compensation information and the second compensation information.

Optionally, determining target compensation information of the brick laying robot according to the first compensation information and the second compensation information includes:

the first compensation information and the second compensation information are determined as target compensation information of the brick laying robot.

Therefore, when the posture information of the brick laying robot simultaneously comprises chassis posture information and brick grabbing posture information, the target compensation information of the brick laying robot is determined based on the first compensation information and the second compensation information, the accuracy and the comprehensiveness of determining the target compensation information can be improved, and further the accuracy and the intelligence of controlling the brick laying robot are improved.

In this alternative embodiment, the chassis inclination error information of the tiling robot may optionally include one or more of angle inclination error information of the tiling robot's chassis with respect to the horizontal plane, distance error information of the tiling robot's chassis with respect to the ground.

In this alternative embodiment, optionally, when it is determined that the chassis posture information satisfies the preset chassis posture condition, the present flow may be ended.

In this optional embodiment, optionally, when the posture information of the brick laying robot includes the brick grabbing posture information of the brick laying robot and the chassis posture information of the brick laying robot, cameras are mounted on both the mechanical arm of the brick laying robot and the chassis of the brick laying robot. The number of cameras installed on the chassis of the brick laying robot can be two, the number of cameras in each group can be 1 or 2, and the specific number of cameras on the chassis is not limited in the embodiment of the invention. Optionally, when the quantity of installing the camera on the tiling robot chassis is two sets of, its two sets of cameras set up respectively in the place ahead and the rear along chassis advancing direction, and the camera is used for detecting chassis gesture information, also namely: based on real-time image information acquired by two groups of cameras arranged on a chassis of the brick paving robot, brick edge lines of paved bricks or preset reference line information can be identified, and then, based on the identification result, the advancing track of the brick paving robot is determined, namely, the advancing track of the brick paving robot is determined according to the identification result, and the advancing track of the brick paving robot is determined to move along the paved brick edge lines or the preset reference line. And based on a plurality of cameras arranged on the mechanical arm and the chassis of the brick laying robot, the real-time image information of the mechanical arm corresponding to the mechanical arm in the brick laying robot and the real-time image information of the chassis corresponding to the chassis in the brick laying robot can be acquired in real time, and the brick laying robot can be controlled to move and execute brick laying operation based on the real-time image information of the mechanical arm and the real-time image information of the chassis. Like this, can guarantee that the tiling robot walks along the fragment of brick sideline of the tiling piece of having been paved or the straight line of preset reference line through setting up a plurality of cameras on tiling robot chassis, can improve the control tiling robot along the accuracy of advancing the orbit and remove to be favorable to improving the accuracy that the control tiling robot removed, and then be favorable to improving the accuracy that the tiling robot carried out the tiling operation.

It can be seen that, implementing this optional embodiment can judge whether chassis posture information satisfies preset chassis posture condition based on the chassis posture information that the camera gathered to the tile-laying robot, if not, then confirm chassis inclination coefficient according to the chassis posture information of tile-laying robot, and thereby generate second compensation information according to chassis inclination coefficient determination second error information, and then confirm the target compensation information of tile-laying robot according to second compensation information, can confirm the second error information of robot based on the chassis posture information of tile-laying robot that the camera gathered, can improve the accuracy and the reliability of confirming the second error information, and can improve the intelligence of confirming the second error information, thereby be favorable to improving accuracy and the reliability of generating the second compensation information, be favorable to improving accuracy and the reliability of confirming the target compensation information of tile-laying robot, can realize dynamic compensation to the intelligent control of robot based on dynamic vision, and then be favorable to improving accuracy and the reliability of controlling the tile-laying robot.

In yet another alternative embodiment, determining control parameters of the robotic arm based on the image information of the environment to be tiled acquired by the camera includes:

Based on real-time image information of a target object acquired by a camera, calculating a brick joint distance between a first brick site corresponding to a brick to be paved and a predetermined reference object, and judging whether the brick joint distance meets a preset brick joint condition;

when the brick joint distance is judged to not meet the preset brick joint condition, determining dynamic tracking parameters of the mechanical arm according to the brick joint distance and the first brick joint point;

generating movement control parameters of the mechanical arm according to the dynamic tracking parameters; the dynamic tracking parameter is used for representing the dynamic relative position relation between the mechanical arm and the first brick site;

and controlling the mechanical arm to execute the operation matched with the movement control parameter, comprising:

and controlling the mechanical arm to move based on the movement control parameters so as to enable the brick joint distance between the first brick site corresponding to the brick to be paved and the predetermined reference object to meet the preset brick joint condition.

In this alternative embodiment, the predetermined reference may be one of a brick that has been laid in the floor to be constructed, a reference brick placed in advance in the floor to be constructed, a standard line for representing the brick edge of the laid brick, and a standard for representing the brick edge of the laid brick, as an alternative.

In this alternative embodiment, the alternative brick joint distance is used to represent the distance between the brick corresponding to the first brick site and the predetermined reference.

In this alternative embodiment, further alternatively, the predetermined reference object may be an adjacent brick adjacent to the brick to be paved, and further, the number of adjacent bricks adjacent to the brick to be paved may be one or more than one; when the number of adjacent bricks to be tiled is plural, the adjacent bricks to be tiled may be at one of the upper, lower, left, right positions corresponding to the bricks to be tiled. Further optionally, when the number of adjacent bricks to be tiled is plural, each adjacent brick has a second brick site corresponding to the adjacent brick, wherein calculating a brick joint distance between a first brick site corresponding to the brick to be tiled and a predetermined reference object, and determining whether the brick joint distance meets a preset brick joint condition includes: calculating target brick joint distances between a first brick site corresponding to a brick to be paved and each second brick site of each adjacent brick of the brick to be paved, and judging whether all the target brick joint distances meet preset brick joint conditions; when all the target brick joint distances are judged to meet the preset brick joint conditions, determining that the brick joint distances meet the preset brick joint conditions; when the fact that the uneven distances of all the target bricks meet the preset brick joint conditions is judged, determining that the distances of the bricks do not meet the preset brick joint conditions. Therefore, when a plurality of adjacent bricks are arranged, the target brick joint distance between the first brick joint point corresponding to the brick to be paved and the second brick joint point of each adjacent brick is calculated, whether the brick joint distance meets the preset brick joint condition is judged according to all the target brick joint distances, the accuracy and the reliability of judging whether the brick joint distance meets the preset brick joint condition can be improved, and the intelligence and the comprehensiveness of judging whether the brick joint distance meets the preset brick joint condition can be improved.

In this alternative embodiment, optionally, dynamic tracking parameters can be used to represent the dynamic relative positional relationship between the robotic arm and the first brick site during movement of the robotic arm.

In this alternative embodiment, optionally, the method further comprises:

when the brick joint distance is judged to meet the preset brick joint condition, controlling the brick laying robot to execute brick laying operation.

Therefore, by implementing the optional embodiment, the brick joint distance between the first brick site corresponding to the brick to be paved and the predetermined reference object can be calculated based on the real-time ground image information, whether the brick joint distance meets the preset brick joint condition is judged, if not, the dynamic tracking parameter of the mechanical arm is determined, the movement control parameter of the mechanical arm is further generated, and the accuracy and the reliability of judging whether the brick joint distance meets the preset brick joint condition can be improved, so that the accuracy and the reliability of determining the dynamic tracking parameter of the mechanical arm are improved, and the accuracy and the reliability of generating the movement control parameter of the mechanical arm are improved; and moreover, the mechanical arm is controlled to move based on the movement control parameters, so that the brick joint distance between a first brick position point corresponding to the brick to be paved and a predetermined reference object meets the preset brick joint condition, the accuracy and stability of the brick paving robot in performing brick paving operation can be improved, the movement control parameters of the mechanical arm are generated based on the dynamic tracking parameters, quick dynamic tracking movement can be realized, the movement efficiency and accuracy of the mechanical arm are controlled, and the brick paving operation efficiency and accuracy of the brick paving robot are improved.

In yet another alternative embodiment, after the tiling robot reaches the second tiling point, before determining the control parameters of the robotic arm based on the image information of the environment to be tiled acquired by the camera, the method further comprises:

the method comprises the steps that a camera is controlled to collect environmental image information of a brick to be paved, the brick paving position relation between a mechanical arm and the ground of the brick to be paved is determined based on a visual servo technology and the environmental image information of the brick to be paved, and whether the brick paving position relation meets a preset position condition is judged; the brick laying positional relationship is used for representing the relative positional relationship between the mechanical arm and bricks included in the ground to be tiled;

when the brick paving position relation is judged to not meet the preset position condition, triggering and executing the operation of determining the control parameters of the mechanical arm based on the image information of the brick paving environment to be acquired by the camera;

when the brick laying position relation is judged to meet the preset position condition, controlling the brick laying robot to execute brick laying operation.

In this optional embodiment, optionally, the environmental image information to be tiled includes ground image information to be tiled; the to-be-tiled ground image information comprises one or more of tiled information of to-be-tiled ground and ground corner information of to-be-tiled ground.

In this alternative embodiment, it should be noted that the visual servoing technique is an action of automatically receiving and processing an image of a real object by means of optical devices and non-contact sensors, and by means of information fed back by the image, letting the machine system perform further control or corresponding adaptive adjustment of the machine.

In this alternative embodiment, optionally, the relative position between the robotic arm and the brick included in the floor to be tiled includes one or more of a horizontal distance relationship of the robotic arm and the brick included in the floor to be tiled, and an inclination angle relationship of the robotic arm and the brick included in the floor to be tiled.

In this optional embodiment, optionally, determining whether the tile position relationship satisfies a preset position condition includes:

judging whether the brick paving position relationship is used for indicating that the horizontal distance value of the mechanical arm and the brick included in the ground to be paved is larger than a preset distance threshold value and the inclination angle value of the mechanical arm and the brick included in the ground to be paved is smaller than a preset angle threshold value;

when the brick paving position relation is judged to be used for indicating that the horizontal distance value of the mechanical arm and the brick included in the ground to be paved is larger than a preset distance threshold value and the inclination angle value of the mechanical arm and the brick included in the ground to be paved is smaller than a preset angle threshold value, determining that the brick paving position relation meets the preset position condition;

When the brick paving position relation is judged to be used for indicating that the horizontal distance value of the mechanical arm and the brick included in the ground to be paved is not larger than a preset distance threshold value and/or the inclination angle value of the mechanical arm and the brick included in the ground to be paved is not smaller than a preset angle threshold value, determining that the brick paving position relation does not meet preset position conditions.

In this optional embodiment, optionally, the visual servo technology includes a visual servo mode of the camera module and/or a visual servo mode of the mechanical arm module, when the mechanical arm moves to the second brick laying point, the visual servo mode of the camera module is turned on, and after the visual servo mode of the camera module is successfully turned on, the visual servo mode of the mechanical arm module is turned on, so that the mechanical arm completes adjustment of the movement track through visual real-time data acquired by the visual servo mode of the mechanical arm module, whether the mechanical arm has completed adjustment of the movement track is judged, if it is judged that the mechanical arm has completed adjustment of the movement track, the visual servo mode of the mechanical arm module is turned off, and after the visual servo mode of the mechanical arm module is successfully turned off, the visual servo mode of the camera module is turned off. Therefore, the opening and closing of the visual servo mode of the mechanical arm module can be controlled based on the visual servo mode of the camera module, the accuracy and reliability of using the visual servo mode can be improved, and the accuracy of track adjustment of the brick laying robot based on the visual servo mode can be improved.

Therefore, according to the implementation of the optional embodiment, the brick laying position relation between the mechanical arm and the brick laying ground can be determined based on the visual servo technology and the brick laying environment image information, whether the brick laying position relation meets the preset position condition or not is judged, if not, the control parameters of the mechanical arm are determined based on the brick laying environment image information, if yes, the brick laying robot is controlled to execute brick laying operation, the brick laying position relation can be determined based on the visual servo technology and the brick laying environment image information, the accuracy of determining the brick laying position relation can be improved, the accuracy and the reliability of judging whether the brick laying position relation meets the preset position condition or not can be improved, and further the accuracy of determining the control parameters of the mechanical arm and the efficiency of controlling the brick laying robot to execute the brick laying operation can be improved.

In yet another alternative embodiment, the mechanical arm is provided with a sucker, and the sucker is used for grabbing the brick;

judging whether the brick laying robot has completed brick grabbing operation, comprising:

acquiring a vacuum value of the sucker, and judging whether the vacuum value is larger than a preset vacuum threshold value;

when the vacuum value is judged to be larger than a preset vacuum threshold value, determining that the brick laying robot finishes brick grabbing operation;

And when the vacuum value is not larger than the preset vacuum threshold value, determining that the brick-laying robot does not complete brick-grabbing operation.

In this alternative embodiment, optionally, after determining that the tile robot has not completed the tile grabbing operation, the method further comprises:

and controlling the sucker on the mechanical arm to re-execute the brick grabbing operation, and re-triggering and executing the step of judging whether the brick paving robot finishes the brick grabbing operation or not until the vacuum value of the sucker is judged to be larger than the preset vacuum threshold value.

Therefore, by implementing the optional embodiment, whether the vacuum value is larger than a preset vacuum threshold value can be judged based on the acquired vacuum value of the sucker, if so, the brick laying robot is determined to finish brick grabbing operation, and if not, the brick laying robot is determined to finish brick grabbing operation, so that accuracy of judging whether the brick laying robot finishes brick grabbing operation can be improved, reliability of judging whether the brick laying robot finishes brick grabbing operation can be improved, the situation that the brick laying robot fails to grab bricks can be reduced, and further efficiency and accuracy of executing brick laying operation by the brick laying robot are improved.

Example III

Referring to fig. 3, fig. 3 is a schematic structural diagram of another intelligent control device for a mechanical arm based on dynamic vision according to an embodiment of the present invention, where the device described in fig. 3 is used for controlling a tile robot, and the mechanical arm is mounted on the tile robot, and a camera is mounted on the tile robot. As shown in fig. 3, the intelligent control device for a mechanical arm based on dynamic vision may include:

The determining module 301 is configured to determine target compensation information of the brick laying robot based on pose information of the brick laying robot acquired by the camera during the process of moving the brick laying robot to the first brick laying point; the attitude information of the brick laying robot comprises brick grabbing attitude information of the brick laying robot and/or chassis attitude information of the brick laying robot, and the first brick laying point is a brick laying point corresponding to a target moving track generated by a track simulation model;

the correction module 302 is configured to perform a correction operation on the first brick laying point according to the target compensation information, so as to obtain a second brick laying point;

a control module 303 for controlling the tile robot to move to a second tile point;

the determining module 301 is further configured to determine, when the tiling robot reaches the second tiling point, a movement control parameter of the mechanical arm based on the to-be-tiled environment image information acquired by the camera;

the control module 303 is further configured to control the mechanical arm to perform an operation matched with the movement control parameter; the environmental image information to be tiled includes real-time ground image information of the ground to be tiled.

Therefore, the device described in fig. 3 can be implemented to determine the target compensation information of the paving robot based on the posture information of the paving robot collected by the camera in the process that the paving robot moves to the first paving point, can adjust the brick grabbing posture of the mechanical arm in the paving robot according to the target compensation information, can compensate errors generated when the paving robot manually carries out brick stacking, can improve the accuracy of brick grabbing operation of the paving robot, can execute correction operation on the first paving point according to the target compensation information, obtain the second paving point, can control the paving robot to move to the second paving point, can dynamically adjust the movement end point of the paving robot, can improve the accuracy and reliability of determining the movement end point corresponding to the paving robot, and can improve the efficiency of determining the movement end point corresponding to the paving robot, so as to be beneficial to improving the efficiency of executing subsequent brick operation of the paving robot.

In an alternative embodiment, as shown in fig. 4, the apparatus further comprises:

the acquisition module 304 is configured to acquire target environmental information corresponding to a position of a tile to be paved before the tile paving robot moves to a first tile paving point;

the input module 305 is configured to input target environmental information and structural information of the tile robot into a preset trajectory simulation model, so as to obtain a target movement trajectory of the tile robot, where the target movement trajectory includes a target brick taking point and a first tile paving point;

the control module 303 is further configured to control the brick laying robot to move to a target brick taking point based on the target movement track, and when the brick laying robot reaches the target brick taking point, control the brick laying robot to perform brick grabbing operation;

a judging module 306, configured to judge whether the brick laying robot has completed the brick grabbing operation; when it is determined that the brick laying robot does not complete the brick grabbing operation, the brick laying robot is re-triggered to execute the operation of controlling the brick laying robot to execute the brick grabbing operation and whether the brick laying robot has completed the brick grabbing operation is determined;

the control module 303 is further configured to control the brick laying robot to move to the first brick laying point when the judging module 306 judges that the brick laying robot has completed the brick grabbing operation.

As can be seen, implementing the apparatus described in fig. 4 can collect the target environment information corresponding to the position of the tile to be paved, and input the target environment information and the structural information of the tile paving robot into a preset track simulation model to obtain the target movement track of the tile paving robot, so as to generate the target movement track of the tile paving robot based on the track simulation model, improve the intelligence of generating the target movement track, and improve the accuracy and reliability of generating the target movement track; the brick laying robot is controlled to move to a target brick taking point based on the target moving track and is controlled to execute brick grabbing operation, so that the accuracy of brick laying operation executed by the brick laying robot can be improved; and just control the tiling robot to remove to first tiling point when judging that the tiling robot has accomplished the operation of grabbing the brick, can improve the tiling robot and carry out the precision and the reliability of grabbing the brick operation and removing the operation, reduce the condition that the tiling robot grabs the brick and fails, and then be favorable to improving the efficiency and the accuracy that the tiling robot carried out the tiling operation.

In another alternative embodiment, the determining module 301 determines, based on the pose information for the tiling robot acquired by the camera, the specific manner of the target compensation information of the tiling robot includes:

When the posture information of the brick laying robot comprises brick grabbing posture information of the brick laying robot, judging whether the brick grabbing posture information meets preset brick grabbing posture conditions or not based on the brick grabbing posture information of the brick laying robot acquired by a camera;

It can be seen that, implementing the device described in fig. 4 can determine whether the tile grabbing posture information meets the preset tile grabbing posture condition based on the tile grabbing posture information of the tile paving robot acquired by the camera, if so, the first environment information of the first tile paving point is obtained, the first error information of the tile paving robot is determined according to the tile grabbing posture information and the first environment information, the first compensation information is generated according to the first error information, and then the target compensation information of the tile paving robot is determined, the first error information of the robot can be determined based on the tile grabbing posture information of the tile paving robot acquired by the camera and the first environment information, the accuracy and the reliability of determining the first error information can be improved, and the intelligence of determining the first error information can be improved, so that the accuracy and the reliability of generating the first compensation information are improved, the accuracy and the reliability of determining the target compensation information of the tile paving robot are improved, the dynamic compensation can be realized based on intelligent control of the dynamic vision on the tile paving robot, and the accuracy and the reliability of the tile paving robot are improved.

In yet another alternative embodiment, the determining module 301 determines, based on pose information for the tiling robot acquired by the camera, the specific manner of the target compensation information of the tiling robot includes:

It can be seen that the device described in fig. 4 is implemented to determine whether the chassis posture information meets the preset chassis posture condition based on the chassis posture information of the tile-paving robot acquired by the camera, if not, determine the chassis inclination coefficient according to the chassis posture information of the tile-paving robot, and determine the second error information according to the chassis inclination coefficient, thereby generating the second compensation information, further determine the target compensation information of the tile-paving robot according to the second compensation information, determine the second error information of the robot based on the chassis posture information of the tile-paving robot acquired by the camera, improve the accuracy and reliability of determining the second error information, and improve the intelligence of determining the second error information, thereby being beneficial to improving the accuracy and reliability of generating the second compensation information, being beneficial to improving the accuracy and reliability of determining the target compensation information of the tile-paving robot, being beneficial to realizing dynamic compensation based on intelligent control of the robot, and further being beneficial to improving the accuracy and reliability of controlling the tile-paving robot.

In yet another alternative embodiment, the determining module 301 determines, based on the image information of the environment to be tiled acquired by the camera, the specific manner of determining the control parameters of the mechanical arm includes:

based on real-time ground image information of the ground to be tiled acquired by a camera, calculating a brick joint distance between a first brick site corresponding to the brick to be tiled and a predetermined reference object, and judging whether the brick joint distance meets a preset brick joint condition;

and, specific ways of controlling the mechanical arm to perform the operation matched with the movement control parameter by the control module 303 include:

As can be seen, implementing the apparatus described in fig. 4 can calculate the brick joint distance between the first brick site corresponding to the brick to be paved and the predetermined reference object based on the real-time ground image information, and determine whether the brick joint distance meets the preset brick joint condition, if not, determine the dynamic tracking parameter of the mechanical arm and further generate the movement control parameter of the mechanical arm, so as to improve the accuracy and reliability of determining whether the brick joint distance meets the preset brick joint condition, thereby being beneficial to improving the accuracy and reliability of determining the dynamic tracking parameter of the mechanical arm, and further being beneficial to improving the accuracy and reliability of generating the movement control parameter of the mechanical arm; and moreover, the mechanical arm is controlled to move based on the movement control parameters, so that the brick joint distance between a first brick position point corresponding to the brick to be paved and a predetermined reference object meets the preset brick joint condition, the accuracy and stability of the brick paving robot in performing brick paving operation can be improved, the movement control parameters of the mechanical arm are generated based on the dynamic tracking parameters, quick dynamic tracking movement can be realized, the movement efficiency and accuracy of the mechanical arm are controlled, and the brick paving operation efficiency and accuracy of the brick paving robot are improved.

In yet another alternative embodiment, the control module 303 is further configured to control the camera to acquire the environmental image information to be tiled, after the tiling robot arrives at the second tiling point, before the determining module 301 determines the control parameters of the mechanical arm based on the environmental image information to be tiled acquired by the camera;

the determining module 301 is further configured to determine a tile position relationship between the mechanical arm and the ground to be tiled based on the visual servo technology and the environmental image information of the tile to be tiled;

the judging module 306 is further configured to judge whether the brick-laying position relationship meets a preset position condition; the brick laying positional relationship is used for representing the relative positional relationship between the mechanical arm and bricks included in the ground to be tiled; when the brick laying position relation is judged not to meet the preset position condition, triggering the operation of determining the control parameters of the mechanical arm based on the image information of the brick laying environment to be acquired by the camera, which is executed by the determining module 301;

the control module 303 is further configured to control the tile paving robot to perform a tile paving operation when the determining module 306 determines that the tile position relationship meets a preset position condition.

Therefore, the device described in fig. 4 can determine the brick paving position relationship between the mechanical arm and the brick paving ground based on the visual servo technology and the environmental image information of the brick to be paved, and judge whether the brick paving position relationship meets the preset position condition, if not, determine the control parameters of the mechanical arm based on the environmental image information of the brick to be paved, and if yes, control the brick paving robot to execute the brick paving operation, and determine the brick paving position relationship based on the visual servo technology and the environmental image information of the brick to be paved, so that the accuracy and the reliability of determining whether the brick paving position relationship meets the preset position condition can be improved, and further the accuracy of determining the control parameters of the mechanical arm and the efficiency of controlling the brick paving robot to execute the brick paving operation are improved.

the specific ways in which the determining module 306 determines whether the tile robot has completed the tile grabbing operation include:

Therefore, the device described in fig. 4 can be implemented to judge whether the vacuum value is larger than the preset vacuum threshold based on the obtained vacuum value of the suction cup, if so, the brick laying robot is determined to finish the brick grabbing operation, and if not, the brick laying robot is determined to finish the brick grabbing operation, so that the accuracy of judging whether the brick laying robot finishes the brick grabbing operation can be improved, the reliability of judging whether the brick laying robot finishes the brick grabbing operation can be improved, the situation that the brick laying robot fails to grab bricks can be reduced, and further the efficiency and the accuracy of executing brick laying operation by the brick laying robot can be improved.

Example IV

Referring to fig. 5, fig. 5 is a schematic structural diagram of another intelligent control device for a mechanical arm based on dynamic vision according to an embodiment of the present invention. The device described in fig. 5 is used for controlling a brick laying robot, and the mechanical arm is mounted on the brick laying robot, and the camera is mounted on the brick laying robot, so that the device can be integrated in the brick laying robot or can exist independently of the brick laying robot, for example, the device is arranged on a cloud control platform, and the embodiment of the invention is not limited. As shown in fig. 5, the mechanical arm intelligent control device based on dynamic vision may include:

a memory 401 storing executable program codes;

a processor 402 coupled with the memory 401;

the processor 402 invokes executable program codes stored in the memory 401 to execute the steps in the dynamic vision-based robot arm intelligent control method described in the first or second embodiment of the present invention.

Example five

The embodiment of the invention discloses a computer storage medium which stores computer instructions for executing the steps in the intelligent control method of the mechanical arm based on dynamic vision described in the first embodiment or the second embodiment of the invention when the computer instructions are called.

Example six

An embodiment of the present invention discloses a computer program product, which includes a non-transitory computer readable storage medium storing a computer program, and the computer program is operable to cause a computer to perform the steps in the dynamic vision-based robot arm intelligent control method described in the first embodiment or the second embodiment.

Example seven

The embodiment of the invention discloses a brick laying robot, which is provided with a mechanical arm and a camera. The brick laying robot is used for executing part or all of the steps in the dynamic vision-based mechanical arm intelligent control method disclosed in any one of the first embodiment and the second embodiment of the invention, or the brick laying robot can comprise any one of the dynamic vision-based mechanical arm intelligent control devices described in the third embodiment.

The apparatus embodiments described above are merely illustrative, wherein the modules illustrated as separate components may or may not be physically separate, and the components shown as modules may or may not be physical, i.e., may be located in one place, or may be distributed over a plurality of network modules. Some or all of the modules may be selected according to actual needs to achieve the purpose of the solution of this embodiment. Those of ordinary skill in the art will understand and implement the present invention without undue burden.

From the above detailed description of the embodiments, it will be apparent to those skilled in the art that the embodiments may be implemented by means of software plus necessary general hardware platforms, or of course by means of hardware. Based on such understanding, the foregoing technical solutions may be embodied essentially or in part in the form of a software product that may be stored in a computer-readable storage medium including Read-Only Memory (ROM), random-access Memory (Random Access Memory, RAM), programmable Read-Only Memory (Programmable Read-Only Memory, PROM), erasable programmable Read-Only Memory (Erasable Programmable Read Only Memory, EPROM), one-time programmable Read-Only Memory (OTPROM), electrically erasable programmable Read-Only Memory (EEPROM), compact disc Read-Only Memory (Compact Disc Read-Only Memory, CD-ROM) or other optical disc Memory, magnetic disc Memory, tape Memory, or any other medium that can be used for computer-readable carrying or storing data.

Finally, it should be noted that: the embodiment of the invention discloses a mechanical arm intelligent control method and device based on dynamic vision and a brick laying robot, which are disclosed by the embodiment of the invention only as a preferred embodiment of the invention, and are only used for illustrating the technical scheme of the invention, but not limiting the technical scheme; although the invention has been described in detail with reference to the foregoing embodiments, those of ordinary skill in the art will understand that; the technical scheme recorded in the various embodiments can be modified or part of technical features in the technical scheme can be replaced equivalently; such modifications and substitutions do not depart from the spirit and scope of the corresponding technical solutions.

Claims

1. The utility model provides a mechanical arm intelligent control method based on dynamic vision which characterized in that, the mechanical arm is installed on the tiling robot, install the camera on the tiling robot, the method includes:

2. The method of claim 1, further comprising, prior to the tile robot moving to the first tile point:

3. The dynamic vision-based robot arm intelligent control method according to claim 2, wherein when the pose information of the brick laying robot includes the brick gripping pose information of the brick laying robot, the determining the target compensation information of the brick laying robot based on the pose information of the brick laying robot acquired by the camera includes:

4. The intelligent control method of a mechanical arm based on dynamic vision according to claim 2, wherein when the posture information of the brick laying robot includes chassis posture information of the brick laying robot, the determining target compensation information of the brick laying robot based on the posture information of the brick laying robot acquired by the camera includes:

5. The intelligent control method of the mechanical arm based on dynamic vision according to any one of claims 1 to 4, wherein the determining the control parameter of the mechanical arm based on the image information of the environment to be tiled acquired by the camera includes:

based on the real-time image information of the target object acquired by the camera, calculating the brick joint distance between a first brick site corresponding to the brick to be paved and a predetermined reference object, and judging whether the brick joint distance meets a preset brick joint condition;

6. The intelligent control method of a mechanical arm based on dynamic vision according to claim 5, wherein after the brick laying robot reaches the second brick laying point, before determining the control parameters of the mechanical arm based on the environmental image information to be brick laid acquired by the camera, the method further comprises:

7. The intelligent control method for the mechanical arm based on dynamic vision according to claim 2, wherein a sucker is arranged on the mechanical arm and is used for grabbing bricks;

8. Mechanical arm intelligent control device based on dynamic vision, a serial communication port, the device is applied to the tiling robot, the mechanical arm is installed on the tiling robot, install the camera on the tiling robot, the device includes:

9. Mechanical arm intelligent control device based on dynamic vision, characterized in that, the device is applied to tiling robot, the device includes:

a memory storing executable program code;

a processor coupled to the memory;

the processor invokes the executable program code stored in the memory to perform the dynamic vision-based robotic arm intelligent control method of any one of claims 1-7.

10. The brick laying robot is characterized in that the mechanical arm is arranged on the brick laying robot, and a camera is arranged on the brick laying robot;

Wherein the tiling robot is used for executing the intelligent control method of the mechanical arm based on dynamic vision as claimed in any one of claims 1 to 7.