CN113052863A

CN113052863A - Robot-based object surface shallow groove profile extraction method and device, electronic equipment and storage medium

Info

Publication number: CN113052863A
Application number: CN202110426228.7A
Authority: CN
Inventors: 李辉; 魏海永; 王昭钦; 张帅; 丁有爽; 邵天兰
Original assignee: Mech Mind Robotics Technologies Co Ltd
Current assignee: Mech Mind Robotics Technologies Co Ltd
Priority date: 2021-04-20
Filing date: 2021-04-20
Publication date: 2021-06-29

Abstract

The application discloses a robot-based object surface shallow groove contour extraction method, a robot-based object surface shallow groove contour extraction device, electronic equipment and a computer-readable storage medium. The robot-based method for extracting the shallow concave groove profile of the object surface comprises the following steps: acquiring a two-dimensional image of the surface of an object; carrying out image equalization operation on the image; carrying out image segmentation operation on the equalized image; and determining a shallow groove according to the segmented image and extracting a shallow groove profile. The invention can accurately identify and extract even when the groove to be filled is shallow. The invention also discloses a groove filling method based on groove profile identification, which can implement a filling mode special for the groove aiming at different grooves, thereby greatly improving the filling precision. The invention also discloses a groove filling method based on robot moving speed control, which can accurately fill grooves with different widths by controlling the moving speed of the robot according to the width of the groove and improve the filling precision. The invention also discloses a robot-based non-closed groove filling method for the surface of the object, which can improve the filling precision of the grooves. Therefore, the groove filling robot solves the problem of the square surface in the scene of groove filling by using the robot.

Description

Robot-based object surface shallow groove profile extraction method and device, electronic equipment and storage medium

Technical Field

The present application relates to the field of B25 intelligent robots, and more particularly, to a robot-based object surface shallow groove contour extraction method, a robot-based object surface shallow groove contour extraction apparatus, an electronic device, and a storage medium.

Background

At present, with the wide spread of the smart programming robot, the operation of filling the filler in the groove on the surface of the object by means of the smart programming robot has been realized. However, the conventional intelligent robot can only fill the fixed type of object and the fixed scene, in which case the robot must fill along a fixed track at a fixed moving speed and discharging speed, and the method cannot be applied to the fixed object and the industrial scene other than the fixed scene. Even if the existing robot vision technology is used for identifying the grooves on the surfaces of different articles and determining filling tracks for filling, the precise filling of neither material shortage nor material stacking is difficult to achieve. On the one hand, the difficulty is that all grooves cannot be accurately identified, and on the other hand, the difficulty is that the filling degree of the filler in the grooves is easily inconsistent when the grooves with different characteristics are filled by using the same filling rule.

Disclosure of Invention

In view of the above, the present invention has been made to overcome the above problems or at least partially solve the above problems. Specifically, one of the innovations of the present invention is that the applicant found that one of the reasons for the above-mentioned inconsistency of the filling degree is that the prior art does not provide different filling methods for different grooves, for example, when the groove is in a linear shape, the robot must move and fill while moving, and when the groove is a point, the robot must precisely identify and fill the point at the position. Therefore, the invention provides a method for identifying the type of the groove and filling the groove by using a filling scheme corresponding to the type by a robot, thereby greatly improving the accuracy of groove filling.

The second innovation of the present invention is that the applicant found that it is difficult to accurately identify all the grooves because the existing robot vision technology uses three-dimensional point cloud information, i.e. three-dimensional image information, of an object for identification, however, if the grooves are shallow, the grooves cannot be identified according to the three-dimensional point cloud information. Therefore, the shallow groove recognition method and the shallow groove recognition device can accurately recognize and extract the shallow groove by acquiring the two-dimensional image of the surface of the object, and performing equalization and segmentation operation on the two-dimensional image, thereby greatly improving the groove recognition capability.

The third innovation of the present invention is that the applicant found that another reason for the above-mentioned inconsistency of the filling degree is that the moving speed of the robot is fixed and the discharging speed is also fixed in the prior art, so that when a section of the groove is wide or narrow, the robot performs filling at the average moving speed and discharging speed, thereby failing to fill the wide groove and forming a pile of the filling material at the shallow groove. Therefore, the moving speed of the robot is adjusted according to different groove widths on the premise of not changing the discharging speed, and the groove filling accuracy is greatly improved.

The fourth innovation of the present invention is that the applicant found that although the characteristics of the closed and non-closed grooves are totally different, also for the same line-type grooves, the same filling scheme cannot be used, in particular, for the non-closed grooves, with a defined open end, it is not possible to select the starting point of filling as optionally as for the closed grooves, and therefore it is necessary to develop a dedicated filling method for the non-closed grooves. Therefore, the invention provides a special filling method for the non-closed grooves on the surface of the object aiming at the characteristics of the non-closed grooves, thereby greatly improving the accuracy of filling the non-closed grooves.

All the solutions disclosed in the claims and in the description of the present application have one or more of the above-mentioned innovations and, accordingly, are capable of solving one or more of the above-mentioned technical problems.

Specifically, the application provides a robot-based object surface shallow groove contour extraction method, a robot-based object surface shallow groove contour extraction device, an electronic device and a computer-readable storage medium.

The robot-based object surface shallow concave groove profile extraction method of the embodiment of the application comprises the following steps:

acquiring a two-dimensional image of the surface of an object;

carrying out image equalization operation on the image;

carrying out image segmentation operation on the equalized image;

and determining a shallow groove according to the segmented image and extracting a shallow groove profile.

In some embodiments, the method further comprises obtaining a three-dimensional image of the surface of the object.

In some embodiments, the two-dimensional image is obtained by mapping a three-dimensional image.

In some embodiments, the determining the shallow groove and extracting the shallow groove profile comprises removing a non-groove profile.

In some embodiments, the removing the non-groove contours includes removing the non-groove contours according to the number of pixel points.

In some embodiments, the shallow groove profile comprises an outermost groove profile and an inner groove profile.

In some embodiments, the inner layer groove profile is extracted from a two-dimensional image.

In certain embodiments, the outermost groove profile comprises an inner profile and an outer profile.

In some embodiments, the inner contour of the outermost groove is extracted from a two-dimensional image.

In some embodiments, the outer contour of the outermost groove is extracted from a three-dimensional image.

The robot-based object surface shallow groove profile extraction device of the embodiment of the present application includes:

the image acquisition module is used for acquiring a two-dimensional image of the surface of the object;

the image equalization module is used for carrying out image equalization operation on the image;

the image segmentation module is used for carrying out image segmentation operation on the equalized image;

and the contour extraction module is used for determining the shallow groove according to the segmented image and extracting the contour of the shallow groove.

The electronic device of the embodiments of the present application includes a memory, a processor, and a computer program stored on the memory and executable on the processor, and the processor implements the robot-based object surface shallow groove contour extraction method of any one of the above embodiments when executing the computer program.

The computer readable storage medium of the embodiments of the present application has stored thereon a computer program which, when executed by a processor, implements the robot-based object surface shallow groove contour extraction method of any of the above embodiments.

Additional aspects and advantages of the present application will be set forth in part in the description which follows and, in part, will be obvious from the description, or may be learned by practice of the present application.

Drawings

The above and/or additional aspects and advantages of the present application will become apparent and readily appreciated from the following description of the embodiments, taken in conjunction with the accompanying drawings of which:

FIG. 1 is a schematic flow chart of a groove filling method based on groove identification according to some embodiments of the present disclosure;

FIG. 2 is a schematic flow chart of a method for extracting a shallow groove profile from a surface of an object according to some embodiments of the present disclosure;

FIGS. 3 and 4 are schematic illustrations of groove profile identification and extraction according to certain embodiments of the present application;

FIG. 5 is a schematic flow chart of a groove filling method based on robot movement speed control according to some embodiments of the present disclosure;

FIG. 6 is a schematic flow chart of a non-closed groove filling method according to some embodiments of the present application;

FIG. 7 is a schematic illustration of a non-closed groove filling method according to certain embodiments of the present application;

FIG. 8 is a schematic diagram of a groove filling apparatus based on groove identification according to certain embodiments of the present disclosure;

FIG. 9 is a schematic structural diagram of an apparatus for extracting a shallow groove profile from a surface of an object according to some embodiments of the present disclosure;

FIG. 10 is a schematic diagram of a groove filling apparatus based on robotic travel speed control according to certain embodiments of the present application;

FIG. 11 is a schematic structural view of a non-closing groove-filling apparatus according to certain embodiments of the present application;

FIG. 12 is a schematic diagram of an electronic device according to some embodiments of the present application.

Detailed Description

Exemplary embodiments of the present disclosure will be described in more detail below with reference to the accompanying drawings. While exemplary embodiments of the present disclosure are shown in the drawings, it should be understood that the present disclosure may be embodied in various forms and should not be limited to the embodiments set forth herein. Rather, these embodiments are provided so that this disclosure will be thorough and complete, and will fully convey the scope of the disclosure to those skilled in the art.

Fig. 1 shows a groove filling method according to an embodiment of the invention, comprising:

step S100, extracting a surface groove profile of the object;

step S200, identifying the type of the groove profile;

step S300, determining a corresponding groove filling scheme according to the type of the groove profile;

and S400, filling the groove based on the groove filling scheme.

In step S100, 3D point cloud information of the object may be obtained, for example, the point cloud information may be obtained by a 3D industrial camera, the 3D industrial camera is generally equipped with two lenses, the two lenses capture the object group to be grabbed from different angles, and the three-dimensional image of the object can be displayed after the three-dimensional image is processed. And placing the object group to be grabbed below the vision sensor, simultaneously shooting by the two lenses, and calculating X, Y, Z coordinate values of all points of the object to be filled and the coordinate directions of all points by using a general binocular stereo vision algorithm according to the relative posture parameters of the two obtained images so as to convert the object group to be grabbed into point cloud data of the object group to be grabbed. In specific implementation, the point cloud may also be generated by using elements such as a visible light detector such as a laser detector and an LED, an infrared detector, and a radar detector.

The point cloud data acquired by the method is three-dimensional data, so that the data processing is facilitated, the efficiency is improved, and the acquired three-dimensional point cloud data can be orthographically mapped onto a two-dimensional plane.

As an example, a depth map corresponding to the forward projection may also be generated. A two-dimensional color map corresponding to a three-dimensional object region and a depth map corresponding to the two-dimensional color map can be acquired in a direction perpendicular to the depth direction of the object. The two-dimensional color image corresponds to an image of a plane area vertical to a preset depth direction; each pixel point in the depth map corresponding to the two-dimensional color image corresponds to each pixel point in the two-dimensional color image one by one, and the value of each pixel point is the depth value of the pixel point.

The invention can be used for various groove filling scenes in industry, such as a scene of groove gluing on the surface of a table board. In some scenarios, the grooves on the surface of the object are shallow, and in the 3D point cloud or the depth map, the point clouds of the inner and outer contours of the grooves cannot be distinguished, so that the contours of the grooves cannot be identified according to the 3D point cloud or the depth map. In order to extract the contour of such a shallow groove, the applicant has developed a shallow groove contour extraction method in which a two-dimensional image capable of clearly displaying a groove is obtained by processing the two-dimensional image in combination with the obtained two-dimensional image, thereby accurately identifying and extracting the contour of the groove, which is one of the important points of the present invention.

Fig. 2 shows a shallow grooved profile extraction method according to one embodiment of the invention, comprising:

step S110, acquiring a two-dimensional image of the surface of an object;

step S120, carrying out image equalization operation on the image;

step S130, performing image segmentation operation on the equalized image;

step S140, determining a shallow groove according to the segmented image and extracting a shallow groove profile.

In step S110, a two-dimensional image of the surface of the object is acquired. A three-dimensional image of the object, such as 3D point cloud information, may be obtained according to the method of S100 and then mapped into a two-dimensional image.

In step S120, an image equalization operation is performed on the acquired two-dimensional image. Due to the image equalization operation, the gray levels of the original few pixels are distributed to other gray levels, the pixels are relatively concentrated, the gray level range of the processed image is enlarged, the contrast is enlarged, the definition is enlarged, and therefore the image details which cannot be seen originally can be identified. Because the contrast ratio of the object surface and the shallow groove is close in the original two-dimensional image, the shallow groove can not be identified effectively, in order to solve the problem, the image equalization operation can be carried out under the condition of ensuring certain illumination, and after the processing, the shallow groove can be identified more clearly.

In step S130, an image segmentation operation is further performed on the two-dimensional image after the image equalization operation is performed. After the contrast between the object surface and the shallow groove approaches a certain degree, the groove may not be clearly recognized even through image equalization. At this time, image segmentation may be performed on the two-dimensional image subjected to the image equalization operation, and specifically, a figure having a property similar to a shallow groove in the image may be segmented and extracted. In one embodiment, the segmentation extracts form a two-dimensional image as shown in fig. 3, in which a plurality of shallow grooves are clearly visible.

In step S140, a shallow groove is determined from the segmented image and a shallow groove profile is extracted. Although all grooves are obtained, many of them are simply pits of the object surface, which may be produced, for example, by collision and do not need to be filled, as shown in fig. 3. In order to eliminate the grooves which do not need to be filled, a minimum pixel point threshold value can be preset, and the grooves with the pixel quantity smaller than the threshold value are all regarded as pits without being filled. Therefore, when the groove profile is extracted, whether the pixel point quantity of the groove part is larger than the minimum pixel point threshold value or not can be judged for each groove, if the pixel point quantity of the groove part is larger than the minimum pixel point threshold value, the groove is considered to be a groove needing to be filled, and if not, the groove is a concave point needing not to be filled. In addition, a maximum pixel point threshold value can be preset to screen out specific concave structures on the surfaces of the objects, for example, some object surfaces may be provided with sub-packaging grids for placing the objects. Thus, only when the number of pixels in the groove portion is smaller than the threshold value, the groove is identified as a groove to be filled. Further, in the two-dimensional image shown in fig. 3 formed after the image equalization, the image segmentation, and the like, the multi-layer groove can be recognized from the outside to the inside, and the groove positioned inside the outermost groove can be defined as an inner-layer groove. The inner layer grooves are clear in outline and easy to identify and extract. However, for the outermost layer of the groove, only the inner contour can be clearly identified, and the outer contour and the black background are integrated, so that the position of the outer contour cannot be accurately identified. In order to determine the outer contour, the position of the outer edge of the outermost groove, i.e., the position of the outer contour, may be determined from the previously acquired three-dimensional image, and then combined with the two-dimensional image, thereby obtaining the outer contour of the outermost groove.

In step S200, based on the extracted groove profile, the type of the groove profile is identified. In order to accurately fill the groove, the filling material can fill the groove and can not overflow from the groove, so that a plurality of filling methods are designed, and each filling method corresponds to a specific profile type, which is one of the key points of the invention. The grooves may be classified according to the type of groove that can be filled by the groove filling method used. In the embodiment shown in fig. 4, the groove is divided into a closed groove, an open groove and a single-point groove, and roughly, the profile of the single-point groove is in a single-point shape, and the closed groove and the open groove are in a linear shape. The closed groove is in a closed linear shape and has no clear starting point and ending point. The opening groove is in a non-closed linear shape and has a definite starting point and an end point.

In step S300, a corresponding groove filling scheme is determined according to the type of the groove profile. In a preferred embodiment of the invention, the groove types correspond one-to-one to the filling pattern. In other implementations, multiple filling manners may correspond to one type of groove, or multiple types of grooves correspond to one filling manner, which is not limited in the present invention. The key point of the present invention is to invoke different filling schemes for different grooves, so that any filling scheme can be used, which is not limited by the present invention. However, in a preferred embodiment, three different filling schemes, a closed groove filling method, an open groove filling method and a dot groove filling scheme, are designed for the filling of the closed groove, the open groove and the dot groove, respectively.

In step S400, the groove is filled based on the groove filling scheme. The groove filling method can use a robot to fill the groove, so that the moving track, the moving speed, the filling speed and the like of the robot are planned, the robot moves on the surface of an object according to the planned path and speed, and the filling material is filled into the groove according to the planned filling speed. Three different filling schemes, a closed groove filling method, an open groove filling method, and a dot groove filling scheme, are explained below.

Single point groove

The robot can be mapped into three-dimensional point cloud according to the obtained two-dimensional outline of the single-point groove, then pose information of the three-dimensional point cloud is obtained, and the robot is made to move to the position of the single-point groove to be filled according to the pose information.

Closed groove

When the robot is used for filling the groove, the discharge head is controlled to be filled based on a certain discharge rate. The discharge rate, as an inherent property of the robot, affects the effect of filling in this embodiment. Since the discharging rate of the robot is generally fixed and constant, if the width of the groove to be filled is not uniform in the whole path, and the robot moves at the same moving speed at each track point, the situation of adhesive shortage at the wide groove and adhesive stacking at the narrow groove may occur.

Fig. 5 shows a groove filling method of the present invention by controlling a moving speed of a robot, which can be used for filling of a closed groove and an open groove of the present invention, the method comprising:

step 411, extracting the surface groove profile of the object;

step 412, traversing points on the contour at intervals according to preset trace points, and determining trace points moved by the robot at each traversed point;

step 413, determining the moving speed Vk of the robot at the track point based on the groove width Wk at the track point;

and step 414, enabling the robot to perform groove filling according to the determined track points and the moving speed of the track points.

For step 411, a method similar to step S100 may be adopted to extract the surface groove profile of the object, and will not be described herein.

For step 412, the closed groove profile consists of an inner profile and an outer profile. In one embodiment, in order to obtain the motion trajectory of the robot, the trajectory point interval may be preset, and then, based on the inner contour or the outer contour, the points on the reference contour are traversed according to the trajectory point interval. The trace point interval may be a distance, that is, it may be set to generate one trace point at every certain distance, in other words, traverse one point on the contour at every certain distance, in this way, traverse the entire contour. For each point P1 traversed, the point P2 is found for the current point P1 to go in the local normal direction to the other contour, if traversal is performed on the inner contour, P1 is on the inner contour and P2 is on the outer contour. The distance from P1 to P2 is the groove width W here. A point P3 may be selected between P1 and P2 as the fill trace point. Preferably, the filling track point may be the midpoint of the groove width W, i.e. the point is the position of the width W at a distance ratio of 1:1 from the inner and outer contours. Depending on the actual filling requirements, it is also possible to select different distance ratios as the locus points, for example if a groove is deeper on one side and shallower on the other side, and filling at the midpoint of the groove of width W may accumulate outside the outer contour because the cross-sectional area from this point to the inner contour is greater than the cross-sectional area to the outer contour. In this case, in order to fill the groove accurately, the trace point may be set to a position closer to the inner contour, and for example, the distance between the trace point and the inner and outer contours may be set to 2: 3. In other embodiments, instead of a ratio, the position of the track point may be adjusted by a distance, for example, the track point is set to 1mm from the inner contour. All track points of the robot on the two-dimensional plane are obtained in the above mode, and after a complete moving track is formed, three-dimensional track points are obtained through the mapping relation from 2D to 3D of the two-dimensional track points, so that a three-dimensional moving track is formed.

For step 413, when the robot fills the groove, the robot controls the stub bar to fill based on a certain filling rate. The discharge rate, as an inherent property of the robot, affects the effect of filling in this embodiment. Since the discharge rate of the robot is generally constant, if the width of the groove to be filled is not uniform over the entire path, and the robot moves at the same movement speed at each track point, the situation of material shortage at the wide groove and material overflow at the narrow groove may occur. In the case of a fixed discharge rate, different robot travel speeds result in different discharge quantities, and in general, in the case of a fixed discharge rate, the faster the robot travel speed, the less the discharge, and vice versa, the more the discharge. Generally, a wider groove requires a greater amount of material to be filled, and thus requires a relatively slow speed of movement of the robot at the groove. In one embodiment, a suitable robot speed V0 for a particular width W0 may be predetermined. Thus, for each track point, the width Wk of the groove at the track point can be obtained first, and if Wk > V0, the robot moving speed Vk at the track point should be less than V0, and vice versa.

For step 414, the ratio Rk of the groove width Wk to the specific width W0 may be calculated as Wk/W0, and the moving speed Vk at the current track point may be a linear proportion or a non-linear proportion of V0 and Rk. In one embodiment, the robot movement speed at each trajectory point can be calculated for that trajectory point by the following formula:

Vk＝V0/Rk.

open groove

The closed groove is formed by two contours, and when track points are arranged, all points on the contour can be traversed from any point on the inner contour or the outer contour according to a preset track point interval. Unlike the closed groove, the open groove has a specific opening end point in shape, so that one opening end point must be used as a starting point and the other opening end point must be used as an end point, so that the contour can be traversed from the opening point to the end point according to a preset track point interval. Therefore, unlike the closed groove, the open groove needs to determine the correct starting point and ending point of the track to accurately form the moving track of the robot. The inventors have performed a lot of labor to design a groove filling scheme that can be used for any groove having a defined starting point and ending point, i.e. a non-closed groove, which is also one of the important points of the present invention.

Fig. 6 illustrates a non-closed groove filling method of the present invention, comprising:

step 421, extracting a non-closed groove profile on the surface of the object;

step 422, generating a circumscribed rectangle of the outline;

step 423, determining two opening end points of the outline based on the generated circumscribed rectangle;

step 424, perform trench filling with the two opening end points as a start point and an end point, respectively.

Fig. 7 shows a schematic diagram of analyzing the opening and contour of a non-closed groove by a circumscribed rectangle according to an embodiment of the present invention, and the non-closed groove filling method of the present invention is explained below with reference to fig. 7.

For step 421, the surface groove profile of the object can be extracted by a method similar to step S100, which is not described in detail here.

For step 422, the minimum circumscribed rectangle is obtained for the outline of the open groove, and any feasible manner can be adopted to obtain the minimum circumscribed rectangle, which is not limited in the invention;

for step 423, the opening endpoint is determined from the minimum bounding rectangle. For example, the opening point is determined to be located on one side of the circumscribed rectangle according to the contour of the groove, and then the point closest to the side on the contour is determined as the opening end point. In a preferred embodiment, two opening end points P1 and P2 may be determined.

For step 424, the entire groove profile is broken at P1 and P2, forming two separate profile segments S1 and S2, in the embodiment shown in fig. 7, S1 is the outer profile and S2 is the inner profile. Thus, the starting point P1 and the ending point P2 of the traversal are obtained, as well as the outer contour S1 where P1 and P2 are located and the inner contour S2 opposite to the outer contour. From P1 to P2, S1 is subjected to traversal sampling according to a preset trace point interval. The trace point interval may be a distance, that is, it may be set to generate one trace point at every certain distance, in other words, to traverse contour points at every certain distance. For each point P1 traversed, the point P3 is found for the current point P1 to correspond to another contour along the local normal direction. The distance from P1 to P3 is the groove width W here. A point P4 is selected between P1 and P3, and the point is used as a filling track point. Preferably, the filling track point may be the midpoint of the groove width W, i.e., the point is a distance of 1:1 from the inner and outer contours across the width W. According to the actual filling requirement, positions with different distance ratios can also be selected as track points, for example, if the cross-sectional area from the middle point of the width W to the inner contour is larger than that from the middle point of the width W to the outer contour, the track points can be set to be closer to the inner contour for accurate filling, and for example, the distance from the track points to the inner and outer contours can be set to be 2: 3. In other embodiments, instead of a ratio, the position of the track point may be adjusted by a distance, for example, the track point is set to 1mm from the inner contour. After the complete movement track of the robot on the two-dimensional plane is obtained in the mode, the two-dimensional track points are used for obtaining three-dimensional track points through a 2D-to-3D mapping relation.

When the robot is filling the recess, can control out the stub bar based on certain filling rate and fill. The discharge rate, as an inherent property of the robot, affects the effect of filling in this embodiment. Since the discharge rate of the robot is generally constant, if the width of the groove to be filled is not uniform over the entire path, and the robot moves at the same movement speed at each track point, the situation of material shortage at the wide groove and material overflow at the narrow groove may occur.

In the case of a fixed discharge rate, different robot travel speeds result in different discharge quantities, and in general, in the case of a fixed discharge rate, the faster the robot travel speed, the less the discharge, and vice versa, the more the discharge. Generally, a wider groove requires a greater amount of material to be filled, and thus requires a relatively slow speed of movement of the robot at the groove. In one embodiment, a suitable robot speed V0 for a particular width W0 may be predetermined. Thus, for each track point, the width Wk of the groove at the track point can be obtained first, and if Wk > V0, the robot moving speed Vk at the track point should be less than V0, and vice versa.

The ratio Rk of the groove width Wk to the specific width W0 may be calculated as Wk/W0, and the moving speed Vk at the current track point may be a linear proportion or a non-linear proportion of V0 and Rk. In one embodiment, the robot movement speed at each trajectory point can be calculated for that trajectory point by the following formula:

Vk＝V0/Rk.

according to the embodiment, firstly, the special filling mode for the groove can be implemented for different grooves, so that the filling precision is greatly improved; secondly, even if the groove to be filled is shallow, the groove can be accurately identified and extracted; thirdly, the moving speed of the robot is controlled according to the width of the groove, the grooves with different widths can be accurately filled, and the filling precision is improved; fourthly, for special open grooves, the invention also develops a special method, and the filling precision of the grooves can be improved. Therefore, the groove filling robot solves the problem of the square surface in the groove filling process by using the robot.

In addition, various modifications and alterations can be made by those skilled in the art with respect to the above-described embodiments:

the robots in various embodiments of the present invention may be industrial robot arms that may be generic or may be dedicated to groove filling. The invention can be used for filling grooves on the surface of any object, such as glass, table plates, steel plates and the like, can be filled by using any filler, such as glue, various chemical fillers and the like, does not limit the specific application field, and is particularly suitable for gluing in the grooves of the table plates as a preferred embodiment.

In order to make the robot walk less redundant tracks, the initial point of the track point can be set at a position on the track path which is closest to the initial pose of the robot, for example: the initiation point is set in the middle of the side near the robot. That is, after the initial pose of the robot is determined, the intermediate point on the trajectory path of the side closest to the initial pose of the robot may be used as the initial point of the trajectory point, and then other trajectory points may be set on the trajectory path according to the inherent attributes of the robot, so that the trajectory point information may be obtained. It should be noted that the track point information may include, but is not limited to, coordinates of the track points, initial track points of the track points, and trends of the track points (i.e., track point walking sequence). After obtaining track point information, can adopt communication mode to send track point information to the robot. When receiving the track point information, the robot can control the material spraying nozzle to fill the groove based on the track point information.

In some embodiments, generating trajectory point information on the trajectory path according to the inherent attributes of the robot and the initial pose of the robot includes:

determining corners and straight lines in the trajectory path;

setting track points at corresponding densities at the turning part and the straight line part according to the discharging speed and the moving speed of the robot;

and determining the walking sequence of the track points according to the initial pose of the robot to obtain track point information.

Specifically, the determination of the corners and the straight lines in the trajectory path may be determined based on the relationship between the coordinate values of the points on the trajectory path. The X and Y coordinates of adjacent points at a corner may be different, while the X or Y coordinates of adjacent points on a straight line may be the same. For example: assuming that the shape of the object to be filled is rectangular, in the trajectory path of the object to be filled, the X coordinates and the Y coordinates of adjacent points at the four corners are different, the Y coordinates of adjacent points on the upper straight line are the same and the X coordinates are different, the Y coordinates of adjacent points on the lower straight line are the same and the X coordinates are different and the value of the Y coordinates is small relative to the value of the upper straight line, the X coordinates of adjacent points on the left straight line are the same and the Y coordinates are different, the X coordinates of adjacent points on the right straight line are the same and the Y coordinates are different and the value of the X coordinates is small relative to the value of the left straight line.

When the robot is used for filling, the material discharging head is controlled to fill based on a certain discharging rate. The discharge rate is an inherent property of the robot, and affects the filling effect in this embodiment. In order to conveniently set track points on a track path by referring to the discharging speed of the robot so as to avoid the stacking condition, the discharging speed of the robot can be determined.

The inherent property of the robot motion is also represented by that if the robot sets the same motion speed parameters at the corners and the straight lines, the motion speeds at the corners and the straight lines are different, and the motion speed at the specific corners is slower than that at the straight lines. In practical situations, the discharge rate of the robot is unchanged due to another inherent property, so that the stacking situation can be caused at a corner for the discharge rate and the movement speed parameters of a proper straight line. In some embodiments, on the premise of ensuring that the robot moves along the determined trajectory path, the distance between the trajectory points arranged at the corners on the trajectory path may be larger than the distance between the trajectory points arranged at the straight line, so as to achieve the balance between the movement speed at the straight line and the movement speed at the corners, and further solve the problem of material piling possibly caused by the corners. Can set up a minimum interval in straight line department and be used for injecing the interval of straight line department track point, prevent straight line department because the robot because track point quantity is too much and the condition of card windrow appears. And different moving speed parameters with different numerical values can be set at the straight line and the corner to achieve the balance of the moving speed at the straight line and the moving speed at the corner, and the problem of material piling caused by inherent properties is solved.

And determining the walking sequence of the track points according to the initial pose of the robot to obtain the track point information. It can be understood that, in order to make the robot walk less redundant tracks, the initial point of the track point is set to a point close to the initial pose of the robot, for example: the corresponding track point can be the middle part of the edge of the object to be filled, which is close to the robot. That is, after the initial pose of the robot is determined, the track point corresponding to the middle point on the track path of the side closest to the initial pose of the robot (or the track point closest to the middle point) may be used as the initial track point of the track point, and then, other track points may be walked clockwise or counterclockwise.

In some embodiments, the track point information may specifically include a track point coordinate, an initial track point coordinate, a walking order of the track point, a movement speed parameter of the track point, and the like.

In some embodiments, the trace point information further includes: and normal information corresponding to the contour points.

Specifically, the normal information may be an angle value of a normal vector corresponding to each contour point cloud with respect to a fixed amount, or may be a deviation angle value of a point cloud in a subsequent walking order in each contour point cloud with respect to a previous point cloud.

Fig. 8 is a schematic structural diagram of a groove filling apparatus based on groove profile recognition according to still another embodiment of the present invention, the apparatus including:

a contour extraction module 500 for extracting a contour of a body surface groove, i.e. for implementing step S100;

a contour identification module 600 for identifying the type of the groove contour, i.e. for implementing step S200;

a filling scheme determining module 700, configured to determine a corresponding groove filling scheme according to the type of the groove profile, that is, to implement step S300;

a filling module 800, configured to fill the groove based on the groove filling scheme, that is, configured to implement step S400.

Optionally, the contour extraction module 500 is further configured to remove non-groove contours, for example, the non-groove contours may be removed according to the number of pixel points.

Optionally, the type of groove profile that can be identified by the profile identification module 600 includes at least one of a closed groove, an open groove, and a single point groove.

Optionally, when the filling scheme determining module 700 determines to use a single-point groove filling scheme, the filling module 800 acquires a 3D point cloud of the single point, and fills the groove according to the pose of the 3D point cloud.

Optionally, when the filling scheme determining module 700 determines to use a closed groove and/or open groove filling scheme, the filling module 800 traverses points on the inner contour or the outer contour according to a preset trace point interval, finds, at each traversed point, a point from the point to the corresponding outer contour or the inner contour along the local normal direction, and selects a point between the point and the corresponding outer contour or the corresponding point on the inner contour as a trace point. The trace points may be the midpoints between the traversed point and the corresponding point on the outer contour or the inner contour.

Optionally, the filling module 800 determines the moving speed Vk of the filling tool at the locus point according to the groove width Wk at the locus point. A moving speed V0 corresponding to the specific width W0 may be preset, and the moving speed Vk at the locus point is a linear proportion or a non-linear proportion of V0 to Rk, wherein Rk is Wk/W0.

Fig. 9 is a schematic structural diagram of an object surface shallow groove profile extraction apparatus according to still another embodiment of the present invention, the apparatus including:

an image obtaining module 510, configured to obtain a two-dimensional image of the surface of the object, that is, to implement step S110;

an image equalization module 520, configured to perform an image equalization operation on the image, that is, to implement step S120;

an image segmentation module 530, configured to perform an image segmentation operation on the equalized image, that is, to implement step S130;

and a contour extraction module 540, configured to determine a shallow groove according to the segmented image and extract a shallow groove contour, that is, to implement step S140.

Optionally, the shallow groove profile extraction device further includes a three-dimensional image acquisition module, configured to acquire a three-dimensional image of the surface of the object.

Alternatively, the image acquisition module 510 may obtain the two-dimensional image by mapping the three-dimensional image.

Optionally, the contour extraction module 540 is further configured to remove non-groove contours, for example, the non-groove contours may be removed according to the number of pixel points.

Optionally, the shallow groove profile comprises an outermost groove profile and an inner groove profile, and the outermost groove profile comprises an inner profile and an outer profile.

Optionally, the inner contour of the inner layer groove and the inner contour of the outermost layer groove may be extracted according to a two-dimensional image, and the inner contour of the outermost layer groove may be extracted according to a three-dimensional image.

Fig. 10 is a schematic structural diagram of a groove filling apparatus based on robot movement speed control according to still another embodiment of the present invention, the apparatus including:

a contour extraction module 811 for extracting the contour of the body surface groove, i.e. for implementing step S411;

a trace point determining module 812, configured to traverse points on the contour at intervals according to preset trace points, and determine trace points moved by the robot at each traversed point, that is, to implement step S412;

a moving speed determining module 813 for determining the robot moving speed Vk at the locus point based on the groove width Wk at the locus point, i.e. for implementing step S413;

and a filling module 814, configured to enable the robot to perform groove filling according to the determined track point and the moving speed at the track point, that is, to implement step S414.

Optionally, the contour extraction module 811 is further configured to remove the non-groove contour, for example, the non-groove contour may be removed according to the number of pixel points.

Alternatively, the moving speed determination module 814 determines the moving speed Vk of the filling tool at the locus point according to the groove width Wk at the locus point. A moving speed V0 corresponding to the specific width W0 may be preset, and the moving speed Vk at the locus point is a linear proportion or a non-linear proportion of V0 to Rk, wherein Rk is Wk/W0.

Optionally, the trace point determining module 812 traverses the inner contour or the points on the outer contour according to the preset trace point interval, and at each traversed point, finds a point from the point to the corresponding outer contour or the corresponding inner contour along the local normal direction, and selects a point between the point and the corresponding outer contour or the corresponding points on the inner contour as the trace point. The trace points may be the midpoints between the traversed point and the corresponding point on the outer contour or the inner contour.

Fig. 11 is a schematic structural view showing a non-closed groove filling apparatus for a surface of an object according to still another embodiment of the present invention, the apparatus comprising:

a contour extraction module 821 for extracting the contour of the surface groove of the object, that is, for implementing the step S421;

a circumscribed rectangle generating module 822 for generating a circumscribed rectangle of the outline, that is, for implementing step S422;

an opening determination module 823, configured to determine two opening end points of the outline based on the generated circumscribed rectangle, that is, to implement step S423;

a filling module 824, configured to perform groove filling with the two opening endpoints as a starting point and an end point respectively, that is, to implement step S424.

Optionally, the filling module 824 breaks the groove profile at the start and end points to form separate inner and outer profiles.

Optionally, the filling module 824 determines the moving speed Vk of the filling tool at the locus point according to the groove width Wk at the locus point. A moving speed V0 corresponding to the specific width W0 may be preset, and the moving speed Vk at the locus point is a linear proportion or a non-linear proportion of V0 to Rk, wherein Rk is Wk/W0.

Optionally, the filling module 824 traverses the points on the inner contour or the outer contour according to the preset trace point interval, obtains, at each traversed point, a point from the point to the corresponding outer contour or the corresponding inner contour along the local normal direction, and selects a point between the point and the corresponding point on the outer contour or the corresponding inner contour as the trace point. The trace points may be the midpoints between the traversed point and the corresponding point on the outer contour or the inner contour.

In the device embodiments shown in fig. 8 to fig. 11, only the main functions of the modules are described, all the functions of each module correspond to the corresponding steps in the method embodiment, and the working principle of each module may also refer to the description of the corresponding steps in the method embodiment, which is not described herein again. In addition, although the correspondence between the functions of the functional modules and the method is defined in the above embodiments, it can be understood by those skilled in the art that the functions of the functional modules are not limited to the correspondence, that is, a specific functional module can also implement other method steps or a part of the method steps. For example, the above embodiment describes the method of the contour recognition module 600 for implementing the step S200, however, the contour recognition module 600 may also be used for implementing the method or part of the method of the step S100, S300 or S400 according to the needs of the actual situation.

The present application also provides a computer-readable storage medium having stored thereon a computer program which, when executed by a processor, implements the method of any of the above embodiments. It should be noted that the computer program stored in the computer-readable storage medium of the embodiments of the present application may be executed by a processor of an electronic device, and the computer-readable storage medium may be a storage medium built in the electronic device or a storage medium that can be plugged into the electronic device in an attachable and detachable manner.

Fig. 12 is a schematic structural diagram of an electronic device according to an embodiment of the present invention, and the specific embodiment of the present invention does not limit the specific implementation of the electronic device.

As shown in fig. 12, the electronic device may include: a processor (processor)902, a communication Interface 904, a memory 906, and a communication bus 908.

Wherein:

the processor 902, communication interface 904, and memory 906 communicate with one another via a communication bus 908.

A communication interface 904 for communicating with network elements of other devices, such as clients or other servers.

The processor 902 is configured to execute the program 910, and may specifically perform the relevant steps in the above method embodiments.

In particular, the program 910 may include program code that includes computer operating instructions.

The processor 902 may be a central processing unit CPU, or an Application Specific Integrated Circuit (ASIC), or one or more Integrated circuits configured to implement an embodiment of the invention. The electronic device comprises one or more processors, which can be the same type of processor, such as one or more CPUs; or may be different types of processors such as one or more CPUs and one or more ASICs.

A memory 906 for storing a program 910. The memory 906 may comprise high-speed RAM memory, and may also include non-volatile memory (non-volatile memory), such as at least one disk memory.

The program 910 may be specifically configured to cause the processor 902 to perform the operations in the above-described method embodiments.

Broadly, the inventive content of the invention comprises:

a groove filling method based on groove contour identification comprises the following steps:

extracting a body surface groove profile;

identifying a type of groove profile;

determining a corresponding groove filling scheme according to the type of the groove profile;

filling the groove based on the groove filling scheme.

Optionally, extracting the body surface groove profile comprises removing a non-groove profile.

Optionally, the removing the non-groove contour includes removing the non-groove contour according to the number of the pixel points.

Optionally, the groove types comprise closed grooves and/or open grooves.

Optionally, the groove type comprises a single point groove.

Optionally, the groove profile comprises an inner profile and an outer profile.

Optionally, points on the inner contour or the outer contour are traversed according to preset trace point intervals.

Optionally, at each point traversed, a point along the local normal direction to the corresponding outer contour or inner contour is found.

Optionally, at each point traversed, a point between the point and a point on the corresponding outer contour or inner contour is selected as a trajectory point.

Optionally, the trace point is a midpoint between the traversed point and a corresponding point on the outer contour or the inner contour.

Alternatively, the moving speed Vk of the filling tool at the locus point is determined from the groove width Wk at the locus point.

Optionally, a moving speed V0 corresponding to the specific width W0 is preset, and the moving speed Vk at the track point is a linear proportion or a non-linear proportion of V0 to Rk, where Rk is Wk/W0.

Optionally, a 3D point cloud of the single point is obtained, and the groove is filled according to the pose of the 3D point cloud.

A method for extracting a shallow groove profile of an object surface comprises the following steps:

acquiring a two-dimensional image of the surface of an object;

carrying out image equalization operation on the image;

carrying out image segmentation operation on the equalized image;

Optionally, acquiring a three-dimensional image of the surface of the object is further included.

Optionally, the two-dimensional image is obtained by mapping a three-dimensional image.

Optionally, the determining the shallow groove and extracting the shallow groove profile includes removing a non-groove profile.

Optionally, the shallow groove profile comprises an outermost groove profile and an inner groove profile.

Optionally, the inner layer groove profile is extracted according to a two-dimensional image.

Optionally, the outermost groove profile comprises an inner profile and an outer profile.

Optionally, the inner contour of the outermost groove is extracted from a two-dimensional image.

Optionally, the outer contour of the outermost groove is extracted according to a three-dimensional image.

A groove filling method based on robot moving speed control comprises the following steps:

extracting a body surface groove profile;

traversing points on the contour at intervals according to preset trace points, and determining trace points moved by the robot at each traversed point;

determining the moving speed Vk of the robot at the track point based on the groove width Wk at the track point;

and enabling the robot to perform groove filling according to the determined track points and the moving speed of the track points.

Optionally, a moving speed V0 corresponding to the specific width W0 is preset, and a robot moving speed Vk at the trajectory point is calculated according to W0, V0 and Wk.

Optionally, the moving speed Vk at the trajectory point is a linear proportion or a non-linear proportion of V0 to Rk, where Rk is Wk/W0.

Optionally, the groove profile comprises an inner profile and an outer profile.

Optionally, at each traversed point, a point from the point to the corresponding outer contour or inner contour along the local normal direction is obtained, and a point between the point and the corresponding point on the outer contour or inner contour is selected as a track point.

Method for filling non-closed groove on surface of object

Extracting a non-closed groove profile of a body surface;

generating a circumscribed rectangle of the outline;

determining two opening end points of the outline based on the generated circumscribed rectangle;

groove filling is performed with the two opening end points as a start point and an end point, respectively.

Optionally, the groove profile is interrupted at the starting and ending points to form separate inner and outer profiles.

Optionally, points on the inner contour or the outer contour are traversed from the starting point to the end point according to a preset trace point interval.

In the description herein, reference to the description of the terms "one embodiment," "some embodiments," "an illustrative embodiment," "an example," "a specific example" or "some examples" or the like means that a particular feature, structure, material, or characteristic described in connection with the embodiment or example is included in at least one embodiment or example of the application. In this specification, schematic representations of the above terms do not necessarily refer to the same embodiment or example. Furthermore, the particular features, structures, materials, or characteristics described may be combined in any suitable manner in any one or more embodiments or examples.

Any process or method descriptions in flow charts or otherwise described herein may be understood as representing modules, segments, or portions of code which include one or more executable instructions for implementing specific logical functions or steps of the process, and the scope of the preferred embodiments of the present application includes other implementations in which functions may be executed out of order from that shown or discussed, including substantially concurrently or in reverse order, depending on the functionality involved, as would be understood by those reasonably skilled in the art of the present application.

The logic and/or steps represented in the flowcharts or otherwise described herein, such as an ordered listing of executable instructions that can be considered to implement logical functions, can be embodied in any computer-readable medium for use by or in connection with an instruction execution system, apparatus, or device, such as a computer-based system, processing module-containing system, or other system that can fetch the instructions from the instruction execution system, apparatus, or device and execute the instructions. For the purposes of this description, a "computer-readable medium" can be any means that can contain, store, communicate, propagate, or transport the program for use by or in connection with the instruction execution system, apparatus, or device. More specific examples (a non-exhaustive list) of the computer-readable medium would include the following: an electrical connection (electronic device) having one or more wires, a portable computer diskette (magnetic device), a Random Access Memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber device, and a portable compact disc read-only memory (CDROM). Additionally, the computer-readable medium could even be paper or another suitable medium upon which the program is printed, as the program can be electronically captured, via for instance optical scanning of the paper or other medium, then compiled, interpreted or otherwise processed in a suitable manner if necessary, and then stored in a computer memory.

The Processor may be a Central Processing Unit (CPU), other general purpose Processor, a Digital Signal Processor (DSP), an Application Specific Integrated Circuit (ASIC), a Field-Programmable gate array (FPGA) or other Programmable logic device, discrete gate or transistor logic device, discrete hardware component, etc. A general purpose processor may be a microprocessor or the processor may be any conventional processor or the like.

It should be understood that portions of the embodiments of the present application may be implemented in hardware, software, firmware, or a combination thereof. In the above embodiments, the various steps or methods may be implemented in software or firmware stored in memory and executed by a suitable instruction execution system. For example, if implemented in hardware, as in another embodiment, any one or combination of the following techniques, which are known in the art, may be used: a discrete logic circuit having a logic gate circuit for implementing a logic function on a data signal, an application specific integrated circuit having an appropriate combinational logic gate circuit, a Programmable Gate Array (PGA), a Field Programmable Gate Array (FPGA), or the like.

It will be understood by those skilled in the art that all or part of the steps carried by the method for implementing the above embodiments may be implemented by hardware related to instructions of a program, which may be stored in a computer readable storage medium, and when the program is executed, the program includes one or a combination of the steps of the method embodiments.

In addition, functional units in the embodiments of the present application may be integrated into one processing module, or each unit may exist alone physically, or two or more units are integrated into one module. The integrated module can be realized in a hardware mode, and can also be realized in a software functional module mode. The integrated module, if implemented in the form of a software functional module and sold or used as a stand-alone product, may also be stored in a computer readable storage medium.

The storage medium mentioned above may be a read-only memory, a magnetic or optical disk, etc.

Although embodiments of the present application have been shown and described above, it is understood that the above embodiments are exemplary and should not be construed as limiting the present application, and that variations, modifications, substitutions and alterations of the above embodiments may be made by those of ordinary skill in the art within the scope of the present application.

Claims

1. A robot-based method for extracting a shallow groove profile of an object surface is characterized by comprising the following steps:

acquiring a two-dimensional image of the surface of an object;

carrying out image equalization operation on the image;

carrying out image segmentation operation on the equalized image;

2. The contour extraction method according to claim 1, characterized in that: also includes acquiring a three-dimensional image of the surface of the object.

3. The contour extraction method according to claim 2, characterized in that: the two-dimensional image is obtained by mapping a three-dimensional image.

4. The groove-filling method according to claim 1, wherein: the determining a shallow groove and extracting a shallow groove profile includes removing a non-groove profile.

5. The groove-filling method according to claim 4, wherein: the removing the non-groove contour includes removing the non-groove contour according to the number of pixel points.

6. The contour extraction method according to any one of claims 1 to 5, characterized in that: the shallow groove profile includes an outermost groove profile and an inner groove profile.

7. The contour extraction method according to claim 6, characterized in that: and extracting the contour of the inner layer groove according to the two-dimensional image.

8. The contour extraction method according to claim 6, characterized in that: the outermost groove profile includes an inner profile and an outer profile.

9. The contour extraction method according to claim 8, characterized in that: the inner contour of the outermost groove is extracted from a two-dimensional image.

10. The contour extraction method according to claim 8, characterized in that: and extracting the outer contour of the outermost groove according to the three-dimensional image.

11. The utility model provides an object surface shallow groove profile extraction element based on robot which characterized in that includes:

12. An electronic device, comprising: a memory, a processor and a computer program stored on the memory and executable on the processor, the processor when executing the computer program implementing the robot-based object surface shallow groove profile extraction method of any one of claims 1 to 10.

13. A computer-readable storage medium, on which a computer program is stored, which, when being executed by a processor, implements the robot-based object surface shallow groove profile extraction method of any one of claims 1 to 10.