CN114620479B

CN114620479B - Mechanical arm control system and method for improving stacking efficiency of rectangular packaging boxes

Info

Publication number: CN114620479B
Application number: CN202210435375.5A
Authority: CN
Inventors: 龙腾发; 雷勤; 柳建雄; 吴凌云; 陈志满; 陈统书
Original assignee: Guangdong Tiantai Robot Co Ltd
Current assignee: Guangdong Tiantai Robot Co Ltd
Priority date: 2022-04-24
Filing date: 2022-04-24
Publication date: 2022-09-30
Anticipated expiration: 2042-04-24
Also published as: CN114620479A

Abstract

The invention relates to the technical field of mechanical arm control, and discloses a mechanical arm control system and a mechanical arm control method for improving stacking efficiency of rectangular packing boxes, wherein the mechanical arm control system comprises packing box state detection equipment and a mechanical arm electrically connected with the packing box state detection equipment; the mechanical arms comprise a plurality of grabbing mechanical arms and pushing mechanical arms, wherein the grabbing mechanical arms are sequentially arranged at the front end of the production line, and the pushing mechanical arms are positioned at the rear end of the production line; the packing box state detection equipment comprises a packing box acquisition module, a pose recognition module, an instruction analysis module and an instruction sending module; the pose identification module is used for analyzing and obtaining pose information of each rectangular packing box in the same row on the production line according to the image information, wherein the pose information comprises poses of the rectangular packing boxes and pose distance difference values, the mechanical arms are grabbed according to the pose combinations of the rectangular packing boxes, and the mechanical arms are pushed to package and stack the rectangular packing boxes, so that manual correction of the rectangular packing boxes is avoided, and stacking efficiency is greatly improved.

Description

Mechanical arm control system and method for improving stacking efficiency of rectangular packaging boxes

Technical Field

The invention relates to the technical field of mechanical arm control, in particular to a mechanical arm control system and method for improving stacking efficiency of rectangular packing boxes.

Background

With the continuous development of mechanical arm technology and industrial mechanical arm, the application of mechanical arm to industrial production is basically realized. The mechanical arm can efficiently protect and transport goods, and is one of mechanical parts in stacking. And a lot of commodities will be placed on the assembly line after being packed into the packing carton by the workman, still is provided with the arm on the assembly line, and the arm can snatch the packing carton, places in the container. And finally, carrying through a mechanical arm to realize quick stacking.

In the production and packaging process of candies or tea, most of the products are packaged by rectangular packaging boxes and then are transversely placed on a production line row by row, but due to the height difference between the production line and a transportation piece and the shaking relation of the crawler belts on the production line during placement, the rectangular packaging boxes can deviate in position and posture after being placed on the production line, and therefore the packaging boxes with changed positions and postures cannot be grabbed by the mechanical arm. Workers are required to pick up the remaining packages at the end of the production line and place the packages into containers, and the stacking efficiency is greatly reduced.

Disclosure of Invention

In view of the above defects, the present invention provides a mechanical arm control system and method for improving stacking efficiency of rectangular packing boxes, so as to improve stacking efficiency of rectangular packing boxes.

In order to achieve the purpose, the invention adopts the following technical scheme: a mechanical arm control system for improving stacking efficiency of rectangular packing boxes comprises packing box state detection equipment and a mechanical arm electrically connected with the packing box state detection equipment;

the mechanical arms comprise a plurality of grabbing mechanical arms and pushing mechanical arms, wherein the grabbing mechanical arms are sequentially arranged at the front end of the production line, and the pushing mechanical arms are positioned at the rear end of the production line;

the packing box state detection equipment comprises a packing box acquisition module, a pose recognition module, an instruction analysis module and an instruction sending module;

the packing box acquisition module is used for acquiring image information of the same row of rectangular packing boxes on the production line;

the pose identification module is used for analyzing and obtaining pose information of each rectangular packing box in the same row on the production line according to the image information, wherein the pose information comprises poses of the rectangular packing boxes and pose distance difference values;

the instruction analysis module is used for analyzing and obtaining a combined control instruction of the grabbing mechanical arm and the pushing mechanical arm according to the pose and the pose distance difference value of the rectangular packaging box;

the command sending module is used for sending the combined control command to the mechanical arm;

the mechanical arm receives and analyzes the combined control instruction to obtain a first grabbing instruction and a pushing instruction, the grabbing mechanical arm is controlled according to the first grabbing instruction to grab all rectangular packaging boxes meeting the pose condition, the pushing mechanical arm is controlled according to the pushing instruction, and the rest rectangular packaging boxes are pushed out of the assembly line.

Preferably, the packaging box acquisition module comprises a first picture acquisition sub-module and a second picture acquisition sub-module;

the first picture acquisition submodule is used for shooting image information of the rectangular packaging box after entering the assembly line and sending the image information to the pose recognition module;

the second picture acquisition submodule is used for shooting image information of the grabbing mechanical arm after the grabbing mechanical arm grabs the rectangular packaging box and sending the image information to the pose recognition module.

Preferably, the pose recognition module comprises a target frame body acquisition submodule, a matching submodule and a marking submodule;

the target frame body acquisition submodule is used for identifying and extracting the frame body of the image information by using an One-Stage algorithm and displaying the rectangular packaging box on the production line in a frame body mode;

the matching submodule is used for carrying out posture matching recognition on the rectangular packing boxes in the frame body to obtain the postures of all the rectangular packing boxes in the same row;

the marking submodule is used for sequentially judging whether the frame body of the rectangular packaging box falls into a grabbing pose gear from one side to the other side in the width direction of the production line, if the frame body of the rectangular packaging box falls into the grabbing pose gear, the frame body is marked with symbols, and the symbols are marked with one or more combinations of numbers, English letters and Greek letters.

Preferably, the matching sub-module comprises a template making sub-unit, an identification feature extraction sub-unit, a storage sub-unit, an attitude score calculation sub-unit and a judgment sub-unit;

the template making subunit is used for making 360 template drawings, wherein each template drawing corresponds to a different integer angle;

the identification feature extraction subunit comprises a gradient quantization module and a lifting module;

the gradient quantization module is used for carrying out first-layer pyramid direction gradient quantization and second-layer pyramid direction gradient quantization on the 360 template pictures, and the lifting module is used for respectively obtaining identification features corresponding to the 360 template pictures according to the quantized template pictures;

the identification feature extraction subunit is used for acquiring the identification features by taking the current angle as a list and storing the identification features;

the storage subunit is used for storing all the identification features in the different angle list;

the gesture score calculating subunit is used for calling the identification features of each angle in the storage subunit, calculating the similarity score of the frame body and each angle identification feature, judging whether the highest similarity score meets a score threshold value, if so, taking the angle corresponding to the highest similarity score as the gesture angle of the frame body, and if not, sending an error instruction to a manager;

the judgment sub-unit is used for acquiring initial poses of all rectangular packing boxes according to the image information of the first picture acquisition sub-module; acquiring the carried poses of all the rectangular packing boxes according to the image information of the second picture acquisition submodule;

and comparing the pose of the rectangular packing box after being conveyed with the initial pose of the rectangular packing box or the pose of the rectangular packing box after being conveyed with the pose of the rectangular packing box after the next time of conveying, judging whether the poses of the remaining rectangular packing boxes are changed or not, and if the poses are changed, performing symbol marking on the frame body of the rectangular packing box again.

Preferably, the instruction analysis module comprises a mechanical arm instruction analysis submodule and an information updating submodule;

the mechanical arm instruction analysis submodule is used for acquiring symbol marks of the frame body, judging whether a rectangular packaging box meeting the grabbing mechanical arm grabbing gesture exists in a current row of rectangular packaging boxes or not, if the rectangular packaging box meeting the grabbing mechanical arm grabbing gesture exists, generating a grabbing sequence of the grabbing mechanical arm to the rectangular packaging boxes according to the sequence of the symbol marks of the rectangular packaging boxes, and generating a first grabbing instruction;

if not, acquiring the frame body position of the rectangular packaging box in the same row, obtaining a transverse pose distance difference value according to the frame body position, and judging whether the transverse pose distance difference value exceeds a distance threshold value or not;

if the distance exceeds the distance threshold, acquiring the positions where the two rectangular packaging boxes with the farthest distance from the frame body position are grabbed by the grabbing mechanical arm, generating a second grabbing instruction, sending the second grabbing instruction to the grabbing mechanical arm to grab the two rectangular packaging boxes with the farthest distance from the frame body position, and acquiring and judging the pose distance difference again after grabbing until the pose distance difference does not exceed the distance threshold;

if the distance does not exceed the distance threshold, generating a pushing instruction for starting the pushing mechanical arm;

combining the first grabbing command, the second grabbing command and the pushing command to form the combined control command;

the information updating submodule is used for updating the pose and the pose distance difference value of the rectangular packaging box by using the image information of the second picture acquisition submodule after the grabbing mechanical arm grabs the rectangular packaging box, and sending the updated result to the mechanical arm instruction analysis submodule.

Preferably, the mechanical arm further comprises an instruction receiving module and an instruction analyzing module;

the instruction receiving module is used for receiving the combined control instruction and sending the combined control instruction to the instruction analyzing module;

the instruction analysis module is used for judging whether the first grabbing instruction exists in the combined control instruction or not, if so, analyzing the first grabbing instruction to obtain the grabbing sequence of the grabbing mechanical arm to the rectangular packaging box and the position of a grabbed object, and controlling the grabbing mechanical arm to grab the grabbed object;

if the second grabbing instruction exists, the second grabbing instruction is analyzed to obtain the positions of the two rectangular packing boxes with the largest difference value of the grabbing pose distances of the grabbing mechanical arms, and the grabbing mechanical arms are controlled to grab the rectangular packing boxes;

if the first grabbing instruction and the second grabbing instruction do not exist, the pushing instruction is triggered to call the pushing mechanical arm to push, and the rectangular packaging boxes in the same row are pushed out of the assembly line.

A mechanical arm control method for improving the stacking efficiency of rectangular packing boxes is applied to the mechanical arm control system for improving the stacking efficiency of the rectangular packing boxes, and the system comprises packing box state detection equipment and a mechanical arm electrically connected with the packing box state detection equipment;

the control method comprises the following steps:

step S1: acquiring image information of the same row of rectangular packaging boxes on the production line through the packaging box state detection equipment, wherein the image information comprises image information obtained after the rectangular packaging boxes are shot and enter the production line and image information obtained after the rectangular packaging boxes are shot by a shooting and grabbing mechanical arm;

step S2: analyzing and obtaining pose information of each rectangular packaging box in the same row on the production line according to the image information, wherein the pose information comprises poses of the rectangular packaging boxes and pose distance difference values;

step S3: analyzing according to the pose and the pose distance difference value of the rectangular packaging box to obtain a combined control instruction of the grabbing mechanical arm and the pushing mechanical arm;

step S4: the mechanical arm receives and analyzes the combined control instruction to obtain a first grabbing instruction and a pushing instruction, the grabbing mechanical arm is controlled according to the first grabbing instruction to grab all rectangular packaging boxes meeting the pose condition, the pushing mechanical arm is controlled according to the pushing instruction, and the rest rectangular packaging boxes are pushed out of the assembly line.

Preferably, the step S1 further includes the steps of:

acquiring initial poses of all rectangular packaging boxes according to image information of the rectangular packaging boxes entering a production line for the first time; acquiring the carried poses of all rectangular packing boxes according to the image information of the rectangular packing boxes after being grabbed by the grabbing mechanical arm;

and comparing the carried pose with the initial pose or the carried pose with the pose after the next carrying, judging whether the poses of the remaining rectangular packing boxes are changed, if so, executing the step S3 again, and updating the combination control command.

Preferably, the specific steps of step S2 are as follows:

step S21: using One-Stage algorithm to identify a frame body and extract image information, and displaying a rectangular packaging box on the production line in a frame body mode;

step S22: performing posture matching recognition on the rectangular packing boxes in the frame body to obtain the postures of all the rectangular packing boxes in the same row;

step S23: whether a frame body of the rectangular packaging box falls into a grabbing position and posture gear is sequentially judged from one side to the other side in the width direction of the assembly line, if the frame body of the rectangular packaging box falls into the grabbing position and posture gear, the frame body is marked with symbols, and the symbols are marked with one or more combinations of numbers, English letters and Greek letters;

the following steps are also required before step S22 is executed:

step A1: making 360 template drawings, wherein each template drawing corresponds to a different integer angle;

step A2: carrying out first-layer pyramid direction gradient quantization and second-layer pyramid direction gradient quantization on 360 template pictures to obtain identification features corresponding to the 360 template pictures respectively, and obtaining and storing the identification features by taking the current angle as a list;

the specific steps of step S22 are as follows: calling the identification feature of each angle, calculating the similarity score of the frame body and each angle identification feature, judging whether the highest similarity score meets a score threshold value, if so, taking the angle corresponding to the highest similarity score as the posture angle of the frame body, and if not, sending an error instruction to a manager, wherein the similarity score calculation formula is as follows:

where L is a frame, T denotes a template drawing, c is a position of the recognition feature in the template drawing, P denotes a field centered on c, o is a position of the recognition feature in the frame, r is an offset position after the frame recognition feature is compared with the template drawing recognition feature, and Sori () denotes a gradient amplitude.

Preferably, the specific steps of step S3 are as follows:

step S31: obtaining symbol marks of a frame body, judging whether a rectangular packaging box meeting the grabbing gesture of the grabbing mechanical arm exists in a current row of rectangular packaging boxes or not, if the rectangular packaging box meeting the grabbing gesture of the grabbing mechanical arm exists, generating the grabbing sequence of the grabbing mechanical arm to the rectangular packaging boxes according to the sequence of the symbol marks of the rectangular packaging boxes, and generating a first grabbing instruction;

step S32: and when the poses of the remaining rectangular packaging boxes after being grabbed by the grabbing mechanical arm are changed, the step S31 is executed again.

The specific steps of step S4 are as follows:

step S41: judging whether the first grabbing instruction exists in the combined control instruction or not, if so, analyzing the first grabbing instruction to obtain the grabbing sequence of the grabbing mechanical arm to the rectangular packaging box and the position of a grabbed object, and controlling the grabbing mechanical arm to grab the grabbed object;

step S42: judging whether the second grabbing instruction exists in the combined control instruction, if so, analyzing the second grabbing instruction to obtain the positions of the two rectangular packing boxes with the largest difference of the grabbing pose distances of the grabbing mechanical arms, and controlling the grabbing mechanical arms to grab the packing boxes;

step S43: and judging whether the second grabbing instruction exists in the combined control instruction, if not, triggering a pushing instruction to call the pushing mechanical arm to push, and pushing the rectangular packaging boxes in the same row out of the assembly line.

One of the above technical solutions has the following advantages or beneficial effects: the pose of rectangle packing carton can be discerned to this application to snatch the arm according to the pose combination of rectangle packing carton and promote the arm and encapsulate the pile up neatly to rectangle packing carton, avoided the manual work to revise it, improved the efficiency of pile up neatly greatly.

Drawings

FIG. 1 is a schematic block diagram of one embodiment of the system of the present invention.

Fig. 2 is a schematic diagram illustrating a process in which a rectangular packing box is pushed by a pushing robot according to an embodiment of the present invention.

Figure 3 is a top perspective view of a rectangular package symbol marking process in accordance with one embodiment of the present invention.

Fig. 4 is a top view of a pose distance difference acquisition process in an embodiment of the invention.

Fig. 5 is a flowchart of an embodiment of a control method of the present invention.

Detailed Description

Reference will now be made in detail to embodiments of the present invention, examples of which are illustrated in the accompanying drawings, wherein like reference numerals refer to the same or similar elements or elements having the same or similar functions throughout. The embodiments described below with reference to the accompanying drawings are illustrative only for the purpose of explaining the present invention, and are not to be construed as limiting the present invention.

In the description of the present invention, it is to be understood that the terms "center", "longitudinal", "lateral", "length", "width", "thickness", "upper", "lower", "left", "right", "vertical", "horizontal", "top", "bottom", "inner", "outer", "axial", "radial", "circumferential", and the like, indicate orientations and positional relationships based on those shown in the drawings, and are used merely for convenience of description and for simplicity of description, and do not indicate or imply that the device or element being referred to must have a particular orientation, be constructed and operated in a particular orientation, and thus, should not be considered as limiting the present invention.

Furthermore, the terms "first", "second" and "first" are used for descriptive purposes only and are not to be construed as indicating or implying relative importance or implicitly indicating the number of technical features indicated. Thus, a feature defined as "first" or "second" may explicitly or implicitly include one or more of that feature. In the description of the present invention, "a plurality" means two or more unless otherwise specified.

In the description of the present invention, it should be noted that, unless otherwise explicitly specified or limited, the terms "mounted," "connected," and "connected" are to be construed broadly, e.g., as meaning either a fixed connection, a removable connection, or an integral connection; they may be connected directly or indirectly through intervening media, or they may be interconnected between two elements. The specific meanings of the above terms in the present invention can be understood in specific cases to those skilled in the art.

As shown in fig. 1 to 4, a mechanical arm control system for improving stacking efficiency of rectangular packing boxes comprises packing box state detection equipment and a mechanical arm electrically connected with the packing box state detection equipment;

Because when placing, the offset of position appearance can take place for rectangular packing carton after placing the assembly line because the difference in height of assembly line and transportation piece and the shake relation of track on the assembly line, has leaded to the arm can't snatch to the packing carton that the position appearance changed. Workers are required to pick up the remaining packages at the end of the production line and place the packages into containers, and the stacking efficiency is greatly reduced. In order to solve the above problem, a packing box state detection device is provided in the present application, and the packing box state detection device includes a packing box acquisition module, a pose recognition module, an instruction analysis module, and an instruction issue module. The packing carton obtains the spare part that the module was provided with a plurality of shootings, can shoot the assembly line. At the moment that the rectangular packaging boxes enter the assembly line in rows, the packaging box acquisition module shoots the row of rectangular packaging boxes entering the assembly line, and the shooting is carried out to obtain image information. And in this application position appearance identification module can carry out the discernment of gesture to the rectangle packing carton in the same row, and instruction analysis module judges through the gesture of rectangle packing carton whether have the rectangle packing carton that needs to snatch in the rectangle packing carton of current row, and owing to this application can discern the gesture of rectangle packing carton in the position appearance identification module, snatch the arm and can be according to the gesture of rectangle packing carton makes self-adaptation rotatory, makes grab the tongs on the arm and can grab rectangle packing carton. And the grabbing mechanical arm can be placed in a container after grabbing the rectangular packaging box for next packaging. When all rectangular packing boxes needing to be grabbed in the same row of rectangular packing boxes are completely grabbed. The remaining rectangular packing boxes are not greatly displaced in posture, so that the pushing robot arm can be used to push out the remaining rectangular packing boxes from one side of the production line. The remaining rectangular packing boxes are required to be arranged side by side when being pushed, so that the subsequent containers are more convenient to install and place. The mechanical arm can effectively grab the rectangular packaging box with the deviated pose, manual intervention is not needed, and stacking efficiency is greatly improved. In addition because use in this application to promote the arm and release the assembly line with the packing carton, if the form of packing carton is non-rectangle, probably the packing carton takes place to overturn when promoting, reduces the efficiency of pile up neatly.

It should be noted that the grabbing mechanical arm and the pushing mechanical arm mentioned in the present invention are both in the prior art, and the structure thereof is also in the prior art, and can be obtained by outsourcing, and the structure thereof is not in the protection scope of the present invention, so that the present invention is not explained much.

The first picture acquisition submodule acquires image information of the row of rectangular packaging boxes after entering the assembly line for the first time and then transmits the image information to the pose recognition module to perform pose recognition on the rectangular packaging boxes after entering the assembly line. And analyzing whether the rectangular packing box needs to be grabbed or not according to the pose of the rectangular packing box. And the second picture acquisition submodule is used for shooting image information of the rectangular packaging box grabbed by the grabbing mechanical arm. The rectangular packaging boxes can be grabbed one by one because the grabbing mechanical arm can only grab one rectangular packaging box, and the mechanical arm can only be arranged in a plurality of directions of a flow line because the mechanical arm has a certain volume. When the grabbing robot arm grabs the rectangular packing boxes next to the rectangular packing boxes to be grabbed, the position and the posture of the rectangular packing boxes next to the rectangular packing boxes to be grabbed can be possibly touched, so that the position and the posture of the rectangular packing boxes next to the rectangular packing boxes to be grabbed are changed, the second picture acquisition sub-module is required to be arranged, after the grabbing motion of the grabbing robot arm is executed each time, the image information obtained after the rectangular packing boxes are obtained is input into the position and posture identification module and the instruction analysis module again, and the combination control instruction is updated. The fault tolerance rate of the system is improved, manual correction of the fault tolerance rate is avoided, and stacking efficiency is greatly improved.

the target frame body acquisition submodule is used for identifying frame body extraction on image information by using an One-Stage algorithm and displaying a rectangular packing box on the assembly line in a frame body mode;

If the matching submodule is directly used for matching and identifying the image information, the image information can be interfered by caterpillar tracks on the production line, and bulges exist on some caterpillar tracks so as to facilitate the placement of objects. These bumps increase the number of identifications of the matching sub-modules, affecting processing speed. Therefore, the image information is extracted by adopting the existing One-Stage algorithm, wherein the One-Stage algorithm is trained through a virtual model of the assembly line to obtain a training result of the One-Stage algorithm, an object on the assembly line can be identified, and a frame of the object on the assembly line is obtained through the One-Stage algorithm. The size or shape of the frame may be limited herein. In one embodiment, the protrusions on the crawler are arranged in an extending mode along the width direction of the crawler, and only the frame body with the length close to the width of the crawler needs to be removed, so that the identification number of the matching sub-modules can be reduced. The matching sub-module is provided with a trained matching sub-module, the matching template can be used for realizing quick matching of the pose, and the recognition efficiency is improved. After the postures of the rectangular packaging boxes are obtained through matching, the marking sub-module can judge the position and the posture of the rectangular packaging boxes, wherein the position and the posture are obtained by pushing the side central points of the rectangular packaging boxes with different postures in a simulation environment, if the posture of the rectangular packaging boxes is changed to the vertical direction in the pushing process, the posture gears in the postures do not fall into the grabbing range, and otherwise, the posture gears in the postures do not fall into the grabbing range.

The following is an explanation of one embodiment:

when there is a rectangular packing box with an offset angle of 15 ° viewed from right to left with the vertical direction being 0 ° as shown in fig. 2, the rectangular packing box is gradually offset to the vertical direction due to the frictional force of the track of the line when the center of the side thereof is pushed. When pushed at the center of the side of a rectangular pack having an offset angle of 45 °, it may be offset in the direction of 0 °, 90 ° or remain unchanged. Only in 90% of cases is the angle at which the attitude change tends to the vertical direction not fall within the grip range. Tests in the virtual environment of the application show that only the postures with the offset angles of [ -19 degrees to 19 degrees ] do not fall into the grabbing posture gear.

After the rectangular packing boxes are identified to be in the grabbing position, the frames are marked, for example, rectangular packing boxes with offset angles of 45 degrees, 15 degrees, 27 degrees and 12 degrees exist in sequence from left to right, wherein the rectangular packing boxes with offset angles of 45 degrees and 27 degrees meet the position condition, the rectangular packing boxes with offset angles of 45 degrees and 27 degrees fall into the grabbing position, after the rectangular packing boxes are identified, the frame corresponding to the rectangular packing box with offset angle of 45 degrees is marked as A1, the frame corresponding to the rectangular packing box with offset angle of 27 degrees is marked as A2, and the frame corresponding to the rectangular packing box with offset angles of 15 degrees and 12 degrees is not marked with symbols. The frame body after the symbol mark is more easily identified by a subsequent instruction analysis module, the data processing speed is improved, and then the first grabbing instructions are sequentially generated according to the size of the symbol mark, for example, a1 is the first grabbing instruction, and a2 is the second grabbing instruction.

Preferably, the matching sub-module comprises a template making sub-unit, an identification feature extraction sub-unit, a storage sub-unit, a posture score calculation sub-unit and a judgment sub-unit;

360 template drawings are manufactured in the template manufacturing subunit, the 360 template drawings correspond to each angle of-180 degrees respectively, and the number of matched template drawings is increased, so that the template drawings can cover the placing angle of each rectangular packaging box, and the rectangular packaging boxes cannot be identified due to the existence of partial angles.

The gradient quantization module can perform gradient quantization on the template graph, so that the identification features in the template graph can be better acquired. In one embodiment, the first layer of pyramid direction gradient quantization and the second layer of pyramid direction gradient quantization are performed as follows:

calculating the gradient of the gradient image through sobel, and if the template picture is a three-channel image, extracting an image matrix with the maximum single-channel gradient amplitude by a non-maximum suppression algorithm of the sum of squares of the gradients in the X direction and the Y direction;

obtaining an angle image matrix from the gradient image matrices in the X and Y directions;

quantizing an angle image matrix from 0-360 degrees into an integer of 1-15, continuously quantizing 7 remainder in 8 directions, taking pixels larger than a threshold value in an amplitude image matrix, then taking a quantized image matrix corresponding to 3 x 3 in the pixel field to form a histogram, taking more than 5 same directions in the pixel field, assigning values to the directions, and performing shift coding on indexes of 00000001-10000000;

wherein the gradient amplitude maximum image matrix calculation formula is as follows:

x represents the position of the object to be measured,

for x-position gradient values, { R, G, B } for R channel, G channel, B channel, ori () for gradient direction.

After the gradient quantization is performed, the identification features in the template graph are obviously different from other pixel points in terms of pixel point values, and therefore, the process of the module for identifying the features is mentioned in the application as follows: traversing the image matrix with the maximum gradient amplitude value, finding out pixel points with the maximum gradient amplitude value in each field of the image matrix with the maximum gradient amplitude value, and if the pixel points with the maximum gradient amplitude value are found out in the field, setting the gradient amplitude values of the pixel points except the pixel points with the maximum gradient amplitude value in the field to be zero;

judging whether the gradient amplitude of the pixel point with the maximum gradient amplitude in all the fields is larger than a gradient amplitude threshold value or not, and if so, marking the pixel point as an identification feature;

acquiring the quantity of all identification features, judging whether the quantity of all identification features is larger than a quantity threshold value, if so, adding all identification features into a feature set and storing the feature set in a configuration file; if not, judging whether the identification feature has at least another identification feature in the range of the distance threshold, if so, rejecting the identification feature and the identification feature in the distance threshold, and if not, storing the identification feature in the storage subunit.

The identifying features in the storage subunit store groupings of identifying features in each group at an angle. In the process of recognition by the gesture score calculation subunit recognition module, the recognition features in the storage subunit are called, and recognition matching is carried out on the recognition features of each group and the frame body on the image information of the first image acquisition submodule. In the application, whether a rectangular packing box exists in the first picture is calculated in a similarity calculation mode,

the similarity score calculation formula in the application is as follows:

wherein L is a frame, T is a template drawing, c is a position of a recognition feature in the template drawing, P represents a region centered on c, _o the position of the identification feature in the frame body is shown, r is the offset position of the frame body identification feature after being compared with the template drawing identification feature, and Sori () represents the gradient amplitude;

and respectively carrying out similarity calculation on the identification features in the 360 template graphs to obtain 360 similarity scores, finding out 360 numerical values with the maximum similarity scores, judging whether the numerical values are greater than a score threshold value, if so, indicating that the content in the input frame is a rectangular packing box, and the angle corresponding to the numerical value with the maximum similarity score is the offset angle corresponding to the posture of the rectangular packing box. Otherwise, the content in the frame is a non-rectangular packaging box, and foreign matters enter the assembly line. It is necessary to notify the manager of the manual intervention.

When a rectangular packing box enters the assembly line for the first time, the judging subunit firstly acquires the pose condition of the image information of the first picture acquisition submodule in the pose score calculating subunit, and when the rectangular packing box is grabbed by the grabbing mechanical arm, the judging subunit inputs the image information of the second picture acquisition submodule into the pose score calculating subunit to perform the pose condition of the rectangular packing box, and then compares the pose condition with the pose condition to judge whether the pose of the rest rectangular packing boxes is changed. For example, after the image information input by the first picture obtaining submodule for calculating the pose status obtains the pose transition of 4 rectangular packing boxes, the offset angles of the rectangular packing boxes from left to right are 45 °, 15 °, 27 °, and 12 °, at this time, the marking submodule marks the frame bodies of the rectangular packing boxes with the offset angles of 45 ° and 27 ° as a1 and a2, and when the grabbing arm grabs the a1 rectangular packing box, the remaining angles are stored, that is, 15 °, 27 °, and 12 ° are recorded. The judging subunit inputs the image information of the second picture acquisition submodule into the pose score calculation subunit to acquire the pose state of the rectangular packing box, for example whether the left-to-right order or the offset angle is 15 deg., 27 deg. -12 deg. as analyzed from the image information of the second picture acquisition sub-module, if the pose is not changed, the rectangular packing box is not marked again, if the image information of the second picture acquisition submodule is analyzed to be obtained from left to right or the offset angle is 26 degrees, 27 degrees and 12 degrees, the frame bodies of the rest rectangular packing boxes need to be marked again due to the change of the offset angle, the frame strips corresponding to the rectangular packaging boxes marked with the symbols A3, A2, -12 degrees and-12 degrees can also be marked without repeated symbols.

the mechanical arm instruction analysis submodule is used for acquiring symbol marks of the frame body, judging whether a rectangular packaging box meeting the grabbing gesture of the grabbing mechanical arm exists in a current row of rectangular packaging boxes or not, and if the rectangular packaging box meeting the grabbing gesture of the grabbing mechanical arm exists, generating the grabbing sequence of the grabbing mechanical arm to the rectangular packaging boxes according to the sequence of the symbol marks of the rectangular packaging boxes to generate a first grabbing instruction;

if the distance difference value does not exceed the distance threshold value, acquiring the frame body position of the rectangular packaging box in the same row, obtaining a transverse pose distance difference value according to the frame body position, and judging whether the transverse pose distance difference value exceeds the distance threshold value or not;

And carrying out symbol marking on the frame body falling into the grabbing position and position gear rectangular packaging box in the marking submodule. The mechanical arm instruction analysis submodule firstly obtains the symbol marks of the frame body, judges whether the rectangular packaging boxes in the row have the rectangular packaging boxes needing to be grabbed by the mechanical arm, and correspondingly generates a first grabbing instruction according to the sequence of the symbol marks to grab the rectangular packaging boxes if the rectangular packaging boxes in the row have the symbol marks. The specific position of the frame body can be spatially positioned through the existing positioning sensor, wherein the positioning sensor is the existing technical equipment used for maturity, and the positioning sensor can be arranged right above the assembly line to acquire the position of the rectangular packaging box on the assembly line in real time. And after the rectangular packing boxes corresponding to all the frame bodies with the symbol marks are grabbed by the grabbing mechanical arm. The mechanical arm instruction analysis submodule can obtain the transverse pose distance difference value of the rest frame body, and then judges whether the pose distance difference value exceeds a distance threshold value or not, because the pushing mechanical arm pushes the central point of the side face of the edge rectangular packaging box from one side, when the rectangular packaging box with the transverse pose distance exceeding the distance threshold value exists, the rectangular packaging box exceeding the distance threshold value can be overturned in the transverse direction when being pushed, so that part of rectangular packaging boxes cannot be pushed out of the assembly line or the rectangular packaging boxes pushed out of the assembly line do not have the same direction, and the rectangular packaging boxes are inconvenient to be subsequently placed into a container for stacking. Therefore, a second grabbing instruction needs to be generated to grab the two rectangular packing boxes with the farthest frame position distance until the pose distance difference does not exceed the distance threshold. After the first grabbing instruction and the second grabbing instruction are generated, if rectangular packaging boxes exist on the water flow line, the mechanical arm instruction analysis submodule generates the pushing instruction and controls the pushing mechanical arm to push the remaining rectangular packaging boxes out of the flow line. The grabbing mechanical arm is not needed to be used for grabbing the rectangular packaging boxes one by one, and the sorting efficiency of the rectangular packaging boxes in the same posture is greatly improved. The information updating submodule sends the image information of the second picture acquisition submodule to the mechanical arm instruction analysis submodule after acquiring the image information of the second picture acquisition submodule, the mechanical arm instruction analysis submodule judges whether a first grabbing instruction or a second grabbing instruction needs to be generated by updating according to the logic of the image information of the second picture acquisition submodule again, and the first grabbing instruction or the second grabbing instruction after being generated by updating needs to be recombined to form a combined control instruction and is sent to the instruction receiving module again.

Taking the rectangular packing boxes with the offset angles of 45 degrees, 15 degrees, 27 degrees and 12 degrees as examples, as shown in fig. 3, the frames corresponding to the rectangular packing boxes with the offset angles of 45 ° and 27 ° are marked as a1 and a2, and the manipulator command analysis submodule generates the first grasping commands for a1 and a2 in numerical order, sequentially grasps a1 and a2, of course, after grabbing a1, the second picture is obtained, and the second picture is input to the information updating sub-module, carrying out pose recognition again on the rectangular packing boxes with the residual angles of 15 degrees, 27 degrees and 12 degrees, if pose changes exist, and marking the frame again, if the pose is not changed, continuing to execute the first grabbing instruction operation generated by the original symbol mark, continuing to grab the frame of the A2, and similarly, using the second picture to update the pose after grabbing. After the rectangular packing boxes corresponding to the A1 and the A2 frame bodies are grabbed. And acquiring a pose distance difference value, wherein H is the pose distance difference value as shown in FIG. 4, and if the H exceeds a distance threshold value, generating a second grabbing instruction to grab the rectangular packing boxes of 15 degrees and 12 degrees, wherein the manipulator instruction analysis submodule does not generate a pushing instruction because the rectangular packing boxes do not exist on the assembly line after grabbing. And if the H does not exceed the distance threshold value, not generating the second grabbing instruction, but generating a pushing instruction, and controlling the pushing mechanical arm to push the residual rectangular packaging boxes out of the production line.

It is worth mentioning that the conveying speed of the crawler on the production line applying the system of the invention is not suitable to be too high, otherwise, when the pushing mechanical arm pushes the rectangular packing box, the rectangular packing box which is away from the pushing mechanical arm is displaced, so that the position of the rectangular packing box exceeds the distance threshold.

the control method comprises the following steps:

Preferably, the step S1 further includes the steps of:

Preferably, the specific steps of step S2 are as follows:

step S21: using a 0ne-Stage algorithm to identify a frame body and extract image information, and displaying a rectangular packaging box on the production line in a frame body mode;

step S22: carrying out posture matching identification on the rectangular packing boxes in the frame body to obtain the postures of all the rectangular packing boxes in the same row;

the following steps are also required before step S22 is executed:

wherein L is a frame, T is a template drawing, c is a position of a recognition feature in the template drawing, P represents a region centered on c, _o the position of the identification feature in the frame, and r is the offset position of the frame identification feature compared with the template pattern identification feature.

Preferably, the specific steps of step S3 are as follows:

step S32: and when the posture of the residual rectangular packaging box after being grabbed by the grabbing mechanical arm is changed, the step S31 is executed again.

The specific steps of step S4 are as follows:

In the description herein, references to the description of the term "one embodiment," "some embodiments," "an illustrative embodiment," "an example," "a specific example," or "some examples" or the like mean that a particular feature, structure, material, or characteristic described in connection with the embodiment or example is included in at least one embodiment or example of the invention. In this specification, the schematic representations of the terms used above do not necessarily refer to the same embodiment or example. Furthermore, the particular features, structures, materials, or characteristics described may be combined in any suitable manner in any one or more embodiments or examples.

While embodiments of the invention have been shown and described, it will be understood by those of ordinary skill in the art that: various changes, modifications, substitutions and alterations can be made to the embodiments without departing from the principles and spirit of the invention, the scope of which is defined by the claims and their equivalents.

Claims

1. A mechanical arm control system for improving stacking efficiency of rectangular packing boxes is characterized by comprising packing box state detection equipment and a mechanical arm electrically connected with the packing box state detection equipment;

the instruction analysis module comprises a mechanical arm instruction analysis submodule

combining the first grabbing command, the second grabbing command and the pushing command to form a combined control command;

the instruction sending module is used for sending the combined control instruction to the mechanical arm;

the mechanical arm receives and analyzes the combined control instruction to obtain a first grabbing instruction, a second grabbing instruction and a pushing instruction, the grabbing mechanical arm is controlled according to the first grabbing instruction or the first grabbing instruction and the second grabbing instruction to grab all rectangular packaging boxes meeting the pose condition, the pushing mechanical arm is controlled according to the pushing instruction, and the rest rectangular packaging boxes are pushed out of the production line.

2. The mechanical arm control system for improving the stacking efficiency of rectangular packing boxes according to claim 1, wherein the packing box acquisition module comprises a first picture acquisition sub-module and a second picture acquisition sub-module;

3. The mechanical arm control system for improving the stacking efficiency of rectangular packing boxes according to claim 2, wherein the pose recognition module comprises a target frame body acquisition sub-module, a matching sub-module and a marking sub-module;

the target frame body acquisition submodule is used for identifying and extracting the frame body of the image information by using a 0ne-Stage algorithm and displaying the rectangular packaging box on the production line in a frame body mode;

4. The mechanical arm control system for improving the stacking efficiency of rectangular packing boxes according to claim 3, wherein the matching sub-module comprises a template making sub-unit, an identification feature extraction sub-unit, a storage sub-unit, a posture score calculation sub-unit and a judgment sub-unit;

5. The mechanical arm control system for improving the stacking efficiency of rectangular packing boxes according to claim 4, wherein the command analysis module further comprises an information updating submodule;

6. The mechanical arm control system for improving the stacking efficiency of the rectangular packing boxes according to claim 4, wherein the mechanical arm further comprises a command receiving module and a command analyzing module;

if the first grabbing instruction and the second grabbing instruction do not exist, a pushing instruction is triggered to call the pushing mechanical arm to push, and the rectangular packing boxes in the same row are pushed out of the assembly line.

7. A mechanical arm control method for improving stacking efficiency of rectangular packing boxes is applied to the mechanical arm control system for improving stacking efficiency of rectangular packing boxes according to any one of claims 1 to 6, and is characterized in that the system comprises packing box state detection equipment and a mechanical arm electrically connected with the packing box state detection equipment;

the control method comprises the following steps:

step S1: acquiring image information of the same row of rectangular packing boxes on the assembly line through the packing box state detection equipment, wherein the image information comprises image information after the rectangular packing boxes are shot and enter the assembly line and image information after the rectangular packing boxes are shot by a shooting and grabbing mechanical arm;

step S3 includes the steps of:

if the distance exceeds the distance threshold, acquiring the positions where the two rectangular packing boxes with the farthest distance from the frame body position are grabbed by the grabbing mechanical arm, generating a second grabbing instruction, sending the second grabbing instruction to the grabbing mechanical arm to grab the two rectangular packing boxes with the farthest distance from the frame body position, and acquiring and judging the pose distance difference again after grabbing until the pose distance difference does not exceed the distance threshold;

step S32: when the poses of the remaining rectangular packaging boxes after being grabbed by the grabbing mechanical arm are changed, the step S31 is executed again;

step S4 includes the steps of:

step S42: judging whether the second grabbing instruction exists in the combined control instruction or not, if so, analyzing the second grabbing instruction to obtain the positions of the two rectangular packing boxes with the largest difference value of the grabbing pose and position of the grabbing mechanical arm, and controlling the grabbing mechanical arm to grab the rectangular packing boxes;

8. The mechanical arm control method for improving the stacking efficiency of rectangular packing boxes according to claim 7, wherein the step S1 further comprises the following steps:

9. The mechanical arm control method for improving the stacking efficiency of rectangular packing boxes according to claim 7, wherein the specific steps of the step S2 are as follows:

step S21: using an One-Stage algorithm to identify a frame body of the image information and extract the frame body, and displaying a rectangular packing box on the assembly line in a frame body mode;

the following steps are also required before step S22 is executed: