CN115556094A

CN115556094A - Material taking method and device based on three-axis manipulator and computer readable storage medium

Info

Publication number: CN115556094A
Application number: CN202211145615.4A
Authority: CN
Inventors: 翟晓杭; 崔晓天
Original assignee: Suzhou Shixun Intelligent Technology Co ltd
Current assignee: Suzhou Shixun Intelligent Technology Co ltd
Priority date: 2022-09-20
Filing date: 2022-09-20
Publication date: 2023-01-03

Abstract

The application provides a material taking method and device based on a three-axis manipulator and a computer readable storage medium, which are used for automatically taking materials from a material placing area, and the method comprises the following steps: s1: detecting whether the materials in the material placing area meet preset material taking conditions or not; if yes, executing S2; if not, executing S4; s2: acquiring a movement strategy of the three-axis manipulator based on the height distribution information of the material in the material placement area; s3: based on the movement strategy of the three-axis manipulator, driving a target picking assembly of the three-axis manipulator to move above one or more materials and pick one or more materials by using a driving assembly of the three-axis manipulator, and executing S1; s4: and after the preset time length, executing the S1 again. The material taking sequence of each material is intelligently determined, the material taking efficiency is high, and the materials are stacked attractively.

Description

Material taking method and device based on three-axis manipulator and computer readable storage medium

Technical Field

The application relates to the technical field of mechanical automation and intelligent manufacturing, in particular to a material taking method and device based on a three-axis manipulator and a computer readable storage medium.

Background

Robotic arms (english) are automatic control devices that mimic human arm functions and perform a variety of tasks, allowing motion in a plane or three-dimensional space or movement using linear displacement. The mechanical arm mainly demands to complete the actions of a wrist and a hand, and is distinguished by shape and size, and the mechanical arm comprises a linear arm, a SCARA arm, a joint multi-axis mechanical arm and the like. A three-axis manipulator for carrying out get material task can adopt linear arm, particularly, can adopt three-axis connecting mechanism, and the appearance is similar with planer-type mechanism, cooperation sucking disc or clamping jaw etc. pick up the mechanism and constitute the linkage manipulator.

Patent CN213415470U discloses a visual guidance automatic feeding mechanism, which includes a feeding conveying assembly, a guiding detection assembly located above the feeding conveying assembly, a material taking assembly located at one side of the feeding conveying assembly and used for taking materials on the feeding conveying assembly, and a feeding conveying assembly used for placing products grabbed by the material taking assembly; the guide detection assembly comprises a guide straight line module, a guide transportation frame arranged on the guide straight line module, and at least one group of detection cameras arranged on the guide transportation frame, and the detection ends of the detection cameras face towards the feeding conveying assembly. This mechanism carries out visual inspection to the product in the transportation process, pick the product through getting the material subassembly after detecting the product position and place the material feeding transportation subassembly on and carry, however when the material is placed the region and is provided with a plurality of materials, can't once get when putting and finish, then can't confirm the order of getting of every material intelligently, if random pickup arbitrary material at every turn, then the complexity of calculation process and control process is high, the control degree of difficulty is big, lead to getting the material inefficiency, the material pile is also not pleasing to the eye.

Based on this, the application provides a material taking method and device based on a three-axis manipulator and a computer readable storage medium, so as to improve the prior art.

Disclosure of Invention

The application aims to provide the material taking method and device based on the three-axis manipulator and the computer readable storage medium, the material taking sequence of each material is intelligently determined, the material taking efficiency is high, and the materials are stacked attractively.

The purpose of the application is realized by adopting the following technical scheme:

in a first aspect, the application provides a material taking method based on a three-axis manipulator, which is used for automatically taking materials from a material placing area, and the method includes:

s1: detecting whether the materials in the material placing area meet preset material taking conditions or not; if yes, executing S2; if not, executing S4;

s2: acquiring a movement strategy of the three-axis manipulator based on the height distribution information of the material in the material placement area, wherein the movement strategy is used for indicating the X-axis displacement, the Y-axis displacement and the Z-axis displacement of a target picking assembly of the three-axis manipulator in a preset rectangular coordinate system;

s3: based on the movement strategy of the three-axis manipulator, driving a target picking assembly of the three-axis manipulator to move above one or more materials and pick one or more materials by using a driving assembly of the three-axis manipulator, and executing S1;

s4: and after the preset time length, executing the S1 again.

The technical scheme has the beneficial effects that: the material taking sequence of each material is intelligently determined, the material taking efficiency is high, and the materials are stacked attractively.

Firstly, detecting whether a material placing area meets preset material taking conditions (for example, materials in the material placing area are stacked more or have larger mass) and respectively executing different subsequent steps according to a judgment result; if the current time is met, the material is taken by using the three-axis manipulator, so that the planning step of the movement strategy is carried out; if the time length does not meet the preset time length, the detection step is executed again.

In the planning step of the movement strategy, the planning is based on height distribution information of the material placement area (the height distribution information is used for indicating the height of the material at each point on the bearing surface of the material placement area). This has the advantage that the material or materials to be picked up at each time is/are selected based on the height distribution, so that the remaining materials in the material placement area meet different requirements in practical application, for example, the remaining materials in the material placement area can be always in a condition of relatively uniform height distribution (uniform height distribution means that the difference between the maximum value and the minimum value of the material height on the carrying surface of the material placement area is not greater than a preset height threshold), or the remaining materials in the material placement area can be picked up in regions (that is, after one sub-region is picked up, another sub-region is picked up).

The material height is evenly distributed, the kinematic constraint condition of the three-axis manipulator is easily met, and the problem that the picking task cannot be completed due to motion interference is avoided. Interference means that the part is in contact with other parts (distance is less than the set clearance value, not necessarily zero). The motion interference refers to the condition that interference occurs in the motion process of the part, such as collision obstruction between the picking assembly and other materials (materials which are not picked up this time).

The materials are picked in different regions, the material taking and placing device is suitable for the condition that the materials need to be taken and placed to external equipment, the stroke of the three-axis manipulator can be greatly reduced, and the material taking efficiency of the materials is improved. For example, the material placement area can be divided into a plurality of sub-areas, namely a sub-area A, a sub-area B, a sub-area C \8230; moving the AGV to a position near the subregion A, and picking the material of the subregion A from the material placing area to the AGV by using a three-axis manipulator; after the materials in the sub-area A are picked up, moving the AGV to a position close to the sub-area B, and picking up the materials in the sub-area B from the material placing area to the AGV by using the three-axis manipulator; after the materials in the sub-area B are picked up, moving the AGV to a position close to the sub-area C, and picking up the materials in the sub-area C from the material placing area to the AGV by using a three-axis manipulator; by analogy, the AGV can be moved to the position near each sub-area in sequence so as to complete the automatic transportation process of the materials of the corresponding sub-areas. Compared with the method that the AGV is parked at the fixed position, the travel of the three-axis manipulator for placing the materials after the materials are picked up is reduced; compare and all move AGV to near position gradually according to the material position that picks up once according to every, AGV's the removal number of times is few, has reduced the linkage degree of difficulty between triaxial manipulator and the AGV.

After the movement strategy is obtained, the driving assembly of the three-axis manipulator can be utilized to drive the target picking assembly to move to the position above the material and pick the material, and one material or a plurality of materials can be picked at a time. Wherein the three-axis robot is provided with a plurality of picking assemblies, and the target picking assembly is one of the plurality of picking assemblies, i.e. the three-axis robot drives the picking assembly to which the assembly is connected when performing a picking task.

The material taking process is circulated based on the detection result whether the material taking conditions are met, and the material taking process has the advantages that aiming at the material taking requirements of different materials or the material taking requirements of different stages of the same material, the performance requirements and the cost requirements in practical application are combined, different material taking processes can be realized by setting different material taking conditions, different preset durations, different single picking quantities and the like, the material taking sequence of each material is intelligently determined, the intelligent degree is high, the material taking efficiency is high, the application range is wide, the material taking requirements of various materials are met, the fine control of the whole material taking process can be realized, and the visual effect formed by stacking the materials (compared with randomly picking the materials at each time) is attractive.

In some optional embodiments, the process of detecting whether the material in the material placement area meets the material taking condition includes:

acquiring image information of the material placement area by using image acquisition equipment;

inputting the image information of the material placement area into an image recognition model to obtain the quantity of the materials in the material placement area;

when the quantity of the materials in the material placing area is not less than a preset quantity threshold value, determining that the materials in the material placing area meet the material taking conditions;

and when the quantity of the materials in the material placing area is smaller than the preset quantity threshold value, determining that the materials in the material placing area do not meet the material taking condition.

The technical scheme has the beneficial effects that: the image recognition model can achieve or even surpass the image recognition capability of human beings in the aspect of image recognition, and the image recognition model is used for carrying out image recognition on the collected image information so as to obtain the quantity of materials, so that the labor cost can be greatly reduced, and the accuracy of the image recognition result is high, thereby being convenient for popularization and copying. In addition, a preset quantity threshold value is preset, the quantity of the materials is used as a judgment basis for judging whether the materials are taken or not, and excessive material accumulation in a material placing area can be avoided.

In some optional embodiments, the inputting the image information of the material placement area to an image recognition model to obtain the quantity of the material in the material placement area includes:

and inputting the image information of the material placement area into the image recognition model to obtain the quantity and height distribution information of the materials in the material placement area.

The technical scheme has the beneficial effects that: on one hand, the quantity and height distribution information of the materials are output by using the image recognition model, so that the calculation efficiency is high; on the other hand, multiplexing of image information is achieved, namely, the image information is collected once, the collected image information is used for obtaining the quantity and the height distribution information at the same time, and compared with the method that the image information is obtained respectively and is used for obtaining the quantity and the height distribution information independently, the image collecting times are reduced, and the overall material taking efficiency is improved.

In some optional embodiments, the training process of the image recognition model includes:

acquiring a training set, wherein the training set comprises a plurality of training data, and each training data comprises sample image information of a sample material placement area and labeling data of the quantity and height distribution information of the materials in the sample material placement area;

for each training data in the training set, performing the following:

inputting sample image information in the training data into a preset deep learning model to obtain prediction data of the quantity and height distribution information of the materials in the sample material placement area;

updating model parameters of the deep learning model based on the prediction data and the labeling data of the quantity and height distribution information of the materials in the sample material placement area;

detecting whether a preset training end condition is met; if yes, taking the trained deep learning model as the image recognition model; and if not, continuing to train the deep learning model by using the next training data.

The technical scheme has the beneficial effects that: through design, a proper amount of neuron calculation nodes and a multilayer operation hierarchical structure are established, a proper input layer and an output layer are selected, a preset deep learning model can be obtained, through learning and tuning of the deep learning model, a function relation from input to output is established, although the function relation between the input and the output cannot be found out in 100%, the function relation can be close to a real association relation as far as possible, the image recognition model obtained through training can obtain corresponding output data (namely quantity and height distribution information) based on any input data (namely image information), the application range is wide, and the calculation result is high in accuracy and reliability.

In some optional embodiments, the step of detecting whether the material in the material placement area meets the material taking condition includes:

acquiring the mass of the material in the material placing area by using a weighing sensor;

when the mass of the materials in the material placing area is not less than the preset mass threshold, determining that the materials in the material placing area meet the material taking conditions;

and when the mass of the material in the material placing area is smaller than the preset mass threshold, determining that the material in the material placing area does not meet the material taking condition.

The technical scheme has the beneficial effects that: on one hand, the mass of the material is acquired by using the weighing sensor, so that the quality acquisition efficiency is high, and the real-time performance is strong; on the other hand, preset quality threshold values are preset, the quality of the materials is used as the basis for judging whether the materials are taken, and the situation that the materials in the material placing area are too heavy to be stacked and damage the bearing surface of the material placing area to cause the materials to be poured into the surrounding area can be avoided.

In some optional embodiments, before S1, the method further comprises:

obtaining a pick strategy of the three-axis manipulator based on the type of the material in the material placement area, wherein the pick strategy is used for indicating one or more of the following pick parameters: the pick-up type, number, shape, size and material type of the pick-up mechanism;

determining one of the picking assemblies from a plurality of picking assemblies as the target picking assembly based on a picking strategy of the three-axis robot;

connecting a drive assembly of the three-axis robot to the target picking assembly;

determining a drive strategy for the three-axis manipulator based on the target picking assembly;

when the drive assembly of the three-axis manipulator employs a motor, the drive strategy is used to indicate one or more of the following drive parameters: speed, torque, output power, and power factor;

when the driving assembly of the three-axis manipulator adopts the air cylinder, the driving strategy is used for indicating one or more of the following driving parameters: output force, piston stroke, and piston movement velocity.

The technical scheme has the beneficial effects that: a plurality of picking assemblies are preset, and each picking assembly corresponds to different picking parameters, namely, the picking parameters correspond to different picking mechanisms, and the picking parameters are different, namely, the picking parameters correspond to different picking mechanisms, the picking parameters are different, and the picking parameters are different. Aiming at different types of materials, different picking strategies are adopted to determine a target picking assembly connected in the material taking process, a driving strategy corresponding to the target picking assembly is adopted to realize stable and efficient operation of the material taking process, fine control is facilitated to be realized, the material taking efficiency and the safety in the material taking process (including the safety of the materials and the safety of a picking mechanism) are both considered, the materials are prevented from being damaged, and meanwhile the service life of the picking mechanism is prolonged. For example, for materials which are easy to deform under stress, a lifting type picking mechanism can be selected, and a lifting mode is adopted to pick the materials so as to avoid deformation of the materials; for materials which are easy to slip, a picking mechanism with an anti-slip function can be selected for picking so as to avoid the materials from slipping; aiming at the materials arranged in the regular shape, a plurality of picking mechanisms can be selected, and a plurality of materials can be picked each time so as to improve the material taking efficiency; aiming at materials with different shapes and sizes, a picking mechanism with corresponding shapes and sizes can be selected to improve the stability in the picking process and the moving process; for materials of different material types, the picking mechanisms of different material types can be selected to avoid damaging the materials; to the material that takes place to empty easily, can choose for use and prevent empting the mechanism of picking up that has the function to pick up the subassembly with suitable drive parameter drive target, in order to realize lower moving speed, avoid the material to take place to empty.

In some optional embodiments, the moving the target picking assembly of the three-axis robot to the above of one or more of the materials and pick one or more of the materials by using the driving assembly of the three-axis robot based on the moving strategy of the three-axis robot comprises:

determining a target amount of material to pick at a time based on the target picking assembly;

based on the movement strategy of the three-axis manipulator, a driving assembly of the three-axis manipulator is utilized to drive a target picking assembly of the three-axis manipulator to move above the target amount of materials and pick the target amount of materials.

The technical scheme has the beneficial effects that: after the target picking assembly is determined, the target quantity of the materials picked each time is determined, so that the target picking assembly can be used for picking the target quantity of the materials after the three-axis manipulator is moved to the designated position corresponding to the moving strategy.

In some optional embodiments, the method further comprises:

and controlling the three-axis manipulator to place the picked material into a bearing surface of the AGV so that the AGV transports the placed material to a target position.

The technical scheme has the beneficial effects that: the AGV can be used for moving the picked materials to a preset target position, and the automatic transportation process of the materials is achieved. The AGV is an automatic Guided Vehicle, is commonly called an AGV trolley, and refers to a transport Vehicle which is provided with an electromagnetic or optical automatic navigation device, can run along a preset running path, has safety protection and various transfer functions, can flexibly change the running path according to different requirements, has the advantages of high automation degree, quick action, small floor area, high positioning precision and the like, greatly reduces the labor cost, is simple to install and debug, is suitable for the development trend of flexible manufacturing, and improves the safety and the normativity of industrial production.

In a second aspect, the present application provides a material taking device based on a three-axis manipulator, for automatically taking materials from a material placement area, the device comprising a processor configured to implement the following steps:

s2: acquiring a movement strategy of the three-axis manipulator based on the height distribution information of the material in the material placing area, wherein the movement strategy is used for indicating the X-axis displacement, the Y-axis displacement and the Z-axis displacement of a target picking assembly of the three-axis manipulator in a preset rectangular coordinate system;

s3: based on the movement strategy of the three-axis manipulator, driving a target picking assembly of the three-axis manipulator to move above one or more materials and pick the one or more materials by using a driving assembly of the three-axis manipulator, and executing S1;

s4: and after the preset time length, executing the S1 again.

In some optional embodiments, the processor is configured to detect whether the material in the material placement area satisfies the material taking condition by:

when the quantity of the materials in the material placing area is not less than a preset quantity threshold value, determining that the materials in the material placing area meet the material taking condition;

In some optional embodiments, the processor is configured to obtain the amount of material in the material placement area by:

for each training data in the training set, performing the following:

In some optional embodiments, the processor is configured to detect whether the material of the material placement area satisfies the material taking condition by:

and when the mass of the material in the material placing area is smaller than the preset mass threshold value, determining that the material in the material placing area does not meet the material taking condition.

In some optional embodiments, the processor is further configured to implement the following steps before implementing the step S1:

In some optional embodiments, the processor is configured to pick up one or more of the materials by:

and based on the movement strategy of the three-axis manipulator, driving a target picking assembly of the three-axis manipulator to move above the target amount of the materials by utilizing a driving assembly of the three-axis manipulator and picking the target amount of the materials.

In some optional embodiments, the processor is further configured to implement the steps of:

In a third aspect, the present application provides a computer readable storage medium storing a computer program which, when executed by a processor, implements the steps of any of the methods described above or implements the functions of any of the apparatus described above.

Drawings

The present application is further described below with reference to the accompanying drawings and embodiments.

Fig. 1 shows a schematic structural diagram of a three-axis manipulator provided in an embodiment of the present application.

Fig. 2 shows a schematic flow chart of a material taking method based on a three-axis manipulator according to an embodiment of the present application.

Fig. 3 shows a schematic flowchart for determining a driving strategy of a three-axis manipulator according to an embodiment of the present disclosure.

Fig. 4 shows a structural block diagram of a material taking device based on a three-axis manipulator according to an embodiment of the present application.

Fig. 5 shows a schematic structural diagram of a program product provided in an embodiment of the present application.

Detailed Description

The technical solutions in the present application will be described below with reference to the drawings and the detailed description of the present application, and it should be noted that, in the present application, new embodiments can be formed by any combination of the following described embodiments or technical features without conflict.

In the embodiments of the present application, "at least one" means one or more, "a plurality" means two or more. "and/or" describes the association relationship of the associated objects, meaning that there may be three relationships, e.g., a and/or B, which may mean: a exists alone, A and B exist simultaneously, and B exists alone, wherein A and B can be singular or plural. The character "/" generally indicates that the former and latter associated objects are in an "or" relationship. "at least one of the following" or similar expressions refer to any combination of these items, including any combination of the singular or plural items. For example, at least one (one) of a, b, or c, may represent: a, b, c, a and b, a and c, b and c, a and b and c, wherein a, b and c can be single or multiple. It is to be noted that "at least one item" may also be interpreted as "one or more item(s)".

It should also be noted that in the embodiments of the present application, words such as "exemplary" or "for example" are used to indicate examples, illustrations or descriptions. Any embodiment or design described herein as "exemplary" or "e.g.," is not necessarily to be construed as preferred or advantageous over other embodiments or designs. Rather, use of the word "exemplary" or "such as" is intended to present relevant concepts in a concrete fashion.

(method embodiment)

Referring to fig. 1 and fig. 2, fig. 1 shows a schematic structural diagram of a three-axis manipulator provided in an embodiment of the present application, and fig. 2 shows a schematic flow diagram of a material taking method based on a three-axis manipulator provided in an embodiment of the present application.

The three-axis manipulator in the embodiment of the application can move in the X-axis direction, the Y-axis direction and the Z-axis direction. Specifically, the three-axis manipulator is provided with a driving assembly and a picking assembly, and the driving assembly drives the picking assembly to move in the X-axis direction, the Y-axis direction and the Z-axis direction.

The embodiment of the application provides a material taking method based on a three-axis manipulator, which is used for automatically taking materials from a material placing area, and comprises the following steps:

s4: and after the preset time length, executing the S1 again.

Therefore, the material taking sequence of each material is intelligently determined, the driving assembly is controlled to drive the target picking assembly to move to the position above the corresponding material so as to pick up the material, the material taking efficiency is high, and the material stacking is attractive.

In the planning step of the movement strategy, the planning basis is height distribution information of the material placement area (the height distribution information is used for indicating the height of the material at each point on the bearing surface of the material placement area). This has the advantage that the material or materials to be picked up at each time is/are selected based on the height distribution, so that the remaining materials in the material placement area meet different requirements in practical application, for example, the remaining materials in the material placement area can be always in a condition of relatively uniform height distribution (uniform height distribution means that the difference between the maximum value and the minimum value of the material height on the carrying surface of the material placement area is not greater than a preset height threshold), or the remaining materials in the material placement area can be picked up in regions (that is, after one sub-region is picked up, another sub-region is picked up).

The material height is evenly distributed, the kinematic constraint condition of the three-axis manipulator is easily met, and the problem that the picking task cannot be completed due to motion interference is avoided. Interference means that the part is in contact with other parts (distance is less than a set gap value, not necessarily zero). The motion interference refers to the condition that the part interferes in the motion process, such as collision resistance between the picking assembly and other materials (materials which are not picked at this time).

The materials are picked up in different areas, the material taking and placing device is suitable for the condition that the materials need to be taken and placed to external equipment, the stroke of the three-axis manipulator can be greatly reduced, and the material taking efficiency of the materials is improved. For example, the material placement area can be divided into a plurality of sub-areas, namely a sub-area A, a sub-area B, a sub-area C \8230; moving the AGV to a position near the subregion A, and picking the material of the subregion A from the material placing area to the AGV by using a three-axis manipulator; after the materials in the sub-area A are picked up, moving the AGV to a position close to the sub-area B, and picking up the materials in the sub-area B from the material placing area to the AGV by using a three-axis manipulator; after the materials in the sub-area B are picked up, the AGV is moved to a position close to the sub-area C, and the three-axis manipulator is used for picking up the materials in the sub-area C from the material placing area to the AGV; by analogy, the AGV can be moved to the position near each sub-area in sequence so as to complete the automatic transportation process of the materials of the corresponding sub-areas. Compared with the method that the AGV is parked at the fixed position, the travel of the three-axis manipulator for placing the materials after the materials are picked up is reduced; compare and all move AGV to near position gradually according to the material position that picks up once according to every, AGV's the removal number of times is few, has reduced the linkage degree of difficulty between triaxial manipulator and the AGV.

After the movement strategy is obtained, the driving assembly of the three-axis manipulator can be utilized to drive the target picking assembly to move to the position above the material and pick the material, and one material or a plurality of materials can be picked at a time.

In the embodiment of the present application, the material placement area may be set indoors or outdoors, for example. As an example, the material placement area may be located inside a factory building.

The material placing area is provided with a bearing surface, and the bearing surface is used for bearing materials. The shape of the bearing surface can be circular, oval, polygonal, arcuate, annular, race track, etc. Wherein the polygon includes triangle, quadrangle, pentagon, hexagon, octagon, decagon, etc.

The number of the materials in the material placement area is not limited in the embodiment of the present application, and may be, for example, 0, 1, 3, 5, 10, 50, 100, 1000, 10000, 100000, and the like.

The shape of the material in the material placing area is not limited in the embodiment of the application, and the material placing area can be a cylinder, a cone, a rotating body, a cross-sectional body or an irregular shape. The cylinder comprises a cylinder and a prism, the cone comprises a cone and a pyramid, the rotating body comprises a cylinder, a circular truncated cone, a ball, an ellipsoid, a spherical crown, a bow ring, a circular ring, an embankment ring, a sector ring, a date pit shape and the like, and the section body comprises a truncated pyramid, a circular truncated cone, an oblique truncated cylinder, an oblique truncated prism, an oblique truncated cone, a spherical crown, a spherical segment and the like.

The size of the material in the material placing area is not limited in the embodiment of the application, and the size of the material can be millimeter order of magnitude, centimeter order of magnitude, decimeter order of magnitude, meter order of magnitude and the like.

The material type of the material in the material placement area is not limited in the embodiment of the present application, and may be, for example, a metal material, an organic polymer material, an inorganic non-metal material, a composite material, and the like. The metal material includes metal and alloy, the organic polymer material includes synthetic plastic, fiber, rubber, natural wool, cotton, etc., the inorganic non-metal material includes glass, ceramic, etc. and the composite material consists of two or more kinds of material, such as cement, wood, etc.

In the embodiments of the present application, the material may be packaged or unpackaged. As an example, the material in the material placing area is lipstick or sneakers with a packing box. As another example, the items in the item placement area are packages having packaging bags. As yet another example, the material of the material placement area is an unpackaged chip. As yet another example, the material of the material placement area is bagged rice.

In the embodiment of the application, the height distribution information of the material in the material placing area is used for indicating the height of the material at each point on the bearing surface of the material placing area. Each point on the bearing surface refers to each sampling point on the bearing surface, the sampling points on the bearing surface of the material placement area can be arranged in a shape of M rows and N columns, the row spacing and the column spacing between the sampling points can adopt preset distances, the preset distances can be 1 micrometer, 1 millimeter or 1 centimeter and the like, and M and N are integers greater than 1. Alternatively, the sampling points on the bearing surface of the material placement area may be arranged in the shape of a plurality of concentric circles, for example.

In the embodiment of the present application, the rectangular coordinate system, i.e. the cartesian coordinate system, is an orthogonal coordinate system in mathematics. The X-axis displacement, the Y-axis displacement and the Z-axis displacement in the preset rectangular coordinate system respectively refer to displacements in the X-axis direction, the Y-axis direction and the Z-axis direction in the preset rectangular coordinate system.

In the embodiment of the application, the three-axis manipulator is provided with a plurality of picking assemblies, and the target picking assembly is one of the plurality of picking assemblies, namely the three-axis manipulator drives the picking assembly connected with the assembly when executing a picking task (namely, a material taking task).

The preset time period is not limited in the embodiment of the present application, and may be, for example, 3 seconds, 5 seconds, 10 seconds, 30 seconds, 1 minute, 3 minutes, 5 minutes, 10 minutes, or the like. The preset time period is not set too long or too short. If the preset time is set to be too long, whether the material taking condition is met or not cannot be detected in real time, and material taking is carried out in time; if the preset duration is set to be too short, more calculation resources are consumed, and energy conservation and emission reduction are not facilitated.

Therefore, the image recognition model can achieve or even surpass the image recognition capability of human beings in the aspect of image recognition, the image recognition model is used for carrying out image recognition on the collected image information so as to obtain the quantity of materials, the labor cost can be greatly reduced, the accuracy of the image recognition result is high, and the popularization and the copying are facilitated. In addition, a preset quantity threshold value is preset, the quantity of the materials is used as a judgment basis for judging whether the materials are taken or not, and excessive material accumulation in a material placing area can be avoided.

The preset number threshold is not limited in the embodiment of the present application, and may be, for example, 1, 2, 3, 5, 10, 30, 50, 100, 1000, or the like.

The image capturing device according to the embodiment of the present application is not limited, and may include, for example, one or more cameras. The camera comprises an optical camera and/or an infrared camera.

Therefore, on one hand, the quantity and height distribution information of the materials are output simultaneously by using the image recognition model, and the calculation efficiency is high; on the other hand, multiplexing of image information is achieved, namely, the image information is collected once, the collected image information is used for obtaining the quantity and the height distribution information at the same time, and compared with the method that the image information is obtained respectively and is used for obtaining the quantity and the height distribution information independently, the image collecting times are reduced, and the overall material taking efficiency is improved.

for each training data in the training set, performing the following:

Therefore, through design, a proper amount of neuron computing nodes and a multilayer operation hierarchical structure are established, a proper input layer and a proper output layer are selected, a preset deep learning model can be obtained, through learning and tuning of the deep learning model, a functional relation from input to output is established, although the functional relation between the input and the output cannot be found 100%, the functional relation can be close to a real association relation as much as possible, the image recognition model obtained through training can obtain corresponding output data (namely quantity and height distribution information) based on any input data (namely image information), the application range is wide, and the accuracy and the reliability of computing results are high.

In some alternative embodiments, the image recognition model may be obtained by training in the embodiments of the present application, and in other alternative embodiments, the image recognition model may be obtained by training in advance.

In some alternative embodiments, for example, historical data may be data mined to obtain sample image information for sample material placement areas in a training set. That is, the sample image information of the sample material placement areas may be image information obtained by image-capturing the real material placement areas. In addition, the sample image information of the sample material placement area may be automatically generated by using a GAN model generation network.

The GAN model is a Generative adaptive Network (generic adaptive Network), and consists of a generating Network and a discriminating Network. The generation network takes random samples from the latent space (latency) as input, and its output needs to mimic the real samples in the training set as much as possible. The input of the discrimination network is the real sample or the output of the generation network, and the purpose is to distinguish the output of the generation network from the real sample as much as possible. The generation network should cheat the discrimination network as much as possible. The two networks resist each other and continuously adjust parameters, and the final purpose is to make the judgment network unable to judge whether the output result of the generated network is real. The GAN model can be used for generating sample image information of a plurality of sample material placing areas for the training process of the image recognition model, so that the data volume of the original data acquisition can be effectively reduced, and the data acquisition and labeling cost is greatly reduced.

The method for acquiring the annotation data is not limited in the present application, and for example, a manual annotation method, an automatic annotation method, or a semi-automatic annotation method may be adopted. When the sample image information of the sample material placement area is image information obtained by image acquisition of the real material placement area, real data can be obtained from historical data in a keyword extraction mode to serve as annotation data.

The training process of the image recognition model is not limited in the present application, and may be, for example, the above-described training mode of supervised learning, or may be a training mode of semi-supervised learning, or may be a training mode of unsupervised learning.

The preset training end condition is not limited in the present application, and may be, for example, that the number of training times reaches a preset number of times (the preset number of times is, for example, 1 time, 3 times, 10 times, 100 times, 1000 times, 10000 times, and the like), or may be that training data in a training set all complete one or more times of training, or may be that a total loss value obtained by this training is not greater than a preset loss value.

Therefore, on one hand, the weighing sensor is used for acquiring the mass of the material, the mass acquisition efficiency is high, and the real-time performance is strong; on the other hand, a preset quality threshold value is preset, the quality of the materials is used as a judgment basis for judging whether the materials are taken or not, and the situation that the materials in the material placing area are piled up too heavily and damage the bearing surface to cause the materials to be poured to the surrounding area can be avoided.

The preset mass threshold is not limited in the embodiments of the present application, and may be, for example, 1 mg, 2 mg, 3 mg, 5 mg, 10 mg, 30 mg, 50 mg, 100 mg, 1 kg, 2 kg, 3 kg, 5 kg, 10 kg, 30 kg, 50 kg, 100 kg, 1000 kg, etc.

calculating the quantity of the materials in the material placing area by using the mass of the materials in the material placing area and the mass of a single material;

That is, the mass of the material in the material placement area and the mass of the individual material can be used to calculate the corresponding quantity. The mass of a single material can be manually input, or can be obtained by measuring the mass of several randomly extracted materials and averaging the masses.

Referring to fig. 3, fig. 3 shows a schematic flowchart for determining a driving strategy of a three-axis manipulator according to an embodiment of the present disclosure.

In some optional embodiments, before S1, the method further comprises:

f1: obtaining a pick strategy of the three-axis manipulator based on the type of the material in the material placement area, wherein the pick strategy is used for indicating one or more of the following pick parameters: the pick-up type, number, shape, size and material type of the pick-up mechanism;

f2: determining one of the picking assemblies from a plurality of picking assemblies as the target picking assembly based on a picking strategy of the three-axis robot;

f3: connecting a drive assembly of the three-axis robot to the target picking assembly;

f4: determining a drive strategy for the three-axis manipulator based on the target picking assembly;

when the driving assembly of the three-axis manipulator adopts the air cylinder, the driving strategy is used for indicating one or more of the following driving parameters: output force, piston stroke, and piston movement speed.

Thereby, a plurality of picking assemblies are provided in advance, each picking assembly corresponding to different picking parameters, i.e. picking type, number, shape, size or material type corresponding to different picking mechanisms.

In the embodiment of the present application, the picking type of the picking mechanism may include, for example, one or more of suction, grasping, and hooking.

In the embodiment of the present application, the number of the picking mechanisms may be, for example, 1, 2, 3, 4, or the like. Each suction pick-up mechanism may be provided with one or more air holes, each grabbing pick-up mechanism may be provided with 2, 4, 8 or more jaws, and each hooking pick-up mechanism may be provided with one or more hooking ends.

The shape of picking up the mechanism is not restricted in this application embodiment, and the formula of picking up of suction type picks up the mechanism and can be flat sucking disc, oval sucking disc, ripple sucking disc, dysmorphism sucking disc etc. for example, snatchs the formula and picks up the mechanism and can be provided with two fingers clamping jaw, three fingers clamping jaw or dysmorphism clamping jaw for example, and the hook end that the formula of picking up of hook picked up the mechanism can be drawn close to or outwards extend.

The size of the picking mechanism is not limited in the embodiments of the present application, and the size may be, for example, millimeter order, centimeter order, decimeter order, meter order, and the like.

The material type of the pick-up mechanism is not limited in the embodiments of the present application, and may be, for example, a metal material, an organic polymer material, an inorganic non-metal material, a composite material, or the like.

Aiming at different types of materials, different picking strategies are adopted to determine a target picking assembly connected in the material taking process, a driving strategy corresponding to the target picking assembly is adopted to realize stable and efficient operation of the material taking process, fine control is facilitated, the material taking efficiency and the safety in the material taking process (including the safety of the materials and the safety of a picking mechanism) are both considered, the materials are prevented from being damaged, and the service life of the picking mechanism is prolonged.

For example, for materials which are easy to deform under stress, a lifting type picking mechanism can be selected, and a lifting mode is adopted to pick the materials so as to avoid deformation of the materials; aiming at materials which are easy to slip, a picking mechanism with an anti-slip function can be selected for picking so as to avoid the materials from slipping; aiming at the materials arranged in the regular shape, a plurality of picking mechanisms can be selected, and a plurality of materials can be picked each time so as to improve the material taking efficiency; aiming at materials with different shapes and sizes, a picking mechanism with corresponding shapes and sizes can be selected to improve the stability in the picking process and the moving process; for materials of different material types, the picking mechanisms of different material types can be selected to avoid damaging the materials; to the material that takes place to empty easily, can choose for use and prevent empting the mechanism of picking up that has the function to pick up the subassembly with suitable drive parameter drive target, in order to realize lower moving speed, avoid the material to take place to empty.

As an example, the kind of the material placing area is lipstick and has a package, and the package is a rectangular parallelepiped paper box. The pick-up strategy of the three-axis manipulator corresponding to the material type may be, for example, "the pick-up type of the pick-up mechanism is a suction type, the number is 4, the shape is a flat sucker, the size is 3 cm, and the material type is Nitrile Butadiene Rubber (NBR)". Determining one picking assembly with qualified picking parameters from a plurality of picking assemblies as a target picking assembly, connecting a driving assembly of the three-axis manipulator to the target picking assembly, and determining the driving strategy of the three-axis manipulator based on the target picking assembly, wherein the output force is 0.5 Newton, the piston stroke is 10 millimeters, and the piston motion speed is 50-500 millimeters per second.

In some optional embodiments, the moving the target picking assembly of the three-axis robot to move above one or more of the materials and pick one or more of the materials by using the driving assembly of the three-axis robot based on the moving strategy of the three-axis robot (i.e., step S3) includes:

Therefore, after the target picking assembly is determined, the target quantity of the materials picked each time is determined, and therefore after the three-axis manipulator is moved to the designated position corresponding to the moving strategy, the target picking assembly can be used for picking the target quantity of the materials.

As an example, when the target picking assembly is provided with 4 vacuum flat suction cups, the target number of items picked at a time is 4.

In some optional embodiments, the method further comprises:

Therefore, the AGV can be used for moving the picked materials to a preset target position, and the automatic transportation process of the materials is achieved. The AGV is an Automated Guided Vehicle, is generally also called an AGV trolley, is provided with an electromagnetic or optical automatic navigation device, can run along a preset running path, has safety protection and various transfer functions, can flexibly change the running path according to different requirements, has the advantages of high automation degree, quick action, small floor area, high positioning precision and the like, greatly reduces the labor cost, is simple to install and debug, is suitable for the development trend of flexible manufacturing, and improves the safety and the standardability of industrial production.

In the embodiment of the present application, the target position may be set manually or may be set intelligently. When an intelligent setting mode is adopted, the AGV-based intelligent storage management system takes the AGV as a bearing platform, an intelligent storage design and management optimization algorithm is used as a core, the target position of each AGV is intelligently set through cooperation of multiple AGVs and a scheduling technology, and functions of automatic AGV automatic transportation, automatic sorting and distribution and the like can be realized by combining storage management software and an automatic logistics equipment interface, so that high automation of the whole processes of warehousing, loading and unloading, carrying, stacking, storing, picking, packaging, ex-warehouse, delivery and the like is achieved, the logistics turnover efficiency is further improved, the timeliness and the accuracy of logistics supply are ensured, and a flexible storage function is realized.

In a specific application scenario, an embodiment of the present application further provides a material taking method based on a three-axis manipulator, which is used for automatically taking materials from a material placement area, and the method includes:

f4: determining a drive strategy for the three-axis manipulator based on the target picking assembly; when the drive assembly of the three-axis manipulator employs a motor, the drive strategy is used to indicate one or more of the following drive parameters: speed, torque, output power, and power factor; when the driving assembly of the three-axis manipulator adopts the air cylinder, the driving strategy is used for indicating one or more of the following driving parameters: output force, piston stroke and piston movement speed;

s4: after the preset time length, executing the S1 again;

after the S3, the method further comprises: and controlling the three-axis manipulator to place the picked material into a bearing surface of the AGV so that the AGV transports the placed material to a target position.

Wherein, whether the process of detecting the material of the material placing area meets the material taking condition comprises the following steps:

inputting the image information of the material placement area into the image recognition model to obtain the quantity and height distribution information of the materials in the material placement area;

Wherein the training process of the image recognition model comprises the following steps:

for each training data in the training set, performing the following:

Wherein the driving component of the three-axis manipulator is driven by the driving component of the three-axis manipulator to move above one or more materials and pick one or more materials based on the movement strategy of the three-axis manipulator, and the method comprises the following steps: determining a target amount of material to pick at a time based on the target picking assembly; based on the movement strategy of the three-axis manipulator, a driving assembly of the three-axis manipulator is utilized to drive a target picking assembly of the three-axis manipulator to move above the target amount of materials and pick the target amount of materials.

(apparatus embodiment)

The embodiment of the application further provides a material taking device based on the three-axis manipulator, and the specific embodiment of the material taking device is consistent with the embodiment and the achieved technical effect recorded in the embodiment of the method, and part of the content is not repeated.

The three-axis manipulator-based material taking device is used for automatically taking materials from a material placement area, and comprises a processor, wherein the processor is configured to realize the following steps:

s4: and after the preset time length, executing the S1 again.

In some alternative embodiments, the processor is configured to obtain the quantity of material of the material placement area by:

In some optional embodiments, the training process of the image recognition model comprises:

for each training data in the training set, performing the following:

when the mass of the materials in the material placement area is not less than the preset mass threshold, determining that the materials in the material placement area meet the material taking condition;

Referring to fig. 4, fig. 4 shows a block diagram of a material taking device based on a three-axis manipulator according to an embodiment of the present application.

A three axis robot based reclaimer assembly, for example, may include at least one memory 210, at least one processor 220, and a bus 230 connecting the different platform systems.

The memory 210 may include readable media in the form of volatile memory, such as Random Access Memory (RAM) 211 and/or cache memory 212, and may further include Read Only Memory (ROM) 213.

The memory 210 further stores a computer program, and the computer program can be executed by the processor 220, so that the processor 220 implements the functions of any one of the methods, and the specific embodiments thereof are consistent with the embodiments described in the embodiments of the methods and the achieved technical effects, and some contents are not described again.

Memory 210 may also include a utility 214 having at least one program module 215, such program modules 215 including, but not limited to: an operating system, one or more application programs, other program modules, and program data, each of which, or some combination thereof, may include an implementation of a network environment.

Accordingly, the processor 220 may execute the computer programs described above, and may execute the utility 214.

The processor 220 may employ one or more Application Specific Integrated Circuits (ASICs), DSPs, programmable Logic Devices (PLDs), complex Programmable Logic Devices (CPLDs), field-Programmable Gate arrays (FPGAs), or other electronic components.

Bus 230 may be one or more of any of several types of bus structures including a memory bus or memory controller, a peripheral bus, an accelerated graphics port, and a local bus using any of a variety of bus architectures.

The three-axis robot-based material extraction apparatus may also communicate with one or more external devices 240, such as a keyboard, pointing device, bluetooth device, etc., one or more devices capable of interacting with the three-axis robot-based material extraction apparatus, and/or any device (e.g., router, modem, etc.) that enables the three-axis robot-based material extraction apparatus to communicate with one or more other computing devices. Such communication may be through input-output interface 250. Also, the three-axis robot-based reclaimer assembly may also communicate with one or more networks (e.g., a Local Area Network (LAN), a Wide Area Network (WAN), and/or a public network, such as the internet) via network adapter 260. The network adapter 260 can communicate with other modules of the three axis robot based reclaimer assembly via the bus 230. It should be understood that although not shown, other hardware and/or software modules may be used in conjunction with a three-axis robot-based material extraction apparatus, including but not limited to: microcode, device drivers, redundant processors, external disk drive arrays, RAID systems, tape drives, and data backup storage platforms, to name a few.

(media embodiment)

An embodiment of the present application further provides a computer-readable storage medium, where a computer program is stored, and when the computer program is executed by a processor, the computer program implements the steps of any one of the methods or implements the functions of any one of the apparatuses, and a specific embodiment of the computer program is consistent with the embodiments and achieved technical effects described in the foregoing method embodiments, and some details are not repeated.

The computer readable storage medium may be, for example, a program product.

Referring to fig. 5, fig. 5 shows a schematic structural diagram of a program product provided in an embodiment of the present application.

The program product is for implementing the steps of any of the methods described above. The program product may employ a portable compact disc read only memory (CD-ROM) and include program code, and may be run on a terminal device, such as a personal computer. However, the program product of the present invention is not limited in this respect, and in the embodiments of the present application, the readable storage medium may be any tangible medium that can contain, or store a program for use by or in connection with an instruction execution system, apparatus, or device. The program product may employ any combination of one or more readable media. The readable medium may be a readable signal medium or a readable storage medium. The readable storage medium may be, for example, but not limited to, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any combination of the foregoing. More specific examples (a non-exhaustive list) of the readable storage medium include: an electrical connection having one or more wires, a portable disk, a hard disk, a Random Access Memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disc read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing.

A computer readable storage medium may include a propagated data signal with readable program code embodied therein, for example, in baseband or as part of a carrier wave. Such a propagated data signal may take many forms, including, but not limited to, electro-magnetic, optical, or any suitable combination thereof. A readable storage medium may also be any readable medium that can communicate, propagate, or transport the program for use by or in connection with the instruction execution system, apparatus, or device. Program code embodied on a readable storage medium may be transmitted using any appropriate medium, including but not limited to wireless, wireline, optical fiber cable, RF, etc., or any suitable combination of the foregoing. Program code for carrying out operations for aspects of the present invention may be written in any combination of one or more programming languages, including an object oriented programming language such as Java, C + + or the like and conventional procedural programming languages, such as the C language or similar programming languages. The program code may execute entirely on the user's computing device, partly on the user's device, as a stand-alone software package, partly on the user's computing device and partly on a remote computing device, or entirely on the remote computing device or server. In the case of a remote computing device, the remote computing device may be connected to the user computing device through any kind of network, including a Local Area Network (LAN) or a Wide Area Network (WAN), or may be connected to an external computing device (e.g., through the internet using an internet service provider).

While the present application is described in terms of various aspects, including exemplary embodiments, the principles of the invention should not be limited to the disclosed embodiments, but are also intended to cover various modifications, equivalents, and alternatives falling within the spirit and scope of the invention as defined by the appended claims.

Claims

1. A material taking method based on a three-axis manipulator is characterized by being used for automatically taking materials from a material placing area, and the method comprises the following steps:

s4: and after the preset time length, executing the S1 again.

2. The three-axis manipulator-based material taking method according to claim 1, wherein the step of detecting whether the material in the material placement area meets the material taking condition comprises the steps of:

3. The three-axis manipulator-based material taking method according to claim 2, wherein the inputting of the image information of the material placement area to an image recognition model to obtain the quantity of the materials in the material placement area comprises:

4. The three-axis manipulator-based material taking method according to claim 3, wherein the training process of the image recognition model comprises the following steps:

for each training data in the training set, performing the following:

detecting whether a preset training end condition is met; if yes, taking the trained deep learning model as the image recognition model; if not, continuously training the deep learning model by using the next training data.

5. The three-axis manipulator-based material taking method according to claim 1, wherein the process of detecting whether the material in the material placement area meets the material taking condition comprises:

6. The three-axis robot-based reclaiming method according to claim 1, wherein prior to the S1, the method further comprises:

7. The three-axis robot-based reclaiming method according to claim 6, wherein the utilizing the driving assembly of the three-axis robot to drive the target picking assembly of the three-axis robot to move above and pick one or more of the materials based on the movement strategy of the three-axis robot comprises:

determining a target amount of material to pick up at a time based on the target picking assembly;

8. The three-axis robot-based reclaiming method according to claim 1, further comprising:

9. A three-axis manipulator-based material extraction apparatus for automatically extracting material from a material placement area, the apparatus comprising a processor configured to implement the steps of:

s4: and after the preset time length, executing the S1 again.

10. A computer-readable storage medium, characterized in that the computer-readable storage medium stores a computer program which, when being executed by a processor, carries out the steps of the method according to any one of claims 1 to 8.