CN114758236A - Non-specific shape object identification, positioning and manipulator grabbing system and method - Google Patents

Abstract

The invention provides a non-specific shape object recognition, positioning and manipulator grabbing system and method, belonging to the field of manipulator grabbing. The system includes a multi-degree-of-freedom manipulator, a machine vision imaging module and a central processing module. The modules are connected, and the machine vision imaging module is connected with the central processing module to transmit the collected image data to the central processing module. The central processing module processes the image data, extracts the target contour, and combines the calibrated object space according to the target contour. The mapping relationship between the size and the number of pixels in the image space realizes the measurement of the external size of each target, and is also used to classify the target according to the target size by using the deep neural network model to control the multi-degree-of-freedom manipulator to identify, locate and pick up the target. . The invention also provides a method for identifying, locating and picking up the target. The method and system of the invention have strong adaptability and can flexibly pick up the target.

Description

A non-specific shape object recognition, positioning and robotic grasping system and method

technical field

The invention belongs to the field of manipulator grabbing, and more particularly, relates to a system and method for identifying, positioning and manipulating an object of non-specific shape.

Background technique

In recent years, Internet technology has developed rapidly, and automation has become the mainstream of industrial production. Traditional manual production has gradually disappeared from the field of vision, and more advanced production equipment such as emerging industrial robots has entered people's field of vision. For the part production industry, many problems follow. How should a fully automated production line identify different types of parts? How does the manipulator grab it?

At present, when the traditional manipulator of industrial production line automation grasps objects, it is necessary to analyze and program the characteristics of the target to be grasped in advance, and then cooperate with the machine vision system to realize the position positioning of the specific target, and finally guide the manipulator to complete the grasping process. Moreover, in the prior art, the visual recognition algorithm is only for parts with certain characteristics, and cannot completely recognize random objects, and the system has a certain projection error in the calculation of coordinates, which causes the robot to grasp the object. There is an error of about a few millimeters. In addition, the grasping strategy of the manipulator is not optimal, and the manipulator still has some redundant actions during the grasping process, so the grasping efficiency is not very high. When the manipulator grasps the object, it will have an unsatisfactory grasping posture, which will cause the grasping error and cause the manipulator to stop. The manipulator has a size limit on the grasped object, and the grasped object can neither be too large nor too small. Occasionally, when the manipulator grasps the object, the grasping is unstable and the object falls off.

In view of the above defects of the prior art, it is necessary to develop a non-specific shape object recognition, positioning and grasping system and method for a manipulator.

SUMMARY OF THE INVENTION

In view of the defects of the prior art, the purpose of the present invention is to provide a non-specific shape object recognition, positioning and manipulator grasping system and method, which realizes the automatic judgment and classification of the target by combining with deep neural network learning, and guides the manipulator to achieve certain Targets with random positions within the range are picked up, which aims to solve the problems that the grasping of the manipulator in the prior art is not flexible enough and the adaptability is insufficient.

In order to achieve the above object, the present invention provides a non-specific shape object recognition, positioning and manipulator grasping system, which includes a multi-degree-of-freedom manipulator, a machine vision imaging module and a central processing module, wherein,

The multi-degree-of-freedom manipulator includes jaws or/and suction nozzles, and the multi-degree-of-freedom manipulator is connected to the central processing module to be controlled by the central processing module, and pick up the target according to the plane motion coordinates and the peripheral size of the target given by the central processing module. ,

The machine vision imaging module includes an industrial camera and an imaging lens. The machine vision imaging module is connected to the central processing module to transmit the image data collected by itself to the central processing module.

The central processing module is used to receive the image data collected by the machine vision imaging module, process the image data, extract the target contour, and realize the shape of each target according to the target contour combined with the mapping relationship between the calibrated object space size and the number of pixels in the image space. The measurement of size is also used to classify the target according to the size of the target by using the deep neural network model, and then guide and control the multi-degree-of-freedom manipulator to identify, locate and pick up the target.

Further, an image processing submodule is integrated in the central processing module, and the image processing submodule is used for image data processing. The target to be picked up in the image is segmented, and all target contours in the image of the working area are extracted to provide a data basis for subsequent analysis and processing.

Further, a size measurement sub-module is also integrated in the central processing module, which is used to measure the target size information according to the target contour, and the target size information includes the aspect ratio of the circumscribed rectangle, the rectangularity and the roundness, and is also used for measuring according to the target contour information. The actual space coordinate of the center position of the target plane, which is used to guide the pick-up of the manipulator later.

Further, a target classification sub-module is also integrated in the central processing module, which is used to perform target classification on the target through the trained deep neural network model according to the target outline information extracted by the image processing sub-module and the target size information obtained by the size measurement sub-module. Classification, and when the target cannot be classified into the known target feature classification library, it is added to the classification library as a new target feature.

Further, the central processing module is also integrated with a picking and guiding sub-module, which is used to guide the manipulator to complete the picking work of the target according to the set picking mode according to the position information and classification information of each target in the working area.

According to the second aspect of the present invention, there is also provided a method for identifying, locating and grasping an object with a non-specific shape, which comprises the following steps:

S1: collect and obtain image data, the image data is in digital format,

S2: Perform image preprocessing on the image data in the data format, obtain the target image, obtain the edge information of the target in the target image by edge extraction and sub-pixel analysis, and determine its circumscribed rectangle according to the edge information,

S3: Pair the edge information of each target with the circumscribed rectangle, store the pairing result in the target information list, analyze each information in the list, and calculate the length, width and circumscribing rectangle of each target circumscribing rectangle. center position coordinate information, and store the information in the target information list for classifying the target,

S4: Use the pre-trained deep neural network model for classification to classify the target information, use a deep neural network composed of an input layer, multiple hidden layers, and an output layer as the classification network, and the network input information is the target outline and size information, The output is the classification result,

When classifying targets, if a new type of target is found, it is first ruled out that it is a known target that overlaps its location. If it still exhibits the characteristics of a new target, it is regarded as a new target that appears for the first time. The target data is added to the training set, and a large amount of target data with similar characteristics is generated through the generative adversarial neural network, and then the parameters of the deep neural network model are updated through machine learning training to achieve accurate classification of new non-specific objects. ,

S5: Instruct and control the multi-degree-of-freedom manipulator to identify, locate and pick up the target according to the set method, and when picking up a non-specific target, place it in the corresponding collection space according to the predetermined collection method.

Further, in step S4, the specific operation of excluding the new type of target from overlapping the known target position is as follows: first, according to the position measurement result, guide the manipulator to run near the new type of target, perform a touch operation on the new type of target, and change the target position, Then re-acquire the image of the working area, and then perform target recognition on the area where the new type of target appears. If it still shows the characteristics of the new target, it is regarded as the new target that appears for the first time.

Further, in step S3, when calculating the length and width of each target circumscribed rectangle and the coordinate information of the center position of the circumscribed rectangle, the first coordinate system is a space coordinate system, the second coordinate system is a picture coordinate system, and the third coordinate system is The coordinate system of the manipulator, the picture coordinate system is based on the fixed photographed image, and the space coordinate system is based on the specific position of the object on the object placement plane. The coordinate system of the manipulator is based on the difference between the two feature points before and after the movement of the manipulator. A matrix completes the transformation of the coordinates in the first coordinate system to the third coordinate system.

Further, in step S5, after placing the target, the multi-DOF manipulator returns its own pose to the external central processing module to confirm whether the coordinates of the manipulator are correct. The multi-DOF manipulator follows the target before grabbing the target. The size of the objects is sorted, so that the objects are grasped, and different objects are placed in corresponding different positions according to their types and sizes.

Further, in step S2, the image preprocessing includes one or more of grayscale stretching, histogram equalization, smooth filtering, distortion correction, and white balance correction. In step S5, instructing and controlling the multi-degree-of-freedom manipulator to follow Setting method When identifying, locating and picking up the target, the setting method refers to the method after the coordinates are sorted according to the size, shape or/and color gamut of the target.

The invention identifies different parts by extracting the information of the parts in the field of view, and the main methods include the calculation of the aspect ratio of the circumscribed rectangle of the part, and the calculation of the rectangularity and roundness of the part. Considering that the parameters required for template matching are many and complex, it takes more time to calculate, and the parameter settings have a greater impact on the speed and results. Therefore, template matching is not selected to identify the object to be grasped, but It is to distinguish the aspect ratio, squareness and roundness of the parts, which undoubtedly makes the method and system of the present invention more efficient. As for the grasping of the manipulator, the algorithm design of the present application enables the manipulator to automatically adjust the position and posture during grasping, making the grasping more precise, and at the same time, the control panel and the program can realize more precise control of the manipulator.

After the manipulator places the object, it will return its own posture to the computer or industrial computer, so a certain degree of monitoring can be done to confirm whether the coordinates of the manipulator are correct. Before grasping the objects, the manipulator can sort the objects according to the size of the objects to grasp the objects, and classify different objects according to their types and sizes to place different objects in different positions.

In general, compared with the prior art, the above technical solutions conceived by the present invention have the following beneficial effects:

In the system of the present invention, by combining machine vision and the idea of deep learning method, a deep neural network model is constructed to realize automatic judgment and classification of targets that appear for the first time, and these features are automatically merged into the long-term accumulated target feature information database. In subsequent applications, intelligent identification, classification and location measurement can be implemented adaptively for non-specific targets. The system and method of the present invention are based on the image of the working area captured by the machine vision system, and realize the size measurement, appearance feature analysis, occlusion recognition, feature classification and position measurement of the target by combining the image processing algorithm of artificial intelligence technology. On this basis, the optimal target picking scheme is given in real time, and the manipulator is guided to pick up random positions, random types, and possibly occluded targets quickly, accurately and orderly.

Description of drawings

FIG. 1 is a schematic flowchart of the operation of a system for identifying, positioning and manipulating an object with a non-specific shape provided by an embodiment of the present invention.

Detailed ways

In order to make the objectives, technical solutions and advantages of the present invention clearer, the present invention will be further described in detail below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present invention, but not to limit the present invention.

The non-specific shape object recognition, positioning and manipulator grasping system of the present invention mainly includes a functional platform part and a sub-module part integrated in the functional platform part, and the functional platform part includes a multi-degree-of-freedom manipulator, a machine vision imaging module and a central processing module. The sub-module part of the functional platform part includes an image processing sub-module, a size measurement sub-module, a target classification sub-module and a pick-up guide sub-module. , is located in the function of grabbing.

Among them, the multi-degree-of-freedom manipulator can choose products with different composition methods, different load capacities and different motion accuracy according to the needs of the task. Claws, suction nozzles and other components achieve accurate grasping of the target. The machine vision imaging module consists of a high-resolution industrial camera, an imaging lens, an illumination light source and its installation structure. The resolution of the high-resolution industrial camera is determined by the minimum size of the target to be picked up and the focal length parameter of the imaging lens, while the imaging lens The parameters such as focal length and aperture need to be calculated according to the scope of the working area to be detected. The illumination light source is used to effectively illuminate the target and the working area, and is calculated according to the surface color and reflectance characteristics of the target to be measured and the background reflection characteristics of the working area. get. The installation structure in the module is responsible for ensuring that the machine vision imaging module and the manipulator are in a relatively fixed spatial position, which is convenient for determining the coordinate transformation matrix and improving the guidance accuracy when guiding the manipulator in the later stage. The central processing module is mainly responsible for the operation of the software system, including collecting the image data of the camera, running the analysis software to give the guidance data, and controlling the manipulator and the components of the machine vision imaging module to work normally according to the given parameters and coordinate positions.

The central processing module integrates multiple sub-modules, including an image processing sub-module, a size measurement sub-module, a target classification sub-module and a pick-up guide sub-module. The specific functions of each module are as follows:

The image processing sub-module performs image processing on the image of the working area collected by the machine vision system, obtains the image of the working area with improved contrast through image preprocessing methods such as histogram equalization, and then realizes the segmentation of the target to be picked through the edge extraction algorithm. Accurately extract all target contours in the work area to provide a data basis for subsequent analysis and processing.

The size measurement sub-module realizes the measurement of the outer size of each target through the mapping relationship between the calibrated object space size and the number of pixels in the image space on the basis that the image processing module obtains all the target contour information in the working area, and at the same time according to the target contour The information measures the actual space coordinates of the center position of the plane for the subsequent guidance of the pick-up by the manipulator.

The target classification sub-module uses the target outline information extracted by the image processing module and the target size information obtained by the size measurement module to classify it through the trained deep neural network. If the target cannot be classified into the known target feature classification library, then Add it to the classification library as a new target feature.

The picking and guiding sub-module guides the manipulator to complete the work of picking up the target according to the established logic through the position information and classification information of each target in the working area obtained after processing by all the previous modules.

FIG. 1 is a schematic flow chart of the operation of a non-specific shape object recognition, positioning and robotic grasping system provided by an embodiment of the present invention. Combined with this figure and the non-specific shape object recognition, positioning and robotic grasping method, it can be known that the method of the present invention mainly includes: Follow the steps below:

(1) The machine vision imaging module performs imaging detection on the working area, the light source illuminates the target in the working area uniformly, and is imaged on the image surface of the high-resolution industrial camera through the imaging lens, and the formed image is transmitted to the central processing unit through the digital signal transmission line. module, which is collected and stored as a digital format image for use in subsequent steps.

(2) The digital format image is sent to the image processing module. First, the image preprocessing methods such as grayscale stretching, histogram equalization, smooth filtering, distortion correction, and white balance correction in computer graphics are used to obtain high contrast and low noise. , distortion-free target image; then use edge extraction and sub-pixel analysis algorithms to analyze and obtain high-precision edge information of the target in the image; finally, use the analyzed target edge to determine its circumscribed rectangle for further size measurement.

(3) Pair the edge contour and the circumscribed rectangle of each analyzed target, store it in the target information list, use the size measurement module to analyze each information in the list, and calculate the length of the circumscribed rectangle, Width information and rectangular center position coordinate information, and store the information in the target list for the next step to classify the target.

(4) Use the pre-trained classification to classify the target information with a deep neural network. The classification is based on the measured size and the specific contour information of the target. Since the randomly placed target may be in various poses when it is photographed , the outer contour obtained by the processing of the captured image may have various shapes, and the measured size will also be different, so it cannot be directly classified by a simple linear model. Consider using a deep neural network composed of an input layer, multiple hidden layers, and an output layer as the classification network. The network input information is the target outline and size information, and the output is the classification result. The prior measurement data is used as the training data to train the model. The resulting neural network model is able to classify known objects with over 99% accuracy. If there is a situation that cannot be accurately classified, consider the new type of target that may be the first occurrence, and go to the next step.

(5) For new types of targets that appear in the classification, it is necessary to first rule out that they are misclassified due to different features on the two-dimensional image after the overlapping positions of the known targets. Therefore, every time a new target is found, first The position measurement results guide the manipulator to run near the target, touch the target slightly, change the target position slightly, and then use the machine vision system to take pictures of the working area, and then focus on the target recognition of the area where the new target appears. After it is still a target and still exhibits the characteristics of a new target, it is regarded as a new target that appears for the first time, and the acquired new target data is added to the training set, and a large number of corresponding Target data with similar characteristics is used to enrich new types of training data, and then the parameters of the deep neural network are updated through machine learning training to achieve accurate classification of emerging non-specific target targets.

(6) After obtaining the outline, position and classification information list of all targets in the working area, the control software determines the picking sequence according to the picking logic, and instructs the manipulator to move to the corresponding actual coordinate position according to the sequence, pick up non-specific targets, and then Place it in the corresponding collection space according to the agreed collection method.

In the present invention, "eye" is such as the visual part of the system, which determines the target position through a series of image processing, "hand" is such as the moving part of the robot arm of the system, including the grasping and placing of the robot arm, and the part of hand-eye combination, including data transmission and Coordinate transformations combine the "hands" and "eyes" of the system.

In an embodiment of the present invention, halcon software is used to complete related image processing. Specifically, under the condition of a given background and sufficient light source, the original picture is obtained by taking pictures, and halcon is used to correct the image, eliminate aberration, and obtain the basic object plane. The rectangular field of view picture on the halcon, through the n*n standard calibration plate downloaded from the halcon, according to its size to determine the correct field of view, that is, the image position and the actual position match. Through the powerful visual processing capability of halcon, the circumscribed rectangle of each object to be grasped in the field of view is extracted, and the recognition of different types of objects is realized by the aspect ratio and area of the rectangle. The scanned objects are classified based on factors such as size, shape, color gamut, etc., and then the coordinates of these pictures are sorted in the order required by the user, such as size, shape, and color gamut. At present, the objects used in the project are mainly common parts in the industry, such as nuts and nuts, which have obvious characteristics in shape and have relatively regular shapes, which are relatively easy to identify and classify.

When the manipulator grasps and places the target object, it is necessary to set the motion path of the manipulator. Due to the influence of the manipulator itself, the restriction of the surrounding metal frame and the obstruction of small objects to be grasped on the storage plane, planning the movement path of the manipulator before implementing the grasping can avoid the movement of the manipulator being blocked and the damage of the original scene by the manipulator. When grabbing, the robotic arm obtains the initial coordinates of the part in sequence, and moves the robotic arm to the initial coordinates. The mechanical arm calculates the reasonable grasped angle of the object to be measured through the preset length of the picture, and controls the steering of the two-finger mechanical claw. The robot arm calculates the vertical coordinate landing point of the preset length according to the difference between the horizontal and vertical coordinates of the current feature point, adjusts the vertical coordinate to move to the target position, and tightens the mechanical claw. When the robotic arm recognizes that the object to be grasped is not grasped, it will obtain the coordinates of the next target until it grasps the object. After the robotic arm picks up the object to be grasped, it identifies the path to the placement position of the object according to the position of the current feature point, and plans the direction of travel of the robotic arm to prevent the movement of the robotic arm from being blocked or destroying the original storage scene. When the recognition feature point of the robot arm reaches the predetermined position, it will fall slowly. During the fall, the force of the robot arm will be checked all the time. If the robot arm receives an upward force, it will be judged that the robot arm has descended to the bottom, and the robot arm will release the robot. Grab, grab, and the robot will place it in different places according to the type of parts. In addition, the manipulator will be classified according to the size of the object to be grasped, and the parts will be placed in different places according to different areas of the area. The industrial computer that controls the manipulator will display the coordinates of the manipulator grabbing the target object and the coordinates of the manipulator placing the object at the target position in real time, which has the effect of monitoring.

In the present invention, converting machine vision information into information usable by a robotic arm is equivalent to combining "hands" and "eyes", mainly requiring coordinate conversion and data transmission. When performing coordinate conversion, the first coordinate system is the space coordinate system, the second coordinate system is the picture coordinate system, and the third coordinate system is the manipulator coordinate system. The coordinate system is based on the specific position of the object on the object placement plane. The object placement plane is the xy plane, and the vertical direction upward is the positive direction of the z-axis. When the manipulator moves, the difference between the two feature points before and after the motion is The system is constructed in the actual space, and the transformation of the coordinates in the second coordinate system to the first coordinate system is completed through a matrix, and the transformation of the coordinates in the first coordinate system to the third coordinate system is completed through the second matrix. After obtaining the coordinates in the data array of image processing, perform coordinate transformation, use the redis database to perform image processing and rapid transmission of thermal data of the robotic arm system, and perform coordinate data interaction in the database.

In an embodiment of the present invention, based on Universal Robots e5 of Universal Robots, it is equipped with a two-finger robot gripper to perform a grasping task with an effective working radius of up to 850 mm and a maximum weight of five kilograms. Set up a metal frame around the manipulator, which ensures that the manipulator is located at the midpoint of one side of the rectangular projection of the metal frame in the top view, and ensures that the vertical distance between the CCD camera and the object plane is, for example, 88cm. The GO-5100M-USB camera from JAI company was selected. The CCD target surface of the camera is 2/3 inch. According to the calculation and the final actual matching, a lens with a focal length of f=16mm was used to obtain a rectangle with a coverage of 50cm*40cm. field of view. Luminous flux E=(1/(D*D))*lnx, D is the distance from the light source to the object to be measured, and x is the distance from the object to the lens (working distance) WD from the luminous intensity. Calculate the image magnification PMAG by using the known sensor imaging surface height Hi and the measured object size (field of view height) Ho, PMAG=Sensor Size(mm)/Field of View(mm)=Hi/Ho. Use the formula to calculate the required focal length f, f=WD*PMAG/(1+PMAG), select the standard lens product that is closest to the calculated value, and take its focal length value. Examples of standard lens focal lengths are: 8mm, 12.5mm, 16mm, 25mm and 50mm. Recalculate the lens-to-object distance WD based on the selected lens focal length. Among them, LE=Di-f=PMAG*f, PMAG=Di/WD or WD=f*(1+PMAG)/PMAG, converted into Chinese as: resolution=photosensitive chip size/pixel size=field length or width/ Detection accuracy. Based on the mounted Feichuang Yida optical platform, a pvc version and a pure black curtain can be used to build a storage area such as 90cm*90cm on the object plane. At the same time, under such a comparison, no additional light source is added.

In the present invention, a set of object recognition, positioning and grasping system combining machine vision imaging and non-specific shape target self-adaptive identification intelligent algorithm is constructed, which realizes that there is no need to analyze or program the grasped target, and the artificial intelligence algorithm is directly used to cooperate. The edge computing module automatically classifies and recognizes emerging objects with different shapes and sizes, achieves high-precision positioning through positioning algorithms, and guides the manipulator to accurately grasp non-specific targets of any shape and size.

The target self-adaptive identification intelligent algorithm designed by the present invention utilizes the design idea of deep neural network and machine learning, and realizes the automatic extraction and clustering of shape features and size parameters by constructing and training the neural network, and self-adapting according to the classification results. Determine whether to increase the types of objects, so as to achieve efficient adaptive recognition of non-specific objects with any new characteristics.

In the present invention, the designed machine vision imaging module and the space installation reference of the manipulator are relatively fixed, and the high-precision calibration algorithm realizes the rapid and efficient conversion of the image coordinates and the space coordinates of the manipulator, and realizes the artificial intelligence object recognition and positioning combined with the image processing algorithm. The function guides the manipulator to obtain specific feature targets according to the instructions or to classify and pick the targets according to the target characteristics.

The present invention has the following two relatively ingenious and novel aspects: (1) intelligent self-adaptive new target classification. The demand for flexible production in the field of intelligent manufacturing is increasing. In modern intelligent production lines, there will be a need to carry and assemble a large number of small batches, multi-specification, and random parts or components. When traditional manipulators carry out parts handling, The size and shape of the object to be handled must be known first, and accurate and reliable grasping can only be achieved after analysis and pre-programming. The application of the present invention constructs a deep neural network by combining machine vision and deep learning methods to realize automatic judgment and classification of targets that appear for the first time, and automatically merge these features into the long-term accumulated target feature information database, which can be automatically used in subsequent applications. Adaptive intelligent identification, classification and location measurement of non-specific targets. (2) Self-adaptive and real-time guidance of the manipulator to automatically pick up the target. At this stage, most of the methods of manipulators picking targets on industrial automatic production lines are based on fixed-point picking at fixed target positions. In very few applications, machine vision-guided manipulators are used to achieve the picking function of random targets within a certain range. With a large number, random placement, or even mutual occlusion, the system cannot effectively identify and pick up. The system of the present invention is based on the image of the working area captured by the machine vision system, and realizes the size measurement, appearance feature analysis, occlusion recognition, feature classification and position measurement of the target through the image processing algorithm combined with artificial intelligence technology. Here On this basis, the optimal target picking scheme is given in real time, and the manipulator is guided to pick up random positions, random types, and possibly occluded targets quickly, accurately and orderly.

The system and method of the present invention can be applied in the following fields: (1) Parts sorting on an automatic production line: Because of the fineness of the parts, different parts can be sorted by size, weight, shape and color through the system, and the automatic sorting In different areas, manpower and material resources are greatly reduced; (2) Intelligent classification in flexible manufacturing: The flexible manufacturing system is an automated production system controlled by a computer based on group technology, which can process a group of similar shapes at the same time. or a class of products. The high-efficiency manufacturing mode is suitable for multiple varieties and small batches. Under different varieties, according to the individual differences of the varieties, the system of the application of the present invention can be integrated into it to realize intelligent classification, reduce the inventory of blanks and work-in-process, and reduce direct labor. Besides, in many places where classification is required, the system and method of the present application can be applied.

Those skilled in the art can easily understand that the above are only preferred embodiments of the present invention, and are not intended to limit the present invention. Any modifications, equivalent replacements and improvements made within the spirit and principles of the present invention, etc., All should be included within the protection scope of the present invention.

Claims (10)

1. A non-specific shaped object recognition, positioning and manipulator grabbing system is characterized by comprising a multi-degree-of-freedom manipulator, a machine vision imaging module and a central processing module, wherein,

the multi-degree-of-freedom manipulator comprises a clamping jaw or/and a suction nozzle, is connected with the central processing module to be controlled by the central processing module, picks up the target according to the plane motion coordinate and the peripheral size of the target given by the central processing module,

the machine vision imaging module comprises an industrial camera and an imaging lens, the machine vision imaging module is connected with the central processing module to transmit the image data acquired by the machine vision imaging module to the central processing module,

the central processing module is used for receiving the image data acquired by the machine vision imaging module, processing the image data, extracting the outline of the target, combining the mapping relation between the object space size and the pixel number of the image space after calibration according to the outline of the target to realize the measurement of the outline dimension of each target, and is also used for classifying the target according to the outline dimension of the target by adopting a deep neural network model so as to guide and control the multi-degree-of-freedom manipulator to identify, position and pick up the target.

2. The system for non-specific shaped object recognition, positioning and manipulator grabbing according to claim 1, wherein the central processing module is integrated with an image processing sub-module, the image processing sub-module is used for processing image data, specifically, a working area image with improved contrast is obtained through histogram equalization, then an edge extraction algorithm is used to segment an object to be picked up in the working area image, and all object contours in the working area image are extracted to provide a data basis for subsequent analysis and processing.

3. A non-specific shaped object recognition, positioning and robot grasping system according to claim 2, wherein the central processing module further integrates a dimension measurement sub-module for measuring target dimension information including aspect ratio, rectangularity and roundness of the circumscribed rectangle based on the target profile, and for measuring actual spatial coordinates of the center position of the target plane based on the target profile information for subsequent guidance of robot picking.

4. The system for non-specific shaped object recognition, positioning and manipulator grabbing according to claim 3, wherein the central processing module further integrates a target classification sub-module, which is used to classify the targets by the trained deep neural network model according to the target contour information extracted by the image processing sub-module and the target dimension information obtained by the dimension measurement sub-module, and to add the targets as new target features to the classification library if the targets cannot be classified into the known target feature classification library.

5. A non-specific shaped object recognition, positioning and manipulator grabbing system according to claim 4, wherein the central processing module further comprises a picking guide sub-module integrated therein for guiding the manipulator to complete the picking operation of the object according to the set picking mode based on the position information and classification information of each object in the working area.

6. A non-specific shape object identification, positioning and mechanical arm grabbing method is characterized by comprising the following steps:

s1: acquiring image data, wherein the image data is in a digital format,

s2: image preprocessing is carried out on the image data in the data format to obtain a target image, edge information of a target in the target image is obtained by utilizing edge extraction and sub-pixel analysis, a circumscribed rectangle of the target image is determined according to the edge information,

s3: the edge information and the circumscribed rectangle of each target are paired, the pairing result is stored in a target information list, each piece of information in the list is analyzed, the length and width information of the circumscribed rectangle of each target and the coordinate information of the central position of the circumscribed rectangle are calculated, the information is stored in the target information list and is used for classifying the targets,

S4: classifying the target information by using a pre-trained deep neural network model for classification, adopting a deep neural network consisting of an input layer, a plurality of hidden layers and an output layer as a classification network, inputting the information of the network as target contour and size information, outputting the information as a classification result,

when the target classification is carried out, if a new type target is found, the overlapping of the positions of the known targets is firstly eliminated, if the characteristics of the new target are still expressed, the new target is regarded as a new target appearing for the first time, the obtained new target data is added into a training set, a large amount of target data similar to the characteristics of the antagonistic neural network is generated, the parameters of a deep neural network model are updated in a machine learning training mode, the accurate classification of the newly appeared non-specific target is realized,

and S5, guiding and controlling the multi-degree-of-freedom manipulator to identify, position and pick the target according to a set mode, and placing the non-specific target into a corresponding collection space according to a preset collection mode when the non-specific target is picked up.

7. The method as claimed in claim 6, wherein the step S4 of excluding the overlapping of the positions of the objects of the new type and the known objects is to guide the robot to move to the vicinity of the objects of the new type according to the position measurement result, touch the objects of the new type, change the positions of the objects, re-acquire the images of the working area, recognize the objects in the areas where the objects of the new type appear, and if the objects still show the characteristics of the new objects, regard the objects as the new objects appearing for the first time.

8. The non-specific shaped object identification, positioning and robot grasping method according to claim 7, wherein, when the length and width of each target circumscribed rectangle and the coordinate information of the center position of the circumscribed rectangle are calculated in step S3, the first coordinate system is a space coordinate system, the second coordinate system is a picture coordinate system, the third coordinate system is a manipulator coordinate system, the picture coordinate system takes the fixed image as a standard system, the space coordinate system takes the specific position of the object on the object plane as a standard, a system is established by taking the object placing plane as an xy plane and taking the vertical direction upwards as the positive direction of a z axis, a system is established in an actual space by taking the difference position of two characteristic points before and after movement of the manipulator when the manipulator moves, the transformation of the coordinates in the second coordinate system into the first coordinate system is done by means of one matrix and the transformation of the coordinates in the first coordinate system into the third coordinate system is done by means of the second matrix.

9. The non-specific shaped object recognition, positioning and manipulator grabbing method of claim 8, wherein in step S5, after the object is placed, the multi-degree of freedom manipulator returns its pose to the external central processing module to confirm whether the coordinates of the manipulator are correct,

The multi-degree-of-freedom manipulator sorts the objects according to the sizes of the object objects before grabbing the objects, so that the objects are grabbed, and different objects are placed at different positions correspondingly according to the types and sizes of the objects.

10. The non-specific shaped object recognition, localization and manipulator capture method according to claim 6, wherein in step S2, the image pre-processing comprises one or more of gray scale stretching, histogram equalization, smoothing filtering, distortion correction, white balance correction,

in step S5, when the multi-degree-of-freedom manipulator is guided and controlled to recognize, position, and pick up the target according to the setting mode, the setting mode is a mode in which coordinates are sorted according to the size, shape, or/and color gamut of the target.

Images