CN114952809A - Workpiece recognition and pose detection method, system and grasping control method of robotic arm - Google Patents

Abstract

The invention discloses a workpiece identification and pose detection method, a system and a grasping control method of a mechanical arm. The workpiece recognition and pose detection method includes: collecting 2D images and 3D point cloud images in the scene to be recognized; recognizing the target workpiece in the scene to be recognized based on the 2D image, and performing instance segmentation based on the mapping relationship to obtain The point cloud area corresponding to the target workpiece; based on a deep learning algorithm, pose detection is performed in the point cloud area to obtain the pose information of the target workpiece. The workpiece identification and pose detection method provided by the present invention avoids the problem of cross-modal data feature extraction and matching in the grasping scene where small workpieces are scattered and stacked, and at the same time avoids overly complicated data processing calculations. The 3D point cloud image provides an optimized solution for the application scenarios of workpiece stack recognition and grasping in the direction of effectively improving the recognition efficiency and improving the grasping efficiency.

Description

Workpiece recognition and pose detection method, system and grasping control method of robotic arm

technical field

The invention relates to the technical field of image recognition and mechanical control, in particular to a workpiece recognition and pose detection method, system and grasping control method of a mechanical arm.

Background technique

Robotic grasping technology based on 2D/3D vision has been widely used in simple scenarios such as logistics and express delivery, warehouse handling, and depalletizing. Vision-guided robots enhance the perception ability of complex environments. In industrial grabbing scenarios, two-dimensional images can provide dense and rich texture information, and after image processing and recognition, the position (two-dimensional coordinates) of the grabbed workpiece can be obtained, but depth information cannot be obtained; three-dimensional images can provide grabbing scenes However, rich detailed information cannot be obtained, which leads to the reduction of the grasping accuracy. The two types of data have good complementarity, and the fusion of the data of the two modalities can realize a more comprehensive perception of the workpiece grasping scene. In recent years, with the increasing research on 6D pose estimation algorithms of workpieces and the increasing computing power of equipment, robotic grasping systems have made creative breakthroughs in related fields such as disordered workpiece stacking, disordered workpiece assembly, and flexible grasping.

Among them, target object recognition and pose detection are key prerequisites for robot grasping tasks. Since the early days of computer vision, target 6D pose detection and estimation is a fixed coordinate system relative to a given reference frame, and the object pose information is described by a translation vector t ∈ R3 and a rotation matrix R ∈ SO(3), is a long-standing challenge and an open field of research.

Due to the diversity of objects in the real world, potential object symmetry, clutter and occlusion in the scene, and changing lighting conditions, the core step of the task of target 6D pose detection and estimation is to first obtain the target object through various algorithms. The centroid position coordinates (x, y, z) in the camera coordinate system, and then match the model to the centroid position to obtain the rotation pose (R _x , R _y , R _z ) of the current target object in the camera coordinate system, based on the hand-eye It is challenging to calibrate the matrix transformation, obtain the position of the target object in the base coordinates of the manipulator, and finally control the motion of the manipulator for grasping. From a technical point of view, 3D point cloud data and 2D image data belong to different modalities. How to skillfully integrate the two data, and analyze reliable geometric features according to the scattered stacking of workpieces in the captured scene, and finally Identifying graspable workpieces in the scene and obtaining pose information is a research direction that domestic and foreign scientific and technological workers focus on exploring.

The existing workpiece recognition and pose detection methods can be divided into the following two categories according to different input image data types: based on 2D visual data (using RGB or RGBD data as input), and based on 3D visual data (using point cloud data as the input). enter). Due to the lack of scene depth information, the simple recognition method based on 2D data can only grasp plane objects and cannot handle stacked scenes. Therefore, workpiece recognition and pose detection methods based on 3D visual data have gradually become the mainstream. Detection methods based on 3D visual data can be roughly divided into the following two categories according to the difference in implementation principle: template matching method and deep learning method. The first type of template matching method is usually based on the PPF (Point Pair Future) algorithm (such as Drost B, Ulrich M, Navab N. et al. Model globally, match locally: Efficient and robust 3D objectrecognition [C]. IEEE computer society conference on computer vision and pattern recognition. Piscataway: IEEE Press, 2010: 998-1005.), the algorithm is a description method based on point-to-point features. According to the 3D model data of the target, the point-to-point features are extracted and the model of the target is trained. Based on PPF The feature descriptor detects and matches 3D feature points in the target scene, obtains an initial estimate of the pose and iteratively votes, and finally uses the ICP algorithm to perform a Refinement operation on the result to obtain a more accurate pose result and output. The biggest defect of the template matching method is the phenomenon of mismatching. However, when the workpiece is too simple and the features are not obvious, the above methods often get wrong recognition results. The second type of deep learning method generates a simulation data set in a simulation scene, then learns the data features in the network, and finally obtains the result of pose detection in the test data set. For example, literature (Dong Z, Liu S, Zhou T. et al. PPR-Net: point-wise pose regression network for instance segmentation and 6d pose estimation in bin-picking scenarios [C]. IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS). Piscataway: IEEE Press, 2019: 1773-1780.) proposes a novel point-wise pose regression network PPR-Net (Point-wise Pose Regression Network). This method, the winner in the IROS2019 "bin-pick pose estimation challenge", uses PointNet++ as the backbone network to perform 6D pose estimation for each point in the point cloud of the object instance to which it belongs, and then cluster-based in space The method averages each identified predicted pose to obtain the final pose hypothesis. However, the shortcomings of this method are: the processing efficiency of the global 3D point cloud image of the workpiece scene is low, and the analysis and detection time is long.

Therefore, how to provide a high-efficiency target object recognition and pose detection method suitable for the scene where small workpieces are stacked and occluded is an urgent problem to be solved.

SUMMARY OF THE INVENTION

In view of the deficiencies of the prior art, the purpose of the present invention is to provide a workpiece identification and pose detection method, a system and a grasping control method of a mechanical arm.

In order to realize the foregoing invention purpose, the technical scheme adopted in the present invention includes:

In a first aspect, the present invention provides a workpiece identification and pose detection method, including:

S1, collect 2D images and 3D point cloud images in the scene to be recognized;

S2, identify the target workpiece in the scene to be identified based on the 2D image, and based on the mapping relationship between the 2D image and the 3D point cloud image, perform an example on the area where the target workpiece is located in the 3D point cloud image Segmentation to obtain the point cloud area corresponding to the target workpiece;

S3, based on a deep learning algorithm, perform pose detection in the point cloud area, and obtain pose information of the target workpiece.

In a second aspect, the present invention also provides a workpiece recognition and pose detection system, including:

The image acquisition module is used to collect 2D images and 3D point cloud images in the scene to be recognized;

an area acquisition module, used for identifying the target workpiece in the scene to be identified based on the 2D image, and based on the mapping relationship between the 2D image and the 3D point cloud image, for the target workpiece in the 3D point cloud image Instance segmentation is performed in the area where it is located, and the point cloud area corresponding to the target workpiece is obtained;

The pose acquisition module is used to perform pose detection in the point cloud area based on a deep learning algorithm, and obtain pose information of the target workpiece.

In a third aspect, the present invention also provides a grasping control method for a robotic arm, comprising:

Obtain the target workpiece in the scene to be recognized and its pose information based on the workpiece recognition and pose detection methods;

The target workpiece to be grasped is selected, and the robotic arm is controlled to perform grasping action based on the pose information.

Based on the above technical solutions, compared with the prior art, the beneficial effects of the present invention at least include:

The workpiece identification and pose detection method provided by the present invention avoids the problem of cross-modal data feature extraction and matching in the grab scene of scattered stacking of small workpieces, and at the same time avoids overly complicated data processing and calculation. The 3D point cloud image provides an optimized solution in the direction of effectively improving the recognition efficiency and improving the grasping efficiency for the application scene of workpiece stack recognition and grasping.

The above description is only an overview of the technical solutions of the present invention. In order to enable those skilled in the art to understand the technical means of the present application more clearly, and to implement them in accordance with the contents of the description, the following preferred embodiments of the present invention are used in conjunction with the detailed descriptions. The accompanying drawings are described below.

Description of drawings

1 is a schematic flowchart of a workpiece identification and pose detection method provided by an exemplary embodiment of the present invention;

FIG. 2 is a partial schematic flowchart of a workpiece identification and pose detection method provided by an exemplary embodiment of the present invention;

FIG. 3 is a partial flowchart of a workpiece identification and pose detection method provided by an exemplary embodiment of the present invention;

FIG. 4 is a partial schematic flowchart of a workpiece identification and pose detection method provided by an exemplary embodiment of the present invention;

5 is a schematic structural diagram of a workpiece recognition and pose detection system provided by an exemplary embodiment of the present invention

6 is a schematic structural diagram of a simulation data set generation system provided by an exemplary embodiment of the present invention

7 is a schematic structural diagram of a 2D/3D deep learning network provided by an exemplary embodiment of the present invention;

FIG. 8 is an example diagram of the recognition and detection effects of the workpiece recognition and pose detection method provided by an exemplary embodiment of the present invention.

Detailed ways

In view of the deficiencies in the prior art, the inventor of the present application was able to propose the technical solution of the present invention after long-term research and extensive practice. The technical solution, its implementation process and principle will be further explained as follows.

Many specific details are set forth in the following description to facilitate a full understanding of the present invention. However, the present invention can also be implemented in other ways different from those described herein. Therefore, the protection scope of the present invention is not limited by the specific implementation disclosed below. example limitations.

Referring to FIG. 1 to FIG. 4 , an embodiment of the present invention provides a workpiece identification and pose detection method, which specifically includes the following steps S1-S3:

S1, collect 2D images and 3D point cloud images in the scene to be recognized.

Specifically, in this embodiment, the used image information includes a 2D image and a 3D image. Specifically, a 2D visible light camera is used to photograph the workpiece in the stacked scene to obtain a 2D image, and a 3D visible light camera is used to photograph the workpiece in the stacked scene. Get 3D images.

Therefore, in some embodiments, step S1 may specifically include: using a 2D camera to acquire the 2D image in the to-be-recognized scene, and using a 3D camera to acquire the 3D image information in the to-be-recognized scene.

In some embodiments, the to-be-identified scene may preferably include a workpiece stacking scene, more preferably a workpiece scattered stacking scene. The above method is especially suitable for the application scenarios where small workpieces are stacked randomly, for example, some smaller workpieces are scattered randomly on the pallet or in the container, such as the disorderly stacking of common semi-finished products on the pallet during industrial production, or When some materials are transported, for example, workpieces such as wrenches are scattered and piled up in the material frame.

S2, identify the target workpiece in the scene to be identified based on the 2D image, and based on the mapping relationship between the 2D image and the 3D point cloud image, perform an example on the area where the target workpiece is located in the 3D point cloud image Segmentation to obtain the point cloud area corresponding to the target workpiece. This step recognizes the target workpiece in the scene based on the 2D image, and uses the mapping relationship between the 2D image and the 3D point cloud image to segment the identified target workpiece area.

Specifically, as shown in FIG. 2, the step S2 may include steps S21-S24:

S21 , collecting several 2D images, marking the closed area of the contour of the workpiece, and making a training data set.

Specifically, in this embodiment, the contour data of the workpiece can be obtained from the 2D image, the contours of the workpiece with the same upward facing are marked as the same class, the position of the marked workpiece area in the global image is recorded, and several images are collected and marked. , to complete the training data set. The above-mentioned annotations can be manually annotated, or can be annotated by methods such as manual annotation and machine annotation in machine adversarial learning.

S22, build a deep learning 2D image target segmentation network model, and train the network model based on the training data set.

Specifically, in this embodiment, a 2D image target segmentation network model based on the Mask R-CNN convolutional neural network can be built, and the network model can be trained and learned based on the prepared training data set.

S23, import the 2D image collected in real time into the trained network model for identification, and obtain the region position of the identified workpiece.

Specifically, in this embodiment, the 2D image target segmentation network model that has been learned can be used, the 2D image collected in real time can be imported into the network model, and Mask R-CNN can be used to make the objects recognized in the 2D image realize pixel stage segmentation, and obtain the location of the region in the segmented 2D image.

S24, map the identified 2D workpiece area position to the 3D point cloud scene area, and perform instance segmentation on the target workpiece area in the 3D point cloud scene.

Specifically, in this embodiment, the mapping relationship between the 2D image and the 3D point cloud can be used to perform point cloud instance segmentation on the same location area in the 3D point cloud data scene based on the target workpiece area after the instance segmentation, Obtain a point cloud set of identified artifacts.

Therefore, in some embodiments, step S2 specifically includes the following steps:

The 2D image is imported into the target segmentation model for identification, and the region position of the target workpiece is obtained.

The region position is mapped to the corresponding region in the 3D point cloud image, and instance segmentation is performed on the region where the target workpiece is located in the 3D point cloud image.

In some embodiments, the training method of the target segmentation model specifically includes the following steps:

A 2D training data set is provided, the 2D training data set includes a plurality of 2D images used for training and their corresponding marking information, the marking information indicating at least the contour enclosed area of the workpiece in the 2D image.

An initial target segmentation model is constructed, and based on the 2D training data set, the initial target segmentation model is trained to obtain the target segmentation model.

S3, based on a deep learning algorithm, perform pose detection in the point cloud area, and obtain pose information of the target workpiece.

Specifically, in this embodiment, after the 2D image is segmented and mapped to a 3D point cloud, a collection of point clouds for identifying the workpiece can be obtained, and based on the PPR-Net deep learning network that has been built and trained and learned, the pose of the workpiece can be determined. Perform detection to obtain the pose information of the workpiece.

Continuing to refer to FIG. 3, step S3 may specifically include the following steps S31-S34:

S31, build a deep learning training simulation data set generation system based on V-REP.

Specifically, in this embodiment, a deep learning training simulation data set generation system can be built based on the V-REP simulation software, which includes building a Kinect simulation vision sensor, importing a 3D model of the workpiece, importing a 3D model of the material frame, and writing the drop and drop of the workpiece. The image data acquisition program and other content, the built simulation system is shown in Figure 6.

S32, producing and generating a simulated 3D training data set.

S33, build a deep learning 3D pose detection neural network model, and train the network model based on the training data set.

Specifically, in this embodiment, a 3D pose detection neural network model based on the PPR-Net deep learning network can be built, and the network model can be trained and learned based on the produced simulation training data set.

S34, import the 3D point cloud image after instance segmentation into the trained network model for workpiece pose detection, and obtain workpiece pose information.

Specifically, as shown in FIG. 4, in this embodiment, step S32 may include the following steps S321-S326:

S321, set that there are n workpieces in the scene.

In a preferred embodiment, the number of workpieces may be n=27, for example.

S322 , randomly drop i workpieces from a certain position in the work area based on the idea of domain randomization (set an initial integer i=0), and assign different color information to different workpieces.

S323, collect and save the depth image and the rgb image in the scene based on the simulated vision sensor.

In a preferred embodiment, the simulated vision sensor may be a Kinect depth camera.

S324, record and save the pose information of each workpiece dropped obtained in the V-REP.

S325, based on the collected rgb image information, perform visualization degree analysis on each workpiece, and record visualization degree data.

In a preferred embodiment, the visualization degree analysis method may be: introducing the visibility degree v∈[0,1] of the workpiece, this parameter reflects the occlusion degree of the predicted object, when v=0, it is completely invisible, and v=1 completely unobstructed, and so on. The degree of visualization of a workpiece in the scene is: v=N/N _max .

Among them, N is the area value of the color area of a certain instance workpiece, and N _max is the value of the largest area value of the color area in the global workpiece.

S326, if the integer i<n, repeat the steps from S322 to S326; if the integer i=n, stop the workpiece from dropping, and create a data set label file.

More specifically, in this embodiment, after the 2D image is segmented and mapped to the 3D point cloud, the point cloud collection for identifying the workpiece can be obtained. The pose is detected to obtain the pose information of the workpiece. The schematic diagram of the 2D/3D deep learning network based on the Mask R-CNN convolutional neural network and the PPR-Net deep learning network is shown in Figure 7. The 2D image detection instance segmentation effect And the schematic diagram of the 3D point cloud pose detection effect is shown in Figure 8.

As shown in Figure 5, a deep learning training simulation data set generation system disclosed in the present invention may include:

The image acquisition device is used for acquiring image information of the workpiece in the stacking scene.

Wherein, the image acquisition device includes a 2D image acquisition unit and a 3D image acquisition unit, and the 2D image acquisition unit is configured to use a 2D camera to take pictures of the workpieces in the stacked scene to acquire a 2D image. The 3D image acquisition unit is used for using a 3D camera to take pictures of the workpieces in the stacked scene to obtain a 3D image.

The area limiting device is used for physically limiting the falling range of the workpiece according to the size of the field of view of the image acquisition device.

Among them, the area limiting device mainly includes a material frame setting unit and a camera setting unit. The material frame setting unit is used to draw and import the 3D model of the material frame, adjust the appropriate position, and facilitate the physical limitation of the drop range of the workpiece. The camera setting unit is used for adjustment. The camera internal parameters and external parameters are guaranteed to be consistent with the camera parameters of the real scene, thereby generating an effective data set.

The workpiece pose acquisition device is used to record the workpiece pose information at that time after the random drop of the workpiece is completed.

A workpiece visualization degree analysis device is used to calculate and analyze the visualization degree of each workpiece in the simulation scene.

Among them, the workpiece visualization degree analysis device mainly includes a color pixel acquisition unit and a visualization degree calculation unit. The color pixel acquisition unit is used to count the area value of each segmented instance color pixel in the scene, and the visualization degree calculation unit uses the color pixel acquisition unit. The provided statistical value is used to calculate the visualization degree and output the visualization degree value.

The data set label integration device is used to integrate information such as image information, workpiece pose information, and visualization degree, and make it into a data set label.

Therefore, in some embodiments, step S3 may specifically include the following steps:

The point cloud area is imported into a pose detection model to perform pose detection of the target workpiece, and the pose information of the target workpiece is acquired.

In some embodiments, the training method of the pose detection model may include the following steps:

A 3D training data set is provided, and the 3D training data set includes at least 3D training images and their corresponding workpiece pose labels and visualization degree labels.

An initial pose detection model is constructed, and based on the 3D training data set, the pose detection initial model is trained to obtain the pose detection model.

In some embodiments, the 3D training dataset is simulated by a simulation dataset generation system.

In some embodiments, the simulation generation may specifically include the following steps:

A simulation scene is constructed, and it is assumed that there are n virtual workpieces in the simulation scene.

Based on the domain randomization method, i virtual workpieces are randomly dropped from a selected position in the working area of the simulation scene, and different color information is assigned to different virtual workpieces, wherein i is iterated incrementally from zero. For example, it is incremented from 0 to +1.

The depth image and rgb image in the scene are collected and saved based on the simulated vision sensor.

Record and save the pose information of each virtual workpiece dropped in the simulation scene as the workpiece pose label.

Based on the acquired depth image and/or rgb image, a visualization degree analysis is performed on each virtual workpiece, and the visualization degree data is recorded as the visualization degree label.

When iterating until the integer i is not less than n, the virtual workpiece is stopped from falling, and the 3D training data set is generated based on the workpiece pose label and the visualization degree label.

In some embodiments, the simulation dataset generation system may include:

The image acquisition device is used for acquiring the 3D training image of the virtual workpiece in the simulation scene.

The area limiting device is used for obtaining the size of the field of view according to the 3D training image, and limiting the falling range of the virtual workpiece.

The workpiece pose acquisition device is used to record the pose information of the target workpiece at that time after the random drop action of the virtual workpiece is completed.

A workpiece visualization degree analysis device is used to calculate and analyze the visualization degree of each virtual workpiece in the simulation scene.

The data set label integration device is used for integrating 3D training images, workpiece pose labels, and visualization degree labels to generate the 3D training data set.

Based on the above method, another embodiment of the present invention also provides a workpiece recognition and pose detection system, which includes:

The image acquisition module is used to collect 2D images and 3D point cloud images in the scene to be recognized.

an area acquisition module, used for identifying the target workpiece in the scene to be identified based on the 2D image, and based on the mapping relationship between the 2D image and the 3D point cloud image, for the target workpiece in the 3D point cloud image Instance segmentation is performed in the region where it is located, and the point cloud region corresponding to the target workpiece is obtained.

The pose acquisition module is used to perform pose detection in the point cloud area based on a deep learning algorithm, and obtain pose information of the target workpiece.

Similarly, an embodiment of the present invention also provides an electronic device that may be applied to the above-mentioned system, which includes a processor and a memory, and the memory stores a computer program, and the computer program is executed to perform the above-mentioned workpiece recognition and pose detection. steps of the method.

Meanwhile, an embodiment of the present invention also provides a readable storage medium, in which a computer program is stored, and when the computer program is run, the steps of the above method for workpiece identification and pose detection are executed.

The above embodiment provides a method and system for workpiece recognition and pose detection, and a simulation data set generation system to which the method and system are applied. As a further application of the above method and system, another embodiment of the present invention also provides a robotic arm The grab control method includes the following steps:

The target workpiece in the scene to be recognized and its pose information are acquired based on the workpiece identification and pose detection method in any of the above embodiments.

The target workpiece to be grasped is selected, and the robotic arm is controlled to perform grasping action based on the pose information.

That is: to obtain the position and attitude information of the workpiece, and control the robotic arm to perform the grasping operation.

It should be noted that the main technical idea of the present invention is how to efficiently and accurately obtain the pose information of the workpiece. As for how to plan the path and/or motion of the robotic arm according to the pose information, it is not the focus of the present invention. The technical solutions have been found in many existing technologies, and those skilled in the art can carry out combination or adaptive research and development without obstacles. The control method should all fall within the protection scope of the present invention.

The method for workpiece identification and pose detection and the applied deep learning training simulation data set generation system disclosed by the present invention avoids the problem of cross-modal data feature extraction and matching in the grabbing scene where small workpieces are scattered and stacked. , while avoiding overly complex data processing calculations. By combining 2D images and 3D point cloud data, it provides an optimized solution for the application scenario of workpiece stack recognition and grasping, in the direction of effectively improving the recognition efficiency and improving the grasping efficiency, and at the same time, it avoids the manual labeling of conventional samples. , automatically make and generate training data sets, which greatly improves work efficiency.

Although the present invention has been described with reference to illustrative embodiments, those skilled in the art will understand that various other changes, omissions and/or additions and the like may be made without departing from the spirit and scope of the invention Effects replace elements of the described embodiments. In addition, many modifications may be made to adapt a particular situation or material to the teachings of the invention without departing from its scope. Therefore, it is not intended herein to limit the invention to the particular embodiments disclosed for carrying out the invention, but it is intended that this invention include all embodiments falling within the scope of the appended claims.

It should be understood that the above-mentioned embodiments are only intended to illustrate the technical concept and characteristics of the present invention, and the purpose thereof is to enable those who are familiar with the art to understand the content of the present invention and implement it accordingly, and cannot limit the protection scope of the present invention. All equivalent changes or modifications made according to the spirit of the present invention should be included within the protection scope of the present invention.

Claims (10)

1. a workpiece recognition and pose detection method, is characterized in that comprising: S1, collect 2D images and 3D point cloud images in the scene to be recognized; S2, identify the target workpiece in the scene to be identified based on the 2D image, and based on the mapping relationship between the 2D image and the 3D point cloud image, perform an example on the area where the target workpiece is located in the 3D point cloud image Segmentation to obtain the point cloud area corresponding to the target workpiece; S3, based on a deep learning algorithm, perform pose detection in the point cloud area, and obtain pose information of the target workpiece. 2. workpiece identification and pose detection method according to claim 1, is characterized in that, step S1 specifically comprises: Use a 2D camera to obtain the 2D image in the to-be-identified scene, and use a 3D camera to obtain the 3D image information in the to-be-identified scene; Preferably, the to-be-identified scene includes a workpiece stacking scene, more preferably a workpiece stacking scene in an orderly manner. 3. The workpiece identification and pose detection method according to claim 1, wherein step S2 specifically comprises: The 2D image is imported into the target segmentation model for identification, and the regional position of the target workpiece is obtained; The region position is mapped to the corresponding region in the 3D point cloud image, and instance segmentation is performed on the region where the target workpiece is located in the 3D point cloud image. 4. workpiece identification and pose detection method according to claim 1, is characterized in that, the training method of described target segmentation model comprises: Provide a 2D training data set, the 2D training data set includes a plurality of 2D images for training and their corresponding marking information, the marking information at least indicates the contour enclosed area of the workpiece in the 2D image; An initial target segmentation model is constructed, and based on the 2D training data set, the initial target segmentation model is trained to obtain the target segmentation model. 5. The workpiece identification and pose detection method according to claim 1, wherein step S3 specifically comprises: The point cloud area is imported into a pose detection model to perform pose detection of the target workpiece, and the pose information of the target workpiece is acquired. 6. The workpiece identification and pose detection method according to claim 5, wherein the training method of the pose detection model comprises: Provide a 3D training data set, the 3D training data set includes at least 3D training images and their corresponding workpiece pose labels and visualization degree labels; An initial pose detection model is constructed, and based on the 3D training data set, the pose detection initial model is trained to obtain the pose detection model. 7. The workpiece identification and pose detection method according to claim 6, wherein the 3D training data set is simulated and generated by a simulation data set generation system; Preferably, the simulation generation specifically includes: constructing a simulation scene, and setting that there are n virtual artifacts in the simulation scene; Based on the domain randomization method, i virtual workpieces are randomly dropped from the selected position in the working area of the simulation scene, and different color information is given to different virtual workpieces; Collect and save the depth image and rgb image in the scene based on the simulated vision sensor; Record and save the pose information of each virtual workpiece dropped in the simulation scene, as the workpiece pose label; Based on the depth image and/or the rgb image, perform visualization degree analysis on each virtual workpiece, and record the visualization degree data as the visualization degree label; When iterating until the integer i is not less than n, the virtual workpiece is stopped from falling, and the 3D training data set is generated based on the workpiece pose label and the visualization degree label. 8. The workpiece identification and pose detection method according to claim 7, wherein the simulation data set generation system comprises: an image acquisition device for acquiring a 3D training image of a virtual workpiece in a simulation scene; an area limiting device, used for obtaining the size of the field of view according to the 3D training image, and limiting the falling range of the virtual workpiece; The workpiece pose acquisition device is used to record the pose information of the target workpiece at that time after the random drop action of the virtual workpiece is completed; A workpiece visualization degree analysis device, used for calculating and analyzing the visualization degree of each virtual workpiece in the simulation scene; The data set label integration device is used for integrating 3D training images, workpiece pose labels, and visualization degree labels to generate the 3D training data set. 9. A workpiece identification and pose detection system is characterized in that comprising: The image acquisition module is used to collect 2D images and 3D point cloud images in the scene to be recognized; an area acquisition module, used for identifying the target workpiece in the scene to be identified based on the 2D image, and based on the mapping relationship between the 2D image and the 3D point cloud image, for the target workpiece in the 3D point cloud image Instance segmentation is performed in the area where it is located, and the point cloud area corresponding to the target workpiece is obtained; The pose acquisition module is used to perform pose detection in the point cloud area based on a deep learning algorithm, and obtain pose information of the target workpiece. 10. A grasping control method for a robotic arm, comprising: Obtain the target workpiece and its pose information in the scene to be recognized based on the workpiece identification and pose detection method according to any one of claims 1-8; The target workpiece to be grasped is selected, and the robotic arm is controlled to perform grasping action based on the pose information.

Images