CN117444990A

CN117444990A - Mechanical arm injection control method and system based on 3D modeling

Info

Publication number: CN117444990A
Application number: CN202311791272.3A
Authority: CN
Inventors: 张良; 凌兴鑫; 吴达; 文海
Original assignee: Shenzhen Pulang Medical Technology Development Co ltd
Current assignee: Shenzhen Pulang Medical Technology Co ltd
Priority date: 2023-12-25
Filing date: 2023-12-25
Publication date: 2024-01-26
Anticipated expiration: 2043-12-25
Also published as: CN117444990B

Abstract

The application relates to the technical field of 3D modeling, and discloses a mechanical arm injection control method and system based on 3D modeling. The method comprises the following steps: acquiring a plurality of continuous mechanical arm injection images of a target mechanical arm injector; carrying out mechanical arm injection recognition through a mechanical arm injection recognition model, and outputting a plurality of mechanical arm injection characteristic distribution graphs; constructing an initial mechanical arm injection finite element model; performing mechanical arm injection center calculation, generating a plurality of target center calculation results, and performing finite element model fusion to obtain a target mechanical arm injection finite element model; calculating a first mechanical arm injection control parameter of the target mechanical arm injector according to the calculation results of the plurality of target centers; and performing simulation test and control parameter correction on the target mechanical arm injection finite element model to obtain corresponding second mechanical arm injection control parameters.

Description

Mechanical arm injection control method and system based on 3D modeling

Technical Field

The application relates to the technical field of 3D modeling, in particular to a mechanical arm injection control method and system based on 3D modeling.

Background

In recent years, researchers have conducted extensive research on a mechanical arm injection control method. The method comprises the steps of carrying out injection treatment and analysis on the mechanical arm by means of a 3D modeling technology and a deep learning algorithm, so that the mechanical arm is accurately controlled.

However, conventional methods often rely on manual programming and experience, which can be difficult to handle for different environments and tasks, resulting in insufficient accuracy and stability. The control of the prior art mechanical arm injector is affected by a number of parameters and factors, such as vibrations, deformations, etc., which lead to a reduced control accuracy.

Disclosure of Invention

The application provides a mechanical arm injection control method and system based on 3D modeling, which are used for improving the precision of mechanical arm injection control by adopting a 3D modeling technology.

In a first aspect, the present application provides a robotic arm injection control method based on 3D modeling, where the robotic arm injection control method based on 3D modeling includes:

acquiring mechanical arm injection video data of a target mechanical arm injector, and performing video segmentation and continuous image frame processing on the mechanical arm injection video data to obtain a plurality of continuous mechanical arm injection images;

inputting the continuous mechanical arm injection images into a preset mechanical arm injection recognition model to perform mechanical arm injection recognition, and outputting a plurality of mechanical arm injection characteristic distribution graphs;

Respectively carrying out 3D modeling on the plurality of mechanical arm injection characteristic distribution graphs to construct an initial mechanical arm injection finite element model corresponding to each mechanical arm injection characteristic distribution graph;

performing mechanical arm injection center calculation on each initial mechanical arm injection finite element model respectively, generating a plurality of target center calculation results, and performing finite element model fusion on the initial mechanical arm injection finite element models according to the plurality of target center calculation results to obtain target mechanical arm injection finite element models;

calculating a first mechanical arm injection control parameter of the target mechanical arm injector according to the calculation results of the plurality of target centers;

and performing simulation test and control parameter correction on the target mechanical arm injection finite element model to obtain corresponding second mechanical arm injection control parameters.

In a second aspect, the present application provides a robotic arm injection control system based on 3D modeling, the robotic arm injection control system based on 3D modeling comprising:

the acquisition module is used for acquiring the mechanical arm injection video data of the target mechanical arm injector, and carrying out video segmentation and continuous image frame processing on the mechanical arm injection video data to obtain a plurality of continuous mechanical arm injection images;

The identification module is used for inputting the continuous mechanical arm injection images into a preset mechanical arm injection identification model to carry out mechanical arm injection identification and outputting a plurality of mechanical arm injection characteristic distribution diagrams;

the modeling module is used for respectively carrying out 3D modeling on the plurality of mechanical arm injection characteristic distribution graphs and constructing an initial mechanical arm injection finite element model corresponding to each mechanical arm injection characteristic distribution graph;

the fusion module is used for carrying out mechanical arm injection center calculation on each initial mechanical arm injection finite element model respectively, generating a plurality of target center calculation results, and carrying out finite element model fusion on the initial mechanical arm injection finite element models according to the plurality of target center calculation results to obtain target mechanical arm injection finite element models;

the calculation module is used for calculating a first mechanical arm injection control parameter of the target mechanical arm injector according to the calculation results of the plurality of target centers;

and the correction module is used for carrying out simulation test and control parameter correction on the target mechanical arm injection finite element model to obtain corresponding second mechanical arm injection control parameters.

According to the technical scheme, the method can realize high-precision control of the mechanical arm injector by collecting the continuous mechanical arm injection images and performing deep learning processing. The deep learning model can accurately identify the injection characteristic distribution diagram of the mechanical arm, so that the real-time monitoring and adjustment of the injection process are realized, and the injection accuracy is improved. The mechanical arm injection recognition model and the finite element model are utilized, so that different environments and task requirements can be met. The control parameters can be automatically adjusted according to different injector characteristics and working conditions, so that the dependence on manual programming and manual intervention is reduced, and the system is more flexible and has stronger adaptability. By performing simulation test and parameter correction on the target mechanical arm injection finite element model, the method can optimize mechanical arm injection control parameters. This helps to improve stability and reliability of the system, reduces errors and uncertainty in the actual injection process, and ensures the success rate of injection. By using the deep learning technology, the method can intelligently model and identify the characteristic distribution of the mechanical arm injector, thereby realizing automatic control. This makes arm injection process more intelligent and high-efficient, has reduced manual operation's risk and complexity, therefore, the application adopts 3D modeling technique to improve the precision of arm injection control.

Drawings

In order to more clearly illustrate the technical solutions of the embodiments of the present invention, the drawings required for the description of the embodiments will be briefly described below, and it is obvious that the drawings in the following description are some embodiments of the present invention, and other drawings may be obtained based on these drawings without inventive effort for a person skilled in the art.

FIG. 1 is a schematic diagram of one embodiment of a robotic arm injection control method based on 3D modeling in an embodiment of the present application;

FIG. 2 is a schematic diagram of one embodiment of a robotic arm injection control system based on 3D modeling in an embodiment of the present application.

Detailed Description

The embodiment of the application provides a mechanical arm injection control method and system based on 3D modeling. The terms "first," "second," "third," "fourth" and the like in the description and in the claims of this application and in the above-described figures, if any, are used for distinguishing between similar objects and not necessarily for describing a particular sequential or chronological order. It is to be understood that the data so used may be interchanged where appropriate such that the embodiments described herein may be implemented in other sequences than those illustrated or otherwise described herein. Furthermore, the terms "comprises," "comprising," or any other variation thereof, are intended to cover a non-exclusive inclusion, such that a process, method, system, article, or apparatus that comprises a list of steps or elements is not necessarily limited to those steps or elements expressly listed or inherent to such process, method, article, or apparatus.

For ease of understanding, a specific flow of an embodiment of the present application is described below, referring to fig. 1, and an embodiment of a method for controlling injection of a robotic arm based on 3D modeling in an embodiment of the present application includes:

s101, acquiring mechanical arm injection video data of a target mechanical arm injector, and performing video segmentation and continuous image frame processing on the mechanical arm injection video data to obtain a plurality of continuous mechanical arm injection images;

it can be appreciated that the execution subject of the present application may be a robotic arm injection control system based on 3D modeling, and may also be a terminal or a server, which is not limited herein. The embodiment of the present application will be described by taking a server as an execution body.

In particular, the deployment of high resolution and high frame rate cameras is arranged around the robotic arm injector, ensuring that each detail of the injection process is captured from multiple angles, while the cameras need to have good dynamic range and low light performance to accommodate different lighting conditions and accurately record the dynamic changes of the robotic arm. In the video acquisition link, the definition and the continuity of the image are ensured, the influence caused by camera shake or ambient light change is avoided, and the recording effect can be optimized by using a stable bracket and an automatic exposure control technology. The acquired video data then needs to undergo a specialized video processing flow, including video segmentation and processing of successive image frames. Video segmentation is the breaking up of long-time video recordings into a series of short-duration segments, which are typically based on the start and stop of the robot arm motion, with the segmentation points being determined by automatically detecting the critical motion of the robot arm, such as the start and end of an injection. This step may utilize a machine learning algorithm, such as a deep learning based motion recognition model, to automatically recognize and label key motions during the injection process, thereby achieving accurate segmentation. Successive image frames are extracted from each video segment for subsequent analysis. This step requires an efficient frame extraction algorithm to ensure that each frame extracted from the video accurately reflects the state of the robotic arm at a particular moment. To ensure the quality of the image frames, an inter-frame interpolation technique may be employed to increase the frame rate so that every minute motion of the robotic arm is captured and recorded. Meanwhile, the extracted image frames are subjected to noise reduction and enhancement processing through image processing algorithms such as sharpening, contrast adjustment and the like, so that each image frame is clearer, and subsequent analysis and processing are facilitated.

Step S102, inputting a plurality of continuous mechanical arm injection images into a preset mechanical arm injection recognition model to perform mechanical arm injection recognition, and outputting a plurality of mechanical arm injection characteristic distribution diagrams;

specifically, a plurality of continuous mechanical arm injection images are input into a preset mechanical arm injection recognition model, and the model is constructed based on a deep learning technology and comprises a convolution layer, a deconvolution layer, a grid dividing layer and a distribution feature extraction layer. The convolution layer extracts important features in the image, such as edges, textures, etc., while the deconvolution layer is used to restore these features to a form closer to the original image, which helps to more accurately identify the dynamics of the robotic arm. Through the encoding and decoding operations of these layers, the model is able to generate manipulator injection bounding boxes corresponding to each successive manipulator injection image, which accurately mark the position and shape of the manipulator. The grid division layer in the model performs grid division on each image according to the mechanical arm injection bounding box. Each image is divided into a number of small areas, each of which contains a part of the information of the robot arm. This meshing helps to analyze the pose and motion of the robotic arm more carefully, as it allows the model to focus on specific parts of the robotic arm, thereby improving the accuracy of overall recognition. The distribution feature extraction layer extracts the distribution features of the mechanical arm injection from the grid image, wherein the distribution features comprise the position, the motion track, the action mode and the like of the mechanical arm. Through careful analysis and processing of these features, the model is able to generate a series of mechanical arm injection feature profiles. These profiles not only show the specific state of the robotic arm in each image frame, but also provide an understanding of the dynamics of the robotic arm.

Step S103, respectively carrying out 3D modeling on a plurality of mechanical arm injection characteristic distribution graphs, and constructing an initial mechanical arm injection finite element model corresponding to each mechanical arm injection characteristic distribution graph;

specifically, each mechanical arm injection characteristic distribution map is converted into a target three-dimensional model. By 3D modeling software such as SolidWorks or AutoCAD, an accurate three-dimensional model is created from the two-dimensional feature profile. In the process, the proportion, the position and the interrelationship of each part of the mechanical arm are required to be paid attention to, so that the three-dimensional model can truly reflect the structure and the dynamic characteristics of the mechanical arm. And carrying out grid discretization processing on each target three-dimensional model. Grid discretization is the conversion of a continuous three-dimensional model into a grid structure consisting of a finite number of small cells. By this process, complex three-dimensional models can be reduced to a form that is easy to calculate and analyze, while retaining sufficient detail to perform accurate simulation. Grid discretization requires the selection of appropriate grid sizes and shapes to balance the relationship between computational efficiency and simulation accuracy. Too large grid cells may result in loss of detail, while too small grid cells may result in too great a calculation. Grid optimization is performed to improve the computational efficiency of the model and the accuracy of the results, which generally involves adjusting the size, shape and distribution of the grid cells, particularly in critical areas of the model, such as joints and active sites of the robotic arm. Through optimization, the model can more accurately reflect the physical characteristics and behaviors of the mechanical arm while the calculation efficiency is maintained. And finally, generating an initial mechanical arm injection finite element model corresponding to each mechanical arm injection characteristic distribution map. The models not only show the structure and the dynamics of the mechanical arm in detail, but also provide a foundation for subsequent mechanical analysis and control algorithm design.

Step S104, respectively carrying out mechanical arm injection center calculation on each initial mechanical arm injection finite element model to generate a plurality of target center calculation results, and carrying out finite element model fusion on the initial mechanical arm injection finite element models according to the plurality of target center calculation results to obtain target mechanical arm injection finite element models;

specifically, each initial mechanical arm injection finite element model is input into a preset mechanical arm injection center calculation function. The purpose of this function is to determine the central coordinates of the robotic arm when performing the injection task, which involves an accurate analysis of the geometry and dynamics of the robotic arm. By this calculation, the precise position and orientation of each manipulator model at the time of the injection action can be obtained. And identifying the mechanical arm injection type of each initial model. Different injection actions are classified based on structural features and modes of operation of the robotic arm. For example, different types of injections involve different speeds, forces, or angles. By identifying these categories, the specific requirements and constraints of each model can be more accurately understood, making more appropriate adjustments in model fusion. And generating a target center calculation result according to the center coordinates and the injection category of each model. These results provide a comprehensive data set for each model, including information on location, orientation, and type of operation. These data help to understand the performance of the robotic arm in different injection tasks and are the basis for performing efficient fusion. And then, based on a target center calculation result, calculating the finite element fusion parameter data of each initial mechanical arm injection finite element model respectively. These parameter data are key to the tuning and optimization of each model and determine how to effectively combine the different models into a unified whole. This step needs to take into account the differences and interactions between the individual models, ensuring that the key characteristics of each model are properly considered and preserved during the fusion process. And fusing the initial manipulator injection finite element model according to the finite element fusion parameter data, so as to obtain the target manipulator injection finite element model. This fusion process takes into account not only the physical and geometric characteristics of the individual models, but also their dynamic behavior in performing the injection task. By means of the fusion, a comprehensive mechanical arm model capable of representing various injection tasks can be obtained.

Step S105, calculating a first mechanical arm injection control parameter of the target mechanical arm injector according to the calculation results of the plurality of target centers;

specifically, according to the preset variable distribution characteristics and the center coordinates in the target center calculation result, a plurality of initial control parameter values of the target mechanical arm injector are calculated respectively. And analyzing the motion characteristics of the mechanical arm under different operation situations, wherein the motion characteristics comprise key parameters such as the moving speed, the acceleration, the rotation angle and the like of the mechanical arm. From this analysis, it is possible to determine the basic law of motion that the robotic arm should follow when performing a particular injection task. And carrying out average value operation on the initial control parameter values to obtain a unified target control parameter value. By balancing the control parameters under different operating scenarios, it is ensured that the robotic arm can exhibit stable and consistent performance in different injection tasks. By calculating the average value of all the initial control parameter values, a control parameter set which can reflect various operation situations and is universal enough can be obtained, and the design of the mechanical arm control system with strong universality is facilitated. And (3) performing control weight calculation according to the mechanical arm injection category in the calculation result of the target center, and further optimizing control parameters. Different injection categories require different operational precision and effort, and thus each category is assigned a different control weight. The weight of each control parameter can be adjusted according to the specific requirements of the injection category, such as injection speed, strength, precision and the like, so as to better adapt to different types of injection tasks. And carrying out weighted analysis on the target control parameter values according to the control weight data to obtain weighted control parameter values. The control parameters are adjusted by comprehensively considering the specific requirements of different injection types, so that the control parameters are more accurately adapted to different operation conditions. The weighted analysis is helpful to improve the precision and efficiency of the mechanical arm injection, and ensures that the optimal performance can be achieved in practical application.

And S106, performing simulation test and control parameter correction on the target mechanical arm injection finite element model to obtain corresponding second mechanical arm injection control parameters.

Specifically, simulation test is carried out on the target mechanical arm injection finite element model, and the performance of the constructed mechanical arm injection finite element model is verified and evaluated. The target simulation test data is generated by simulating actual operating conditions, such as movement of the mechanical arm, injection action and the like. The data comprising the behavior of the model under various operating scenarios is key to assessing the accuracy and practicality of the model. And extracting features of the target simulation test data to obtain key information capable of representing the performance of the model. This includes various parameters such as the movement trajectory, speed, accuracy, etc. of the robotic arm. After extracting these features, they are vector coded so that they can be efficiently processed by the robotic injection analysis model. Vector coding is the conversion of these complex features into a standardized numerical form that enables them to be identified and analyzed by subsequent machine learning models. These simulated test encoded vectors are input into a pre-set robotic injection analysis model that includes an encoding network and a decoding network. In the coding network, a bidirectional LSTM (long-short-term memory) unit is used to extract test features, which is an efficient sequential data processing method capable of extracting key time-series features from the input coded vector. The bidirectional LSTM unit can capture the dependency relationship of the front and back time points, so that the dynamic characteristics of the model performance can be accurately identified. And inputting the extracted simulation test feature vector into a unidirectional LSTM unit of a decoding network to perform feature decoding. And the complex feature vectors are converted into more visual and easier to understand simulation test evaluation indexes. These evaluation indexes include comprehensive evaluation of the model performance, such as accuracy, stability, response speed, and the like. And according to the simulation test evaluation indexes, carrying out control parameter correction on the first mechanical arm injection control parameters of the target mechanical arm injector. And adjusting and optimizing control parameters according to the performance of the model in the simulation test so as to ensure that the mechanical arm can achieve the optimal performance in actual operation. Control parameter correction involves adjusting various aspects of the movement speed, acceleration, rotation angle, etc. of the robotic arm in order to ensure the accuracy and efficiency of the injection action. By this correction, a more accurate and reliable second robot arm injection control parameter can be obtained, thereby improving the performance and reliability of the whole system.

And calculating a loss value of the simulation test evaluation index, and judging whether the loss value exceeds a preset threshold value. This determination is made to evaluate whether the performance of the current arm control parameters meets a predetermined criterion, and the loss value represents a deviation between the performance of the arm in the simulation test and the ideal state. If the loss value exceeds a preset threshold, the current mechanical arm control parameters need to be further optimized, and a preset genetic optimization algorithm is adopted to adjust the parameters. The genetic optimization algorithm is an efficient searching method, and can quickly find the optimal solution in a wide solution space. An initialized manipulator injection control parameter population is created from the first manipulator injection control parameters, the population comprising a plurality of first candidate injection control parameters, each parameter representing a solution. And then, setting the iteration times of the genetic optimization algorithm according to the loss value, and determining the length of the optimization process to ensure that the optimal solution can be found in a reasonable time. In the iterative optimization process, a plurality of first candidate injection control parameters are subjected to steps of selection, crossing, mutation and the like, and a plurality of second candidate injection control parameters are gradually generated. Each iteration is to screen out parameter combinations with higher fitness in the process of simulating natural selection. And calculating the fitness value of each second candidate injection control parameter, and selecting an optimized parameter combination according to the fitness values to finally obtain the corresponding second mechanical arm injection control parameters. The series of operations ensure the high efficiency and accuracy of the parameter optimization process, and the performance of the mechanical arm can be obviously improved. If the loss value does not exceed the preset threshold, this indicates that the current robot arm control parameters are relatively satisfactory, but there is room for further optimization. At this time, control feedback data target values corresponding to the first arm injection control parameters are searched, and a second arm injection control parameter of the target arm injector is generated according to the target values. The optimization in this case is mainly fine tuning, ensuring that the control parameters are more tailored to the actual operating requirements, while maintaining the existing performance criteria.

In the embodiment of the application, the method can realize high-precision control of the mechanical arm injector by collecting the continuous mechanical arm injection images and performing deep learning processing. The deep learning model can accurately identify the injection characteristic distribution diagram of the mechanical arm, so that the real-time monitoring and adjustment of the injection process are realized, and the injection accuracy is improved. The mechanical arm injection recognition model and the finite element model are utilized, so that different environments and task requirements can be met. The control parameters can be automatically adjusted according to different injector characteristics and working conditions, so that the dependence on manual programming and manual intervention is reduced, and the system is more flexible and has stronger adaptability. By performing simulation test and parameter correction on the target mechanical arm injection finite element model, the method can optimize mechanical arm injection control parameters. This helps to improve stability and reliability of the system, reduces errors and uncertainty in the actual injection process, and ensures the success rate of injection. By using the deep learning technology, the method can intelligently model and identify the characteristic distribution of the mechanical arm injector, thereby realizing automatic control. This makes arm injection process more intelligent and high-efficient, has reduced manual operation's risk and complexity, therefore, the application adopts 3D modeling technique to improve the precision of arm injection control.

In a specific embodiment, the process of executing step S102 may specifically include the following steps:

(1) Inputting a plurality of continuous mechanical arm injection images into a preset mechanical arm injection recognition model, wherein the mechanical arm injection recognition model comprises a convolution layer, a deconvolution layer, a grid dividing layer and a distribution feature extraction layer;

(2) Coding and decoding a plurality of continuous mechanical arm injection images through a convolution layer and a deconvolution layer in the mechanical arm injection recognition model respectively to obtain a mechanical arm injection boundary frame corresponding to each continuous mechanical arm injection image;

(3) Performing image grid division on each continuous mechanical arm injection image according to the mechanical arm injection boundary frame through grid division layers in the mechanical arm injection recognition model to obtain mechanical arm injection grid images of each continuous mechanical arm injection image;

(4) And carrying out mechanical arm injection distribution identification on the mechanical arm injection grid image through a distribution feature extraction layer in the mechanical arm injection identification model to generate a plurality of mechanical arm injection feature distribution graphs.

Specifically, a plurality of continuous mechanical arm injection images are input into a preset mechanical arm injection recognition model, and image processing and feature extraction are performed through a plurality of layers of the model. The mechanical arm injection recognition model is a complex deep learning network and comprises a convolution layer, a deconvolution layer, a grid dividing layer and a distributed feature extraction layer. Basic visual features such as edges, textures, shapes, etc. are extracted from the input sequential mechanical arm injection image by the convolution layer. The convolution operation slides over the image through a filter (or convolution kernel), extracts features of the local region, and generates a feature map. As the network deepens, the convolution layer is able to capture more abstract and complex features. For example, when analyzing the mechanical arm injection image, the primary convolution layer only recognizes the contour of the mechanical arm, while the deeper convolution layer recognizes the specific components and pose of the mechanical arm. The deconvolution layer maps the extracted features back into higher resolution space by upsampling the feature maps to facilitate subsequent accurate positioning. This allows the model to accurately identify and locate the robotic arm injection site in the image. Through these convolution and deconvolution operations, the model can obtain the manipulator injection bounding boxes corresponding to each successive manipulator injection image, which are the exact positions and ranges of the manipulator and injection site in the image. The grid division layer further subdivides the image areas within these bounding boxes into grids. Such partitioning enables the model to analyze the image at a finer granularity, capturing specific movements and changes of the robotic arm in different grids. For example, if the robotic arm rotates or expands during injection, these actions may be manifested in different grids. Through the grid division, the model can not only identify the whole position of the mechanical arm, but also understand the specific dynamic state of the mechanical arm in a local area. The distribution feature extraction layer extracts the distribution features of the mechanical arm injection from the grid images. The layer carries out deep analysis on the images after grid division, and identifies the injection action modes and characteristics of the mechanical arm in different grids, such as injection speed, injection force, injection angle and the like. Through the analysis, the model can generate a plurality of mechanical arm injection characteristic distribution diagrams, and the diagrams not only show the spatial distribution of the mechanical arm, but also reflect the specific behavior mode of the mechanical arm when the mechanical arm performs an injection task.

In a specific embodiment, the process of executing step S103 may specifically include the following steps:

(1) 3D modeling is conducted on the injection characteristic distribution diagrams of the plurality of mechanical arms respectively, and a target three-dimensional model corresponding to the injection characteristic distribution diagram of each mechanical arm is obtained;

(2) Performing grid discretization processing on the target three-dimensional model corresponding to the injection characteristic distribution map of each mechanical arm to obtain a target discrete grid corresponding to the injection characteristic distribution map of each mechanical arm;

(3) And carrying out grid optimization on the target discrete grid corresponding to the injection characteristic distribution map of each mechanical arm, and generating an initial mechanical arm injection finite element model corresponding to the injection characteristic distribution map of each mechanical arm.

Specifically, 3D modeling is performed on the plurality of mechanical arm injection feature profiles, respectively. Each mechanical arm injection characteristic distribution map contains key visual information of the mechanical arm in the injection process, such as position, gesture, movement track and the like. This information is converted into three-dimensional data by image processing techniques. In this process, a stereo matching technique similar to that in computer vision can be used to reconstruct the three-dimensional structure of the robotic arm by analyzing the feature profiles for multiple perspectives. For example, if there are a series of images of the robotic arm taken from different angles, the shape and position of the robotic arm in three-dimensional space may be calculated using a three-dimensional reconstruction algorithm by analyzing the relative positions and orientations of the various portions of the robotic arm in these images. And carrying out grid discretization processing on each target three-dimensional model. Grid discretization refers to the conversion of a continuous three-dimensional model into a model consisting of a series of small, discrete grid cells. This process typically involves defining the size and shape of the grid cells and how the cells are distributed throughout the model. For example, the articulating and moving portions of the robotic arm require finer mesh to capture details, while other portions may use larger meshes. And optimizing each discrete grid, so that the calculation efficiency and the analysis precision of the model are improved. Grid optimization refers to adjusting the layout of grid cells to reduce errors in computation and improve accuracy of the results. This means that a higher density grid is used at critical locations of the robotic arm (e.g. near the injection site) while a lower density grid is used at less critical locations. Through the optimization, the model is accurate and efficient in analysis of mechanical behaviors in the mechanical arm injection process. For example, a high density grid of critical locations may help to accurately calculate the pressure distribution of the injection against the skin, while a lower density grid of other locations is sufficient to analyze the overall motion of the robotic arm. By combining the steps, a finite element model which can accurately simulate the actual behavior of the mechanical arm and can efficiently carry out numerical analysis is obtained. This model may be used for further arm control algorithm development or for simulating and optimizing the actual injection operation of the arm.

In a specific embodiment, the process of executing step S104 may specifically include the following steps:

(1) Inputting each initial mechanical arm injection finite element model into a preset mechanical arm injection center calculation function to calculate the mechanical arm injection center, so as to obtain center coordinates corresponding to each initial mechanical arm injection finite element model;

(2) Respectively carrying out mechanical arm injection type identification on each initial mechanical arm injection finite element model to obtain mechanical arm injection types;

(3) Generating a target center calculation result according to the center coordinates corresponding to each initial mechanical arm injection finite element model and the mechanical arm injection category, and obtaining a plurality of target center calculation results;

(4) Respectively calculating the finite element fusion parameter data of each initial mechanical arm injection finite element model according to the calculation results of the plurality of target centers;

(5) And carrying out finite element model fusion on the initial mechanical arm injection finite element model according to the finite element fusion parameter data to obtain a target mechanical arm injection finite element model.

Specifically, each initial mechanical arm injection finite element model is input into a preset mechanical arm injection center calculation function to calculate the mechanical arm injection center, and the center coordinates of each model are determined, wherein the coordinates are geometric center points of the model in space. The calculation of the center coordinates is typically based on geometric properties of the model, such as shape and size. For example, if the model represents an injection robot, the center coordinates of the model are located at the geometric center of the robot or at a specific functional location, such as an injection head. And identifying the mechanical arm injection type of each model. By analyzing specific features of the model, such as shape, size or configuration, it is determined which type of robotic arm injection system it belongs to. For example, different types of robotic arms are specifically designed for different types of medical injection tasks, such as intravenous or intramuscular injection. By identifying these categories, the functional and operational characteristics of each model can be better understood. Based on the center coordinates and injection category of each model, a target center calculation result is generated. And comprehensively considering the central coordinates and the injection type information to generate a result set which comprehensively reflects the characteristics of each model. These result sets include the spatial positioning of the model, the operating range, and specific functional attributes. For example, a robotic arm model for precision small-scale injection would have a target center calculation corresponding to its precision handling capabilities. And calculating the finite element fusion parameter data of each initial mechanical arm injection finite element model, and determining how to fuse different models into a unified finite element model with complete functions. The fusion parameters are calculated taking into account the physical and functional properties of the individual models and how they interact in one integrated model. For example, if there are multiple different types of injection robot arm models, the calculation of the fusion parameters needs to take into account their respective mechanical properties and modes of operation to ensure that the fused models accurately reflect the function of all individual models. And according to the fusion parameter data, performing finite element model fusion on the initial mechanical arm injection finite element model. Multiple models are integrated into a unified model while retaining their respective key characteristics. The fusion process aims to accurately control the fusion mode and degree. For example, if the goal is to create an integrated robotic arm system capable of performing multiple injection tasks, the fusion process needs to ensure that the critical operating characteristics of each individual model are preserved and integrated into the final model. Through the steps, a plurality of different mechanical arm injection finite element models are fused into a comprehensive and functional target mechanical arm injection finite element model. The model not only integrates the characteristics of each individual model, but also provides a more comprehensive and accurate view to understand and analyze the performance of the mechanical arm in various injection tasks.

In a specific embodiment, the process of executing step S105 may specifically include the following steps:

(1) Calculating a plurality of initial control parameter values of the target mechanical arm injector according to preset variable distribution characteristics and center coordinates in a plurality of target center calculation results;

(2) Performing average value operation on a plurality of initial control parameter values to obtain a target control parameter value of a target mechanical arm injector;

(3) Performing control weight calculation on the target control parameter values according to the mechanical arm injection categories in the calculation results of the plurality of target centers to obtain target control weight data, and performing weighted analysis on the target control parameter values according to the target control weight data to obtain weighted control parameter values of the target mechanical arm injector;

(4) And generating a first mechanical arm injection control parameter of the target mechanical arm injector according to the weighted control parameter value.

Specifically, according to preset variable distribution characteristics and center coordinates in a plurality of target center calculation results, a plurality of initial control parameter values of the target mechanical arm injector are calculated respectively. These parameters include the velocity of movement of the robotic arm, acceleration, rotation angle, injection pressure, etc. And the motion characteristics of the mechanical arm under different injection conditions are known by analyzing the performance of the mechanical arm in actual operation. For example, if the target center calculation indicates that the robotic arm requires a quick and accurate motion in a particular injection task, the corresponding initial control parameter will be set to the parameter value that supports such motion. And carrying out average value operation on the plurality of initial control parameter values to determine the target control parameter value of the target mechanical arm injector. The performance of the mechanical arm under different conditions is synthesized to obtain a group of control parameters which can be widely suitable for various operation requirements. For example, if the robotic arm needs to perform injection tasks in different types of medical environments, the control parameters calculated by the averages will satisfy the requirements of various environments, so as to ensure that the robotic arm can work stably under different conditions. And then, according to the mechanical arm injection type in the calculation result of the target center, carrying out control weight calculation on the target control parameter values. The importance of the control parameters is adjusted based on the specific requirements of the robotic arm in the different injection categories. For example, some injection tasks require greater precision, while others are more speed-intensive. By distributing different weights for different types of tasks, the control parameters can be ensured to reflect the actual demands of the mechanical arm in various tasks more accurately. And carrying out weighted analysis on the target control parameter value according to the control weight data to obtain a final weighted control parameter value. The importance and actual performance of the control parameters are combined to generate a set of accurate and practical robot arm control parameters. For example, if a particular injection task requires very fine motion control, the weighted control parameters will emphasize those parameters that are relevant to motion accuracy, such as motion speed and stability, while those parameters that are less relevant to the task are given lower weights.

In a specific embodiment, the process of executing step S106 may specifically include the following steps:

(1) Performing simulation test on the injection finite element model of the target mechanical arm to obtain target simulation test data;

(2) Extracting features of the target simulation test data to obtain a simulation test feature set, and vector encoding the simulation test feature set to obtain a simulation test encoding vector;

(3) Inputting the simulation test coding vector into a preset mechanical arm injection analysis model, wherein the mechanical arm injection analysis model comprises: an encoding network and a decoding network;

(4) Performing test feature extraction on the simulation test coding vector through a bidirectional LSTM unit in the coding network to obtain a simulation test feature vector;

(5) Inputting the simulation test feature vector into a unidirectional LSTM unit in a decoding network to perform feature decoding to obtain a simulation test evaluation index;

(6) And correcting the control parameters of the first mechanical arm injection control parameters of the target mechanical arm injector according to the simulation test evaluation indexes to obtain corresponding second mechanical arm injection control parameters.

Specifically, a simulation test is performed on the injection finite element model of the target mechanical arm, and simulation software is used for simulating the performance of the mechanical arm under the actual working condition. This includes the movement trajectory, speed, precision, injection motion, etc. of the robotic arm. For example, the process of injecting under different conditions, such as different injection angles, forces and speeds, of the robotic arm may be simulated by simulation software. These simulation tests are capable of generating large amounts of data, including performance information of the robotic arm under various operating scenarios, which is the basis for subsequent analysis and optimization. And extracting features of the target simulation test data, and identifying factors with the greatest influence on the performance of the mechanical arm from a large amount of data. The feature extraction process needs to screen out key parameters which can most represent the performance of the mechanical arm, such as smoothness of movement, response speed, precision and the like. After extracting the features, vector encoding is performed on the feature sets, and the features are converted into a digital form so as to facilitate the processing of the machine learning model. For example, the movement speed and accuracy of the robot arm may be converted into numerical data to form a feature vector representing the performance of the robot arm. These feature vectors are input into a preset mechanical arm injection analysis model. This model typically includes an encoding network and a decoding network, and a two-way long short-term memory (LSTM) unit is used in the encoding network. The bi-directional LSTM unit is able to learn information from the front-to-back time series of data, which helps to understand the behavior pattern of the robotic arm at different points in time. The simulated test feature vector is input to unidirectional LSTM cells in the decoding network for decoding. The complex feature vectors are converted into more direct and easier to understand evaluation indexes. These evaluation indexes are direct quantification of the performance of the mechanical arm, such as scoring of injection accuracy or ranking of response speed. And correcting the first mechanical arm injection control parameters of the target mechanical arm injector based on the simulation test evaluation indexes to generate corresponding second mechanical arm injection control parameters. The correction process is based on comprehensive evaluation of the performance of the mechanical arm in simulation test, and aims to adjust key parameters affecting the performance of the mechanical arm so as to be more suitable for actual operation requirements. For example, if the evaluation index shows that the accuracy of the robotic arm in a particular injection task is insufficient, the accuracy thereof may be improved by adjusting control parameters such as the moving speed and the force.

In a specific embodiment, the performing step performs control parameter correction on the first mechanical arm injection control parameter of the target mechanical arm injector according to the simulation test evaluation index, and the process of obtaining the corresponding second mechanical arm injection control parameter may specifically include the following steps:

(1) Calculating a loss value of the simulation test evaluation index, and judging whether the loss value exceeds a preset threshold value;

(2) If the number exceeds the number, creating an initialized mechanical arm injection control parameter population according to the first mechanical arm injection control parameters through a preset genetic optimization algorithm, wherein the initialized mechanical arm injection control parameter population comprises a plurality of first candidate injection control parameters;

(3) Setting iteration times corresponding to a genetic optimization algorithm according to the loss value, and carrying out iterative optimization on a plurality of first candidate injection control parameters according to the iteration times to generate a plurality of second candidate injection control parameters;

(4) Calculating the fitness value of each second candidate injection control parameter respectively, and selecting an optimized parameter combination according to the fitness value to obtain corresponding second mechanical arm injection control parameters;

(5) If the control data target value does not exceed the control data target value, searching for a control feedback data target value corresponding to the first mechanical arm injection control parameter, and generating a second mechanical arm injection control parameter of the target mechanical arm injector according to the control feedback data target value.

Specifically, a loss value of the simulation test evaluation index is calculated. This loss value is the difference between the performance of the evaluation robot arm in the simulation test and the expected target. For example, if the goal is to have the robotic arm reach a certain accuracy and speed when performing an injection, then the loss value will represent the deviation between the actual performance of the robotic arm in the test and these goals. The calculation of the loss value typically involves a series of mathematical and statistical methods, such as Mean Square Error (MSE) or cross entropy loss, etc. It is determined whether this loss value exceeds a preset threshold. The threshold is set in advance according to the performance requirement of the mechanical arm and the actual application scene. If the loss value exceeds this threshold, this means that there is room for improvement in the performance of the robot arm, requiring further optimization. If it is determined that optimization is required, an initialized manipulator injection control parameter population is created from the first manipulator injection control parameters using a preset genetic optimization algorithm. Genetic algorithm is an optimization method imitating natural selection process, and is suitable for solving the optimization problem. This population contains a plurality of first candidate injection control parameters, each of which is a variant of one of the original control parameters. For example, if the original control parameter is a particular speed and effort combination, then the population contains a combination of parameters that differ slightly in speed and effort. Then, the iteration number of the genetic optimization algorithm is set according to the loss value. The number of iterations determines the length of the algorithm run, and in general, a larger penalty requires more iterations to find a good solution. During these iterations, genetic algorithms continually refine the parameter population by selecting, crossing, and mutating, etc. The selection operation retains those parameters that perform well, the crossover operation combines the two parameters to produce new parameters, and the mutation operation randomly alters some parameters to explore more solutions. For example, the server explores different speed and force combinations by crossover and mutation to find parameter settings more suitable for the robotic arm. After each iteration, the server calculates an fitness value for each of the second candidate injection control parameters. The fitness value is a measure of the fitness of each parameter combination to the current optimization objective. The high fitness of the parameter combination means that it is closer to the optimization objective of the server. Based on these fitness values, the algorithm will choose the optimized combination of parameters as the final second mechanical arm injection control parameter. For example, by comparing the performance of different parameter combinations in simulating an injection task, a genetic algorithm can determine which set of parameters is most effective in reducing injection errors and improving efficiency. If the initial loss value does not exceed the preset threshold, this indicates that the current performance of the robotic arm has been relatively satisfactory, but a fine tuning is still required to further optimize performance. In this case, a control feedback data target value corresponding to the first arm injection control parameter may be searched. The control feedback data target values are determined based on the performance of the robot arm in actual operation, reflecting the performance criteria that the robot arm should achieve in the optimal state. Then, a second arm injection control parameter of the target arm injector is generated from these target values. This step involves minor adjustments to the current parameters to ensure that the performance of the robotic arm more accurately matches the needs of the actual operation. For example, if the control feedback data indicates that the robotic arm is slightly biased in a particular injection session, this bias may be corrected by fine tuning the speed or force parameters.

The method for controlling the injection of the mechanical arm based on the 3D modeling in the embodiment of the present application is described above, and the following describes a mechanical arm injection control system based on the 3D modeling in the embodiment of the present application, referring to fig. 2, an embodiment of the mechanical arm injection control system based on the 3D modeling in the embodiment of the present application includes:

the acquisition module 201 is configured to acquire arm injection video data of a target arm injector, and perform video segmentation and continuous image frame processing on the arm injection video data to obtain a plurality of continuous arm injection images;

the recognition module 202 is configured to input the plurality of continuous mechanical arm injection images into a preset mechanical arm injection recognition model to perform mechanical arm injection recognition, and output a plurality of mechanical arm injection feature distribution graphs;

the modeling module 203 is configured to perform 3D modeling on the plurality of mechanical arm injection feature profiles, and construct an initial mechanical arm injection finite element model corresponding to each mechanical arm injection feature profile;

the fusion module 204 is configured to perform mechanical arm injection center calculation on each initial mechanical arm injection finite element model, generate a plurality of target center calculation results, and perform finite element model fusion on the initial mechanical arm injection finite element models according to the plurality of target center calculation results to obtain target mechanical arm injection finite element models;

A calculation module 205, configured to calculate a first mechanical arm injection control parameter of the target mechanical arm injector according to the calculation results of the plurality of target centers;

and the correction module 206 is used for performing simulation test and control parameter correction on the target manipulator injection finite element model to obtain corresponding second manipulator injection control parameters.

Through the cooperation of the components, the method can realize high-precision control of the mechanical arm injector by collecting the injection images of the continuous mechanical arm and performing deep learning processing. The deep learning model can accurately identify the injection characteristic distribution diagram of the mechanical arm, so that the real-time monitoring and adjustment of the injection process are realized, and the injection accuracy is improved. The mechanical arm injection recognition model and the finite element model are utilized, so that different environments and task requirements can be met. The control parameters can be automatically adjusted according to different injector characteristics and working conditions, so that the dependence on manual programming and manual intervention is reduced, and the system is more flexible and has stronger adaptability. By performing simulation test and parameter correction on the target mechanical arm injection finite element model, the method can optimize mechanical arm injection control parameters. This helps to improve stability and reliability of the system, reduces errors and uncertainty in the actual injection process, and ensures the success rate of injection. By using the deep learning technology, the method can intelligently model and identify the characteristic distribution of the mechanical arm injector, thereby realizing automatic control. This makes arm injection process more intelligent and high-efficient, has reduced manual operation's risk and complexity, therefore, the application adopts 3D modeling technique to improve the precision of arm injection control.

It will be clearly understood by those skilled in the art that, for convenience and brevity of description, the specific working processes of the above-described systems, systems and units may refer to the corresponding processes in the foregoing method embodiments, which are not repeated herein.

The integrated units, if implemented in the form of software functional units and sold or used as stand-alone products, may be stored in a computer readable storage medium. Based on such understanding, the technical solution of the present application may be embodied in essence or a part contributing to the prior art or all or part of the technical solution in the form of a software product stored in a storage medium, including several instructions to cause a computer device (which may be a personal computer, a server, or a network device, etc.) to perform all or part of the steps of the methods described in the embodiments of the present application. And the aforementioned storage medium includes: a U-disk, a removable hard disk, a read-only memory (ROM), a random access memory (random acceS memory, RAM), a magnetic disk, or an optical disk, or other various media capable of storing program codes.

The above embodiments are merely for illustrating the technical solution of the present application, and not for limiting the same; although the present application has been described in detail with reference to the foregoing embodiments, it should be understood by those of ordinary skill in the art that: the technical scheme described in the foregoing embodiments can be modified or some technical features thereof can be replaced by equivalents; such modifications and substitutions do not depart from the spirit and scope of the corresponding technical solutions.

Claims

1. The mechanical arm injection control method based on 3D modeling is characterized by comprising the following steps of:

2. The method for controlling arm injection based on 3D modeling according to claim 1, wherein inputting the plurality of continuous arm injection images into a preset arm injection recognition model for arm injection recognition, outputting a plurality of arm injection feature profiles, comprises:

inputting the continuous mechanical arm injection images into a preset mechanical arm injection recognition model, wherein the mechanical arm injection recognition model comprises a convolution layer, a deconvolution layer, a grid dividing layer and a distribution feature extraction layer;

coding and decoding the continuous mechanical arm injection images through a convolution layer and a deconvolution layer in the mechanical arm injection recognition model to obtain mechanical arm injection bounding boxes corresponding to each continuous mechanical arm injection image;

Performing image grid division on each continuous mechanical arm injection image according to the mechanical arm injection boundary frame through a grid division layer in the mechanical arm injection identification model to obtain mechanical arm injection grid images of each continuous mechanical arm injection image;

and carrying out mechanical arm injection distribution identification on the mechanical arm injection grid image through a distribution feature extraction layer in the mechanical arm injection identification model to generate a plurality of mechanical arm injection feature distribution graphs.

3. The robotic arm injection control method based on 3D modeling of claim 1, wherein 3D modeling the plurality of robotic arm injection feature profiles, respectively, to construct an initial robotic arm injection finite element model corresponding to each robotic arm injection feature profile, comprises:

respectively carrying out 3D modeling on the injection characteristic distribution diagrams of the plurality of mechanical arms to obtain a target three-dimensional model corresponding to each mechanical arm injection characteristic distribution diagram;

performing grid discretization processing on the target three-dimensional model corresponding to the injection characteristic distribution map of each mechanical arm to obtain a target discrete grid corresponding to the injection characteristic distribution map of each mechanical arm;

and carrying out grid optimization on the target discrete grid corresponding to the injection characteristic distribution map of each mechanical arm, and generating an initial mechanical arm injection finite element model corresponding to the injection characteristic distribution map of each mechanical arm.

4. The method for controlling the injection of the mechanical arm based on the 3D modeling according to claim 1, wherein the performing mechanical arm injection center calculation on each initial mechanical arm injection finite element model to generate a plurality of target center calculation results, and performing finite element model fusion on the initial mechanical arm injection finite element models according to the plurality of target center calculation results to obtain target mechanical arm injection finite element models, includes:

inputting each initial mechanical arm injection finite element model into a preset mechanical arm injection center calculation function to calculate the mechanical arm injection center, so as to obtain center coordinates corresponding to each initial mechanical arm injection finite element model;

respectively carrying out mechanical arm injection type identification on each initial mechanical arm injection finite element model to obtain mechanical arm injection types;

generating a target center calculation result according to the center coordinates corresponding to each initial mechanical arm injection finite element model and the mechanical arm injection category, and obtaining a plurality of target center calculation results;

respectively calculating the finite element fusion parameter data of each initial mechanical arm injection finite element model according to the calculation results of the plurality of target centers;

And carrying out finite element model fusion on the initial manipulator injection finite element model according to the finite element fusion parameter data to obtain a target manipulator injection finite element model.

5. The method of claim 4, wherein calculating the first arm injection control parameter of the target arm injector based on the plurality of target center calculations comprises:

calculating a plurality of initial control parameter values of the target mechanical arm injector according to preset variable distribution characteristics and center coordinates in the calculation results of the target centers;

performing average value operation on the plurality of initial control parameter values to obtain a target control parameter value of the target mechanical arm injector;

performing control weight calculation on the target control parameter values according to the mechanical arm injection categories in the calculation results of the plurality of target centers to obtain target control weight data, and performing weighted analysis on the target control parameter values according to the target control weight data to obtain weighted control parameter values of the target mechanical arm injector;

and generating a first mechanical arm injection control parameter of the target mechanical arm injector according to the weighted control parameter value.

6. The method for controlling the injection of the mechanical arm based on the 3D modeling according to claim 1, wherein the performing the simulation test and the control parameter correction on the target mechanical arm injection finite element model to obtain the corresponding second mechanical arm injection control parameter comprises:

performing simulation test on the target mechanical arm injection finite element model to obtain target simulation test data;

extracting features of the target simulation test data to obtain a simulation test feature set, and performing vector coding on the simulation test feature set to obtain a simulation test coding vector;

inputting the simulation test coding vector into a preset mechanical arm injection analysis model, wherein the mechanical arm injection analysis model comprises: an encoding network and a decoding network;

performing test feature extraction on the simulation test coding vector through a bidirectional LSTM unit in the coding network to obtain a simulation test feature vector;

inputting the simulation test feature vector into a unidirectional LSTM unit in the decoding network to perform feature decoding to obtain a simulation test evaluation index;

and correcting the control parameters of the first mechanical arm injection control parameters of the target mechanical arm injector according to the simulation test evaluation indexes to obtain corresponding second mechanical arm injection control parameters.

7. The method for controlling the injection of the mechanical arm based on the 3D modeling according to claim 6, wherein the performing the control parameter correction on the first mechanical arm injection control parameter of the target mechanical arm injector according to the simulation test evaluation index to obtain the corresponding second mechanical arm injection control parameter comprises:

calculating a loss value of the simulation test evaluation index, and judging whether the loss value exceeds a preset threshold value;

if the first mechanical arm injection control parameter is more than the second mechanical arm injection control parameter, creating an initialized mechanical arm injection control parameter population according to the first mechanical arm injection control parameter through a preset genetic optimization algorithm, wherein the initialized mechanical arm injection control parameter population comprises a plurality of first candidate injection control parameters;

setting iteration times corresponding to the genetic optimization algorithm according to the loss value, and carrying out iterative optimization on the first candidate injection control parameters according to the iteration times to generate second candidate injection control parameters;

calculating the fitness value of each second candidate injection control parameter respectively, and selecting an optimized parameter combination according to the fitness value to obtain corresponding second mechanical arm injection control parameters;

If the control data target value does not exceed the control data target value, searching for a control feedback data target value corresponding to the first mechanical arm injection control parameter, and generating a second mechanical arm injection control parameter of the target mechanical arm injector according to the control feedback data target value.

8. The utility model provides a robotic arm injection control system based on 3D modeling which characterized in that, robotic arm injection control system based on 3D modeling includes: