CN116052864A - Digital twinning-based puncture operation robot virtual test environment construction method - Google Patents
Digital twinning-based puncture operation robot virtual test environment construction method Download PDFInfo
- Publication number
- CN116052864A CN116052864A CN202310076661.1A CN202310076661A CN116052864A CN 116052864 A CN116052864 A CN 116052864A CN 202310076661 A CN202310076661 A CN 202310076661A CN 116052864 A CN116052864 A CN 116052864A
- Authority
- CN
- China
- Prior art keywords
- model
- puncture
- human body
- constructing
- sub
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Granted
Links
Images
Classifications
-
- G—PHYSICS
- G16—INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR SPECIFIC APPLICATION FIELDS
- G16H—HEALTHCARE INFORMATICS, i.e. INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR THE HANDLING OR PROCESSING OF MEDICAL OR HEALTHCARE DATA
- G16H40/00—ICT specially adapted for the management or administration of healthcare resources or facilities; ICT specially adapted for the management or operation of medical equipment or devices
- G16H40/40—ICT specially adapted for the management or administration of healthcare resources or facilities; ICT specially adapted for the management or operation of medical equipment or devices for the management of medical equipment or devices, e.g. scheduling maintenance or upgrades
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F30/00—Computer-aided design [CAD]
- G06F30/20—Design optimisation, verification or simulation
- G06F30/23—Design optimisation, verification or simulation using finite element methods [FEM] or finite difference methods [FDM]
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F30/00—Computer-aided design [CAD]
- G06F30/20—Design optimisation, verification or simulation
- G06F30/27—Design optimisation, verification or simulation using machine learning, e.g. artificial intelligence, neural networks, support vector machines [SVM] or training a model
-
- G—PHYSICS
- G16—INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR SPECIFIC APPLICATION FIELDS
- G16H—HEALTHCARE INFORMATICS, i.e. INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR THE HANDLING OR PROCESSING OF MEDICAL OR HEALTHCARE DATA
- G16H10/00—ICT specially adapted for the handling or processing of patient-related medical or healthcare data
- G16H10/40—ICT specially adapted for the handling or processing of patient-related medical or healthcare data for data related to laboratory analysis, e.g. patient specimen analysis
-
- G—PHYSICS
- G16—INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR SPECIFIC APPLICATION FIELDS
- G16H—HEALTHCARE INFORMATICS, i.e. INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR THE HANDLING OR PROCESSING OF MEDICAL OR HEALTHCARE DATA
- G16H50/00—ICT specially adapted for medical diagnosis, medical simulation or medical data mining; ICT specially adapted for detecting, monitoring or modelling epidemics or pandemics
- G16H50/50—ICT specially adapted for medical diagnosis, medical simulation or medical data mining; ICT specially adapted for detecting, monitoring or modelling epidemics or pandemics for simulation or modelling of medical disorders
-
- Y—GENERAL TAGGING OF NEW TECHNOLOGICAL DEVELOPMENTS; GENERAL TAGGING OF CROSS-SECTIONAL TECHNOLOGIES SPANNING OVER SEVERAL SECTIONS OF THE IPC; TECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS
- Y02—TECHNOLOGIES OR APPLICATIONS FOR MITIGATION OR ADAPTATION AGAINST CLIMATE CHANGE
- Y02P—CLIMATE CHANGE MITIGATION TECHNOLOGIES IN THE PRODUCTION OR PROCESSING OF GOODS
- Y02P90/00—Enabling technologies with a potential contribution to greenhouse gas [GHG] emissions mitigation
- Y02P90/02—Total factory control, e.g. smart factories, flexible manufacturing systems [FMS] or integrated manufacturing systems [IMS]
Landscapes
- Engineering & Computer Science (AREA)
- Health & Medical Sciences (AREA)
- Medical Informatics (AREA)
- Physics & Mathematics (AREA)
- Theoretical Computer Science (AREA)
- Public Health (AREA)
- Biomedical Technology (AREA)
- Epidemiology (AREA)
- General Health & Medical Sciences (AREA)
- Primary Health Care (AREA)
- Evolutionary Computation (AREA)
- General Physics & Mathematics (AREA)
- Computer Hardware Design (AREA)
- Geometry (AREA)
- General Engineering & Computer Science (AREA)
- Business, Economics & Management (AREA)
- General Business, Economics & Management (AREA)
- Pathology (AREA)
- Databases & Information Systems (AREA)
- Data Mining & Analysis (AREA)
- Artificial Intelligence (AREA)
- Computer Vision & Pattern Recognition (AREA)
- Software Systems (AREA)
- Instructional Devices (AREA)
Abstract
The invention is applicable to the technical field of surgical robots, and particularly relates to a puncture surgical robot virtual test environment construction method based on digital twinning, which comprises the following steps: s1, modeling an application scene of a puncture sampling operation environment based on a digital twin technology to obtain a digital twin scene model, wherein the application scene comprises physical elements; s2, virtually packaging the digital twin scene model according to the physical elements to obtain virtual mechanism models of different physical elements; s3, compiling control variables for different virtual mechanism models, and associating the control variables with a data communication protocol, so that the construction of the virtual test environment of the puncture surgical robot is completed. The virtual test environment constructed by the invention can rapidly, safely, repeatedly and highly accurately acquire puncture test data under different conditions.
Description
Technical Field
The invention is suitable for the technical field of surgical robots, and particularly relates to a puncture surgical robot virtual test environment construction method based on digital twinning.
Background
With the continuous development of robot technology, the application of robots further promotes the progress of medical technology, and in the global medical robot market, the share ratio of surgical robots reaches more than 60 percent, so that the robot is the product type with the highest technical content and the largest market demand. The puncture operation robot is one kind of operation robot, and is one robot for positioning target anatomy through MRI, ultrasonic, CT and other imaging technology, guiding the feedback needle to reach the target anatomy structure and assisting in completing puncture operation. Compared with the traditional manual puncturing by doctors, the puncturing operation robot has the advantages of higher stability and higher accuracy. The puncture operation robot has diagnosis and treatment functions, can realize biopsy, drainage, ablation, implantation and other functions, and can puncture organs such as lung, liver, kidney, breast, prostate, pancreas, spine and the like in the diagnosis aspect, reach a target anatomical structure with the assistance of an imaging technology, take out a target tissue sample and carry out pathological examination; in terms of treatment, the puncture surgery robot can be used for kidney stone surgery, tumor ablation surgery, surgery for treating cancers by implanting radioactive particles, and the like.
The surgical robot puncture sampling application mainly comprises the following structures: scanning modeling equipment (CT, MRI), a surgical robot with a controller, a tip puncture sampling instrument with a force sensor, and a patient position calibration camera. Before operation, acquiring images of focus parts of a patient through imaging equipment such as CT, MRI and the like, separating focus part boundaries by a doctor, planning an operation path, avoiding important blood vessels, nerves and other tissues, giving out target areas, completing registration of the patient according to a software algorithm, and calibrating a patient coordinate system, an image coordinate system and a mechanical arm coordinate system; in operation, the mechanical arm reaches the planned needle insertion point according to the planned path of the doctor and the planned needle insertion gesture, and the doctor can acquire the state information of the instrument entering the human body in real time according to the tracking device.
As three types of medical instrument products, the surgical robots are registered and managed, and potential harm to patients, doctors and medical environments for expected use must be considered in the whole life cycle of the products, so that the safety and effectiveness of the instruments are strictly controlled. In the research and development process of the surgical robot system, whether the debugging and verification are performed in the early stage of project development or the simulation optimization and rapid verification test are performed on the robot program after the surgical robot system is put into clinical use, a large amount of puncture test data are required, and the progress of system research and development is severely restricted by the acquisition speed of the test data. Therefore, how to obtain puncture test data in a large amount, quickly, repeatedly and safely becomes a big problem in the development process of the puncture robot system.
Disclosure of Invention
The invention provides a digital twinning-based virtual test environment construction method for a puncture operation robot, and aims to solve the technical problem that test data of the puncture operation robot are difficult to acquire in the prior art.
Specifically, the invention provides a method for constructing a virtual test environment of a puncture surgical robot based on digital twinning, which comprises the following steps:
s1, modeling an application scene of a puncture sampling operation environment based on a digital twin technology to obtain a digital twin scene model, wherein the application scene comprises physical elements;
s2, virtually packaging the digital twin scene model according to the physical elements to obtain virtual mechanism models of different physical elements;
s3, compiling control variables for different virtual mechanism models, and associating the control variables with a data communication protocol, so that the construction of the virtual test environment of the puncture surgical robot is completed.
Still further, the physical elements include a puncture surgical robot, a human body and a position calibration camera, and the digital twin scene model includes a puncture surgical robot model, a human body model and a position calibration camera model.
Still further, step S2 comprises the sub-steps of:
s21, packaging the puncture operation robot model, wherein a body of the puncture operation robot model and a puncture needle force sensor are respectively packaged to obtain a puncture operation robot virtual mechanism model and a puncture needle force sensor virtual mechanism model;
s22, packaging the human body model to obtain a human body virtual mechanism model;
s23, packaging the position calibration camera model to obtain a position calibration camera virtual mechanism model.
Still further, step S21 includes the sub-steps of:
s2111, constructing a kinematic chain of the puncture operation robot model, splitting each joint of a body in the puncture operation robot model into a sub-model, and establishing a child-parent relationship of the sub-model;
s2112, constructing a kinematic relationship according to the child-parent relationship of the child model;
s2113, constructing a dynamic relationship according to the child-parent relationship of the child models, and calculating driving moment between the connected child models, so that the packaging of the virtual mechanism model of the puncture surgical robot is completed.
Still further, step S21 includes the sub-steps of:
s2121, constructing a puncture tissue detection algorithm according to the puncture needle force sensor, wherein the puncture tissue detection algorithm is used for judging the pose of the puncture needle force sensor;
s2122, constructing a puncture force calculation algorithm according to the puncture needle force sensor, wherein the puncture force calculation algorithm is used for calculating the puncture and the puncture force of the puncture needle force sensor, so that the packaging of the virtual mechanism model of the puncture needle force sensor is completed.
Still further, step S22 includes the sub-steps of:
s221, dividing the human body model into a plurality of human body structure sub-models;
s222, adjusting the pose of the human body model, and establishing a human body model coordinate system;
s223, constructing a plurality of scaling sub-models with different sizes according to the human body structure sub-model;
s224, constructing a plurality of variable sub-models with different shapes according to the human body structure sub-model;
s225, constructing a human respiratory effect simulation model according to the human structure sub-model;
s226, presetting a human body position calibration point in the human body model, thereby completing the encapsulation of the human body virtual mechanism model.
Further, step S23 specifically includes:
and aligning the position calibration camera model with the human body position calibration point, so that the position calibration camera model can acquire depth information of the human body position calibration point, and packaging of the position calibration camera virtual mechanism model is completed.
The invention has the beneficial effects that the invention provides the construction method of the virtual test environment of the puncture operation robot based on the digital twin technology, puncture test data under different conditions can be rapidly, safely, repeatedly and highly accurately acquired through the virtual test environment, and meanwhile, the problem of the puncture test environment of a real object can be solved based on the virtual environment, so that the development progress of the puncture operation robot is accelerated, and the harm and the risk of an actual puncture test to a patient are avoided.
Drawings
Fig. 1 is a schematic flow chart of steps of a method for constructing a virtual test environment of a puncture surgical robot based on digital twinning according to an embodiment of the present invention.
Detailed Description
The present invention will be described in further detail with reference to the drawings and examples, in order to make the objects, technical solutions and advantages of the present invention more apparent. It should be understood that the specific embodiments described herein are for purposes of illustration only and are not intended to limit the scope of the invention.
Referring to fig. 1, fig. 1 is a schematic step flow diagram of a method for constructing a virtual testing environment of a puncture surgical robot based on digital twinning, which includes the following steps:
s1, modeling an application scene of a puncture sampling operation environment based on a digital twin technology to obtain a digital twin scene model, wherein the application scene comprises physical elements.
Still further, the physical elements include a puncture surgical robot, a human body and a position calibration camera, and the digital twin scene model includes a puncture surgical robot model, a human body model and a position calibration camera model.
Specifically, in the embodiment of the invention, the digital twin scene model is obtained by modeling by using three-dimensional modeling software such as SolidWorks, unigraphics NX, 3D Studio Max and the like. The puncture operation robot model, the human body model and the position calibration camera model are the most common objects in a puncture operation scene, and in the practical implementation process, other medical objects can be modeled, so that a more complex test method is realized. Illustratively, the human body model can be reconstructed by a point cloud model scanned by an MRI (magnetic resonance imaging), an ultrasonic imaging, a CT (computed tomography) imaging technology and the like, but the point cloud model needs to be split into sub-models of muscles, bones, organs and the like.
S2, virtually packaging the digital twin scene model according to the physical elements to obtain virtual mechanism models of different physical elements.
Still further, step S2 comprises the sub-steps of:
s21, packaging the puncture operation robot model, wherein a body of the puncture operation robot model and a puncture needle force sensor are respectively packaged to obtain a puncture operation robot virtual mechanism model and a puncture needle force sensor virtual mechanism model;
s22, packaging the human body model to obtain a human body virtual mechanism model;
s23, packaging the position calibration camera model to obtain a position calibration camera virtual mechanism model.
Still further, step S21 includes the sub-steps of:
s2111, constructing a kinematic chain of the puncture operation robot model, splitting each joint of the body in the puncture operation robot model into sub-models, and establishing a child-parent relationship of the sub-models.
Specifically, in the embodiment of the invention, if the puncture operation robot model is an integral model, each joint of the puncture operation robot model needs to be split into sub-models, and the sub-parent relationship among each joint of the robot is obtained according to the mechanical structure and the motion characteristics of the robot, so that the joint motion chain of the robot is constructed. In a possible embodiment, after the kinematic chain is constructed, the kinematic chain can be corrected according to the actual standard size parameters or DH parameters of the robot, so that the relative positions of the joints completely meet the requirements, and the size error between the virtual robot and the real robot is reduced.
S2112, constructing a kinematic relationship according to the child-parent relationship of the child model.
And in the kinematic relation, a forward kinematics algorithm and an inverse kinematics algorithm of the robot are constructed according to DH parameters of the robot, wherein the forward kinematics algorithm is used for solving the pose of the tail end of the sampling needle of the robot, and the inverse kinematics algorithm is used for solving the pose of the joints of the robot. In the embodiment of the invention, the inverse kinematics algorithm of the robot comprises, but is not limited to, an analytic method such as algebraic method, geometric method and the like and a numerical method, but the analytic method is preferably adopted to obtain a faster solving speed.
S2113, constructing a dynamic relationship according to the child-parent relationship of the child models, and calculating driving moment between the connected child models, so that the packaging of the virtual mechanism model of the puncture surgical robot is completed.
In the embodiment of the invention, firstly, the quality characteristics of each joint component are calculated according to three-dimensional modeling software, then the modeling of the dynamic relation is completed according to DH parameters and the quality characteristics by a Newton-Euler equation method or Lagrange method and other dynamic modeling methods, when the robot moves, the driving moment of each joint is solved according to the position, speed and acceleration data of the front joint and the rear joint, and the joint moment sensor of the cooperative robot and corresponding control measures are simulated according to the driving moment.
Still further, step S21 includes the sub-steps of:
s2121, constructing a puncture tissue detection algorithm according to the puncture needle force sensor, wherein the puncture tissue detection algorithm is used for judging the pose of the puncture needle force sensor.
In an actual puncture operation scene, the puncture needle needs to sense the current punctured tissue and the corresponding puncture depth, and the current punctured tissue and the corresponding puncture depth are used as the basis for calculating the puncture force subsequently. In actual implementation, in a virtual test environment, a directional fixed-length ray is used for simulating a puncture needle, the initial pose of the ray is the pose of the root point of the puncture needle, the length of the ray is the length of the puncture needle, and whether the tissue model is punctured or not is judged by judging whether the ray interferes with the tissue model or not.
One embodiment of the penetration tissue detection algorithm is as follows:
executing the ray detection of the puncture needle, if no ray contact point exists, continuing to execute the ray detection of the puncture needle for the next time until the ray contact point exists;
after the contact point appears, traversing the contact point, acquiring and storing a tissue model to which the contact point belongs, if a repeated tissue model appears, indicating that the tissue model is penetrated, wherein the penetration depth is a coordinate difference value of two ray contact points, if the number of the ray contact points is an odd number, indicating that the puncture needle is currently penetrating a certain tissue, the current puncture tissue is the tissue model to which the furthest contact point belongs, and the penetration depth is a difference value between the tail end coordinate of the puncture needle and the coordinate of the furthest contact point;
and circularly executing the puncture needle ray detection until the puncture test is finished.
S2122, constructing a puncture force calculation algorithm according to the puncture needle force sensor, wherein the puncture force calculation algorithm is used for calculating the puncture and the puncture force of the puncture needle force sensor, so that the packaging of the virtual mechanism model of the puncture needle force sensor is completed.
Illustratively, there are two ways of puncture force calculation algorithms, one is a finite element fit and the other is a simple numerical calculation fit. The finite element fitting is used, a finite element calculation model is established according to medical data and model data, and the direction and the magnitude of the puncture force are calculated according to the pose and the speed of the puncture needle during the puncture test; when simple numerical fitting calculation is used, the method is divided into calculation of directions and sizes, the directions are compositely fitted according to the pose of the puncture needle during puncture and the movement and deformation directions of the tissue model, and the sizes are the sum of the pressure and friction force of the tissue model to the puncture needle.
Still further, step S22 includes the sub-steps of:
s221, dividing the human body model into a plurality of human body structure sub-models.
Illustratively, the mannequin is divided into individual sub-models of muscle, bone, organ and the like, the names and the tissue types of the sub-models are identified, the sub-models are classified into sub-classes of the tissue types, and the hierarchical structure of the sub-parent class of the mannequin is organized.
S222, adjusting the pose of the human body model, and establishing a human body model coordinate system.
After the pose of the human body model is adjusted to the required pose, the pose is recorded as the initial pose of the human body model, the direction of a coordinate system of the human body model is set to be consistent with the direction of a world coordinate system, and the position of the coordinate system is positioned at the center of the human body model. And then recording the pose of each tissue sub-model as an initial pose, wherein the coordinate system direction of the tissue sub-model is unified to the coordinate system direction of the human body model, and the coordinate system position is positioned at the centroid of the sub-model.
After the coordinate system is built, when the pose change increment is transmitted, the pose change increment is converted into a target pose matrix of the tissue sub-model relative to the human body model, and then the tissue pose change simulation can be realized.
S223, constructing a plurality of scaling submodels with different sizes according to the human body structure submodel.
According to the real-time requirement of puncture test, a plurality of scaling sub-models with different sizes can be constructed by two methods, so that the size change of different tissues and organs can be simulated, if the real-time requirement is higher, a plurality of tissue models with different sizes can be preset in the modeling stage, and the tissue models with different sizes are switched when the size change is needed; if the real-time requirement is not high, the tissue model can be scaled equally by taking the model coordinate system as the center.
S224, constructing a plurality of variable sub-models with different shapes according to the human body structure sub-model.
According to the real-time requirement of puncture test, a plurality of different shape change sub-models can be constructed by two methods, so as to simulate the shape change of different tissues and organs, if the real-time requirement is higher, a plurality of different shape tissue models can be preset in the modeling stage, and when the shape change is needed, the tissue models with different shapes are switched. If the real-time requirement is not high, grid level change can be performed on the organization model according to a certain change rule.
S225, constructing a human respiratory effect simulation model according to the human structure sub-model.
Because the human body still maintains the breathing state in the whole puncture operation process, the human body breathing effect simulation needs to be constructed to verify whether the breathing compensation algorithm of the puncture operation robot is effective. In the breathing process, the tissues of the chest cavity mainly change in position and size at a certain breathing frequency, so that the embodiment of the invention constructs a human breathing effect simulation model to simulate the breathing state of a human body.
S226, presetting a human body position calibration point in the human body model, thereby completing the encapsulation of the human body virtual mechanism model.
Further, step S23 specifically includes:
and aligning the position calibration camera model with the human body position calibration point, so that the position calibration camera model can acquire depth information of the human body position calibration point, and packaging of the position calibration camera virtual mechanism model is completed.
Specifically, in the puncture test process, the position calibration camera traverses the human body model first, acquires and stores world coordinates of the model marked as a calibration point, and then converts the world coordinates of the calibration point into relative coordinates relative to the position calibration camera through a homogeneous transformation matrix.
S3, compiling control variables for different virtual mechanism models, and associating the control variables with a data communication protocol, so that the construction of the virtual test environment of the puncture surgical robot is completed.
The embodiment of the invention realizes a test system based on a programming language, and the control variables are used for controlling the forms of different virtual mechanism models. The control variable is packaged into a data interaction interface and is connected to an external puncture operation robot control system through a data communication protocol, the data communication protocol can be realized based on OPC UA, TCP, UDP, MQTT, a database and the like, and through communication with the puncture operation robot control system, instructions are received and related data are sent, so that interaction with a model in a virtual test environment is realized through a variable reading and writing mode, the appointed variable of the model is read, or the data is written into the appointed variable of the model, thereby simulating the test of the puncture operation robot operation, and test data are obtained.
The invention has the beneficial effects that the invention provides the construction method of the virtual test environment of the puncture operation robot based on the digital twin technology, puncture test data under different conditions can be rapidly, safely, repeatedly and highly accurately acquired through the virtual test environment, and meanwhile, the problem of the puncture test environment of a real object can be solved based on the virtual environment, so that the development progress of the puncture operation robot is accelerated, and the harm and the risk of an actual puncture test to a patient are avoided.
Those skilled in the art will appreciate that implementing all or part of the above-described methods in accordance with the embodiments may be accomplished by way of a computer program stored on a computer readable storage medium, which when executed may comprise the steps of the embodiments of the methods described above. The storage medium may be a magnetic disk, an optical disk, a Read-Only Memory (ROM), a random access Memory (Random Access Memory, RAM) or the like.
It should be noted that, in this document, the terms "comprises," "comprising," or any other variation thereof, are intended to cover a non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements does not include only those elements but may include other elements not expressly listed or inherent to such process, method, article, or apparatus. Without further limitation, an element defined by the phrase "comprising one … …" does not exclude the presence of other like elements in a process, method, article, or apparatus that comprises the element.
From the above description of the embodiments, it will be clear to those skilled in the art that the above-described embodiment method may be implemented by means of software plus a necessary general hardware platform, but of course may also be implemented by means of hardware, but in many cases the former is a preferred embodiment. Based on such understanding, the technical solution of the present invention may be embodied essentially or in a part contributing to the prior art in the form of a software product stored in a storage medium (e.g. ROM/RAM, magnetic disk, optical disk) comprising instructions for causing a terminal (which may be a mobile phone, a computer, a server, an air conditioner, or a network device, etc.) to perform the method according to the embodiments of the present invention.
While the embodiments of the present invention have been illustrated and described in connection with the drawings, what is presently considered to be the most practical and preferred embodiments of the invention, it is to be understood that the invention is not limited to the disclosed embodiments, but on the contrary, is intended to cover various equivalent modifications and equivalent arrangements included within the spirit and scope of the appended claims.
Claims (7)
1. The method for constructing the virtual test environment of the puncture surgical robot based on the digital twinning is characterized by comprising the following steps of:
s1, modeling an application scene of a puncture sampling operation environment based on a digital twin technology to obtain a digital twin scene model, wherein the application scene comprises physical elements;
s2, virtually packaging the digital twin scene model according to the physical elements to obtain virtual mechanism models of different physical elements;
s3, compiling control variables for different virtual mechanism models, and associating the control variables with a data communication protocol, so that the construction of the virtual test environment of the puncture surgical robot is completed.
2. The method for constructing a virtual test environment of a puncture surgical robot based on digital twinning according to claim 1, wherein the physical elements comprise the puncture surgical robot, a human body and a position calibration camera, and the digital twinning scene model comprises the puncture surgical robot model, the human body model and the position calibration camera model.
3. The method for constructing a virtual test environment of a puncture surgical robot based on digital twinning as set forth in claim 2, wherein the step S2 comprises the sub-steps of:
s21, packaging the puncture operation robot model, wherein a body of the puncture operation robot model and a puncture needle force sensor are respectively packaged to obtain a puncture operation robot virtual mechanism model and a puncture needle force sensor virtual mechanism model;
s22, packaging the human body model to obtain a human body virtual mechanism model;
s23, packaging the position calibration camera model to obtain a position calibration camera virtual mechanism model.
4. The method for constructing a virtual test environment of a puncture surgical robot based on digital twinning as set forth in claim 3, wherein the step S21 comprises the sub-steps of:
s2111, constructing a kinematic chain of the puncture operation robot model, splitting each joint of a body in the puncture operation robot model into a sub-model, and establishing a child-parent relationship of the sub-model;
s2112, constructing a kinematic relationship according to the child-parent relationship of the child model;
s2113, constructing a dynamic relationship according to the child-parent relationship of the child models, and calculating driving moment between the connected child models, so that the packaging of the virtual mechanism model of the puncture surgical robot is completed.
5. The method for constructing a virtual test environment of a puncture surgical robot based on digital twinning as set forth in claim 3, wherein the step S21 comprises the sub-steps of:
s2121, constructing a puncture tissue detection algorithm according to the puncture needle force sensor, wherein the puncture tissue detection algorithm is used for judging the pose of the puncture needle force sensor;
s2122, constructing a puncture force calculation algorithm according to the puncture needle force sensor, wherein the puncture force calculation algorithm is used for calculating the puncture and the puncture force of the puncture needle force sensor, so that the packaging of the virtual mechanism model of the puncture needle force sensor is completed.
6. The method for constructing a virtual test environment of a puncture surgical robot based on digital twinning as set forth in claim 3, wherein the step S22 comprises the sub-steps of:
s221, dividing the human body model into a plurality of human body structure sub-models;
s222, adjusting the pose of the human body model, and establishing a human body model coordinate system;
s223, constructing a plurality of scaling sub-models with different sizes according to the human body structure sub-model;
s224, constructing a plurality of variable sub-models with different shapes according to the human body structure sub-model;
s225, constructing a human respiratory effect simulation model according to the human structure sub-model;
s226, presetting a human body position calibration point in the human body model, thereby completing the encapsulation of the human body virtual mechanism model.
7. The method for constructing a virtual test environment of a puncture surgical robot based on digital twinning as set forth in claim 6, wherein the step S23 is specifically:
and aligning the position calibration camera model with the human body position calibration point, so that the position calibration camera model can acquire depth information of the human body position calibration point, and packaging of the position calibration camera virtual mechanism model is completed.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202310076661.1A CN116052864B (en) | 2023-02-03 | 2023-02-03 | Digital twinning-based puncture operation robot virtual test environment construction method |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202310076661.1A CN116052864B (en) | 2023-02-03 | 2023-02-03 | Digital twinning-based puncture operation robot virtual test environment construction method |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN116052864A true CN116052864A (en) | 2023-05-02 |
| CN116052864B CN116052864B (en) | 2023-10-20 |
Family
ID=86131196
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202310076661.1A Active CN116052864B (en) | 2023-02-03 | 2023-02-03 | Digital twinning-based puncture operation robot virtual test environment construction method |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN116052864B (en) |
Cited By (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN116822100A (en) * | 2023-08-31 | 2023-09-29 | 西北工业大学太仓长三角研究院 | Digital twin modeling method and simulation test system thereof |
| CN117647952A (en) * | 2024-01-30 | 2024-03-05 | 南京航空航天大学 | Industrial robot-oriented digital modeling position precision compensation method and system |
| CN117994346A (en) * | 2024-04-03 | 2024-05-07 | 华中科技大学同济医学院附属协和医院 | Digital twinning-based puncture instrument detection method, system and storage medium |
Citations (9)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20160008082A1 (en) * | 2014-07-10 | 2016-01-14 | Canon Kabushiki Kaisha | Puncture planning apparatus and puncture system |
| US9811074B1 (en) * | 2016-06-21 | 2017-11-07 | TruPhysics GmbH | Optimization of robot control programs in physics-based simulated environment |
| CN108648821A (en) * | 2018-03-21 | 2018-10-12 | 北京理工大学 | Intelligent operation decision system and its application process towards puncturing operation robot |
| CN110709938A (en) * | 2017-06-28 | 2020-01-17 | 通用电气公司 | Method and system for generating a digital twin of patients |
| CN111210359A (en) * | 2019-12-30 | 2020-05-29 | 中国矿业大学(北京) | Intelligent mine scene oriented digital twin evolution mechanism and method |
| CN113485392A (en) * | 2021-06-17 | 2021-10-08 | 广东工业大学 | Virtual reality interaction method based on digital twins |
| CN115014824A (en) * | 2022-06-02 | 2022-09-06 | 深圳鹏行智能研究有限公司 | Robot test system, method and computer readable storage medium |
| CN115005981A (en) * | 2022-06-07 | 2022-09-06 | 武汉联影智融医疗科技有限公司 | Surgical path planning method, system, equipment, medium and surgical operation system |
| CN115620888A (en) * | 2022-11-07 | 2023-01-17 | 上海微创医疗机器人（集团）股份有限公司 | Digital twin-based surgical robot monitoring method, device and system |
-
2023
- 2023-02-03 CN CN202310076661.1A patent/CN116052864B/en active Active
Patent Citations (9)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20160008082A1 (en) * | 2014-07-10 | 2016-01-14 | Canon Kabushiki Kaisha | Puncture planning apparatus and puncture system |
| US9811074B1 (en) * | 2016-06-21 | 2017-11-07 | TruPhysics GmbH | Optimization of robot control programs in physics-based simulated environment |
| CN110709938A (en) * | 2017-06-28 | 2020-01-17 | 通用电气公司 | Method and system for generating a digital twin of patients |
| CN108648821A (en) * | 2018-03-21 | 2018-10-12 | 北京理工大学 | Intelligent operation decision system and its application process towards puncturing operation robot |
| CN111210359A (en) * | 2019-12-30 | 2020-05-29 | 中国矿业大学(北京) | Intelligent mine scene oriented digital twin evolution mechanism and method |
| CN113485392A (en) * | 2021-06-17 | 2021-10-08 | 广东工业大学 | Virtual reality interaction method based on digital twins |
| CN115014824A (en) * | 2022-06-02 | 2022-09-06 | 深圳鹏行智能研究有限公司 | Robot test system, method and computer readable storage medium |
| CN115005981A (en) * | 2022-06-07 | 2022-09-06 | 武汉联影智融医疗科技有限公司 | Surgical path planning method, system, equipment, medium and surgical operation system |
| CN115620888A (en) * | 2022-11-07 | 2023-01-17 | 上海微创医疗机器人（集团）股份有限公司 | Digital twin-based surgical robot monitoring method, device and system |
Cited By (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN116822100A (en) * | 2023-08-31 | 2023-09-29 | 西北工业大学太仓长三角研究院 | Digital twin modeling method and simulation test system thereof |
| CN116822100B (en) * | 2023-08-31 | 2023-11-24 | 西北工业大学太仓长三角研究院 | Digital twin modeling method and simulation test system thereof |
| CN117647952A (en) * | 2024-01-30 | 2024-03-05 | 南京航空航天大学 | Industrial robot-oriented digital modeling position precision compensation method and system |
| CN117994346A (en) * | 2024-04-03 | 2024-05-07 | 华中科技大学同济医学院附属协和医院 | Digital twinning-based puncture instrument detection method, system and storage medium |
Also Published As
| Publication number | Publication date |
|---|---|
| CN116052864B (en) | 2023-10-20 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN116052864B (en) | Digital twinning-based puncture operation robot virtual test environment construction method | |
| US10453360B2 (en) | Ultrasound simulation methods | |
| TWI711428B (en) | Optical tracking system and training system for medical equipment | |
| JP7240519B2 (en) | Smart monitoring system for pelvic fracture reduction | |
| KR102536732B1 (en) | Device and method for the computer-assisted simulation of surgical interventions | |
| KR20120122542A (en) | Method and System for Providing Rehearsal of Image Guided Surgery and Computer-readable Recording Medium for the same | |
| CN115005981A (en) | Surgical path planning method, system, equipment, medium and surgical operation system | |
| KR101862359B1 (en) | Program and method for generating surgical simulation information | |
| CN112151169B (en) | Autonomous scanning method and system of humanoid-operation ultrasonic robot | |
| JP2013528880A (en) | A method to determine joint bone deformity using motion patterns | |
| CN112587233B (en) | Method and system for simulating endotracheal intubation | |
| CN113662573B (en) | Mammary gland focus positioning method, device, computer equipment and storage medium | |
| CN113100934A (en) | Operation assisting method, device, computer equipment and storage medium | |
| Villamil et al. | Simulation of the human TMJ behavior based on interdependent joints topology | |
| Huang et al. | On mimicking human’s manipulation for robot-assisted spine ultrasound imaging | |
| TWI707660B (en) | Wearable image display device for surgery and surgery information real-time system | |
| CN116224829B (en) | Digital twinning-based surgical robot puncture sampling operation semi-physical simulation method | |
| KR20190088419A (en) | Program and method for generating surgical simulation information | |
| KR101940706B1 (en) | Program and method for generating surgical simulation information | |
| CN113893033B (en) | Pulmonary percutaneous puncture navigation method and system | |
| CN114283179A (en) | Real-time fracture far-near end space pose acquisition and registration system based on ultrasonic images | |
| Dang et al. | Digital twin-based skill training with a hands-on user interaction device to assist in manual and robotic ultrasound scanning | |
| CN117058267B (en) | Autonomous ultrasound scanning system, method, memory and device based on reinforcement learning | |
| KR20190131243A (en) | Simulation tool manufacturing method for arthroscopic surgical and simulation tool for arthroscopic surgical manufactured by the same | |
| Meinhold et al. | A Virtual Reality Guidance System for a Precise MRI Injection Robot |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |