CN118177689A - Wireless endoscope system for AI-assisted 3D modeling and real-time detection - Google Patents
Wireless endoscope system for AI-assisted 3D modeling and real-time detection Download PDFInfo
- Publication number
- CN118177689A CN118177689A CN202410352174.8A CN202410352174A CN118177689A CN 118177689 A CN118177689 A CN 118177689A CN 202410352174 A CN202410352174 A CN 202410352174A CN 118177689 A CN118177689 A CN 118177689A
- Authority
- CN
- China
- Prior art keywords
- real
- modeling
- assisted
- endoscope
- time detection
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
- 238000011897 real-time detection Methods 0.000 title claims abstract description 26
- 230000006698 induction Effects 0.000 claims abstract description 20
- 210000000056 organ Anatomy 0.000 claims abstract description 14
- 239000011159 matrix material Substances 0.000 claims abstract description 13
- 238000012545 processing Methods 0.000 claims abstract description 8
- 230000009466 transformation Effects 0.000 claims abstract description 4
- 239000003550 marker Substances 0.000 claims description 15
- 238000000034 method Methods 0.000 claims description 11
- 239000013256 coordination polymer Substances 0.000 claims description 6
- 230000033001 locomotion Effects 0.000 claims description 6
- 239000004973 liquid crystal related substance Substances 0.000 claims description 5
- 238000006243 chemical reaction Methods 0.000 claims description 3
- 238000013507 mapping Methods 0.000 claims description 3
- 239000008188 pellet Substances 0.000 claims description 3
- 238000010586 diagram Methods 0.000 description 4
- 230000003287 optical effect Effects 0.000 description 3
- 238000013461 design Methods 0.000 description 2
- 238000001514 detection method Methods 0.000 description 2
- 238000013459 approach Methods 0.000 description 1
- 230000009286 beneficial effect Effects 0.000 description 1
- 210000004204 blood vessel Anatomy 0.000 description 1
- 230000008859 change Effects 0.000 description 1
- 230000007547 defect Effects 0.000 description 1
- 238000011161 development Methods 0.000 description 1
- 239000003814 drug Substances 0.000 description 1
- 230000000694 effects Effects 0.000 description 1
- 238000005516 engineering process Methods 0.000 description 1
- 210000003734 kidney Anatomy 0.000 description 1
- 210000004185 liver Anatomy 0.000 description 1
- 230000004807 localization Effects 0.000 description 1
- 238000002324 minimally invasive surgery Methods 0.000 description 1
- 238000012986 modification Methods 0.000 description 1
- 230000004048 modification Effects 0.000 description 1
- 210000000496 pancreas Anatomy 0.000 description 1
- 230000002093 peripheral effect Effects 0.000 description 1
- 230000002980 postoperative effect Effects 0.000 description 1
- 230000008569 process Effects 0.000 description 1
- 238000011084 recovery Methods 0.000 description 1
- 238000006467 substitution reaction Methods 0.000 description 1
- 230000000007 visual effect Effects 0.000 description 1
Landscapes
- Endoscopes (AREA)
Abstract
The invention discloses a wireless endoscope system for AI-assisted 3D modeling and real-time detection, which relates to the technical field of medical instruments and comprises a coordinate registration and image processing unit, a magnetic induction positioning unit, a display module, a wireless endoscope and a power supply module, wherein the coordinate registration and image processing unit uses a VTK, reconstructs a three-dimensional image of a patient operation position based on a three-dimensional image of the patient operation position, constructs a CT image coordinate system, uses a reference object to register the image coordinate system and a magnetic field coordinate system uniformly based on a position relation transformation matrix, and also uses yolo-v8 for real-time detection of organs and focus objects in a cavity mirror picture to assist doctors in real-time positioning operation positions and identifying organ types, and the magnetic induction positioning unit consists of a magnetic field generator, a microsensor and a controller. The invention can accurately construct the three-dimensional image model of the surgical region, display the three-dimensional position of the surgical operation of the doctor in real time, and improve the accuracy and safety of the surgical operation of the doctor.
Description
Technical Field
The invention relates to the technical field of medical instruments, in particular to a wireless endoscope system for AI-assisted 3D modeling and real-time detection.
Background
With the development and progress of medical science and technology, many operations can be treated in a minimally invasive manner, such as heart operations, liver, gall, pancreas, kidney operations and the like, in the process, an endoscope system is widely applied, a clear intraoperative field of view can be provided for doctors, the intraoperative wound area of patients can be reduced, and the postoperative recovery effect is improved.
However, when the existing endoscope system is used, the doctor can only see the content of the current layer, a plurality of peripheral key organs and blood vessels are numerous when the doctor operates instruments, the organs are closely connected, the risk of false touch exists in the operation, and the risks all easily lead to medical accidents. There are many applications of optical navigation in the market at present, but the problem of shielding in the minimally invasive surgery exists in the optical positioning, and meanwhile, the optical positioning reference frame brings inconvenience to the operation in the doctor.
To solve the above problems, a wireless endoscope system based on AI-assisted 3D modeling and real-time detection based on magnetic induction is proposed to change the current situation.
Disclosure of Invention
The invention aims to solve the defects in the prior art, and provides a wireless endoscope system for AI-assisted 3D modeling and real-time detection. The method has the advantages that the three-dimensional image model of the surgical region can be accurately constructed, the three-dimensional position of the surgical operation of the doctor can be displayed in real time, and the accuracy and safety of the surgical operation of the doctor are improved.
In order to achieve the above purpose, the present invention adopts the following technical scheme:
The wireless endoscope system for AI-assisted 3D modeling and real-time detection comprises a coordinate registration and image processing unit, a magnetic induction positioning unit, a display module, a wireless endoscope and a power supply module;
The coordinate registration and image processing unit uses the VTK to reconstruct a three-dimensional image of a patient operation position based on a CT three-dimensional image before the patient operation to construct a CT image coordinate system; registering and unifying an image coordinate system and a magnetic field coordinate system by using a reference object based on a position relation transformation matrix, and detecting organs and focus objects in a cavity mirror picture in real time by using yolo-v8 by using the module to assist doctors in locating operation positions and identifying organ types in real time;
the magnetic induction positioning unit consists of a magnetic field generator, a microsensor and a controller, wherein the magnetic field generator generates a weak safe magnetic field, the microsensor generates specific frequency current in the weak magnetic field, the microsensor is implanted into the instrument and the endoscope in the operation, the controller can control the intensity and the frequency of the magnetic field along with the movement of the endoscope and the instrument, and meanwhile, the positioning of the instrument and the endoscope is realized by detecting the tiny current generated by the microsensor.
The invention is further arranged that the display module is a liquid crystal display screen, the liquid crystal display screen is used for displaying a three-dimensional picture and a first view picture of the endoscope in real time, and the wireless endoscope provides a first view operation picture in operation for the system.
The invention is further arranged that the power supply module is a storage battery, and the storage battery is used for supplying power to the system and providing power supply guarantee for the system.
The invention is further arranged such that the magnetic induction positioning unit is capable of providing reliable position data and tracking the device in spatially restricted, difficult to view and access areas, and which is kept continuously tracking, even if the sensor is not visible, without the need for a line of sight.
The method for using the wireless endoscope system for AI-assisted 3D modeling and real-time detection comprises the following steps:
Step one: before operation, placing a corresponding small ball marker frame in an operation area of a patient, recording SF points at the placement positions, reconstructing three-dimensional images of the operation area containing markers by using VTK, obtaining coordinates TPi under a coordinate CT coordinate system of each marker point to form a point set T, and respectively implanting micro sensors in a magnetic induction unit into a surgical instrument and a cavity mirror for positioning in the cavity mirror and the surgical instrument operation;
Step two: in the operation, firstly, a small ball marker is placed at a pre-operation placement point SF, and then a surgical instrument containing a microsensor and a cavity mirror are sequentially moved to the small ball points on a pre-operation marker frame to obtain coordinates CPj of each marker point under a magnetic induction coordinate system to form a point set C; converting the point sets T and C into point cloud data Ty and Cy respectively, obtaining a registration conversion matrix M0 between the two groups of point clouds Ty and Cy by using a least square method, and obtaining three-dimensional coordinates of the instrument and the endoscope in the CT image after obtaining the registration matrix;
step three: and mapping the instrument and the endoscope coordinates to CT coordinates based on the registration matrix M0, so that the real-time display of the instrument motion track on the CT three-dimensional image is realized.
The invention is further arranged that the pellets are connected by using a bracket and form a certain fixed included angle with each other.
The invention is further arranged that the value range of i is 0< i <6, and the value range of j is 0< j <6.
The invention further provides that the expression of the three-dimensional coordinate is CP TPi＝CPj*M0.
The beneficial effects of the invention are as follows:
1. This wireless endoscope system of AI auxiliary 3D modeling and real-time detection uses the real-time picture of chamber mirror as doctor's operation first art field picture, and the system uses yolo-v8 target detection module to carry out real-time identification and location to organ and focus target in the art field, reminds doctor's target type and position in the art field, and this system chamber mirror is based on wireless design simultaneously, gets rid of traditional circuit constraint, and operation is lighter in art traditional chinese medicine life, can promote the convenience in the art greatly.
2. According to the wireless endoscope system with the AI-assisted 3D modeling and the real-time detection, the three-dimensional image model of the region in the operation can be accurately constructed through the AI-assisted 3D modeling and the coordinate registration, the three-dimensional position of the operation in the operation of a doctor can be displayed in real time, and the accuracy and the safety of the operation in the operation of the doctor are improved.
Drawings
FIG. 1 is a schematic diagram of the overall structure of a wireless endoscope system for AI-aided 3D modeling and real-time detection;
FIG. 2 is a schematic diagram of organ target recognition and localization of the wireless endoscope system for AI-assisted 3D modeling and real-time detection according to the present invention;
FIG. 3 is a schematic view of a reconstructed three-dimensional image of a wireless endoscope system for AI-aided 3D modeling and real-time detection;
FIG. 4 is a schematic diagram of a magnetic induction unit of a wireless endoscope system for AI-assisted 3D modeling and real-time detection according to the present invention;
fig. 5 is a schematic diagram of a wireless endoscope unit of a wireless endoscope system for AI-assisted 3D modeling and real-time detection according to the present invention.
Detailed Description
The technical scheme of the patent is further described in detail below with reference to the specific embodiments.
Embodiments of the present patent are described in detail below, examples of which are illustrated in the accompanying drawings, wherein the same or similar reference numerals refer to the same or similar elements or elements having the same or similar functions throughout. The embodiments described below by referring to the drawings are exemplary only for explaining the present patent and are not to be construed as limiting the present patent.
In the description of this patent, it should be understood that the terms "center," "upper," "lower," "front," "rear," "left," "right," "vertical," "horizontal," "top," "bottom," "inner," "outer," and the like indicate orientations or positional relationships based on the orientation or positional relationships shown in the drawings, merely to facilitate describing the patent and simplify the description, and do not indicate or imply that the devices or elements being referred to must have a particular orientation, be configured and operated in a particular orientation, and are therefore not to be construed as limiting the patent.
In the description of this patent, it should be noted that, unless explicitly stated and limited otherwise, the terms "mounted," "connected," and "disposed" are to be construed broadly, and may be fixedly connected, disposed, detachably connected, disposed, or integrally connected, disposed, for example. The specific meaning of the terms in this patent will be understood by those of ordinary skill in the art as the case may be.
1-5, The AI-assisted 3D modeling and real-time detection wireless endoscope system comprises a coordinate registration and image processing unit, a magnetic induction positioning unit, a display module, a wireless endoscope and a power supply module;
The coordinate registration and image processing unit uses the VTK to reconstruct a three-dimensional image of the surgical position of the patient based on the CT three-dimensional image before the patient is operated, and a CT image coordinate system is constructed; registering and unifying an image coordinate system and a magnetic field coordinate system by using a reference object based on a position relation transformation matrix, and detecting organs and focus objects in a cavity mirror picture in real time by using yolo-v8 by using the module to assist doctors in locating operation positions and identifying organ types in real time;
the magnetic induction positioning unit consists of a magnetic field generator, a microsensor and a controller, wherein the magnetic field generator generates a weak safe magnetic field, the microsensor can generate specific frequency current in the weak magnetic field, the microsensor is implanted into the instrument and the endoscope in the operation, the controller can control the intensity and the frequency of the magnetic field along with the movement of the endoscope and the instrument, and meanwhile, the instrument and the endoscope are positioned by detecting the tiny current generated by the microsensor, the magnetic induction positioning unit can provide reliable position data in a region which is spatially compact and difficult to view and approach and track equipment, and can keep continuous tracking even if the sensor is invisible without aiming lines.
In this embodiment, the display module is a liquid crystal display, and the liquid crystal display is used for displaying a three-dimensional image and a first view image of the endoscope in real time, and the wireless endoscope provides a first view operation image in operation for the system; the power supply module is a storage battery, and the storage battery is used for supplying power to the system and providing power supply guarantee for the system.
The method for using the wireless endoscope system for AI-assisted 3D modeling and real-time detection comprises the following steps:
Step one: before operation, placing a corresponding small ball marker frame in an operation area of a patient, recording SF points at the placement positions, reconstructing three-dimensional images of the operation area containing markers by using VTK, obtaining coordinates TPi under a coordinate CT coordinate system of each marker point to form a point set T, and respectively implanting micro sensors in a magnetic induction unit into a surgical instrument and a cavity mirror for positioning in the cavity mirror and the surgical instrument operation;
Step two: in the operation, firstly, a small ball marker is placed at a pre-operation placement point SF, and then a surgical instrument containing a microsensor and a cavity mirror are sequentially moved to the small ball points on a pre-operation marker frame to obtain coordinates CPj of each marker point under a magnetic induction coordinate system to form a point set C; converting the point sets T and C into point cloud data Ty and Cy respectively, obtaining a registration conversion matrix M0 between the two groups of point clouds Ty and Cy by using a least square method, and obtaining three-dimensional coordinates of the instrument and the endoscope in the CT image after obtaining the registration matrix;
step three: and mapping the instrument and the endoscope coordinates to CT coordinates based on the registration matrix M0, so that the real-time display of the instrument motion track on the CT three-dimensional image is realized.
Furthermore, the pellets are connected by using a bracket and form a certain fixed included angle with each other; the value range of i is 0< i <6, and the value range of j is 0< j <6; the expression of the three-dimensional coordinates is CP TPi＝CPj*M0.
Acquiring a three-dimensional image of a target area in a patient operation based on 3D reconstruction, acquiring an instrument three-dimensional coordinate operated in a doctor operation based on real-time positioning in a magnetic induction operation by an operation system, fusing the three-dimensional coordinate in the operation with a three-dimensional image before the operation by using coordinate registration, displaying the position of the instrument in the operation in the three-dimensional image in real time, and guiding the doctor to select a proper operation path; based on the organ target detection system, key organ names are marked in real time in the endoscope visual field, so that false collision during doctor operation is avoided; meanwhile, the endoscope adopts a wireless design, so that the operation preparation time can be reduced, and the operation in the operation is more convenient.
The foregoing is only a preferred embodiment of the present invention, but the scope of the present invention is not limited thereto, and any person skilled in the art, who is within the scope of the present invention, should make equivalent substitutions or modifications according to the technical scheme of the present invention and the inventive concept thereof, and should be covered by the scope of the present invention.
Claims (8)
- The wireless endoscope system for AI-assisted 3D modeling and real-time detection is characterized by comprising a coordinate registration and image processing unit, a magnetic induction positioning unit, a display module, a wireless endoscope and a power supply module;The coordinate registration and image processing unit uses the VTK to reconstruct a three-dimensional image of a patient operation position based on a CT three-dimensional image before the patient operation to construct a CT image coordinate system; registering and unifying an image coordinate system and a magnetic field coordinate system by using a reference object based on a position relation transformation matrix, and detecting organs and focus objects in a cavity mirror picture in real time by using yolo-v8 by using the module to assist doctors in locating operation positions and identifying organ types in real time;the magnetic induction positioning unit consists of a magnetic field generator, a microsensor and a controller, wherein the magnetic field generator generates a weak safe magnetic field, the microsensor generates specific frequency current in the weak magnetic field, the microsensor is implanted into the instrument and the endoscope in the operation, the controller can control the intensity and the frequency of the magnetic field along with the movement of the endoscope and the instrument, and meanwhile, the positioning of the instrument and the endoscope is realized by detecting the tiny current generated by the microsensor.
- 2. The AI-assisted 3D modeling and real-time detection wireless endoscope system of claim 1, wherein the display module is a liquid crystal display screen for displaying a three-dimensional view and a first field of view of the endoscope in real time, the wireless endoscope providing an intraoperative first field of view operation view for the system.
- 3. The AI-assisted 3D modeling and real-time detection wireless endoscope system of claim 2, wherein the power module is a battery for powering the system and providing power supply assurance for the system.
- 4. The AI-assisted 3D modeling and real-time detection wireless endoscope system of claim 3, wherein the magnetic induction positioning unit is capable of providing reliable position data and tracking equipment in spatially-constrained, hard-to-view and access areas, maintaining continuous tracking even if the sensor is not visible, without requiring a line of sight.
- 5. A method of using the AI-assisted 3D modeling and real-time detection wireless endoscope system of claim 1, comprising the steps of:Step one: before operation, placing a corresponding small ball marker frame in an operation area of a patient, recording SF points at the placement positions, reconstructing three-dimensional images of the operation area containing markers by using VTK, obtaining coordinates TPi under a coordinate CT coordinate system of each marker point to form a point set T, and respectively implanting micro sensors in a magnetic induction unit into a surgical instrument and a cavity mirror for positioning in the cavity mirror and the surgical instrument operation;Step two: in the operation, firstly, a small ball marker is placed at a pre-operation placement point SF, and then a surgical instrument containing a microsensor and a cavity mirror are sequentially moved to the small ball points on a pre-operation marker frame to obtain coordinates CPj of each marker point under a magnetic induction coordinate system to form a point set C; converting the point sets T and C into point cloud data Ty and Cy respectively, obtaining a registration conversion matrix M0 between the two groups of point clouds Ty and Cy by using a least square method, and obtaining three-dimensional coordinates of the instrument and the endoscope in the CT image after obtaining the registration matrix;step three: and mapping the instrument and the endoscope coordinates to CT coordinates based on the registration matrix M0, so that the real-time display of the instrument motion track on the CT three-dimensional image is realized.
- 6. The method of using a wireless endoscopic system for AI-assisted 3D modeling and real-time detection of claim 5, wherein said pellets are connected by a bracket and form a fixed angle with each other.
- 7. The method of using a wireless endoscope system for AI-assisted 3D modeling and real-time detection according to claim 6, wherein the range of values of i is 0< i <6, and the range of values of j is 0< j <6.
- 8. The method of using a wireless endoscopic system for AI-assisted 3D modeling and real-time detection of claim 7, wherein said three-dimensional coordinates are expressed as CP TPi＝CPj*M0.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202410352174.8A CN118177689A (en) | 2024-03-26 | 2024-03-26 | Wireless endoscope system for AI-assisted 3D modeling and real-time detection |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202410352174.8A CN118177689A (en) | 2024-03-26 | 2024-03-26 | Wireless endoscope system for AI-assisted 3D modeling and real-time detection |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN118177689A true CN118177689A (en) | 2024-06-14 |
Family
ID=91413419
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202410352174.8A Pending CN118177689A (en) | 2024-03-26 | 2024-03-26 | Wireless endoscope system for AI-assisted 3D modeling and real-time detection |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN118177689A (en) |
-
2024
- 2024-03-26 CN CN202410352174.8A patent/CN118177689A/en active Pending
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| Sielhorst et al. | Advanced medical displays: A literature review of augmented reality | |
| JP2966089B2 (en) | Interactive device for local surgery inside heterogeneous tissue | |
| CN110537961B (en) | Minimally invasive intervention guiding system and method for CT and ultrasonic image fusion | |
| JP4822634B2 (en) | A method for obtaining coordinate transformation for guidance of an object | |
| CN103735312B (en) | Multimode image navigation system for ultrasonic guidance operation | |
| CN109223121A (en) | Based on medical image Model Reconstruction, the cerebral hemorrhage puncturing operation navigation system of positioning | |
| CN111281540B (en) | Real-time visual navigation system based on virtual-actual fusion in minimally invasive surgery of orthopedics department | |
| CN101375805A (en) | Method and system for guiding operation of electronic endoscope by auxiliary computer | |
| CN115105207A (en) | Operation holographic navigation method and system based on mixed reality | |
| CN114711969A (en) | Surgical robot system and using method thereof | |
| CN103356284A (en) | Surgical navigation method and system | |
| CN103479431A (en) | Non-intrusive minimally invasive operation navigation system | |
| US20130172730A1 (en) | Motion-Compensated Image Fusion | |
| CN110537980A (en) | puncture surgery navigation method based on motion capture and mixed reality technology | |
| WO2023246521A1 (en) | Method, apparatus and electronic device for lesion localization based on mixed reality | |
| CN112641512A (en) | Spatial registration method applied to surgical robot front planning | |
| CN112168346A (en) | Method for real-time coincidence of three-dimensional medical image and patient and operation auxiliary system | |
| CN111839727A (en) | Prostate particle implantation path visualization method and system based on augmented reality | |
| Esposito et al. | Multimodal US–gamma imaging using collaborative robotics for cancer staging biopsies | |
| CN113648061B (en) | Head-mounted navigation system based on mixed reality and navigation registration method | |
| CN113229937A (en) | Method and system for realizing surgical navigation by using real-time structured light technology | |
| CN116823898A (en) | Mixed reality augmented display method and system | |
| CN209826968U (en) | Surgical robot system | |
| Harders et al. | Multimodal augmented reality in medicine | |
| CN118177689A (en) | Wireless endoscope system for AI-assisted 3D modeling and real-time detection |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination |