CN115399880A - Calibration method, instrument control method, device, electronic equipment and storage medium - Google Patents
Calibration method, instrument control method, device, electronic equipment and storage medium Download PDFInfo
- Publication number
- CN115399880A CN115399880A CN202211160190.4A CN202211160190A CN115399880A CN 115399880 A CN115399880 A CN 115399880A CN 202211160190 A CN202211160190 A CN 202211160190A CN 115399880 A CN115399880 A CN 115399880A
- Authority
- CN
- China
- Prior art keywords
- coordinate
- coordinate system
- marker
- mri
- solid
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
- 238000000034 method Methods 0.000 title claims abstract description 76
- 239000003550 marker Substances 0.000 claims abstract description 188
- 239000007787 solid Substances 0.000 claims abstract description 116
- 239000007788 liquid Substances 0.000 claims abstract description 84
- 230000003287 optical effect Effects 0.000 claims abstract description 44
- 230000009466 transformation Effects 0.000 claims description 37
- 239000011159 matrix material Substances 0.000 claims description 35
- 238000004590 computer program Methods 0.000 claims description 16
- 230000006870 function Effects 0.000 claims description 15
- 239000000126 substance Substances 0.000 claims description 7
- 238000013519 translation Methods 0.000 claims description 2
- 238000006243 chemical reaction Methods 0.000 abstract description 18
- 238000002595 magnetic resonance imaging Methods 0.000 description 130
- 238000010586 diagram Methods 0.000 description 11
- 238000004891 communication Methods 0.000 description 8
- 238000012545 processing Methods 0.000 description 6
- 230000008569 process Effects 0.000 description 5
- 210000000988 bone and bone Anatomy 0.000 description 4
- 238000005516 engineering process Methods 0.000 description 3
- 239000000523 sample Substances 0.000 description 3
- 230000009286 beneficial effect Effects 0.000 description 2
- 230000003993 interaction Effects 0.000 description 2
- 238000012986 modification Methods 0.000 description 2
- 230000004048 modification Effects 0.000 description 2
- 210000004872 soft tissue Anatomy 0.000 description 2
- 238000003491 array Methods 0.000 description 1
- 238000013473 artificial intelligence Methods 0.000 description 1
- 210000004556 brain Anatomy 0.000 description 1
- 230000001413 cellular effect Effects 0.000 description 1
- 238000013170 computed tomography imaging Methods 0.000 description 1
- 238000000354 decomposition reaction Methods 0.000 description 1
- 230000007547 defect Effects 0.000 description 1
- 238000013461 design Methods 0.000 description 1
- 239000012530 fluid Substances 0.000 description 1
- 239000011521 glass Substances 0.000 description 1
- 210000002216 heart Anatomy 0.000 description 1
- 238000005286 illumination Methods 0.000 description 1
- 238000003384 imaging method Methods 0.000 description 1
- 230000006872 improvement Effects 0.000 description 1
- 239000004973 liquid crystal related substance Substances 0.000 description 1
- 230000004807 localization Effects 0.000 description 1
- 238000010801 machine learning Methods 0.000 description 1
- 238000013507 mapping Methods 0.000 description 1
- 238000012544 monitoring process Methods 0.000 description 1
- 210000003205 muscle Anatomy 0.000 description 1
- 239000013307 optical fiber Substances 0.000 description 1
- 238000005457 optimization Methods 0.000 description 1
- 239000004065 semiconductor Substances 0.000 description 1
- 230000001953 sensory effect Effects 0.000 description 1
- 238000006467 substitution reaction Methods 0.000 description 1
- 238000001356 surgical procedure Methods 0.000 description 1
- 210000001519 tissue Anatomy 0.000 description 1
- 230000000007 visual effect Effects 0.000 description 1
Images
Classifications
-
- A—HUMAN NECESSITIES
- A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
- A61B—DIAGNOSIS; SURGERY; IDENTIFICATION
- A61B34/00—Computer-aided surgery; Manipulators or robots specially adapted for use in surgery
- A61B34/20—Surgical navigation systems; Devices for tracking or guiding surgical instruments, e.g. for frameless stereotaxis
-
- A—HUMAN NECESSITIES
- A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
- A61B—DIAGNOSIS; SURGERY; IDENTIFICATION
- A61B5/00—Measuring for diagnostic purposes; Identification of persons
- A61B5/05—Detecting, measuring or recording for diagnosis by means of electric currents or magnetic fields; Measuring using microwaves or radio waves
- A61B5/055—Detecting, measuring or recording for diagnosis by means of electric currents or magnetic fields; Measuring using microwaves or radio waves involving electronic [EMR] or nuclear [NMR] magnetic resonance, e.g. magnetic resonance imaging
-
- A—HUMAN NECESSITIES
- A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
- A61B—DIAGNOSIS; SURGERY; IDENTIFICATION
- A61B34/00—Computer-aided surgery; Manipulators or robots specially adapted for use in surgery
- A61B34/20—Surgical navigation systems; Devices for tracking or guiding surgical instruments, e.g. for frameless stereotaxis
- A61B2034/2046—Tracking techniques
- A61B2034/2055—Optical tracking systems
-
- A—HUMAN NECESSITIES
- A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
- A61B—DIAGNOSIS; SURGERY; IDENTIFICATION
- A61B34/00—Computer-aided surgery; Manipulators or robots specially adapted for use in surgery
- A61B34/20—Surgical navigation systems; Devices for tracking or guiding surgical instruments, e.g. for frameless stereotaxis
- A61B2034/2068—Surgical navigation systems; Devices for tracking or guiding surgical instruments, e.g. for frameless stereotaxis using pointers, e.g. pointers having reference marks for determining coordinates of body points
- A61B2034/207—Divots for calibration
Landscapes
- Health & Medical Sciences (AREA)
- Life Sciences & Earth Sciences (AREA)
- Engineering & Computer Science (AREA)
- Nuclear Medicine, Radiotherapy & Molecular Imaging (AREA)
- Surgery (AREA)
- Animal Behavior & Ethology (AREA)
- Public Health (AREA)
- Heart & Thoracic Surgery (AREA)
- Medical Informatics (AREA)
- Molecular Biology (AREA)
- Physics & Mathematics (AREA)
- General Health & Medical Sciences (AREA)
- Biomedical Technology (AREA)
- Veterinary Medicine (AREA)
- Robotics (AREA)
- High Energy & Nuclear Physics (AREA)
- Radiology & Medical Imaging (AREA)
- Biophysics (AREA)
- Pathology (AREA)
- Magnetic Resonance Imaging Apparatus (AREA)
Abstract
The invention discloses a calibration method, an apparatus control device, electronic equipment and a storage medium, wherein the calibration method is used for calibrating MRI equipment through a marking tool, the marking tool is provided with a liquid marker and a solid marker, the first coordinate of the liquid marker scanned in a coordinate system of the MRI equipment is obtained by controlling the MRI equipment, and the second coordinate of the solid marker in a world coordinate system is obtained through an optical positioning system; through the coordinate comparison of the same liquid marker in different coordinate systems, the conversion relation from the coordinate system of the marking tool to the coordinate system of the MRI equipment can be analyzed, the position data of the solid marker on the marking tool can be converted into the coordinate system of the MRI equipment according to the conversion relation to obtain a third coordinate, finally, the final conversion relation from the coordinate system of the MRI equipment to the world coordinate system can be obtained according to the third coordinate and the second coordinate, and the calibration of the MRI equipment can be realized based on the final conversion relation.
Description
Technical Field
The invention relates to the technical field of surgical navigation, in particular to a calibration method, an instrument control method, a device, electronic equipment and a storage medium.
Background
The current optical positioning system can be applied to the medical field, and is used for assisting in positioning an operation object, tracking the position of a surgical instrument in real time and providing accurate operation navigation.
Currently, a medical robot usually uses a CT imaging technology in combination with an optical positioning system to realize surgical navigation and positioning in bone treatment. The optical positioning system mainly recognizes solid components such as bones, and the markers are easily recognized when the optical positioning system is used in a CT system.
There is also a need for performing surgery on soft tissues (such as brain, muscle, heart, etc.) by medical robots, etc., and the imaging of soft tissues of the body requires the use of MRI (magnetic resonance imaging) technology, so it is very important how to achieve optical localization in MRI systems.
Disclosure of Invention
The invention provides a calibration method, an instrument control device, electronic equipment and a storage medium, which are used for realizing optical positioning in an MRI system.
In a first aspect, the present invention provides a calibration method for calibrating an MRI apparatus by a marker tool, the marker tool having a liquid marker and a solid marker disposed thereon, the calibration method comprising:
controlling the MRI equipment to scan the liquid marker to obtain a first coordinate of the liquid marker in a coordinate system of the MRI equipment, and shooting the solid marker through an optical positioning system to obtain a second coordinate of the solid marker in a world coordinate system;
determining a third coordinate of the solid marker in the coordinate system of the MRI device according to the position data of the liquid marker and the solid marker on the marking tool and/or the first coordinate;
and calibrating the MRI equipment according to the second coordinate and the third coordinate.
In a second aspect, the present invention provides an instrument control method comprising:
acquiring instrument coordinates of an instrument in a world coordinate system;
acquiring a calibration coordinate of the operation object in a coordinate system of MRI equipment from a magnetic resonance image containing the operation object;
taking the calibration coordinates as the coordinates of an operation object in the world coordinate system;
controlling the instrument according to the operation object coordinates and the instrument coordinates;
wherein the MRI apparatus is calibrated by the calibration method of the first aspect.
In a third aspect, the present invention provides a calibration apparatus, including:
the marker coordinate acquisition module is used for controlling the MRI equipment to scan the liquid marker to obtain a first coordinate of the liquid marker in a coordinate system of the MRI equipment, and shooting the solid marker through an optical positioning system to obtain a second coordinate of the solid marker in a world coordinate system;
a third coordinate acquisition module, configured to determine, according to the first coordinate, a third coordinate of the solid marker in a coordinate system of the MRI apparatus;
and the MRI equipment calibration module is used for calibrating the MRI equipment based on the second coordinate and the third coordinate.
In a fourth aspect, the present invention provides an instrument control device comprising:
the instrument coordinate acquisition module is used for acquiring instrument coordinates of the instrument in a world coordinate system;
the MRI calibration coordinate acquisition module is used for acquiring calibration coordinates of the operation object in a coordinate system of MRI equipment from a magnetic resonance image containing the operation object;
the world coordinate acquisition module is used for taking the calibration coordinates as the coordinates of the operation object in the world coordinate system;
an instrument control module for controlling the instrument according to the operation object coordinates and the instrument coordinates;
wherein the MRI apparatus is calibrated by the calibration method of the first aspect.
In a fifth aspect, the present invention provides an electronic device, comprising:
at least one processor; and
a memory communicatively coupled to the at least one processor; wherein the content of the first and second substances,
the memory stores a computer program executable by the at least one processor to enable the at least one processor to perform the calibration method of the first aspect of the invention and/or the instrument control method of the second aspect.
In a sixth aspect, the present invention provides a computer readable storage medium storing computer instructions for causing a processor to implement the calibration method according to the first aspect of the present invention, and/or the instrument control method according to the second aspect of the present invention when executed.
The calibration method provided by the embodiment of the invention is characterized by being used for calibrating MRI equipment by a marking tool, wherein the marking tool is provided with a liquid marker and a solid marker, and the calibration method comprises the following steps: controlling the MRI equipment to scan the liquid marker to obtain a first coordinate of the liquid marker in a coordinate system of the MRI equipment, and shooting the solid marker through an optical positioning system to obtain a second coordinate of the solid marker in a world coordinate system; determining a third coordinate of the solid marker in a coordinate system of the MRI device according to the position data of the liquid marker and the solid marker on the marking tool and/or the first coordinate; and calibrating the MRI equipment according to the second coordinate and the third coordinate. According to the position data of the liquid marker and the solid marker on the marking tool and/or the first coordinate, the conversion relation from the coordinate system of the marking tool to the coordinate system of the MRI device can be analyzed through the coordinate comparison of the same liquid marker under different coordinate systems, then the position data of the solid marker on the marking tool can be converted into the coordinate system of the MRI device according to the conversion relation to obtain a third coordinate, finally the final conversion relation from the coordinate system of the MRI device to the world coordinate system can be obtained according to the third coordinate of the solid marker in the MRI device and the second coordinate of the solid marker under the world coordinate system, and the calibration of the MRI device can be realized based on the final conversion relation. The whole calibration process is simple, so that the realization of the optical positioning system in the MRI equipment is feasible.
It should be understood that the statements in this section do not necessarily identify key or critical features of the embodiments of the present invention, nor do they necessarily limit the scope of the invention. Other features of the present invention will become apparent from the following description.
Drawings
In order to more clearly illustrate the technical solutions in the embodiments of the present invention, the drawings needed to be used in the description of the embodiments will be briefly introduced below, and it is obvious that the drawings in the following description are only some embodiments of the present invention, and it is obvious for those skilled in the art to obtain other drawings based on these drawings without creative efforts.
Fig. 1 is a flowchart of a calibration method according to an embodiment of the present invention;
FIG. 2 is a schematic diagram of an MRI apparatus and an optical positioning system for acquiring coordinates of a marker according to an embodiment of the present invention;
FIG. 3 is a flowchart of a calibration method according to a second embodiment of the present invention;
FIG. 4 is a schematic diagram of a marking tool provided in accordance with a second embodiment of the present invention;
FIG. 5 is a schematic diagram of a coordinate system of a marking tool according to a second embodiment of the present invention;
FIG. 6 is a schematic illustration of another marking tool provided in accordance with a second embodiment of the present invention;
FIG. 7 is a flowchart of an apparatus control method according to a third embodiment of the present invention;
fig. 8 is a schematic structural diagram of a calibration apparatus according to a fourth embodiment of the present invention;
fig. 9 is a schematic structural diagram of an instrument control device according to a fifth embodiment of the present invention;
fig. 10 is a schematic structural diagram of an electronic device according to a sixth embodiment of the present invention.
Detailed Description
In order to make those skilled in the art better understand the technical solutions of the present invention, the technical solutions in the embodiments of the present invention will be clearly and completely described below with reference to the drawings in the embodiments of the present invention, and it is obvious that the described embodiments are only a part of the embodiments of the present invention, and not all of the embodiments. All other embodiments, which can be derived by a person skilled in the art from the embodiments given herein without making any creative effort, shall fall within the protection scope of the present invention.
Example one
Fig. 1 is a flowchart of a calibration method according to an embodiment of the present invention, where this embodiment is applicable to a case where an MRI (magnetic resonance imaging) apparatus is calibrated by a calibration tool, and the method may be executed by a calibration apparatus, which may be implemented in hardware and/or software, and the calibration apparatus may be configured in an electronic device, for example, in a computer for calibrating the MRI apparatus. As shown in fig. 1, the calibration method includes:
s101, controlling the MRI equipment to scan the liquid marker to obtain a first coordinate of the liquid marker in a coordinate system of the MRI equipment, and shooting the solid marker through an optical positioning system to obtain a second coordinate of the solid marker in a world coordinate system.
The optical locating system can identify solid components such as bones, for example, the locating points in bones can be easily observed by using the optical locating system in a CT system. The optical positioning system can be a binocular vision positioning system, and the optical positioning system is based on a parallax principle, two cameras are fixed on the same rigid body at a certain distance and angle, when the optical positioning system works, the two cameras respectively acquire mapping images of the same characteristic point in a view field range, then the specific position of the characteristic point in a three-dimensional space is calculated according to the arrangement positions of the two cameras and the position of the characteristic point acquired in the images through a triangulation principle, and the positions of a plurality of characteristic points can be dynamically acquired.
MRI mainly identifies fluid components, and for example, by performing arteriovenous scans using MRI, clear images of the transverse plane, the sagittal plane, and the like can be obtained.
The MRI equipment, the marking tool and the optical positioning system are respectively provided with different coordinate systems, and the coordinate system where the optical positioning system is located is a world coordinate system, namely an absolute coordinate system. Therefore, the MRI apparatus can be calibrated to convert the coordinates detected by the MRI apparatus into coordinates in the world coordinate system, so that the coordinates of the MRI apparatus and the coordinates of the world coordinate system are placed in the same coordinate system, thereby facilitating the determination of the position of the operation object in the world coordinate system.
Fig. 2 is a schematic diagram of the MRI apparatus and the optical positioning system obtaining coordinates of different markers on the marking tool, as shown in fig. 2, the marking tool 2 is generally a rigid body tool, the marking tool 2 includes a plurality of rigid body rods, each rigid body rod is provided with 1 liquid marker and 1 solid marker, the liquid marker may be a soft container filled with liquid and sealed, or may be a soft substance, and then the liquid marker may be detected by the MRI apparatus 1 to obtain a first coordinate. The solid marker may be a light reflecting marker ball, and the solid marker may be recognized by the optical locating system 3 to obtain the second coordinate. Wherein the optical positioning system 3 is movable, the shooting position and angle can be adjusted according to the position of the marking tool 2.
Specifically, the optical positioning system can form a set of binocular vision system through two near-infrared cameras, infrared LEDs are fixed around optical axes of the two cameras for illumination, and infrared light emitted by the infrared LEDs is reflected back to the camera photosensitive chip through the small reflective balls on the solid markers on the marking tool. The camera is used for collecting the image of the mark point, and the mark point is identified and positioned by a digital image processing technology under the assistance of a computer. The needle point coordinate and the direction of the marking tool can be calibrated according to the three-dimensional coordinate of the marking point, so that the accurate navigation of the operation is realized.
And S102, determining a third coordinate of the solid marker in a coordinate system of the MRI equipment according to the position data of the liquid marker and the solid marker on the marking tool and/or the first coordinate.
The position data of the liquid marker and the solid marker on the marking tool is the coordinates of the liquid marker and the solid marker on the marking tool.
According to the position data of the liquid marker and the solid marker on the marking tool and/or the first coordinate, the conversion relation between the coordinate system of the marking tool and the coordinate system of the MRI device can be analyzed through comparing the liquid coordinate with the liquid coordinate, and then the position data of the solid marker on the marking tool can be converted into the coordinate system of the MRI device according to the conversion relation to obtain a third coordinate.
And S103, calibrating the MRI equipment according to the second coordinate and the third coordinate.
After the third coordinate of the solid marker in the coordinate system of the MRI apparatus is determined, the final transformation relationship between the coordinate system of the MRI apparatus and the world coordinate system can be obtained according to the third coordinate of the solid marker in the MRI apparatus and the second coordinate of the solid marker in the world coordinate system, that is, the transformation relationship between the two coordinate systems is obtained for the coordinates of the same solid marker in the two different coordinate systems, and finally the calibration of the MRI apparatus can be realized based on the final transformation relationship. The coordinate points detected in the MRI apparatus can be directly converted into the world coordinate system. The operator can visually see the coordinates of the soft body or liquid object acquired by the MRI apparatus in the world coordinate system. In addition, a display screen can be arranged in the optical positioning system, the coordinates of the operation object acquired by the MRI equipment under the world coordinate system are displayed in the display screen, and the position of the marking tool under the world coordinate system, which can be detected by the optical positioning system, can be displayed in the display screen, so that the operator or the mechanical arm can be guided to control the marking tool to operate the operation object conveniently.
The calibration method provided by the embodiment of the invention is characterized by being used for calibrating MRI equipment by a marking tool, wherein the marking tool is provided with a liquid marker and a solid marker, and the calibration method comprises the following steps: controlling the MRI equipment to scan the liquid marker to obtain a first coordinate of the liquid marker in a coordinate system of the MRI equipment, and shooting the solid marker through an optical positioning system to obtain a second coordinate of the solid marker in a world coordinate system; determining a third coordinate of the solid marker in a coordinate system of the MRI device according to the position data of the liquid marker and the solid marker on the marking tool and/or the first coordinate; and calibrating the MRI equipment according to the second coordinate and the third coordinate. According to the position data of the liquid marker and the solid marker on the marking tool and/or the first coordinate, the conversion relation from the coordinate system of the marking tool to the coordinate system of the MRI device can be analyzed through the coordinate comparison of the same liquid marker under different coordinate systems, then the position data of the solid marker on the marking tool can be converted into the coordinate system of the MRI device according to the conversion relation to obtain a third coordinate, finally the final conversion relation from the coordinate system of the MRI device to the world coordinate system can be obtained according to the third coordinate of the solid marker in the MRI device and the second coordinate of the solid marker under the world coordinate system, and the calibration of the MRI device can be realized based on the final conversion relation. The whole calibration process is simple, so that the realization of the optical positioning system in the MRI equipment has feasibility.
Example two
Fig. 3 is a flowchart of a calibration method provided in the second embodiment of the present invention, and the first embodiment of the present invention is optimized based on the first embodiment, as shown in fig. 3, the calibration method includes:
s301, controlling the MRI device to scan the liquid marker to obtain a first coordinate of the liquid marker in a coordinate system of the MRI device, and shooting the solid marker through an optical positioning system to obtain a second coordinate of the solid marker in a world coordinate system.
S301 is the same as S101 of the first embodiment, and refer to S101. Wherein the liquid marker is arranged 21 outside the solid marker 22, i.e. for the same pair of liquid marker and solid marker, the coordinates of both on the marking tool are different, as shown in fig. 4.
S302, acquiring a fourth coordinate of the liquid marker in the coordinate system of the marking tool, and acquiring a fifth coordinate of the solid marker in the coordinate system of the marking tool.
The marking tool itself is provided with a set of coordinate system, which is shown in fig. 5 as the coordinate system of the marking tool, and the coordinate system can be established by taking the plane of the rigid rod of the marking tool as a horizontal plane and the intersection point of the rigid rod as an origin. If the rigid rods are not in the same horizontal plane at the same time or if there are multiple intersection points of the rigid rods, the plane where part of the rigid rods are located may be used as the horizontal plane, or the intersection points of part of the rigid rods may be used as the origin.
It should be noted that, when a plurality of liquid markers and a plurality of solid markers are present on the marking tool, the solid/liquid markers described by the MRI apparatus, the optical system, and the marking tool should be the same solid/liquid marker, taking the liquid marker as an example, as shown in fig. 5, in the marking tool, a plurality of liquid markers a are simultaneously present 1 ，A 2 ，A 3 ，A 4 If the coordinate of the liquid marker in the coordinate system of the MRI apparatus is the first coordinate, the first coordinate is for the liquid marker A 1 The coordinate of the liquid marker in the coordinate system of the marking tool is the position data and is also the liquid marker A 1 The corresponding coordinates.
The fourth coordinate and the fifth coordinate are coordinates in a coordinate system of the marking tool, and when the coordinate system of the marking tool is established, the fourth coordinate and the fifth coordinate are determined. It should be noted that a set of monitoring equipment of the marking tool may be provided to specifically establish the coordinate system of the marking tool.
And S303, calculating an intermediate transformation matrix from the coordinate system of the marking tool to the coordinate system of the MRI device according to the first coordinate and the fourth coordinate.
The first coordinate is the coordinate of the liquid marker in the coordinate system of the MRI device, the fourth coordinate is the coordinate of the liquid marker in the coordinate system of the marking tool, and the conversion relation from the coordinate system of the marking tool to the coordinate system of the MRI device can be analyzed according to the relation between the fourth coordinate and the first coordinate to obtain an intermediate transformation matrix.
In particular, an intermediate transformation matrix of the coordinate system of the marking tool to the coordinate system of the MRI device may be calculated from the first and fourth coordinates, comprising:
constructing a first objective function:
wherein, ma i Being the first coordinate, ta, of the liquid marker in the coordinate system of the MRI apparatus i Is a liquid markerAnd a fourth coordinate of the coordinate system of the marking tool, rtm being a rotation matrix, ttm being a translation matrix, and N being the number of liquid markers.
After the first objective function is constructed, rtm and Ttm at which the function value of the first objective function is minimized can be solved as an intermediate transformation matrix from the coordinate system of the marker tool to the coordinate system of the MRI apparatus. ICP (iterative closest point) matching is generally used for solving, and for example, SVD (matrix singular value decomposition), a nonlinear optimization method can be used for solving.
And S304, converting the fifth coordinate into a third coordinate of the solid marker in the coordinate system of the MRI equipment by adopting the intermediate transformation matrix.
And the fifth coordinate is the coordinate of the solid marker in the coordinate system of the marking tool, and after an intermediate transformation matrix from the coordinate system of the marking tool to the coordinate system of the MRI equipment is solved, the fifth coordinate can be converted into the third coordinate of the solid marker in the coordinate system of the MRI equipment by adopting the intermediate transformation matrix.
Specifically, the converting the fifth coordinate into a third coordinate of the solid marker in the coordinate system of the MRI apparatus using the intermediate transformation matrix includes:
the third coordinate of the solid marker in the coordinate system of the MRI device is calculated by the following formula:
Mb i ＝Rtm×Tb i +Ttm
wherein Mb i As third coordinate of the solid marker in the coordinate system of the MRI apparatus, tb i And Rtm and Ttm are the fifth coordinate of the solid marker in the coordinate system of the marking tool, and serve as intermediate transformation matrices from the coordinate system of the marking tool to the coordinate system of the MRI device.
And S305, determining a target transformation matrix from the coordinate system of the MRI device to the world coordinate system according to the second coordinate and the third coordinate.
The second coordinate is the coordinate of the solid marker in the world coordinate system, the third coordinate is the coordinate of the solid marker in the coordinate system of the MRI device, and the conversion relation from the coordinate system of the MRI device to the world coordinate system can be obtained according to the third coordinate and the second coordinate.
Specifically, determining a target transformation matrix from the coordinate system of the MRI apparatus to a world coordinate system according to the second coordinate and the third coordinate includes:
constructing a second objective function:
wherein, mob i As a second coordinate, mb, of the solid marker in the world coordinate system i Is the third coordinate of the solid marker in the coordinate system of the MRI apparatus, rmo and Tmo are the target transformation matrices in the coordinate system of the MRI apparatus to the world coordinate system.
And solving Rmo and Tmo when the function value of the second objective function is minimized as objective transformation matrixes from the coordinate system of the MRI device to the world coordinate system.
And S306, calibrating the MRI equipment based on the target transformation matrix.
The target transformation matrix represents the transformation relation from the coordinate system of the MRI device to the world coordinate system, so the MRI device can be calibrated based on the target transformation matrix. The coordinate points detected in the MRI apparatus may be directly converted into the world coordinate system. The operator can visually see the coordinates of the soft body or liquid object acquired by the MRI apparatus in the world coordinate system. In addition, a display screen can be arranged in the optical positioning system, the coordinates of the operation object acquired by the MRI equipment under the world coordinate system are displayed in the display screen, and the position of the marking tool under the world coordinate system, which can be detected by the optical positioning system, can be displayed in the display screen, so that the operator or the mechanical arm can be guided to control the marking tool to operate the operation object conveniently.
The embodiment of the invention provides a calibration method, which is used for calibrating MRI equipment through a marking tool, wherein the marking tool is provided with a liquid marker and a solid marker, and the calibration method comprises the following steps: controlling the MRI equipment to scan the liquid marker to obtain a first coordinate of the liquid marker in a coordinate system of the MRI equipment, and shooting the solid marker through an optical positioning system to obtain a second coordinate of the solid marker in a world coordinate system; acquiring a fourth coordinate of the liquid marker in the coordinate system of the marking tool and a fifth coordinate of the solid marker in the coordinate system of the marking tool; calculating an intermediate transformation matrix from the coordinate system of the marking tool to the coordinate system of the MRI device according to the first coordinate and the fourth coordinate; and converting the fifth coordinate into a third coordinate of the solid marker in the coordinate system of the MRI equipment by adopting the intermediate transformation matrix, determining a target transformation matrix from the coordinate system of the MRI equipment to a world coordinate system according to the third coordinate and the second coordinate MB, and calibrating the MRI equipment based on the transformation matrix. And finally, a target transformation matrix from the coordinate system of the MRI device to the world coordinate system can be obtained according to the third coordinate of the solid marker in the MRI device and the second coordinate of the solid marker in the world coordinate system, so that the calibration of the MRI device can be realized based on the target transformation matrix pair. The whole calibration process is simple, so that the realization of the optical positioning system in the MRI equipment has feasibility.
In an alternative embodiment of the invention, as shown in FIG. 6, the liquid marker 21 is inside the solid marker 22 and the center of the liquid marker 21 coincides with the center of the solid marker 22.
Determining a third coordinate of the solid marker in the coordinate system of the MRI apparatus based on the position data of the liquid marker and the solid marker on the marking tool, and/or the first coordinate, comprising:
the first coordinate is determined as a third coordinate of the solid marker in the coordinate system of the MRI device.
The centers of the liquid marker and the solid marker on the marking tool are overlapped, so that the coordinates of the liquid marker and the solid marker on the marking tool are the same, further, when the MRI device detects the first coordinate of the liquid marker, the first coordinate can also be regarded as the coordinate of the solid marker, therefore, the first coordinate can be directly determined as the third coordinate of the solid marker under the coordinate system of the MRI device, further, the target conversion matrix can be obtained according to the third coordinate of the solid marker under the coordinate system of the MRI device and the second coordinate of the solid marker under the world coordinate system, the intermediate conversion matrix does not need to be solved, the coordinate of the solid marker on the marking tool does not need to be converted into the coordinate system of the MRI device, the operation resources are saved, and the calibration process of the MRI device is further simplified.
EXAMPLE III
Fig. 7 is a flowchart of an instrument control method according to a third embodiment of the present invention, where this embodiment is applicable to a case where an instrument is controlled to operate an operation object, and the method may be executed by an instrument control device, where the instrument control device may be implemented in a form of hardware and/or software, and the instrument control device may be configured in a surgical navigator for controlling an instrument, as shown in fig. 7, and the instrument control method includes:
s701, acquiring the instrument coordinate of the instrument in a world coordinate system.
The instrument in this embodiment may be a surgical instrument, such as a surgical probe, a scalpel, a manipulator of a medical robot, or the like.
The world coordinate system is the coordinate system of the optical positioning system, the optical positioning system is mainly used for detecting solid, the instrument itself is generally solid, or the instrument can be provided with a solid marker, so that the optical positioning system can identify the fixed marker on the instrument to obtain the instrument coordinate. The solid marker on the instrument may be a tool for performing a surgical operation on the operation target, such as a probe or a puncture needle, and the coordinates of the probe or the puncture needle in the world coordinate system may be used as the instrument coordinates.
S702, acquiring the calibration coordinates of the operation object in the coordinate system of the MRI equipment from the magnetic resonance image containing the operation object.
The operation object is a soft body or a liquid tissue area, so that the MRI apparatus can detect the coordinates of the operation object, wherein the MRI apparatus is calibrated by the calibration method of any of the above embodiments, i.e. the MRI apparatus can also convert the coordinates of the detected liquid substance or soft body substance into a world coordinate system, i.e. the calibrated coordinates are obtained.
And S703, taking the calibration coordinates as the coordinates of the operation object in the world coordinate system.
The calibration coordinates are coordinates in the world coordinate system, and therefore the calibration coordinates of the operation object in the coordinate system of the MRI apparatus can be directly used as the coordinates of the operation object in the world coordinate system.
And S704, controlling the instrument according to the operation object coordinates and the instrument coordinates.
The coordinates of the operation object and the coordinates of the instrument are coordinates in a world coordinate system, and the position relation of the operation object and the coordinates of the instrument can be displayed in the surgical navigator, so that an operator or a mechanical arm moves the instrument to the position of the operation object according to the position relation of the operation object and the instrument, and the operation object is accurately operated.
The embodiment obtains the coordinates of the instrument in a world coordinate system; acquiring a calibration coordinate of an operation object in a coordinate system of MRI equipment from a magnetic resonance image containing the operation object; taking the calibration coordinates as the coordinates of an operation object in the world coordinate system; and controlling the instrument according to the operation object coordinate and the instrument coordinate. The MRI apparatus is calibrated by the calibration method of any one of the above embodiments, and then the MRI apparatus can also convert the coordinates of the detected liquid substance or soft substance into a world coordinate system, so as to obtain calibration coordinates, where the calibration coordinates are coordinates of the operation object in the world coordinate system. Further, the position relationship of the operation object and the instrument in the world coordinate system may be displayed in the surgical navigator, so that the operator or the manipulator moves the instrument to the operation object position according to the position relationship of the operation object and the instrument to perform precise operation on the operation object.
Example four
Fig. 8 is a schematic structural diagram of a calibration apparatus according to a fourth embodiment of the present invention. As shown in fig. 8, the calibration apparatus includes:
a marker coordinate obtaining module 801, configured to control the MRI apparatus to scan the liquid marker to obtain a first coordinate of the liquid marker in a coordinate system of the MRI apparatus, and to obtain a second coordinate of the solid marker in a world coordinate system by shooting the solid marker through an optical positioning system;
a third coordinate obtaining module 802, configured to determine a third coordinate of the solid marker in a coordinate system of the MRI apparatus according to the first coordinate;
and the MRI apparatus calibration module 803 is configured to calibrate the MRI apparatus based on the second coordinate and the third coordinate.
The calibration device provided by the embodiment of the invention can execute the calibration method provided by any embodiment of the invention, and has corresponding functional modules and beneficial effects of the execution method.
EXAMPLE five
Fig. 9 is a schematic structural diagram of an instrument control device according to a fourth embodiment of the present invention. As shown in fig. 9, the instrument control device includes:
an instrument coordinate acquiring module 901, configured to acquire instrument coordinates of an instrument in a world coordinate system;
an MRI calibration coordinate obtaining module 902, configured to obtain calibration coordinates of the operation object in a coordinate system of the MRI apparatus from a magnetic resonance image containing the operation object;
a world coordinate obtaining module 903, configured to use the calibration coordinates as coordinates of an operation object in a world coordinate system;
an instrument control module 904 for controlling the instrument according to the operation object coordinates and the instrument coordinates;
wherein the MRI apparatus is calibrated by the calibration method of any of the above embodiments.
The instrument control device provided by the embodiment of the invention can execute the instrument control method provided by any embodiment of the invention, and has corresponding functional modules and beneficial effects of the execution method.
Example six
FIG. 10 illustrates a block diagram of an electronic device 70 that may be used to implement an embodiment of the invention. Electronic devices are intended to represent various forms of digital computers, such as laptops, desktops, workstations, personal digital assistants, servers, blade servers, mainframes, and other appropriate computers. The electronic device may also represent various forms of mobile devices, such as personal digital assistants, cellular phones, smart phones, wearable devices (e.g., helmets, glasses, watches, etc.), and other similar computing devices. The components shown herein, their connections and relationships, and their functions, are meant to be exemplary only, and are not meant to limit implementations of the inventions described and/or claimed herein.
As shown in fig. 10, the electronic device 70 includes at least one processor 71, and a memory communicatively connected to the at least one processor 71, such as a Read Only Memory (ROM) 72, a Random Access Memory (RAM) 73, and the like, wherein the memory stores computer programs executable by the at least one processor, and the processor 71 may perform various appropriate actions and processes according to the computer programs stored in the Read Only Memory (ROM) 72 or the computer programs loaded from the storage unit 78 into the Random Access Memory (RAM) 73. In the RAM 73, various programs and data necessary for the operation of the electronic apparatus 70 can also be stored. The processor 71, the ROM 72, and the RAM 73 are connected to each other by a bus 74. An input/output (I/O) interface 75 is also connected to bus 74.
A plurality of components in the electronic device 70 are connected to the I/O interface 75, including: an input unit 76 such as a keyboard, a mouse, etc.; an output unit 74 such as various types of displays, speakers, and the like; a storage unit 78, such as a magnetic disk, optical disk, or the like; and a communication unit 79 such as a network card, modem, wireless communication transceiver, etc. The communication unit 79 allows the electronic device 70 to exchange information/data with other devices via a computer network such as the internet and/or various telecommunication networks.
In some embodiments, the calibration method, and or the instrument control method, may be implemented as a computer program tangibly embodied in a computer-readable storage medium, such as the storage unit 78. In some embodiments, part or all of the computer program may be loaded and/or installed onto the electronic device 70 via the ROM 72 and/or the communication unit 79. When loaded into RAM 73 and executed by processor 71, the computer program may perform one or more of the steps of the calibration method, and/or the instrument control method described above. Alternatively, in other embodiments, the processor 71 may be configured to perform the calibration method, and or the instrument control method, in any other suitable manner (e.g., by way of firmware).
Various implementations of the systems and techniques described here above may be implemented in digital electronic circuitry, integrated circuitry, field Programmable Gate Arrays (FPGAs), application Specific Integrated Circuits (ASICs), application Specific Standard Products (ASSPs), system on a chip (SOCs), load programmable logic devices (CPLDs), computer hardware, firmware, software, and/or combinations thereof. These various embodiments may include: implemented in one or more computer programs that are executable and/or interpretable on a programmable system including at least one programmable processor, which may be special or general purpose, receiving data and instructions from, and transmitting data and instructions to, a storage system, at least one input device, and at least one output device.
A computer program for implementing the methods of the present invention may be written in any combination of one or more programming languages. These computer programs may be provided to a processor of a general purpose computer, special purpose computer, or other programmable data processing apparatus, such that the computer programs, when executed by the processor, cause the functions/acts specified in the flowchart and/or block diagram block or blocks to be performed. A computer program can execute entirely on a machine, partly on a machine, as a stand-alone software package partly on a machine and partly on a remote machine or entirely on a remote machine or server.
In the context of the present invention, a computer-readable storage medium may be a tangible medium that can contain, or store a computer program for use by or in connection with an instruction execution system, apparatus, or device. A computer readable storage medium may include, but is not limited to, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any suitable combination of the foregoing. Alternatively, the computer readable storage medium may be a machine readable signal medium. More specific examples of a machine-readable storage medium would include an electrical connection based on one or more wires, a portable computer diskette, a hard disk, a Random Access Memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a compact disc read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing.
To provide for interaction with a user, the systems and techniques described here can be implemented on an electronic device having: a display device (e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor) for displaying information to a user; and a keyboard and a pointing device (e.g., a mouse or a trackball) by which a user can provide input to the electronic device. Other kinds of devices may also be used to provide for interaction with a user; for example, feedback provided to the user can be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user may be received in any form, including acoustic, speech, or tactile input.
The systems and techniques described here can be implemented in a computing system that includes a back-end component (e.g., as a data server), or that includes a middleware component (e.g., an application server), or that includes a front-end component (e.g., a user computer having a graphical user interface or a web browser through which a user can interact with an implementation of the systems and techniques described here), or any combination of such back-end, middleware, or front-end components. The components of the system can be interconnected by any form or medium of digital data communication (e.g., a communication network). Examples of communication networks include: local Area Networks (LANs), wide Area Networks (WANs), blockchain networks, and the internet.
The computing system may include clients and servers. A client and server are generally remote from each other and typically interact through a communication network. The relationship of client and server arises by virtue of computer programs running on the respective computers and having a client-server relationship to each other. The server can be a cloud server, also called a cloud computing server or a cloud host, and is a host product in a cloud computing service system, so that the defects of high management difficulty and weak service expansibility in the traditional physical host and VPS service are overcome.
It should be understood that various forms of the flows shown above may be used, with steps reordered, added, or deleted. For example, the steps described in the present invention may be executed in parallel, sequentially, or in different orders, and are not limited herein as long as the desired results of the technical solution of the present invention can be achieved.
The above-described embodiments should not be construed as limiting the scope of the invention. It should be understood by those skilled in the art that various modifications, combinations, sub-combinations and substitutions may be made in accordance with design requirements and other factors. Any modification, equivalent replacement, and improvement made within the spirit and principle of the present invention should be included in the protection scope of the present invention.
Claims (12)
1. A calibration method for calibrating an MRI apparatus by a marker tool having a liquid marker and a solid marker disposed thereon, comprising:
controlling the MRI equipment to scan the liquid marker to obtain a first coordinate of the liquid marker in a coordinate system of the MRI equipment, and shooting the solid marker through an optical positioning system to obtain a second coordinate of the solid marker in a world coordinate system;
determining a third coordinate of the solid marker in the coordinate system of the MRI device according to the position data of the liquid marker and the solid marker on the marking tool and/or the first coordinate;
and calibrating the MRI equipment according to the second coordinate and the third coordinate.
2. The calibration method according to claim 1, wherein the liquid marker is disposed outside the solid marker, and the determining the third coordinate of the solid marker in the coordinate system of the MRI apparatus according to the position data of the liquid marker and the solid marker on the marking tool and/or the first coordinate comprises:
acquiring a fourth coordinate of the liquid marker in the coordinate system of the marking tool and acquiring a fifth coordinate of the solid marker in the coordinate system of the marking tool;
calculating an intermediate transformation matrix from the coordinate system of the marking tool to the coordinate system of the MRI device according to the first coordinate and the fourth coordinate;
and converting the fifth coordinate into a third coordinate of the solid marker in a coordinate system of the MRI device by using the intermediate transformation matrix.
3. The calibration method according to claim 2, wherein said calculating an intermediate transformation matrix of the coordinate system of the marking tool to the coordinate system of the MRI apparatus based on the first coordinates and the fourth coordinates comprises:
constructing a first objective function:
wherein, ma i Is a first coordinate, ta, of the liquid marker in the coordinate system of the MRI apparatus i Is a fourth coordinate of the liquid marker in the coordinate system of the marking tool, rtm is a rotation matrix, ttm is a translation matrix, and N is the number of liquid markers;
solving Rtm and Ttm when the function value of the first objective function is minimized as an intermediate transformation matrix from the coordinate system of the marking tool to the coordinate system of the MRI apparatus.
4. The calibration method according to claim 2, wherein the converting the fifth coordinate into the third coordinate of the solid marker in the coordinate system of the MRI apparatus using the intermediate transformation matrix comprises:
calculating a third coordinate of the solid marker in the coordinate system of the MRI apparatus by the following formula:
Mb i ＝Rtm×Tb i +Ttm
wherein Mb i As third coordinate, tb, of the solid marker in the coordinate system of the MRI apparatus i And Rtm and Ttm are used as an intermediate transformation matrix from the coordinate system of the marking tool to the coordinate system of the MRI device, wherein the fifth coordinate of the solid marker in the coordinate system of the marking tool is shown.
5. The calibration method according to claim 1, wherein the liquid marker is inside the solid marker, and the center of the liquid marker coincides with the center of the solid marker, and the determining a third coordinate of the solid marker in the coordinate system of the MRI apparatus according to the position data of the liquid marker and the solid marker on the marking tool, and/or the first coordinate comprises:
determining the first coordinate as a third coordinate of the solid marker in a coordinate system of the MRI apparatus.
6. The calibration method according to any one of claims 1 to 5, wherein the calibrating the MRI apparatus according to the second coordinate and the third coordinate comprises:
determining a target transformation matrix from the coordinate system of the MRI device to the world coordinate system according to the second coordinate and the third coordinate;
and calibrating the MRI equipment based on the target transformation matrix.
7. The calibration method according to claim 6, wherein the determining the target transformation matrix from the coordinate system of the MRI apparatus to the world coordinate system according to the second coordinate and the third coordinate comprises:
constructing a second objective function:
wherein, mob i For the second coordinate, mb, of the solid marker in the world coordinate system i Is a third coordinate of the solid marker in the coordinate system of the MRI device, rmo and Tmo are target transformation matrices from the coordinate system of the MRI device to the world coordinate system;
and solving Rmo and Tmo when the function value of the second objective function is minimized as objective transformation matrixes from the coordinate system of the MRI device to the world coordinate system.
8. An instrument control method, comprising:
acquiring instrument coordinates of an instrument in a world coordinate system;
acquiring a calibration coordinate of the operation object in a coordinate system of MRI equipment from a magnetic resonance image containing the operation object;
taking the calibration coordinates as the coordinates of an operation object in the world coordinate system;
controlling the instrument according to the operation object coordinates and the instrument coordinates;
wherein the MRI apparatus is calibrated by the calibration method of any one of claims 1-7.
9. A calibration device, comprising:
the marker coordinate acquisition module is used for controlling the MRI equipment to scan the liquid marker to obtain a first coordinate of the liquid marker in a coordinate system of the MRI equipment, and shooting the solid marker through an optical positioning system to obtain a second coordinate of the solid marker in a world coordinate system;
a third coordinate acquisition module, configured to determine, according to the first coordinate, a third coordinate of the solid marker in a coordinate system of the MRI apparatus;
and the MRI equipment calibration module is used for calibrating the MRI equipment based on the second coordinate and the third coordinate.
10. An instrument control device, comprising:
the instrument coordinate acquisition module is used for acquiring instrument coordinates of the instrument in a world coordinate system;
the MRI calibration coordinate acquisition module is used for acquiring calibration coordinates of the operation object in a coordinate system of the MRI equipment from a magnetic resonance image containing the operation object;
the world coordinate acquisition module is used for taking the calibration coordinates as the coordinates of the operation object in the world coordinate system;
an instrument control module for controlling the instrument according to the operation object coordinates and the instrument coordinates;
wherein the MRI apparatus is calibrated by the calibration method of any one of claims 1-7.
11. An electronic device, characterized in that the electronic device comprises:
at least one processor; and
a memory communicatively coupled to the at least one processor; wherein the content of the first and second substances,
the memory stores a computer program executable by the at least one processor, the computer program being executable by the at least one processor to enable the at least one processor to perform the calibration method of any one of claims 1-7, and/or the instrument control method of claim 8.
12. A computer-readable storage medium storing computer instructions for causing a processor to perform the calibration method of any one of claims 1-7 and/or the instrument control method of claim 8 when executed.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202211160190.4A CN115399880A (en) | 2022-09-22 | 2022-09-22 | Calibration method, instrument control method, device, electronic equipment and storage medium |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202211160190.4A CN115399880A (en) | 2022-09-22 | 2022-09-22 | Calibration method, instrument control method, device, electronic equipment and storage medium |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN115399880A true CN115399880A (en) | 2022-11-29 |
Family
ID=84165113
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202211160190.4A Pending CN115399880A (en) | 2022-09-22 | 2022-09-22 | Calibration method, instrument control method, device, electronic equipment and storage medium |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN115399880A (en) |
-
2022
- 2022-09-22 CN CN202211160190.4A patent/CN115399880A/en active Pending
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN112006779B (en) | Precision detection method of surgical navigation system | |
| RU2478980C2 (en) | System and method for automatic calibration of tracked ultrasound | |
| Harders et al. | Calibration, registration, and synchronization for high precision augmented reality haptics | |
| CN113442169B (en) | Method and device for calibrating hands and eyes of robot, computer equipment and readable storage medium | |
| CN110780285A (en) | Pose calibration method, system and medium for laser radar and combined inertial navigation | |
| US11270465B2 (en) | Multiple camera calibration | |
| CN112155727A (en) | Surgical navigation systems, methods, devices, and media based on three-dimensional models | |
| CN112382359B (en) | Patient registration method and device, electronic equipment and computer readable medium | |
| JP2020521598A (en) | Method and system for retrieving blood vessel corresponding positional relationship in multi-angle imaging | |
| EP3430595B1 (en) | Determining the relative position between a point cloud generating camera and another camera | |
| US10078906B2 (en) | Device and method for image registration, and non-transitory recording medium | |
| CN111973273A (en) | Operation navigation system, method, device and medium based on AR technology | |
| US10179031B2 (en) | Interrelated point acquisition for navigated surgery | |
| CN116269763A (en) | Coordinate conversion relation calibration method and device, operation navigation system and medium | |
| CN115399880A (en) | Calibration method, instrument control method, device, electronic equipment and storage medium | |
| CN116030135A (en) | Real-time attitude measurement system in remote operation | |
| CN116047412A (en) | Artifact coordinate removal method of marker and related device | |
| CN115153632A (en) | Ultrasonic imaging positioning system, method, device, equipment and storage medium | |
| CN110368026B (en) | Operation auxiliary device and system | |
| CN113823000A (en) | Enhanced display method, system, device and storage medium based on head | |
| Pei-Lun et al. | Surgical navigation system based on the visual object tracking algorithm | |
| Knoerlein et al. | Comparison of tracker-based to tracker-less haptic device calibration | |
| CN117679178B (en) | Minimally invasive surgical robot system for traumatic orthopedics department | |
| CN116309449B (en) | Image processing method, device, equipment and storage medium | |
| CN115624384B (en) | Operation auxiliary navigation system, method and storage medium based on mixed reality technology |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination |