CN114581635B - Positioning method and system based on HoloLens glasses - Google Patents
Positioning method and system based on HoloLens glasses Download PDFInfo
- Publication number
- CN114581635B CN114581635B CN202210205741.8A CN202210205741A CN114581635B CN 114581635 B CN114581635 B CN 114581635B CN 202210205741 A CN202210205741 A CN 202210205741A CN 114581635 B CN114581635 B CN 114581635B
- Authority
- CN
- China
- Prior art keywords
- positioning
- patient
- point
- head
- reference virtual
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T19/00—Manipulating 3D models or images for computer graphics
- G06T19/006—Mixed reality
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
- G06F3/01—Input arrangements or combined input and output arrangements for interaction between user and computer
- G06F3/011—Arrangements for interaction with the human body, e.g. for user immersion in virtual reality
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T7/00—Image analysis
- G06T7/30—Determination of transform parameters for the alignment of images, i.e. image registration
- G06T7/33—Determination of transform parameters for the alignment of images, i.e. image registration using feature-based methods
- G06T7/337—Determination of transform parameters for the alignment of images, i.e. image registration using feature-based methods involving reference images or patches
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/10—Image acquisition modality
- G06T2207/10004—Still image; Photographic image
- G06T2207/10012—Stereo images
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/30—Subject of image; Context of image processing
- G06T2207/30004—Biomedical image processing
- G06T2207/30088—Skin; Dermal
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/30—Subject of image; Context of image processing
- G06T2207/30196—Human being; Person
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2210/00—Indexing scheme for image generation or computer graphics
- G06T2210/41—Medical
Abstract
The invention provides a positioning method and a positioning system based on HoloLens glasses, wherein a three-dimensional image is generated in advance according to CTA data of the head of a patient, and the bleeding position and at least three positioning points of the patient are marked on the three-dimensional image; then generating a reference virtual registration point according to the positioning device, wherein the reference virtual registration point can establish virtual and real connection; the user wears the HoloLens glasses, and the positioning points are sequentially moved to the head position of the patient corresponding to the positioning points according to the reference virtual registration points, so that the three-dimensional image and the head of the patient are superposed, and the user is assisted to quickly and accurately position the bleeding position of the patient.
Description
Technical Field
The invention relates to the field of medical treatment, in particular to a positioning method and system based on HoloLens glasses.
Background
The mixed reality product is to superimpose a computer-generated image on the real world and display the image of the virtual space in the real space so that human eyes can see a virtual object in the real world. A relatively representative product is a wearable mixed reality computing device HoloLens released by microsoft corporation. HoloLens is a mixed reality device introduced by Microsoft, and a user can see a real environment through a lens on glasses after wearing the HoloLens, and meanwhile, virtual digital models and animations can be displayed through the lens. The HoloLens can acquire three-dimensional scanning data of a surrounding real scene through a sensor, and the HoloToolKit toolkit is utilized to perform space mapping processing on the three-dimensional data to enable the three-dimensional data to become grid data attached to the surface of an object of the real scene.
Cerebral hemorrhage (cerebral hemorrhage) refers to hemorrhage caused by vascular rupture in non-traumatic brain parenchyma, accounts for 20% -30% of all cerebral apoplexy, the acute phase fatality rate is 30% -40%, therefore, the rapid positioning of the hemorrhage position and the craniotomy of patients with cerebral hemorrhage are particularly important in the rescue process of cerebral hemorrhage, CT angiography (CTA) data has intracranial hemorrhage information of patients, but cannot be directly used for positioning, doctors are still required to judge the hemorrhage position through experience, therefore, how to utilize mixed reality technology to process the CTA data of patients, rapidly match the CTA data to the heads of patients, assist the doctors to rapidly and accurately position the hemorrhage position of the patients is a technical problem to be solved urgently.
Disclosure of Invention
In view of the above-mentioned shortcomings of the prior art, the present invention aims to provide a positioning method and system based on HoloLens glasses, which is used to solve the problem in the prior art that doctors need to determine the bleeding position of patients through experience.
The invention provides a positioning method based on HoloLens glasses, which comprises the following steps:
presenting to a user a three-dimensional image generated from CTA data of a patient's head; the three-dimensional image comprises a marked bleeding position, human skin and at least three positioning points on the marked human skin;
identifying a positioning device, generating a reference virtual registration point based on the positioning device and presenting the reference virtual registration point to a user;
sequentially moving each positioning point to the head position of the patient corresponding to the positioning point according to the reference virtual registration point to realize the superposition of the three-dimensional image and the head of the patient;
hiding the marked human skin and each positioning point, and presenting the head of the patient and the marked bleeding position for the user.
In an embodiment of the present invention, the method further includes: the step of generating a three-dimensional image from CTA data of a patient's head includes:
acquiring CTA data of the patient's head;
marking the bleeding position of the head of the patient and the skin of the human body based on the CTA data, and selecting a plurality of positioning points from different positions of the marked skin of the human body to obtain a three-dimensional image.
In an embodiment of the present invention, the step of identifying the positioning device and generating a reference virtual registration point based on the positioning device includes:
the positioning device is provided with an identification code which can be identified;
and after the identification code is identified, generating a reference virtual registration point which is separated from the identification code by a fixed distance based on the spatial position of the identification code.
In an embodiment of the invention, the step of sequentially moving each positioning point to the head position of the patient corresponding to the positioning point according to the reference virtual registration point includes:
and moving the reference virtual registration point to the head position of the patient corresponding to one positioning point, then moving the positioning point to the position of the reference virtual registration point, and fixing the positioning point after the positioning point is overlapped with the reference virtual registration point until the fixation of all the positioning points is completed.
The second aspect of the present invention also provides a HoloLens glasses-based positioning system, which includes:
HoloLens glasses for presenting a three-dimensional image generated from CTA data of the head of a patient to a user; the three-dimensional image comprises a marked bleeding position, human skin and a plurality of positioning points on the marked human skin;
the system is also used for identifying a positioning device, generating a reference virtual registration point based on the positioning device and presenting the reference virtual registration point to a user;
the positioning points are sequentially moved to the positions of the heads of the patients corresponding to the positioning points according to the reference virtual registration points, so that the three-dimensional images are superposed with the heads of the patients;
the device is also used for hiding the marked human skin and each positioning point and presenting the head of the patient and the marked bleeding position for the user;
and the positioning device is used for providing a positioning function for the HoloLens glasses.
In an embodiment of the present invention, the method further includes:
the processing module is used for obtaining CTA data of the head of the patient, marking the bleeding position of the head of the patient and the skin of the human body based on the CTA data, and selecting a plurality of positioning points from different positions of the marked skin of the human body to obtain a three-dimensional image.
In an embodiment of the present invention, the positioning device is provided with an identification code that can be recognized by the HoloLens glasses;
the HoloLens glasses are further used for generating a reference virtual registration point which is separated from the identification code by a fixed distance based on the spatial position of the identification code after the identification code is identified.
In an embodiment of the present invention, the positioning device is further provided with a plurality of control keys for outputting control commands to the HoloLens glasses;
the HoloLens glasses are further used for moving the reference virtual registration point to the head position of the patient corresponding to one positioning point according to the control instruction, then moving the positioning point to the position where the reference virtual registration point is located, and fixing the positioning point after the positioning point is overlapped with the reference virtual registration point until the fixing of all the positioning points is completed.
The third aspect of the present invention also provides a server, comprising a memory, a processor and a computer program stored on the memory and executable on the processor, wherein the processor executes the program to implement any one of the above methods of the first aspect of the present invention.
The fourth aspect of the present invention also provides a storage medium having stored thereon a computer program which, when executed by a processor, implements the method according to any one of the above-mentioned HoloLens glasses-based positioning methods of the first aspect of the present invention.
As described above, the positioning method and system based on the HoloLens glasses of the present invention have the following beneficial effects:
the invention generates a three-dimensional image in advance according to CTA data of the head of a patient, and marks a bleeding position and at least three positioning points of the patient on the three-dimensional image; then generating a reference virtual registration point according to the positioning device, wherein the reference virtual registration point can establish virtual and real connection; the user wears the HoloLens glasses, and the positioning points are sequentially moved to the head position of the patient corresponding to the positioning points according to the reference virtual registration points, so that the three-dimensional image and the head of the patient are superposed, and the user is assisted to quickly and accurately position the bleeding position of the patient.
Drawings
In order to more clearly illustrate the technical solutions of the embodiments of the present application, the drawings needed to be used in the description of the embodiments of the present application will be briefly introduced below, and it is obvious that the drawings in the following description are only some embodiments of the present application, and it is obvious for those skilled in the art that other drawings can be obtained according to these drawings without inventive exercise.
Fig. 1 is a block diagram showing a positioning system according to a first embodiment of the present invention.
Fig. 2 is a flowchart illustrating a positioning method according to a second embodiment of the invention.
Fig. 3 is a schematic diagram of a server according to a fourth embodiment of the present invention.
Detailed Description
The embodiments of the present invention are described below with reference to specific embodiments, and other advantages and effects of the present invention will be easily understood by those skilled in the art from the disclosure of the present specification. The invention is capable of other and different embodiments and of being practiced or of being carried out in various ways, and its several details are capable of modification in various respects, all without departing from the spirit and scope of the present invention. It is to be noted that the features in the following embodiments and examples may be combined with each other without conflict.
It should be noted that the drawings provided in the following embodiments are only for illustrating the basic idea of the present invention, and the drawings only show the components related to the present invention rather than being drawn according to the number, shape and size of the components in actual implementation, and the type, amount and proportion of each component in actual implementation may be changed arbitrarily, and the layout of the components may be more complicated.
Referring to fig. 1, a first embodiment of the present invention relates to a positioning system based on HoloLens glasses, which is used for superimposing a virtual three-dimensional image and a real head of a patient based on CTA data of the head of the patient, so as to present a bleeding position of the head of the patient on the real head of the patient, thereby facilitating a user to quickly position the bleeding position. The positioning system includes: the processor, the HoloLens glasses, and the positioning device, it should be understood that a Communication connection is established between the processor, the HoloLens glasses, and the positioning device, and the Communication connection may include any one of a Wi-Fi connection, a bluetooth connection, a Universal Serial Bus (USB) connection, an Ultra Wireless Band (UWB) connection, a ZigBee (ZigBee Technology) connection, and a Near Field Communication (NFC) connection.
The processor is used for acquiring CTA data of the head of the patient, marking the bleeding position of the head of the patient and the skin of the human body based on the CTA data, and selecting a plurality of positioning points from different positions of the marked skin of the human body to obtain a three-dimensional image.
Specifically, the CTA data of the patient's head, which is three-dimensional reconstructed mirror image data obtained by CT scanning, may show the bleeding position of the patient's head and the human skin. Based on the CTA data, the processor marks the bleeding position and the human skin, and selects a plurality of positioning points from different positions of the marked human skin to obtain a three-dimensional image. It should be understood that there are various ways to implement the labeling method, for example, the CTA recognition model can be pre-trained for recognition and labeling, and the CTA recognition model can also be manually labeled by a specially trained person and then uploaded to the processor, and the specific labeling method is not limited in this scheme.
It should be understood that the CTA data corresponds to the real head of the patient, and the user may predetermine the positions of the positioning points, for example, in this embodiment, the user selects four positioning positions in advance, which are respectively located on the left tragus, the right tragus, the left canthus and the right canthus; the processor, upon receiving the CTA data of the patient's head, finds the four localized positions from the marked human skin based on the position information.
HoloLens glasses are used for presenting the head of the patient and the marked bleeding position for the user according to instructions of the user. Specifically, after the user wears the HoloLens glasses, the user can input a download instruction to the HoloLens glasses, and after receiving the download instruction, the HoloLens glasses download the three-dimensional image generated according to the CTA data of the head of the patient and display the three-dimensional image to the user.
And the positioning device is used for providing a positioning function for the HoloLens glasses and completing the superposition of the virtual three-dimensional image to the real head of the patient. Specifically, the positioning device is provided with an identification code which can be recognized by the HoloLens glasses, and the identification code can be set in various ways, such as a specific picture, wherein the picture is preset with content which can be recognized by the HoloLens glasses; in addition, the method can be realized by two-dimensional codes or infrared.
The HoloLens glasses are also used for identifying the positioning device, and generating a reference virtual registration point based on the positioning device and presenting the reference virtual registration point to a user. Specifically, after the HoloLens glasses recognize the identification code of the positioning device, a reference virtual registration point which is separated from the identification code by a fixed distance is generated based on the spatial position of the identification code, and when the spatial position of the positioning device changes, the reference virtual registration point can move along with the movement of the positioning device.
The HoloLens glasses are also used for sequentially moving each positioning point to the head position of the patient corresponding to the positioning point along with the movement of the reference virtual registration point, so that the three-dimensional image is coincided with the head of the patient. Specifically, in the actual operation, when the user moves the positioning device, the reference virtual registration point moves along with the positioning device, and by adopting the scheme, the reference virtual registration point can be sequentially moved to the four positioning positions in a mode of moving the positioning device, for example, the user moves the positioning device, moves the reference virtual registration point presented in the HoloLens glasses to the left tragus position of the patient, when the reference virtual registration point and the reference virtual registration point are overlapped, the positioning point of the marked human skin corresponding to the left tragus position is moved to the position where the reference virtual registration point is located through the HoloLens glasses, and when the positioning point and the reference virtual registration point are overlapped, the positioning point is fixed until all the positioning points are fixed. It should be understood that the above-mentioned coincidence can be preset to a tolerance range as long as it is within the tolerance range, i.e., any two coincide. In addition, the three-dimensional image in this embodiment is a whole, that is, when the positioning points are moved to the corresponding left tragus position, the whole three-dimensional image is also moved integrally therewith, because the number of the positioning points in this embodiment is at least three, the three-dimensional image can be coincided with the real head of the patient.
The HoloLens glasses are also used for hiding the marked human skin and each positioning point and presenting the head of the patient and the marked bleeding position for the user. Specifically, after the three-dimensional image is overlapped with the real head of the patient, the user can hide the marked human skin and the positioning points through the HoloLens glasses, and only the head of the patient and the marked bleeding position are displayed to the user, so that the user can conveniently and quickly and accurately position the bleeding position of the patient.
Further, in order to facilitate a user to quickly operate the HoloLens glasses, the positioning device in this embodiment is further provided with a plurality of keys for outputting each operation instruction to the HoloLens glasses through communication, for example, a display switching key is provided to switch the display or hidden state of the skin of the human body and each positioning point; a reset key is arranged, so that the generated reference virtual registration point can be eliminated; the cancel key is arranged, so that a last fixed positioning point can be eliminated; the positioning point can be fixed after the positioning point is superposed with the head position of the patient corresponding to the positioning point.
Further, in order to facilitate the user to quickly move the reference virtual registration point to a specific position on the head of the patient, a prompt icon may be pasted on the head of the patient when four positioning positions are selected in advance.
As can be seen, the present embodiment generates a three-dimensional image in advance from CTA data of the head of a patient, and marks a bleeding position and at least three positioning points of the patient on the three-dimensional image; then generating a reference virtual registration point according to the positioning device, wherein the reference virtual registration point can establish virtual and real connection; the user wears HoloLens glasses, and sequentially moves each positioning point to the head position of the patient corresponding to the positioning point according to the reference virtual registration point, so that the three-dimensional image is superposed with the head of the patient, and the user is assisted to quickly and accurately position the bleeding position of the patient.
Referring to fig. 2, a second embodiment of the present invention relates to a positioning method based on HoloLens glasses, which is used for superimposing a virtual three-dimensional image and a real head of a patient by using the positioning system of the first embodiment, so as to present a bleeding position of the head of the patient on the real head of the patient, thereby facilitating the user to quickly position the bleeding position. The positioning method comprises the following steps:
s201, a three-dimensional image generated from CTA data of the head of the patient is presented to the user. The three-dimensional image comprises a marked bleeding position, human skin and at least three positioning points on the marked human skin.
Specifically, the CTA data of the patient's head, which is three-dimensional reconstructed mirror image data obtained by CT scanning, may show the bleeding position of the patient's head and the human skin. Based on the CTA data, the processor marks the bleeding position and the human skin, and selects a plurality of positioning points from different positions of the marked human skin to obtain a three-dimensional image. It should be understood that there are various ways to implement the labeling method, for example, the CTA recognition model can be pre-trained for recognition and labeling, and the CTA recognition model can also be manually labeled by a specially trained person and then uploaded to the processor, and the specific labeling method is not limited in this scheme.
It should be understood that the CTA data corresponds to the real head of the patient, and the user may predetermine the positions of the positioning points, for example, in this embodiment, the user selects four positioning positions in advance, which are respectively located on the left tragus, the right tragus, the left canthus and the right canthus; the processor, upon receiving the CTA data of the patient's head, finds the four localized positions from the marked human skin based on the position information.
HoloLens glasses are used for presenting the head of the patient and the marked bleeding position for the user according to instructions of the user. Specifically, after the user wears the HoloLens glasses, a download instruction can be input into the HoloLens glasses, and after receiving the download instruction, the HoloLens glasses download the three-dimensional image generated according to the CTA data of the head of the patient and present the three-dimensional image to the user.
S202, identifying a positioning device, generating a reference virtual registration point based on the positioning device and presenting the reference virtual registration point to a user; and sequentially moving each positioning point to the head position of the patient corresponding to the positioning point according to the reference virtual registration point, so as to realize the superposition of the three-dimensional image and the head of the patient.
Specifically, the positioning device is provided with an identification code which can be recognized by the HoloLens glasses, and the identification code can be set in various ways, such as preset pictures, two-dimensional codes or infrared rays.
After the HoloLens glasses recognize the identification code of the positioning device, a reference virtual registration point which is separated from the identification code by a fixed distance is generated based on the space position of the identification code, and when the space position of the positioning device changes, the reference virtual registration point can move along with the movement of the positioning device. In actual operation, when a user moves the positioning device, the reference virtual registration point moves along with the positioning device, and by adopting the scheme, the reference virtual registration point can be sequentially moved to the four positioning positions in a mode of moving the positioning device, for example, the user moves the positioning device, the reference virtual registration point displayed in the HoloLens glasses is moved to the left tragus position of a patient, when the reference virtual registration point and the reference virtual registration point are overlapped, the positioning point of the marked human skin corresponding to the left tragus position is moved to the position where the reference virtual registration point is located at the moment through the HoloLens glasses, and when the positioning point is overlapped with the reference virtual registration point, the positioning point is fixed until all the positioning points are fixed. It should be understood that the above-mentioned coincidence can be preset to a tolerance range as long as it is within the tolerance range, i.e., any two coincide. In addition, the three-dimensional image in this embodiment is a whole, that is, when the positioning points are moved to the corresponding left tragus position, the whole three-dimensional image is also moved integrally therewith, because the number of the positioning points in this embodiment is at least three, the three-dimensional image can be coincided with the real head of the patient.
S203, hiding the marked human skin and each positioning point, and presenting the head of the patient and the marked bleeding position for the user.
Specifically, after the three-dimensional image is overlapped with the real head of the patient, the user can hide the marked human skin and the positioning points through the HoloLens glasses, and only the head of the patient and the marked bleeding position are displayed to the user, so that the user can conveniently and quickly and accurately position the bleeding position of the patient.
As can be seen, the present embodiment generates a three-dimensional image in advance from CTA data of the head of a patient, and marks a bleeding position and at least three positioning points of the patient on the three-dimensional image; then generating a reference virtual registration point according to the positioning device, wherein the reference virtual registration point can establish virtual and real connection; the user wears the HoloLens glasses, and the positioning points are sequentially moved to the head position of the patient corresponding to the positioning points according to the reference virtual registration points, so that the three-dimensional image and the head of the patient are superposed, and the user is assisted to quickly and accurately position the bleeding position of the patient.
A third embodiment of the invention relates to a storage medium having stored thereon a computer program which, when executed by a processor, implements any one of the methods described in the second embodiment above. Preferably, the storage medium includes: various media that can store program codes, such as ROM, RAM, magnetic disk, U-disk, memory card, or optical disk.
Any combination of one or more storage media may be employed. The storage medium may be a computer-readable signal medium or a computer-readable storage medium. A computer readable storage medium may be, for example, but not limited to, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any combination of the foregoing. More specific examples (a non-exhaustive list) of the computer readable storage medium would include the following: an electrical connection having one or more wires, a portable computer diskette, a hard disk, a RAM, a ROM, an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disc read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing. In the context of this document, a computer readable storage medium may be any tangible medium that can contain, or store a program for use by or in connection with an instruction execution system, apparatus, or device.
A computer readable signal medium may include a propagated data signal with computer readable program code embodied therein, for example, in baseband or as part of a carrier wave. Such a propagated data signal may take any of a variety of forms, including, but not limited to, electro-magnetic, optical, or any suitable combination thereof. A computer readable signal medium may also be any computer readable medium that is not a computer readable storage medium and that can communicate, propagate, or transport a program for use by or in connection with an instruction execution system, apparatus, or device.
Program code embodied on a computer readable medium may be transmitted using any appropriate medium, including but not limited to wireless, wireline, optical fiber cable, RF, etc., or any suitable combination of the foregoing.
Computer program code for carrying out operations for aspects of the present invention may be written in any combination of one or more programming languages, including an object oriented programming language such as Java, smalltalk, C + + or the like and conventional procedural programming languages, such as the "C" programming language or similar programming languages. The program code may execute entirely on the user's computer, partly on the user's computer, as a stand-alone software package, partly on the user's computer and partly on a remote computer or entirely on the remote computer or server. In the case of a remote computer, the remote computer may be connected to the user's computer through any type of network, including a Local Area Network (LAN) or a Wide Area Network (WAN), or the connection may be made to an external computer (for example, through the Internet using an Internet service provider).
The present invention is described below with reference to flowchart illustrations and/or block diagrams of methods, apparatus (systems) and computer program products according to embodiments of the invention. It will be understood that each block of the flowchart illustrations and/or block diagrams, and combinations of blocks in the flowchart illustrations and/or block diagrams, can be implemented by computer program instructions. These computer program instructions may be provided to a processor of a general purpose computer, special purpose computer, or other programmable data processing apparatus to produce a machine, such that the computer program instructions, which execute via the processor of the computer or other programmable data processing apparatus, create means for implementing the functions/acts specified in the flowchart and/or block diagram block or blocks.
These computer program instructions may also be stored in a computer-readable medium that can direct a computer, other programmable data processing apparatus, or other devices to function in a particular manner, such that the instructions stored in the computer-readable medium produce an article of manufacture including instructions which implement the function/act specified in the flowchart and/or block diagram block or blocks.
The computer program instructions may also be loaded onto a computer, other programmable data processing apparatus, or other devices to cause a series of operational steps to be performed on the computer, other programmable apparatus or other devices to produce a computer implemented process such that the instructions which execute on the computer or other programmable apparatus provide processes for implementing the functions/acts specified in the flowchart and/or block diagram block or blocks.
Referring to fig. 3, a fourth embodiment of the present invention relates to a server, which includes a memory, a processor, and a computer program stored in the memory and running on the processor, and when the processor executes the computer program, the processor implements any one of the methods described in the second embodiment.
Specifically, the memory includes: various media that can store program codes, such as ROM, RAM, magnetic disk, U-disk, memory card, or optical disk.
Specifically, the Processor may be a general-purpose Processor, including a Central Processing Unit (CPU), a Network Processor (NP), and the like; the Integrated Circuit may also be a Digital Signal Processor (DSP), an Application Specific Integrated Circuit (ASIC), a Field Programmable Gate Array (FPGA) or other Programmable logic device, discrete Gate or transistor logic device, or discrete hardware components.
Where the memory and processor are connected by a bus, the bus may comprise any number of interconnected buses and bridges, the buses connecting together one or more of the various circuits of the processor and the memory. The bus may also connect various other circuits such as peripherals, voltage regulators, power management circuits, and the like, which are well known in the art, and therefore, will not be described any further herein. A bus interface provides an interface between the bus and the transceiver. The transceiver may be one element or a plurality of elements, such as a plurality of receivers and transmitters, providing a means for communicating with various other apparatus over a transmission medium. The data processed by the processor is transmitted over a wireless medium through an antenna, which further receives the data and transmits the data to the processor.
The processor is responsible for managing the bus and general processing and may also provide various functions including timing, peripheral interfaces, voltage regulation, power management, and other control functions. And the memory may be used to store data used by the processor in performing operations.
In summary, the positioning method and system based on the HoloLens glasses of the present invention generate a three-dimensional image in advance according to the CTA data of the head of the patient, and mark the bleeding position and at least three positioning points of the patient on the three-dimensional image; then generating a reference virtual registration point according to the positioning device, wherein the reference virtual registration point can establish virtual and real connection; the user wears the HoloLens glasses, and the positioning points are sequentially moved to the head position of the patient corresponding to the positioning points according to the reference virtual registration points, so that the three-dimensional image and the head of the patient are superposed, and the user is assisted to quickly and accurately position the bleeding position of the patient. Therefore, the invention effectively overcomes various defects in the prior art and has high industrial utilization value.
The foregoing embodiments are merely illustrative of the principles and utilities of the present invention and are not intended to limit the invention. Any person skilled in the art can modify or change the above-mentioned embodiments without departing from the spirit and scope of the present invention. Accordingly, it is intended that all equivalent modifications or changes which can be made by those skilled in the art without departing from the spirit and technical spirit of the present invention be covered by the claims of the present invention.
Claims (6)
1. A positioning method based on HoloLens glasses is characterized by comprising the following steps:
presenting to a user a three-dimensional image generated from CTA data of a patient's head; the three-dimensional image comprises a marked bleeding position, human skin and at least three positioning points on the marked human skin;
identifying a positioning device by using HoloLens glasses, generating a reference virtual registration point based on the positioning device, and presenting the reference virtual registration point to a user, wherein an identification code which can be identified by the HoloLens glasses is arranged on the positioning device, the identification code is a picture, and contents which can be identified by the HoloLens glasses are preset in the picture;
sequentially moving each positioning point to the head position of the patient corresponding to the positioning point according to the reference virtual registration point to realize the superposition of the three-dimensional image and the head of the patient;
hiding the marked human skin and each positioning point, and presenting the head of the patient and the marked bleeding position for the user;
the step of identifying the positioning device by using the HoloLens glasses and generating a reference virtual registration point based on the positioning device comprises the following steps:
after the HoloLens glasses recognize the identification code, based on the space position of the identification code, generating a reference virtual registration separated from the identification code by a fixed distance, wherein when the space position of the positioning device changes, the reference virtual registration point can move along with the movement of the positioning device;
the step of sequentially moving each positioning point to the head position of the patient corresponding to the positioning point according to the reference virtual registration point comprises the following steps:
and the user moves the reference virtual registration point to the head position of the patient corresponding to one positioning point in a mode of moving the positioning device, then moves the positioning point to the position of the reference virtual registration point through HoloLens glasses, and fixes the positioning point after the positioning point is superposed with the reference virtual registration point until the fixation of all the positioning points is completed.
2. The positioning method according to claim 1, further comprising: the step of generating a three-dimensional image from CTA data of the patient's head comprises:
obtaining CTA data of a patient's head;
marking the bleeding position of the head of the patient and the skin of the human body based on the CTA data, and selecting a plurality of positioning points from different positions of the marked skin of the human body to obtain a three-dimensional image.
3. A HoloLens glasses-based positioning system, comprising:
HoloLens glasses for presenting a three-dimensional image generated from CTA data of the head of a patient to a user; the three-dimensional image comprises a marked bleeding position, human skin and a plurality of positioning points on the marked human skin;
the system is also used for identifying a positioning device, generating a reference virtual registration point based on the positioning device and presenting the reference virtual registration point to a user;
the positioning points are sequentially moved to the positions of the heads of the patients corresponding to the positioning points according to the reference virtual registration points, so that the three-dimensional images are superposed with the heads of the patients;
the device is also used for hiding the marked human skin and each positioning point and presenting the head of the patient and the marked bleeding position for the user;
the positioning device is used for providing a positioning function for the HoloLens glasses;
the positioning device is provided with an identification code which can be identified by the HoloLens glasses, the identification code is a picture, and the picture is preset with content which can be identified by the HoloLens glasses;
the HoloLens glasses are further used for generating a reference virtual registration point which is separated from the identification code by a fixed distance based on the spatial position of the identification code after the identification code is identified; when the spatial position of the positioning device changes, the reference virtual registration point can move along with the movement of the positioning device;
the positioning device is also provided with a plurality of control keys for outputting control instructions to the HoloLens glasses;
the HoloLens glasses are further used for moving the positioning point to the position where the reference virtual registration point is located when the user moves the positioning device to move the reference virtual registration point displayed in the HoloLens glasses to the position of the head of the patient corresponding to the positioning point, and fixing the positioning point after the positioning point is overlapped with the reference virtual registration point until all the positioning points are fixed.
4. The positioning system of claim 3, further comprising:
the processing module is used for obtaining CTA data of the head of the patient, marking the bleeding position of the head of the patient and the skin of the human body based on the CTA data, and selecting a plurality of positioning points from different positions of the marked skin of the human body to obtain a three-dimensional image.
5. A server comprising a memory, a processor and a computer program stored on the memory and executable on the processor, characterized in that the processor implements the positioning method of claim 1 or 2 when executing the program.
6. A storage medium on which a computer program is stored, which program, when being executed by a processor, is adapted to carry out the positioning method of claim 1 or 2.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202210205741.8A CN114581635B (en) | 2022-03-03 | 2022-03-03 | Positioning method and system based on HoloLens glasses |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202210205741.8A CN114581635B (en) | 2022-03-03 | 2022-03-03 | Positioning method and system based on HoloLens glasses |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN114581635A CN114581635A (en) | 2022-06-03 |
| CN114581635B true CN114581635B (en) | 2023-03-24 |
Family
ID=81777552
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202210205741.8A Active CN114581635B (en) | 2022-03-03 | 2022-03-03 | Positioning method and system based on HoloLens glasses |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN114581635B (en) |
Citations (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| WO2017211087A1 (en) * | 2016-06-06 | 2017-12-14 | 北京理工大学 | Endoscopic surgery navigation method and system |
| JP6453501B1 (en) * | 2018-02-01 | 2019-01-16 | 株式会社Ｃｙｇａｍｅｓ | Mixed reality system, program, method, and portable terminal device |
| JP2019133642A (en) * | 2018-12-12 | 2019-08-08 | 株式会社Ｃｙｇａｍｅｓ | Mixed reality system, program, method, and mobile terminal device |
| CN110786877A (en) * | 2019-11-11 | 2020-02-14 | 北京推想科技有限公司 | Marking method, device and system for medical image |
| CN113168733A (en) * | 2019-01-04 | 2021-07-23 | 沃比帕克公司 | Virtual glasses try-on system and method |
Family Cites Families (12)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US10391277B2 (en) * | 2011-02-18 | 2019-08-27 | Voxel Rad, Ltd. | Systems and methods for 3D stereoscopic angiovision, angionavigation and angiotherapeutics |
| AU2014231341B2 (en) * | 2013-03-15 | 2019-06-06 | Synaptive Medical Inc. | System and method for dynamic validation, correction of registration for surgical navigation |
| GB2568426B (en) * | 2016-08-17 | 2021-12-15 | Synaptive Medical Inc | Methods and systems for registration of virtual space with real space in an augmented reality system |
| TWI634870B (en) * | 2017-03-20 | 2018-09-11 | 承鋆生醫股份有限公司 | Image registration and augmented reality system and method augmented reality thereof |
| US10438415B2 (en) * | 2017-04-07 | 2019-10-08 | Unveil, LLC | Systems and methods for mixed reality medical training |
| US10699448B2 (en) * | 2017-06-29 | 2020-06-30 | Covidien Lp | System and method for identifying, marking and navigating to a target using real time two dimensional fluoroscopic data |
| WO2019021236A1 (en) * | 2017-07-28 | 2019-01-31 | Edda Technology, Inc. | Method and system for surgical planning in a mixed reality environment |
| WO2021180331A1 (en) * | 2020-03-12 | 2021-09-16 | Brainlab Ag | Method for registering a virtual representation of a patient anatomy with a coordinate system of a physical patient anatomy |
| CN111631814B (en) * | 2020-06-11 | 2024-03-29 | 上海交通大学医学院附属第九人民医院 | Intraoperative blood vessel three-dimensional positioning navigation system and method |
| RU2754288C1 (en) * | 2020-10-06 | 2021-08-31 | Владимир Михайлович Иванов | Method for preparing for and performing a surgical operation on the head using mixed reality |
| CN113143457A (en) * | 2021-02-09 | 2021-07-23 | 席庆 | Maxillofacial operation auxiliary system and method based on MR head-mounted equipment |
| CN113893034A (en) * | 2021-09-23 | 2022-01-07 | 上海交通大学医学院附属第九人民医院 | Integrated operation navigation method, system and storage medium based on augmented reality |
-
2022
- 2022-03-03 CN CN202210205741.8A patent/CN114581635B/en active Active
Patent Citations (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| WO2017211087A1 (en) * | 2016-06-06 | 2017-12-14 | 北京理工大学 | Endoscopic surgery navigation method and system |
| JP6453501B1 (en) * | 2018-02-01 | 2019-01-16 | 株式会社Ｃｙｇａｍｅｓ | Mixed reality system, program, method, and portable terminal device |
| JP2019133642A (en) * | 2018-12-12 | 2019-08-08 | 株式会社Ｃｙｇａｍｅｓ | Mixed reality system, program, method, and mobile terminal device |
| CN113168733A (en) * | 2019-01-04 | 2021-07-23 | 沃比帕克公司 | Virtual glasses try-on system and method |
| CN110786877A (en) * | 2019-11-11 | 2020-02-14 | 北京推想科技有限公司 | Marking method, device and system for medical image |
Non-Patent Citations (3)
| Title |
|---|
| A neuroendoscopic navigation system based on dual-mode augmented reality for minimally invasive surgical treatment of hypertensive intracerebral hemorrhage;Tao Zhu等;《Computers in Biology and Medicine》;全文 * |
| Effect of marker position and size on the registration accuracy of HoloLens in a non-clinical setting with implications for high-precision surgical tasks;Laura Pérez-Pachón等;《International Journal of Computer Assisted Radiology and Surgery》;全文 * |
| 基于增强现实的下颌角截骨配准技术的研究;朱明等;《上海口腔医学》;20101215(第06期);全文 * |
Also Published As
| Publication number | Publication date |
|---|---|
| CN114581635A (en) | 2022-06-03 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| Léger et al. | Quantifying attention shifts in augmented reality image‐guided neurosurgery | |
| US20200188028A1 (en) | Systems and methods for augmented reality guidance | |
| US10977866B2 (en) | Output of position information of a medical instrument | |
| JP2022103224A (en) | Augmented reality viewing and tagging for medical procedures | |
| US9990744B2 (en) | Image registration device, image registration method, and image registration program | |
| EP3602492B1 (en) | Augmented reality patient positioning using an atlas | |
| Gsaxner et al. | The HoloLens in medicine: A systematic review and taxonomy | |
| de Oliveira et al. | A hand‐eye calibration method for augmented reality applied to computer‐assisted orthopedic surgery | |
| CN106251284B (en) | Medical image registration method based on facing | |
| Karmonik et al. | Augmented reality with virtual cerebral aneurysms: a feasibility study | |
| CN107789072B (en) | Intracranial lesion body surface holographic projection positioning system based on head-mounted augmented reality equipment | |
| CN111540008B (en) | Positioning method, device, system, electronic equipment and storage medium | |
| CN113893034A (en) | Integrated operation navigation method, system and storage medium based on augmented reality | |
| JP2022541233A (en) | System and method for recommending parameters for surgery | |
| EP3803540B1 (en) | Gesture control of medical displays | |
| Soeiro et al. | Visualizing the brain on a mixed reality smartphone application | |
| JP6933522B2 (en) | Medical image processing device, control method of medical image processing device, and program | |
| Egger et al. | Integration of the HTC Vive into the medical platform MeVisLab | |
| Gsaxner et al. | Augmented reality in oral and maxillofacial surgery | |
| Sugimoto | Cloud XR (extended reality: virtual reality, augmented reality, mixed reality) and 5g mobile communication system for medical image-guided holographic surgery and telemedicine | |
| CN115804652A (en) | Surgical operating system and method | |
| CN114581635B (en) | Positioning method and system based on HoloLens glasses | |
| CN110584780A (en) | Cerebral hemorrhage puncture operation navigation system | |
| US20190388177A1 (en) | Surgical navigation method and system using augmented reality | |
| CN111243713B (en) | Radiotherapy plan simulation method, equipment and medium based on virtual intelligent medical platform |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |