KR102442093B1 - Methods for improving surface registration in surgical navigation systems - Google Patents

Abstract

The present invention relates to a method for improving surface matching in surgical navigation systems to reduce matching errors in deep lesions. According to an embodiment of the present invention, the method for improving surface matching in surgical navigation systems comprises the steps of: acquiring medical image data of an object to be examined from a medical imaging device; acquiring camera image data of the object to be examined from a camera imaging device; extracting medical image point cluster data from the medical image data and extracting camera image point cluster data from the camera image data; training an artificial intelligence prediction model on the basis of the medical image point cluster; inputting the camera image point cluster data in the trained artificial intelligence prediction model to generate a predicted internal feature point cluster for the camera image; and using a medical image point cluster including internal feature point clusters for the medical image point cluster and a camera image point cluster including the predicted internal feature point cluster to perform surface matching so that two point clusters are disposed adjacent to each other.

Description

METHODS FOR IMPROVING SURFACE REGISTRATION IN SURGICAL NAVIGATION SYSTEMS

The present invention relates to a method for improving surface registration in a surgical navigation system, and more particularly, to a method for improving surface registration in a surgical navigation system using internal feature points predicted using artificial intelligence technology as additional registration points. .

Neuronavigation system or surgical navigation system is a medical device that supports accurate surgery by visualizing the three-dimensional position of a surgical tool on the patient's image taken before surgery in real time. It is a device that aligns medical image data acquired using MRI and patient data acquired in an operating room through an optical camera or electromagnetic generator in the same position to track the position of a surgical tool in real time.

Research to verify the clinical usefulness of such a surgical navigation system is continuously being conducted, and recently, it is recognized as an essential device for accurate surgery planning and execution, and its use is continuously increasing. The key to determining the accuracy of the surgical navigation system is matching the patient's anatomical body information acquired during surgery with the medical image information acquired before surgery, and this is usually done by patient-to-medical image matching. Also called -image registration).

The registration method can be largely divided into paired-point registration and contour-based registration. In point registration, a fiducial marker is attached to the patient's head and the image data It is a method of calculating the transformation between the medical image space and the actual patient space by acquiring the position of the corresponding marker marker using a tracking device during surgery. Point registration is currently most used in clinical practice because it provides accurate and stable results. However, additional medical imaging is required immediately before surgery, resulting in increased radiation exposure or medical costs. There is a disadvantage in that the accuracy can be guaranteed only when the marker markers are fixed until they become available.

On the other hand, the surface registration method using point clusters obtained from the face surface has the advantage that it does not require additional medical imaging and is convenient. There is a problem that the application of precision surgery is limited due to the effect. Specifically, the surgical navigation system tracks the relative positions of internal lesions and surgical tools through the matching of the medical image space and the patient space. Patient data (3D information based on a navigation system) and medical image data (3D information based on medical imaging equipment) used as inputs for registration are acquired in the form of a point cluster, which is a set of points distributed in a 3D space.

1 is a flowchart illustrating a conventional surface matching ICP algorithm. Iterative closest points (ICP) is used as a matching algorithm that matches two point clusters to the same location. ICP is an algorithm that calculates a rotation/translation matrix that minimizes the global error value between point clusters composed of different numbers, as shown in Equation 1 below.

R and t denote the rotation matrix and the translation matrix between the two point clusters, and Si and Ci denote the patient space point cluster and the medical image space point cluster. Since the number of medical image point clusters is relatively larger than that of the patient space point clusters, the process of searching for corresponding points on the medical image corresponding to the patient space point clusters is preceded. The registration is performed by obtaining a rotation and translation transformation matrix that minimizes the sum of the Euclidean distances between the corresponding points.

Since the two point clusters used in the ICP algorithm use different measurement equipment, the number and distribution of points constituting each point cluster are obtained differently. Therefore, as shown in FIG. 2, the minute attitude and heading error between the two matched point clusters caused by this shows a small matching error of the lesion located on the surface of the face, but the lesion matching error gradually increases as the depth increases. A residual rotation error (RRE) characteristic is generated. The above problems cause inaccurate positioning of surgical tools inserted inside the human body and a decrease in user reliability in the actual surgical environment that relies on the accuracy of surface registration, which can lead to problems such as medical accidents, postoperative complications, and limiting the scope of surgery. causes

Korean Patent Publication No. 2017-0012077 (2017.02.02)

The present invention provides a method capable of improving surface registration in a surgical navigation system by generating an additional point cluster using internal feature points in order to improve the depth error occurring in the conventional surgical navigation system registration, and using this The purpose.

According to an aspect of the present invention in order to solve the above problems, there is provided a method for improving surface registration in a surgical navigation system, the method comprising:

acquiring medical image data of an examination subject from a medical imaging apparatus;

acquiring camera image data of an inspection object from a camera imaging device;

extracting medical image point cluster data from the medical image data and extracting the camera image point cluster data from the camera image data;

learning an artificial intelligence prediction model based on the medical image point cluster;

generating a predicted internal feature point cluster for the camera image by inputting the camera image point cluster data to the learned artificial intelligence prediction model; and

performing surface registration so that the two point clusters are disposed adjacent to each other using a medical image point cluster including an internal feature point cluster for the medical image point cluster and a camera image point cluster including a predicted internal feature point cluster; includes

In the above-described aspect, the artificial intelligence prediction model includes the steps of extracting medical image point clusters and internal feature point data from a medical image, generating virtual patient point cluster data by downsampling the medical image point cluster data, and It is learned by learning the artificial intelligence prediction model using the relationship between the virtual patient point cluster data and the internal feature point cluster.

In addition, in any one of the above aspects, the medical image is at least one of a computed tomography (CT) device, a magnetic resonance imaging (MRI) device, and a positron emission tomography (PET) device. It may be an image taken from

In addition, in any one of the above aspects, the camera image may be an image taken from at least one of a three-dimensional scanner, an endoscope device, an optical camera, a C-arm device, and an optical coherence tomography device. .

In addition, in any one of the above aspects, the performing the surface registration comprises: a camera image point cluster including an internal feature point cluster predicted by the artificial intelligence model; Registration is performed so that the medical image point clusters including the feature point clusters inside the medical image are close to each other.

According to the present invention, by using an artificial intelligence model, it is possible to acquire internal anatomical feature points that cannot be obtained with a conventional navigation system, and by automatically generating and learning data necessary for the artificial intelligence model, feature point prediction is performed without user intervention. By learning the relative positional relationship between thousands of virtual patient point clusters and anatomical feature points inside the human body, it is possible to obtain internal anatomical feature points with high accuracy in the real patient space point cluster, as well as By using the point cluster data as an additional matching point, the registration error in deep lesions can be further reduced.

1 is a flow chart showing the flow of a conventional surface registration ICP algorithm;
2 is a diagram showing a problem in the prior art surface registration technique;
3 is a block diagram showing the configuration of a navigation system for surgery according to the present invention;
4 is a view showing a detailed configuration of a surface matching device;
5 is a diagram showing an example of an artificial intelligence learning model;
6 is a diagram illustrating an example of generating a virtual patient point cluster using point cluster downsampling;
7 is a diagram for explaining a point cluster center point and a direction vector;
8 is a diagram illustrating a network structure for predicting a three-dimensional movement value;
9 is a diagram illustrating an internal structure and a virtual lesion location used to explain measurement of lesion registration error;
10 is a diagram illustrating point cluster data including an internal feature point point cluster;
11 is a diagram showing a matching result including an internal feature point cluster;
12 is a diagram showing a flow of a method for learning an artificial intelligence model;
13 is a flowchart illustrating a surface registration process between a camera image and a medical image through a trained artificial intelligence prediction model.

Advantages and features of the present invention, and methods of achieving them, will become apparent with reference to the embodiments described below in detail in conjunction with the accompanying drawings. However, the present invention is not limited to the embodiments disclosed below, but will be implemented in various different forms. In the present specification, the present embodiment is provided to complete the disclosure of the present invention, and to completely inform those of ordinary skill in the art to which the present invention pertains to the scope of the present invention. And the invention is only defined by the scope of the claims. Accordingly, in some embodiments, well-known components, well-known operations, and well-known techniques have not been specifically described in order to avoid obscuring the present invention.

All terms (including technical and scientific terms) used in this specification may be used with meanings that can be commonly understood by those of ordinary skill in the art to which the present invention belongs. Also, terms defined in commonly used dictionary are not to be interpreted ideally or excessively unless defined.

3 is a view for explaining the configuration of a navigation system for surgery according to an embodiment of the present invention. As shown in FIG. 3 , the navigation system 100 for surgery according to an embodiment of the present invention includes a camera imaging device 110 , a medical imaging device 120 , and a surface matching device 130 .

The camera imaging apparatus 110 may be any one of an optical camera, a 3D scanner, an endoscope apparatus, a C-arm apparatus, and an optical coherence tomography apparatus provided in an operating space immediately before surgery. The camera imaging apparatus 110 acquires camera image data and transmits it to the surface matching apparatus 130 as patient data. Here, the patient data or camera image data may be camera image data photographed from any one of an optical camera, a three-dimensional scanner, an endoscope device, a C-arm device, and an optical coherence tomography device. .

The medical imaging apparatus 120 may be any one of a computed tomography (CT) apparatus, a magnetic resonance imaging (MRI) apparatus, and a positron emission tomography (PET) apparatus. The medical imaging apparatus 120 acquires medical image data and transmits it to the surface matching apparatus 130 . Here, the medical image data is medical image data captured from any one of a computed tomography (CT) device, a magnetic resonance imaging (MRI) device, and a positron emission tomography (PET) device. It may be, and it is configured to enable three-dimensional reconstruction of the surface and internal organs.

The surface matching device 130 is configured to accurately calculate the lesion location based on camera image data and medical image data, and in the present invention, the surface matching device performs a matching operation by applying artificial intelligence technology to accurately calculate the lesion location. configured to do

Specifically, the surface matching device 130 includes a data extraction unit 210 , an internal feature point generation unit 220 , and a surface matching unit 230 as shown in FIG. 4 . The data extraction unit 210 divides camera image data (patient image data) and medical image data for the surgical target surface, and point cluster data (medical) for each of the divided camera image data and medical image data for the surgical target surface. image point cluster data and camera image point cluster data) are extracted.

The internal feature point generator 220 is configured to predict and generate internal feature points when the camera image point cluster data is input to the learned artificial intelligence model. The artificial intelligence prediction model uses medical image point clusters (42,062 points) downsampling It is learned by the relationship between the generated multiple sets of virtual patient point clusters (591 points) and internal feature points. After learning is completed, when the camera image surface point cluster (591 points) is input to the AI prediction model, the AI prediction model generates internal feature points (121 points) for the camera image point cluster. Here, the internal feature point data is composed of an internal anatomical index of the human body that can be distinguished from the medical image with the naked eye.

Internal feature points existing in a medical image may be classified by an expert for hard tissue in which shape deformation according to an imaging environment is minimized. This is a point corresponding to the joint, protrusion, and pole of the bone, and the present invention is not limited thereto. and may include other clinically defined feature points. Each internal feature point is obtained as a coordinate value in three-dimensional space, which constitutes an artificial intelligence model that learns the relationship between the facial surface point cluster and the internal feature point of the medical image.

The surface matching unit 230 includes: a camera image surface point cluster (591 points) + an internal feature point cluster (121 points) predicted by the artificial intelligence model; Using the medical image point cluster (42,062 points) + medical image internal feature point cluster (121 points), registration is performed so that two point clusters are close together.

Next, the artificial intelligence learning module 500 will be described with reference to FIG. 5 . 5 shows an example of an artificial intelligence learning module for training an internal feature point generator. The artificial intelligence learning module 500 is a device capable of performing machine learning using training data, and may include a device that learns using a model composed of an artificial neural network. That is, the neural network training apparatus may be configured to receive, classify, store and output information to be used for data mining, data analysis, intelligent decision making, and machine learning algorithms. Here, the machine learning algorithm may include a deep learning algorithm.

The learning module 500 may communicate with at least one external device or terminal, and may derive a result by analyzing or learning data on behalf of or with the help of an external device. Here, helping other devices may mean distribution of computing power through distributed processing. The learning module 500 may generally mean a server, or may be referred to as a neural network learning server or the like.

The neural network learning apparatus 100 may include a communication unit 510, an input unit 520, a memory 530, a learning processor 540, and a processor 560. can

The communication unit 510 may refer to a configuration including a wireless communication unit (not shown) and an interface unit (not shown). That is, the communication unit 510 may transmit/receive data to and from the surface matching device through wired/wireless communication or an interface.

The input unit 520 may acquire training data for model learning or input data for acquiring an output using a trained model. The input unit 520 may acquire raw input data, and in this case, the learning processor 540 or the processor 560 pre-processes the acquired data to generate training data or pre-processed input data that can be input to model learning. can do.

The memory 530 may store the model learned by the learning processor 540 or the neural network learning apparatus 500 . In this case, the memory 530 may divide and store the learned model into a plurality of versions according to a learning time point or learning progress, etc. as needed. In this case, the memory 530 may store input data obtained from the input unit 520 , learning data (or training data) used for model learning, a learning history of the model, and the like. In this case, the input data stored in the memory 530 may be unprocessed input data itself as well as data processed to be suitable for model learning.

The memory 530 may include a model storage unit 531 , a database 532 , and the like. The model storage unit 531 stores the neural network model being trained or learned through the learning processor 540 , and when the model is updated through learning, the updated model is stored. In this case, the model storage unit 531 may divide and store the learned model into a plurality of versions according to a learning time point or learning progress, etc. as necessary.

The database 532 may store input data obtained from the input unit 520 , learning data (or training data) used for model learning, a learning history of the model, and the like. The input data stored in the database 532 may be raw input data itself as well as data processed to be suitable for model learning.

The learning processor 540 may train (train, or learn) the artificial neural network using training data or a training set. The learning processor 540 learns the artificial neural network by directly acquiring the data obtained by preprocessing the input data obtained by the processor 560 through the input unit 520 , or by acquiring the preprocessed input data stored in the database 532 to obtain the artificial neural network can learn

Specifically, the learning processor 540 may determine the optimized model parameters of the artificial neural network by repeatedly learning the artificial neural network using the various learning techniques described above. In the present specification, an artificial neural network whose parameters are determined by being trained using training data may be referred to as a learning model or a trained model.

The learning processor 540 may include a memory integrated or implemented in the neural network learning apparatus 500 . In some embodiments, the learning processor 540 may be implemented using the memory 530 . Alternatively or additionally, the learning processor 540 may be implemented using memory associated with the terminal, such as external memory coupled directly to the terminal or memory maintained in a server in communication with the terminal.

In detail, in order to provide an artificial intelligence prediction model (internal feature point generator) in the present invention, a learning data generation step and an AI model learning step for internal feature point prediction are included.

As the first step, the process of generating AI model training data is as follows.

go. To automatically generate time series data depicting patient point clusters without user intervention, a continuous time series data model is generated using medical image point cluster downsampling.

me. Downsampling is performed by randomly extracting surface points as much as a defined ratio from a medical image point cluster. In an embodiment of the present invention, a medical image point cluster consisting of 42,062 points was downsampled to the level of 591 patient point clusters (about 0.01%).

All. Since patient point clusters are acquired in different forms depending on the user's preference, proficiency, and lesion location, as the number of sampling repetitions increases, virtual patient point cluster data in various cases can be generated.

la. The AI model training data consists of a pair of input and output, respectively, and in the present invention, the relative coordinates of internal anatomical feature points are used as output information.

mind. Medical imaging equipment such as CT and MRI can acquire coordinates of internal anatomical features on medical image data using radiation or magnetic field that penetrates the human body.

bar. In an embodiment of the present invention, as shown in FIG. 6 , a relative position value between the coordinates of the center point of a point cluster downsampled from the three-dimensional reconstructed medical image image and the internal anatomical feature point is calculated and set as the output value of each input data. .

The process of learning input and output information using an artificial intelligence model is as follows.

go. Although not limited thereto according to an embodiment of the present invention, the artificial intelligence model has a multiple input structure composed of a network for extracting morphological features of a virtual patient point cluster and a network for processing numerical values of point cluster centers and direction vectors. .

me. The virtual patient point cluster is data showing geometric shapes (images) of points distributed on the surgical target surface, and shape features were extracted using a convolutional neural network (CNN), one of the artificial intelligence models. However, the present invention is not limited thereto, and network models such as Deep Neural Network (DNN), Recurrent Neural Network (RNN), Bidirectional Recurrent Deep Neural Network (BRDNN), and Multilayer Perceptron (MLP) may be included.

A neural network is an information processing system in which a number of neurons called nodes or processing elements are connected in the form of a layer structure by modeling the operating principle of biological neurons and the connection relationship between neurons. An artificial neural network may refer to an overall model having problem-solving ability by changing the strength of synapses through learning in which artificial neurons (nodes) formed a network by combining synapses. The term neural network or artificial neural network may be used interchangeably with the term neural network, and the artificial neural network may include a plurality of layers, and each of the layers may include a plurality of neurons. . Also, the artificial neural network may include neurons and synapses connecting neurons.

All. As shown in FIG. 7 , the point cluster center point and the direction vector are representative values of the downsampled point cluster, and since different values are calculated according to the point cluster distribution, they show a difference in the relative relationship to the same anatomical feature point coordinates.

la. 8 is a diagram illustrating a network structure for predicting a three-dimensional movement value. As a model input, the virtual patient space point cluster '[xi yi zi]xn' and the point cluster center coordinate '[xc yc zc]' & 3-axis direction vector '[ax ay az; bx by bz; cx cy cz]’ is used to repeatedly learn the network weight between the output value of the anatomical feature coordinate relative position ‘[xr yr zr]’.

mind. As such, the learned model is uploaded to the internal feature point generator 220 and predicts the 3-axis coordinate movement value for the patient point cluster input obtained in the actual operating space. can be predicted

In order to verify the improvement of the deep registration accuracy using the present invention described above, as shown in FIG. 9 , the lesion registration error was measured using a head model in which a structure depicting an internal lesion was inserted. The internal structure was grooved at regular intervals of height (7 cm) and depth (4 cm) to describe the lesion according to the location.

After generating 2,000 sets of virtual patient point clusters using the downsampling method for the medical image point clusters of the head model, the model was trained by dividing the training and validation rates by 80% and 20%, respectively.

The coordinates of the internal anatomical feature points were obtained using the predicted coordinate shift values by inputting the actual patient point cluster, which was not used for training, into the trained model.

10 is a diagram illustrating point cluster data including an internal point cluster according to the present invention. The predicted single coordinate is a major indicator that can improve the deep-level registration accuracy, but the rigid body transformation is performed with a high weight on the existing 591 point clusters used for surface registration, so the effect on the registration accuracy improvement is insignificant (591 + 1 points) association). Therefore, in the present invention, a point cluster in the form of concentric circles is generated centering on the predicted feature points to equalize the weights for calculating the rotation movement matrix (591 + 121 point clusters).

The ICP registration accuracy using the medical image point cluster including the internal point cluster and the patient image point cluster is compared with the target registration error of the existing surface registration after calculating the target registration error (TRE) at 27 internal target points. did.

Here, Target Medical P is the location of the internal target point (feature point) in the medical image.

Target Patient P means the location of the internal target point (feature point) in the camera image.

As shown in Fig. 11 and Table 1, the target points used were based on the face surface Proximal: [#1, #4, #7, #10, #13, #,16, #19, #22, # 25], Middle: [#2, #5, #8, #11, #14, #17, #20, #23, #26], Distal: [#3, #6, #9, #12, # 15, #18, 21, #24, #27], and the target registration errors are proximal mean 2.502 mm & 1.663 mm near the surface, middle middle mean 3.297 mm & 1.935 mm, and distal mean 4.383 located at the deepest part. mm & 2.525 mm, with the improvement of the depth error and the improvement of the registration error at all lesion locations.

[Table 1] Comparison of surface point cluster consistency and surface point cluster + internal point cluster consistency

That is, it can be seen that the average registration error of 27 internal target points is 2.041 mm when the method proposed in the present invention is applied, which improves the registration error by about 40% compared to 3.394 mm in the existing method using only surface point clusters.

12 is a diagram illustrating a flow of a learning method of an artificial intelligence model of a surgical navigation system as described above. As shown in FIG. 12 , the method of learning an artificial intelligence model includes first extracting a medical image point cluster and internal feature point data from a medical image in step S110. The medical image data may be medical image data taken from any one of a computed tomography (CT) device, a magnetic resonance imaging (MRI) device, and a positron emission tomography (PET) device. In addition, medical image point cluster data and internal feature point data are extracted from the restored 3D image and restored as a 3D image of the surface and internal organs.

In the subsequent step S120, several sets of virtual patient point clusters (591 points) are generated through downsampling of the medical image point cluster data.

In the subsequent step S130, it is checked whether the virtual patient point cluster data is sufficiently generated for artificial intelligence model learning.

In step S140, the artificial intelligence model is trained on the relationship between the virtual patient point cluster data and the internal feature point data to generate and end the artificial intelligence prediction model.

The generated artificial intelligence prediction model is input to the internal feature point generator 220 and the surface matching device including the internal feature point generator 220 performs the matching operation as shown in FIG. 13 .

13 is a flowchart illustrating a surface registration method using a learned artificial intelligence prediction model. As shown in FIG. 13 , the surface matching method using the artificial intelligence prediction model includes acquiring respective image data from the camera imaging apparatus 110 and the medical imaging apparatus 120 in step S210 .

The camera image data is image data obtained by photographing a body part of a patient, for example, a face surface, and may be referred to herein as patient data or patient image data. In the subsequent step S220 , the data extractor 210 acquires camera image (patient image) point cluster data from the camera image data from the camera image and medical image point cluster data from the medical image data. Medical image data such as CT is restored as a three-dimensional image of the patient's body surface and internal organs, and medical image point cluster data and internal feature point data are extracted from the restored three-dimensional image.

In the subsequent step S230, the patient image point cluster data obtained in step S220 is input to the AI prediction model, and internal feature points are predicted and generated by the AI prediction model.

After the internal feature points for the patient image point cluster are generated, in step S240, the surface matching unit 230 includes the camera image point cluster and the internal feature point cluster predicted from the artificial intelligence model; A matching operation is performed between the medical image point cluster and the medical image internal feature point cluster, that is, the two point clusters are close to each other.

After the registration operation is performed, in step S250, the surface matching unit calculates the registration accuracy and determines whether the calculated registration accuracy satisfies a predetermined reference value. If satisfied, the process is terminated.

According to the present invention, by using an artificial intelligence model, it is possible to acquire internal anatomical feature points that cannot be obtained with a conventional navigation system, and by automatically generating and learning data necessary for the artificial intelligence model, feature point prediction is performed without user intervention. By learning the relative positional relationship between thousands of virtual patient point clusters and anatomical feature points inside the human body, it is possible to obtain internal anatomical feature points with high accuracy in the real patient space point cluster, as well as By using the point cluster data as an additional matching point, the registration error in deep lesions can be further reduced.

According to the present invention, by using an artificial intelligence model, it is possible to acquire internal anatomical feature points that cannot be obtained with a conventional navigation system, and by automatically generating and learning data necessary for the artificial intelligence model, feature point prediction is performed without user intervention. By learning the relative positional relationship between thousands of virtual patient point clusters and anatomical feature points inside the human body, it is possible to obtain internal anatomical feature points with high accuracy in the real patient space point cluster, as well as By using the point cluster data as an additional matching point, the registration error in deep lesions can be further reduced.

The device described above may be implemented as a hardware component, a software component, and/or a combination of the hardware component and the software component. For example, devices and components described in the embodiments may include, for example, a processor, a controller, an arithmetic logic unit (ALU), a digital signal processor, a microcomputer, a field programmable gate array (FPGA). , a programmable logic unit (PLU), microprocessor, or any other device capable of executing and responding to instructions, may be implemented using one or more general purpose or special purpose computers. The processing device may execute an operating system (OS) and one or more software applications running on the operating system. The processing device may also access, store, manipulate, process, and generate data in response to execution of the software. For convenience of understanding, although one processing device is sometimes described as being used, one of ordinary skill in the art will recognize that the processing device includes a plurality of processing elements and/or a plurality of types of processing elements. It can be seen that it can include For example, the processing device may include a plurality of processors or one processor and one controller. Other processing configurations are also possible, such as parallel processors.

Software may comprise a computer program, code, instructions, or a combination of one or more thereof, which configures a processing device to operate as desired or is independently or collectively processed You can command the device. The software and/or data may be any kind of machine, component, physical device, virtual equipment, computer storage medium or device, to be interpreted by or to provide instructions or data to the processing device. may be embodied in The software may be distributed over networked computer systems and stored or executed in a distributed manner. Software and data may be stored in one or more computer-readable recording media.

The method according to the embodiment may be implemented in the form of program instructions that can be executed through various computer means and recorded in a computer-readable medium. The computer-readable medium may include program instructions, data files, data structures, etc. alone or in combination. The program instructions recorded on the medium may be specially designed and configured for the embodiment, or may be known and available to those skilled in the art of computer software. Examples of the computer-readable recording medium include magnetic media such as hard disks, floppy disks and magnetic tapes, optical media such as CD-ROMs and DVDs, and magnetic such as floppy disks. - includes magneto-optical media, and hardware devices specially configured to store and execute program instructions, such as ROM, RAM, flash memory, and the like. Examples of program instructions include not only machine language codes such as those generated by a compiler, but also high-level language codes that can be executed by a computer using an interpreter or the like.

As described above, although the embodiments have been described with reference to the limited embodiments and drawings, various modifications and variations are possible by those skilled in the art from the above description. For example, the described techniques are performed in a different order than the described method, and/or the described components of the system, structure, apparatus, circuit, etc. are combined or combined in a different form than the described method, or other components Or substituted or substituted by equivalents may achieve an appropriate result. Therefore, other implementations, other embodiments, and equivalents to the claims should be construed as falling within the scope of the following claims.

100: surgical navigation system 110: camera imaging device
120: medical imaging device 130: surface matching device
210: data extraction unit 220: internal feature point generation unit
230: surface matching unit 500: learning module

Claims (7)

A method for improving surface registration in a surgical navigation system including a camera imaging device, a medical imaging device, and a surface registration device, the method comprising:
acquiring medical image data of an examination subject from a medical imaging apparatus;
acquiring camera image data of an inspection object from a camera imaging device;
extracting medical image point cluster data from the medical image data using a surface matching device, and extracting the camera image point cluster data from the camera image data;
learning an artificial intelligence prediction model used in a surface matching device based on a medical image point cluster;
generating, in the surface matching device, a predicted human body feature point cluster for the camera image by inputting the camera image point cluster data to the learned artificial intelligence prediction model; and
The surface matching device uses a medical image point cluster including the internal body feature point cluster for the medical image point cluster and the camera image point cluster including the predicted body internal feature point cluster so that the two point clusters are placed adjacent to each other. performing matching; characterized in that it comprises
A method for improving surface registration in surgical navigation systems.
According to claim 1,
The artificial intelligence prediction model used in the surface matching device is,
extracting the medical image point cluster and human body internal feature point data from the medical image;
downsampling the medical image point cluster data to generate virtual patient point cluster data; and
Characterized in that the learning is performed by the step of learning the artificial intelligence prediction model using the relationship between the virtual patient point cluster data and the human body internal feature point cluster.
A method for improving surface registration in surgical navigation systems.
According to claim 1,
The medical image is an image taken from at least one of a computed tomography (CT) device, a magnetic resonance imaging (MRI) device, and a positron emission tomography (PET) device
A method for improving surface registration in surgical navigation systems.
According to claim 1,
The camera image is an image taken from at least one of a three-dimensional scanner, an endoscope device, an optical camera, a C-arm device, and an optical coherence tomography device.
A method for improving surface registration in surgical navigation systems.
3. The method of claim 2,
The performing of the surface registration in the surface matching device may include: a camera image point cluster including a human body internal feature point cluster predicted by an artificial intelligence model; Medical image comprising the internal feature point cluster of a medical image, characterized in that the registration is performed so that the point clusters are close to each other
A method for improving surface registration in surgical navigation systems.
3. The method of claim 2,
In the surface matching device, the downsampling of the medical image point cluster data comprises reducing the medical image point cluster to the level of the camera image point cluster.
A method for improving surface registration in surgical navigation systems.
3. The method of claim 2,
The human body internal feature point data is characterized in that it consists of an internal anatomical index of the human body that can be distinguished from the medical image with the naked eye.
A method for improving surface registration in surgical navigation systems.

Images