KR20180099039A - Image database-based real-time registration method of 2d x-ray image and 3d ct image, and an apparatus thereof - Google Patents

Abstract

A medical image database-based real-time registration method of a 2D X-ray image and a 3D CT image is suggested. The method includes: a step of constructing a database (DB) of a 2D X-ray image and a 3D CT image; a step of dividing the 2D X-ray image and the 3D CT image of the constructed database into various regions; a step of generating a registration prediction model using the divided 2D X-ray image and the 3D CT image; and a step of matching the 2D X-ray image and the 3D CT image in real time using the generated matching prediction model. It is possible to match the 2D X-ray and 3D CT images in real time without delay.

Description

TECHNICAL FIELD [0001] The present invention relates to a real-time matching method and apparatus for a two-dimensional X-ray image and a three-dimensional CT image based on an image database,

The present invention relates to a real-time matching method of a two-dimensional X-ray image and a three-dimensional CT image, and more particularly to a database of a two-dimensional X-ray image and a three-dimensional CT image, Dimensional image and three-dimensional (3D) CT images are divided into regions and machine training is performed to generate a matching prediction model, and a method for real-time image matching between heterogeneous images.

With the rapid development of interventional treatment techniques for coronary artery and vascular diseases and its related apparatuses, there has been actively attempted treatment of coronary artery or vascular diseases, which has been considered impossible in the past. For example, the area of interventions for chronic obstructive pulmonary disease, complex coronary artery disease, or structural heart disease is continuously expanding.

However, as the scope of the intervention is expanded and the difficulty of the procedure is increased, the related complications are inevitably increasing rapidly. The reason for this is that the procedure is performed with reference to the past image in a state in which the operation site can not be observed in real time. Therefore, the operation may be performed at an inappropriate position or the wound may be injured in the normal blood vessel while the prosthetic implant is moved into the blood vessel.

In order to minimize the unavoidable complications caused by these technical limitations, a technique of inter-x-ray matching has been developed to induce accurate procedures at the operation site. However, the present technology uses a CT coronary angiogram And it is still in the level of conducting the matching. Such matching between still images is merely a match between simple images that do not reflect movement of blood vessels due to actual breathing exercise and heartbeat, so that it is still very difficult to induce an operation in a medical field.

In other words, matching images of two-dimensional (2D) x-ray images and three-dimensional (3D) CT images can be very helpful in inducing cardiovascular treatment and intervention procedures. A single 2D X-ray image has the advantage of being able to accurately visualize the lesion and the position of the catheter by imaging the vessel in real time during the procedure. However, because of the limitation of the 2D image, And it is difficult to perform the procedure because of constraints such as overlapping of blood vessels.

In addition, in the case of a single 2D X-ray image, it is not possible to confirm the characteristics of the blood vessel or the lesion developed in the blood vessel. Therefore, a CT image having a three-dimensional structure and considering the characteristics of the inside of the blood vessel is required. Recently, research has been actively carried out to help the procedure through the matching technique of two images, which has advantages of distinguishing the three-dimensional structural feature of CT and the characteristic of blood vessel tissue and the advantage of real-time operation of X-ray imaging.

Currently, the best image matching performance in the world is realized by Siemens with 0.3 image frames per second and Erasmus Medical Center with 0.8 image frames per second. However, There is a limitation in that the performance is difficult to induce the procedure due to high delay in performing the real-time procedure induction.

For reference, this high delay occurs because of the high load of the computational complexity of the algorithms used for matching the image modality, and also because the image within the heartbeat period is at least 10 phases long, . For this reason, existing companies have developed and released a video-matching device designed to maintain the trade-off between the accuracy of matching and the speed of computation at an appropriate level, but the level is far below the level that can be used in clinical practice .

[Patent Literature 1] Korean Patent Registration No. 10-1485900 (entitled " Method of Matching CT Angiographic Image and X-ray Angiographic Image Based on Radiopaque Hemispherical Stereo Tag) "

SUMMARY OF THE INVENTION It is an object of the present invention to provide a method and apparatus for aligning a 2D X-ray image and a 3D CT image in real time without delay.

According to an aspect of the present invention, there is provided a method for real-time matching of a two-dimensional X-ray image and a three-dimensional CT image based on a medical image database, Constructing a database (DB) of the 3D CT image; Dividing the two-dimensional X-ray image and the three-dimensional CT image of the constructed database into regions; Generating a registration prediction model using the segmented two-dimensional X-ray image and the three-dimensional CT image; And matching the two-dimensional X-ray image and the three-dimensional CT image in real time using the generated matching prediction model.

Here, the real-time matching of the two-dimensional X-ray image and the three-dimensional CT image using the generated matching prediction model may include using non-rigid matching.

The step of generating a registration prediction model using the segmented two-dimensional X-ray image and the three-dimensional CT image may include extracting a registration prediction model from the two-dimensional X-ray image and the three- Extracting the blood vessel images using the CNNs; Machine training each of the extracted blood vessel images using a circular neural network (RNN); And constructing a matching prediction model based on the machine-learned blood vessel images.

In addition, the step of mechanically learning each of the extracted blood vessel images using a circular neural network (RNN) includes: applying a long short term memory (LSTM) to each of the extracted blood vessel images; And applying a predicted weight to each of the blood vessel images to which the LSTM is applied.

According to another aspect of the present invention, there is provided an image processing apparatus comprising: an image receiving unit configured to receive a 2D X-ray image and a 3D CT image of a heart; and an image processing unit configured to process a heart image received by the image receiving unit A display unit configured to display a 2D X-ray image and a 3D CT image matched in the image processing unit, and a control unit configured to control the image receiving unit, the image processing unit, and the display unit, The image processing unit includes a database building unit configured to construct a database of the two-dimensional X-ray image and the three-dimensional CT image; An area dividing unit configured to divide the two-dimensional X-ray image and the three-dimensional CT image of the constructed database into regions; A matching prediction model generation unit configured to generate a matching prediction model using the region-divided two-dimensional X-ray image and the three-dimensional CT image; And an image matching unit configured to match the two-dimensional X-ray image and the three-dimensional CT image in real time using the generated matching prediction model.

Here, the image matching unit may be configured to match two-dimensional X-ray images and three-dimensional CT images in real time using non-rigid body matching.

The matching prediction model generation unit may include a blood vessel image extraction unit configured to extract blood vessel images from the two-dimensional X-ray image and the three-dimensional CT image using the composite neural network (CNN), respectively; A machine learning unit configured to machine-learn each of the extracted blood vessel images using a circular neural network (RNN); And a matching prediction model building unit configured to build a matching prediction model based on the machine-learned blood vessel images.

In addition, the machine learning unit may be configured to apply LSTM (Long Short Term Memory) to each of the extracted blood vessel images and to give a predicted weight to each of the blood vessel images to which the LSTM is applied.

According to a method for real-time matching of a two-dimensional X-ray image and a three-dimensional CT image based on a medical image database according to an embodiment of the present invention and an image processing apparatus thereof, It becomes possible to match images in real time without delay.

According to a method for real-time matching of a two-dimensional X-ray image and a three-dimensional CT image based on a medical image database according to an embodiment of the present invention and an image processing apparatus thereof, Real-time matching of 2D CT images enables precise treatment according to the patient's condition.

In addition, according to a method for real-time matching of a two-dimensional X-ray image and a three-dimensional CT image based on a medical image database according to an embodiment of the present invention and an image processing apparatus thereof, It is possible to secure robustness and improve the accuracy of the technology by securing an accurate matching prediction model through the CNN based on the RNN.

1 is an exemplary view for explaining registration of a 2D X-ray image and a 3D CT image according to the prior art.
FIG. 2A is a flowchart illustrating a real-time matching method of a two-dimensional X-ray image and a three-dimensional CT image based on an image database according to an embodiment of the present invention, and FIG. 2B is a detailed flowchart of S230 of FIG. 2A.
3 is a schematic diagram illustrating a matching mechanism according to an embodiment of the present invention.
FIG. 4 is an exemplary view for explaining a method of segmenting a two-dimensional X-ray image and a three-dimensional CT image of a database constructed according to an embodiment of the present invention.
FIG. 5 is an exemplary diagram for illustrating a flow of generating a matching prediction model through a CNN based on a circular neural network (RNN) according to an embodiment of the present invention.
6A is a block diagram of an image processing apparatus 600 configured to implement a real-time matching method of a two-dimensional X-ray image based on an image database and a three-dimensional CT image according to an embodiment of the present invention. The matching prediction model generation unit 633 of FIG.

DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS Hereinafter, a detailed description of preferred embodiments of the present invention will be given with reference to the accompanying drawings. It should be noted that the same configurations of the drawings denote the same reference numerals as possible whenever possible. In the following description, specific details are set forth to provide a better understanding of the present invention. In the following description of the present invention, detailed description of known functions and configurations incorporated herein will be omitted when it may make the subject matter of the present invention rather unclear.

While the present invention has been described in connection with what is presently considered to be the most practical and preferred embodiment, it is to be understood that the invention is not limited to the disclosed embodiments, but, on the contrary, have. In addition, in certain cases, there may be a term selected arbitrarily by the applicant, in which case the meaning thereof will be described in detail in the description of the present invention. Therefore, the terms used in the present specification should be defined based on the meaning of the terms, not on the names of simple terms, and on the contents throughout the specification.

When an element is referred to as "including" an element throughout the specification, it is to be understood that the element may include other elements, without departing from the scope of the present invention. Also, the terms "part," " module, "and the like described in the specification mean units for processing at least one function or operation, which may be implemented in hardware or software or a combination of hardware and software .

1 is an exemplary view for explaining registration of a 2D X-ray image and a 3D CT image according to the prior art. Conventionally, various methods for matching a 2D X-ray image and a 3D CT image exist. In FIG. 1, an image matching method using the stereoscopic sign 10 is illustrated as an example.

1 (a) shows a 2D X-ray image, FIG. 1 (b) shows a 3D CT image, and FIG. 1 (c) shows a matching screen of two images by the three- Are illustrated by way of example.

As shown in the figure, the image matching between the stereoscopic sign 10 shown in Fig. 1A and the stereoscopic sign 10 shown in Fig. 1B is performed, As shown in the figure, the matching of the coronary arteries in the two images is performed as a result of the matching of the stereoscopic signs 10 of the two images.

The conventional matching method using the radiopaque stereoscopic sign 10 has an advantage that it can respond quickly to all external environments such as breathing or patient's movement, but the above-mentioned delay problem is still not solved.

In order to solve the problems of the conventional image matching method, the inventors of the present application intend to provide a method and apparatus for matching a 2D X-ray image and a 3D CT image in real time without delay, A detailed description will be given below.

FIG. 2A is a flowchart illustrating a real-time matching method of a two-dimensional X-ray image and a three-dimensional CT image based on an image database according to an exemplary embodiment of the present invention, and FIG. 2B is a detailed flowchart of S230 of FIG. 2A. For reference, in the following description, a heart image (in particular, a coronary artery image of a heart) is illustrated by way of example for ease of understanding of the present invention, but embodiments of the present invention may be applied to images other than the heart in the same manner It will be apparent.

As shown in FIG. 2A, a method for real-time matching two-dimensional X-ray images and three-dimensional CT images based on a medical image database according to an embodiment of the present invention includes two-dimensional X- Constructing a database (DB) of the 2D CT image (S210); A step (S220) of segmenting the two-dimensional X-ray image and the three-dimensional CT image of the constructed database, respectively; (S230) generating a registration prediction model using the region-partitioned two-dimensional X-ray image and the three-dimensional CT image; And matching the two-dimensional X-ray image and the three-dimensional CT image in real time using the generated matching prediction model (S240).

For reference, a series of steps S210 to S240 shown in FIG. 2A is a method for real-time matching of a two-dimensional X-ray image and a three-dimensional CT image based on a medical image database according to an embodiment of the present invention , And therefore it will be apparent that additional steps not shown in Figure 2a may additionally be performed.

S210 corresponds to a step of constructing a database (DB) of the two-dimensional X-ray image and the three-dimensional CT image. For example, a two-dimensional image of a human heart can be obtained using an X-ray imaging apparatus, and a three-dimensional image of a human heart can be acquired using a CT imaging apparatus.

For reference, throughout this specification, an X-ray image is described for a two-dimensional image and a CT image is described for a three-dimensional image. However, this is merely an example for easy understanding of the present invention, It will be apparent that any two-dimensional and two-dimensional images obtainable by any device / apparatus capable of acquiring images may be similarly applied to the present invention.

A database (DB) according to an embodiment of the present invention may include any type of database such as a hierarchical database, a networked database, a relational database, and a NoSQL database.

The database construction of the two-dimensional X-ray image and the three-dimensional CT image serves as a basis for generation of a large-data-based matching prediction model, which will be described later. Serve as an important foundation for implementing the matching method according to an embodiment of the present invention.

For reference, the construction of such a database may be performed in various ways according to various embodiments, such as by individual, by age, by gender, and the like.

When a database of the two-dimensional X-ray image and the three-dimensional CT image is constructed in step S210, the step S220 of dividing the two-dimensional X-ray image and the three-dimensional CT image of the constructed database may be performed. For example, S220 may include dividing a periodic image of the heart into a predetermined number (e.g., 10) of regions.

For reference, it is apparent that the region segmentation of such an image may be referred to as so-called " sampling ", and that the number of image region segmentations may vary according to various embodiments of the present invention.

The segmentation step S220 of the two-dimensional X-ray image and the three-dimensional CT image corresponds to an important step in that a divided image serving as a basis of the matching prediction model generation step S230, which will be described later, A detailed description of the segmentation of the line image and the three-dimensional CT image will be described later in detail with reference to FIG.

When the two-dimensional X-ray image and the three-dimensional CT image are region-divided in S220, a registration prediction model is generated using the region-divided two-dimensional X-ray image and the three-dimensional CT image (S230 ) Can be performed.

S230 is characterized by constructing a matching prediction model by machine training on a big data basis using a region-divided two-dimensional X-ray image and a three-dimensional CT image. More specifically, as shown in FIG. 2B, a matching prediction model generation step S230 according to an embodiment of the present invention uses a composite neural network (CNN) from a two-dimensional X-ray image and a three-dimensional CT image A step (S231) of extracting a blood vessel image, respectively; A step (S232) of mechanically learning each of the extracted blood vessel images using a circular neural network (RNN); And constructing a matching prediction model based on the machine-learned blood vessel images (S233).

Thus, by constructing the matching prediction model through machine learning using the CNN based on the circular neural network (RNN), accurate prediction model can be secured and the accuracy of the procedure can be improved. A detailed description will be described later in more detail with reference to FIG.

If a matching prediction model by machine learning is generated in step S230, a real-time matching step (S240) of the two-dimensional X-ray image and the three-dimensional CT image may be performed based on the generated matching prediction model.

Here, a series of steps (steps (a) and (b), which constitute a method for real-time matching of a two-dimensional X-ray image and a three-dimensional CT image based on a medical image database according to an embodiment of the present invention shown in Figs. S210 to S240 can be created as a program that can be executed by a computer. For example, the computer can be implemented as an image processing apparatus / device, and can be a general-purpose digital computer Lt; / RTI >

Generally, in the field of medical image processing, the processing of CT images, MRI images, X-ray images, etc. is accompanied by operations such as manipulating images, analyzing images, recognizing images, , This series of image processing operations can be performed by the image processing apparatus. In addition, the series of steps (steps) may be pre-programmed into a storage medium or a recording medium and executed to achieve a predetermined purpose when implemented by a predetermined apparatus (e.g., an image processing apparatus).

A series of steps S210 to S210 constituting a method for real-time matching of a two-dimensional X-ray image and a three-dimensional CT image based on a medical image database according to an embodiment of the present invention shown in Figs. S240) is also performed by a specific means using a combination of hardware and software constituting the image processing apparatus, and the specific configuration and description of the image processing apparatus configured to perform the method steps of the present invention will be described with reference to FIGS. 6A and 6B Will be described later in detail.

3 is a schematic diagram illustrating a matching mechanism according to an embodiment of the present invention.

3 (a) shows a conceptual diagram for generating a prediction model of a three-dimensional CT image 32 on a two-dimensional X-ray image 31. FIG. 3 (b) Non-rigid body matching is performed.

3 (c) exemplarily shows a non-rigid-body-matched image based on the real-time prediction data. 3D, the non-rigid body registration of the three-dimensional CT image 32 is performed on the two-dimensional X-ray image 31 as shown in FIG. 3 (c).

In this regard, according to a further embodiment of the present invention, matching the two-dimensional X-ray image and the three-dimensional CT image in real time (S240) using the generated matching prediction model, rigid registration. < / RTI > For reference, the specific description of the non-rigid body matching is substantially the same as known in the art and will be omitted in this paragraph.

FIG. 4 is an exemplary view for explaining a method of segmenting a two-dimensional X-ray image and a three-dimensional CT image of a database constructed according to an embodiment of the present invention.

The image matching method according to an embodiment of the present invention includes a step S220 of segmenting the two-dimensional X-ray image and the three-dimensional CT image of the constructed database, respectively. The concept of the image region dividing step S220 Is shown in Fig.

Although FIG. 4 illustrates an example of dividing one period of the heart into 10 phases / frames, it will be apparent that the number of image region divisions may vary according to various embodiments.

FIG. 4 shows a two-dimensional X-ray image. FIG. 4 shows a three-dimensional CT image on the lower side. FIG. For easy understanding of the embodiments of the present invention, it is assumed that 0%, 10%, 20%, 30%, 40%, 50%, 60%, 70%, 80% Respectively.

The region-divided two-dimensional X-ray image and the three-dimensional CT image are sources for generating a matching prediction model, and a detailed description thereof will be given below.

FIG. 5 is an exemplary diagram for illustrating a flow of generating a matching prediction model through a CNN based on a circular neural network (RNN) according to an embodiment of the present invention.

For reference, FIG. 4 exemplifies that each image (two-dimensional X-ray image and three-dimensional CT image) is divided into 10 parts, and therefore, the same example is applied to FIG. 5 as well.

As shown in FIG. 5, only the cardiovascular image can be extracted from 0% to 90% of the divided images of the 2-dimensional X-ray image and the 3-dimensional CT image using the composite neural network (CNN) 51.

Convolutional Neural Network (CNN) is a neural network that consists of one or more synthetic product layers and general artificial neural network layers placed on them, and performs preprocessing on the composite product layer. Convolutional neural network, circuit neural network, neural network, and the like.

4, when the machine learning is performed using the deep learning technique of the CNN on each of the two-dimensional X-ray image and the three-dimensional CT image segmented in the region, Only the blood vessel images of the heart can be extracted from the images.

For reference, in the embodiments of the present invention, since the features of the CNN are substantially applied as they are, detailed description of the CNN will be omitted in this section.

When the blood vessel image is extracted from the two-dimensional X-ray image and the three-dimensional CT image using the composite neural network (CNN), each of the extracted blood vessel images is subjected to machine training using a circular neural network (RNN) . For example, as shown in FIG. 5, machine learning using a circular neural network (RNN) includes applying LSTM (Long Short Term Memory) 53 to each extracted blood vessel image, And assigning a predicted weight 54 to each of them.

Recurrent Neural Network (RNN) is a deep learning method that considers current data and past data at the same time. Circular neural network (RNN) ).

For reference, a circular neural network (RNN) can utilize a memory inside a neural network to process an arbitrary input, and the circular neural network (RNN) is utilized in fields such as handwriting recognition and has a high recognition rate .

For example, a full recurrent network, a Hopfield network, an Elman network, an echo state network (ESN) network, and the like may be used in the RNN. , A long short term memory network (LSTM), a bi-directional RNN, a continuous-time RNN (CTRNN), a hierarchical RNN, and a secondary RNN. As a method for learning the RNN, a method such as a descending method, Hessian Free Optimization, or Global Optimization Method can be used.

As shown in FIG. 5, the machine learning according to the embodiment of the present invention is implemented by LSTM (short and long term memory) among various types of circular neural network (RNN) schemes. The LSTM corresponds to a special kind of circular neural network (RNN) capable of learning long-term dependencies, and each of the blood vessel images to which the LSTM 53 is applied can be given a predictive weight 54.

Therefore, the image of a specific time (e.g., t) is machine-learned using a CNN based CNN, thereby generating an image of a specific time (e.g., t-1) 1) temporal image as much as the ratio of the prediction weights, which is clearly contrasted with that the conventional image matching method only includes image information for a specific time.

6A is a block diagram of an image processing apparatus 600 configured to implement a real-time matching method of a two-dimensional X-ray image based on an image database and a three-dimensional CT image according to an embodiment of the present invention. The matching prediction model generation unit 633 of FIG.

For reference, a series of steps constituting a method for real-time matching of a two-dimensional X-ray image and a three-dimensional CT image based on a medical image database according to an embodiment of the present invention shown in Figs. (S210 to S240) can be created as a program that can be executed in a computer. For example, the computer can be implemented as an image processing apparatus, can be implemented in a general-purpose digital computer that operates a program using a computer- 6A is a block diagram of an image processing apparatus 600 configured to implement a real-time matching method of a two-dimensional X-ray image and a three-dimensional CT image based on an image database according to an exemplary embodiment of the present invention.

6A, an image processing apparatus 600 according to an embodiment of the present invention includes a control unit 610, an image receiving unit 620, an image processing unit 630, and a display unit 640 .

6A are merely examples for easy understanding of the present invention, and elements other than the elements shown in FIG. 6A are not shown in the image processing apparatus 600 As will be apparent to those skilled in the art.

The image receiving unit 620 may be configured to receive the two-dimensional X-ray image and the three-dimensional CT image. For example, in order to construct a database of the two-dimensional X-ray image and the three-dimensional CT image in S210, it is necessary to receive these images, and it is possible to receive the images through the image receiving unit 620. [

The image processing unit 630 may be configured to process the heart image received through the image receiving unit 620. 6A, the image processing unit 630 according to an embodiment of the present invention includes a database building unit 631, a region dividing unit 632, a matching prediction model generating unit 633, And a matching unit 634.

The database building unit 631 may be configured to construct a database of two-dimensional X-ray images and three-dimensional CT images. In addition, the region dividing unit 632 may be configured to divide the two-dimensional X-ray image and the three-dimensional CT image of the constructed database into regions. The matching prediction model generating unit 633 may be configured to generate a matching prediction model using the region-divided two-dimensional X-ray image and the three-dimensional CT image. In addition, the image matching unit 634 may be configured to match the two-dimensional X-ray image and the three-dimensional CT image in real time using the generated matching prediction model.

In this regard, the matching prediction model generation unit 633 extracts the blood vessel images from the two-dimensional X-ray image and the three-dimensional CT image using the composite neural network (CNN) as shown in FIG. 6B A blood vessel image extracting unit 633a configured; A machine learning unit 633b configured to machine-learn each of the extracted blood vessel images using a circular neural network (RNN); And a matching prediction model building unit 633c configured to build a matching prediction model based on the machine-learned blood vessel images.

Specific operations and descriptions of the units 631, 632, 633, and 634 of the image processing unit 630 are substantially the same as those described with reference to FIGS. 2 to 5, so that redundant explanations are omitted in this paragraph.

When the X-ray image and the CT image are matched by the image matching unit 634, the matched image is transmitted to the display unit 640, and the matched image can be displayed by the display unit 640. That is, the display unit 640 may be configured to output the matched 2D X-ray image and the 3D CT image.

The control unit 610 may be configured to control the image receiving unit 620, the image processing unit 630, and the display unit 640 as a whole. For example, the controller 610 may be implemented as a single controller, or may be implemented as a plurality of micro-controllers.

Therefore, according to the method for real-time matching of a two-dimensional X-ray image and a three-dimensional CT image based on a medical image database according to an embodiment of the present invention, Dimensional CT images can be matched in real time without delay.

According to a method for real-time matching of a two-dimensional X-ray image and a three-dimensional CT image based on a medical image database according to an embodiment of the present invention and an image processing apparatus thereof, Real-time matching of 2D CT images enables precise treatment according to the patient's condition.

In addition, according to a method for real-time matching of a two-dimensional X-ray image and a three-dimensional CT image based on a medical image database according to an embodiment of the present invention and an image processing apparatus thereof, It is possible to secure robustness and improve the accuracy of the technology by securing an accurate matching prediction model through the CNN based on the RNN.

The above-described embodiments of the present invention can be embodied in a general-purpose digital computer that can be created as a program that can be executed by a computer and operates the program using a computer-readable recording medium.

Computer readable media can be any available media that can be accessed by a computer and includes both volatile and nonvolatile media, removable and non-removable media. In addition, the computer-readable medium may include both computer storage media and communication media. Computer storage media includes both volatile and nonvolatile, removable and non-removable media implemented in any method or technology for storage of information such as computer readable instructions, data structures, program modules or other data. Communication media typically includes any information delivery media, including computer readable instructions, data structures, program modules, or other data in a modulated data signal such as a carrier wave, or other transport mechanism.

It will be understood by those skilled in the art that the foregoing description of the present invention is for illustrative purposes only and that those of ordinary skill in the art can readily understand that various changes and modifications may be made without departing from the spirit or essential characteristics of the present invention. will be. It is therefore to be understood that the above-described embodiments are illustrative in all aspects and not restrictive. For example, each component described as a single entity may be distributed and implemented, and components described as being distributed may also be implemented in a combined form.

The scope of the present invention is defined by the appended claims rather than the detailed description and all changes or modifications derived from the meaning and scope of the claims and their equivalents are to be construed as being included within the scope of the present invention something to do.

31: 2D X-ray image 32: 3D CT image
51: Composite Neural Network 52: Circular Neural Network
53: LSTM 54: predicted weight
600: image processing apparatus 610:
620: image receiving unit 630: image processing unit
631: Database building unit 632: Region dividing unit
633: matching prediction model generation unit 633a: blood vessel image extraction unit
633b: Machine learning unit 633c: Match prediction model building unit
634: Image matching unit 640:

Claims (8)

A method for real-time matching of a two-dimensional X-ray image and a three-dimensional CT image based on a medical image database,
Constructing a database (DB) of the two-dimensional X-ray image and the three-dimensional CT image;
Dividing the two-dimensional X-ray image and the three-dimensional CT image of the constructed database into regions;
Generating a registration prediction model using the segmented two-dimensional X-ray image and the three-dimensional CT image; And
And matching the two-dimensional X-ray image and the three-dimensional CT image in real time using the generated matching prediction model
/ RTI >
A method for real-time registration of a two-dimensional X-ray image and a three-dimensional CT image based on a medical image database. The method according to claim 1,
Wherein real-time matching of the two-dimensional X-ray image and the three-dimensional CT image using the generated matching prediction model comprises using non-rigid matching,
A method for real-time registration of a two-dimensional X-ray image and a three-dimensional CT image based on a medical image database. 3. The method of claim 2,
The step of generating a registration prediction model using the segmented two-dimensional X-ray image and the three-dimensional CT image may include:
Extracting a blood vessel image from a two-dimensional X-ray image and a three-dimensional CT image using a composite neural network (CNN);
Machine training each of the extracted blood vessel images using a circular neural network (RNN); And
Constructing a matching prediction model based on the machine-learned blood vessel images
/ RTI >
A method for real-time registration of a two-dimensional X-ray image and a three-dimensional CT image based on a medical image database. The method of claim 3,
The step of mechanically learning each of the extracted blood vessel images using a circular neural network (RNN)
Applying an LSTM (Long Short Term Memory) to each of the extracted blood vessel images; And
A step of assigning a predicted weight to each of the blood vessel images to which the LSTM is applied
/ RTI >
A method for real-time registration of a two-dimensional X-ray image and a three-dimensional CT image based on a medical image database. An image processing apparatus configured to perform the method according to any one of claims 1 to 4,
An image receiving unit configured to receive a 2D X-ray image and a 3D CT image of the heart;
An image processor configured to process a heart image received by the image receiver;
A display unit configured to display a 2D X-ray image and a 3D CT image matched in the image processing unit, and
A control unit configured to control the image receiving unit, the image processing unit,
Lt; / RTI >
Wherein the image processing unit comprises:
A database building unit configured to construct a database of a two-dimensional X-ray image and a three-dimensional CT image;
An area dividing unit configured to divide the two-dimensional X-ray image and the three-dimensional CT image of the constructed database into regions;
A matching prediction model generation unit configured to generate a matching prediction model using the region-divided two-dimensional X-ray image and the three-dimensional CT image; And
An image matching unit configured to match the two-dimensional X-ray image and the three-dimensional CT image in real time using the generated matching prediction model,
/ RTI >
Image processing apparatus. 6. The method of claim 5,
Wherein the image matching unit is configured to match two-dimensional X-ray images and three-dimensional CT images in real time using non-rigid body matching,
Image processing apparatus. The method according to claim 6,
Wherein the matching prediction model generation unit comprises:
A blood vessel image extraction unit configured to extract blood vessel images from a two-dimensional X-ray image and a three-dimensional CT image using a composite neural network (CNN);
A machine learning unit configured to machine-learn each of the extracted blood vessel images using a circular neural network (RNN); And
A matching prediction model building unit configured to build a matching prediction model based on the machine-
/ RTI >
Image processing apparatus. 8. The method of claim 7,
The machine learning unit comprises:
Applying a LSTM (Long Short Term Memory) to each of the extracted blood vessel images, and applying a predicted weight to each of the blood vessel images to which the LSTM is applied,
Image processing apparatus.

Images