KR20150115438A - Method and apparatus for overlaying medical images included the region of the heart - Google Patents

Abstract

Disclosed are a method and apparatus for overlaying a medical image including a heart region. The method for overlaying a medical image according to an embodiment of the present invention comprises the steps of: receiving a first medical image including an image of a heart region of a testee; receiving a second medical image including an image of a cardiac region of the testee; extracting two-dimensional cardiovascular information from the received first medical image and extracting three-dimensional heart information from the second medical image; extracting information regarding a transformation matrix between the first medical image and the second medical image by matching the extracted two-dimensional cardiovascular information, and three-dimensional cardiovascular information included in the three-dimensional heart information; and applying the extracted transformation matrix information to the three-dimensional heart information and overlaying the three-dimensional heart information applied with the transfer matrix information to the first medical image.

Description

TECHNICAL FIELD [0001] The present invention relates to a medical image overlay method and an apparatus for overlaying a medical image,

The present invention relates to a medical image overlay, and more particularly, to a medical image overlay method and apparatus that can assist in finding an accurate position of a blood vessel during cardiac angiography or stenting.

The present invention is derived from research carried out by the Ministry of Industry, Commerce, and Industry and the Korea Industrial Technology Evaluation and Management Institute as a part of the project for the development of technology for industrial convergence [Project Number: 10044910, Title: Diagnosis of Vascular Disease - Development of Integrated Medical Treatment Support System].

X-ray fluoroscopy (real-time X-ray video) is used for many minimally invasive procedures such as percutaneous coronary interventions (PCI) of chronic total occlusions It is the modality chosen for the guide. During minimally invasive procedures, crossing of the CTO using a guide-wire can prevent occlusion of the contrast agent from spreading and block the occluded portion of the blood vessel by fluoroscopy it can cause difficulties because it makes it invisible in fluoroscopy. In addition, the projective nature of fluoroscopy obscures the interpretation of a three-dimensional (3D) structure, which further complicates the intervention. These problems exist in a number of minimally invasive procedures.

Current pre-interventional planning involves routinely acquiring computed tomography angiography (CTA) or other 3D imaging modalities and may reduce visual uncertainty. The performance characteristics of CTA using blood pool contrast injection make it possible to clearly distinguish the calcifications responsible for CTO, as opposed to what is recognized by interventional X-ray imaging with direct contrast injection. While these CTAs are very useful tools in the planning stage, establishing a correspondence between their data and interventional images has proven to be challenging.

To address this issue, the pre-procedure 3D model may be extracted from the acquired volume, aligned with the view of the 2D fluoroscopy, and overlaid on the live image to enhance the interventional image. However, achieving this alignment is itself challenging because it is non-trivial to find intermodal correspondence and essentially because the optimization process is nonlinear. In addition, simple rigid transforms may not be sufficient to provide satisfactory 2D / 3D registration. The 3D plan images are acquired in a breathhold state and there is a significant shape change when compared to the intraoperative images acquired in free respiratory state. The non-rigid body matching method is useful for aligning images during preoperative and live surgery. However, the non-rigid matching method is a challenging ill-posed inverse task. Nonetheless, the 2D / 3D matching method has the potential to significantly reduce the uncertainty associated with interventional X-ray angiography, merely adding minimal changes to existing clinical flows. From a broad perspective, 2D / 3D matching methods can have a variety of applications in areas such as neurology, orthopedics, and cardiology.

The optimization surface of 2D / 3D matching can be nonlinear for several reasons such as image discretization, the complexity of the structure to be matched, and the non-rigid transformation model. As a result, minimizing the cost function is a challenging problem. The maximum accuracy that can be achieved by the matching process is related to the complexity of the associated transformation model. Many 2D / 3D matching approaches only consider models that are rigid or inflexible. In general, the rigid-body transformation model or the slightly flexible element model is suitable for providing initial alignment of rigid bodies or modalities such as bones, but it has proven to be insufficient to account for the shape change of the flexible structure. As a result, a non-rigid deformation model has been proposed for 2D / 3D matching.

In a clinical situation, one or two transmission surfaces are used for interrogation guidance. Monoplane scenarios have been studied, but the reported errors in the out-of-plane direction are large, and this error in accuracy limits reliability in the registration process. B eplane acquisition significantly reduces uncertainty associated with these interventions and is being performed in complex cardiac interventions.

As an example of the prior art for the conventional 2D / 3D matching method, U.S. Patent Publication No. 2013-0094745 discloses a technique for overlaying a 2D cardiovascular image and a 3D cardiovascular image through non-rigid body matching, to be.

However, conventional prior art overlays only 2D cardiovascular image and 3D cardiovascular image, so the user using it can not know the position of the atrium, ventricle, and myocardium of the heart. The use of pericardiocentesis, pacemaker implantation, myocardial biopsy, and percutaneous transluminal coronary angioplasty can be used for image enhancement of cardiovascular information such as angiography and stent implantation. , And septal ablation (heart ablation), there is a problem that the exact location of the atria, ventricles, and myocardium can not be found during the procedure requiring the structure and location information of the heart.

Therefore, there is a need for a method that can precisely provide the position of the atria, ventricles, and myocardium in the heart during the procedures requiring the structure and position information of the heart.

U.S. Published Patent Application No. 2013-0094745 entitled " NON-RIGID 2D_3D REGISTRATION OF CORONARY ARTERY MODELS WITH LIVE FLUOROSCOPY IMAGES "

SUMMARY OF THE INVENTION Accordingly, the present invention has been made to solve the above-mentioned problems occurring in the prior art, and it is an object of the present invention to provide a cardiac angiogram, a stent, And to provide a medical image overlay method and apparatus thereof that can help in finding the structure and position of a medical image.

Specifically, the present invention overcomes the X-ray angiography image and the 3D cardiac information, thereby accurately grasping the cardiovascular position in the heart, thereby providing not only cardiovascular angiography and stenting, Heart, heart, and myocardial location can be identified during heart surgery such as alcohol, pacemaker implantation, myocardial biopsy, and myocardial necrosis, and accurate technology can be supported.

It is another object of the present invention to provide a medical image overlay method and apparatus that can provide the practitioner with intuitive and easy-to-understand structure and position information of the heart such as cardiovascular, atrial, ventricle, and myocardial.

Further, the present invention aims at selecting the information of the medical image that is adaptively overlaid according to the type and form of the minimally invasive procedure to more effectively perform the guidance of the interventional procedure.

According to another aspect of the present invention, there is provided a medical image overlay method comprising: receiving a first medical image including an image of a heart of a subject; Receiving a second medical image including an image of the heart region of the subject; Extracting two-dimensional cardiovascular information from the received first medical image and extracting three-dimensional cardiac information from the second medical image; Extracting transformation matrix information between the first medical image and the second medical image using the extracted two-dimensional cardiovascular information and matching of three-dimensional cardiovascular information included in the three-dimensional cardiac information; And applying the extracted transformation matrix information to the 3D cardiac information and overlaying the 3D cardiac information to which the transformation matrix information is applied on the first medical image.

Wherein the overlaying step comprises the steps of: selecting at least one image corresponding to a treatment type of the subject out of a plurality of images included in the 3D cardiac information, selecting at least one of the at least one image selected for the treatment type of the subject And may overlay the first medical image considering at least one of the determined layout, structure, and the shape of the at least one image.

The overlaying may include overlaying the thickness information on the outer wall of the heart when the outer wall of the heart is included in the overlayed 3D image information.

The three-dimensional cardiac information may include three-dimensional cardiovascular information, 4-chamber information including an atrium and a ventricle, cardiac periphery information, and myocardial information.

The step of extracting the transformation matrix information may extract the transformation matrix information using the non-rigid registration of the 2D cardiovascular information and the 3D cardiovascular information.

The 3D cardiac information overlaid on the first medical image may be adjusted in transparency by a user's input using a user interface.

A medical image overlay apparatus according to an embodiment of the present invention includes a receiving unit for receiving a first medical image and a second medical image including an image of a heart portion of a subject; An image information extracting unit for extracting two-dimensional cardiovascular information from the received first medical image and extracting three-dimensional cardiac information from the second medical image; A transformation matrix extraction unit for extracting transformation matrix information between the first medical image and the second medical image using the extracted two-dimensional cardiovascular information and the matching of three-dimensional cardiovascular information included in the three-dimensional cardiac information; And an image processor for applying the extracted transformation matrix information to the 3D cardiac information and overlaying the 3D cardiac information to which the transformation matrix information is applied on the first medical image.

According to the present invention, by overlaying cardiac periphery information on cardiovascular images, cardiac angiography, stent procedures, and the like, which can assist in finding the exact structure and position of cardiovascular, atrial, ventricular, and myocardial, , As well as cardiac procedures that require cardiac structure and location information.

In addition, according to the present invention, 3D heart information including cardiac periphery information is overlaid and displayed on an X-ray fluoroscopy, thereby providing a practitioner with intuitive cardiovascular, atrial, ventricular, and myocardial structure and position , Can help a variety of procedures.

For example, the present invention may be useful for performing procedures such as pericardial puncture, pacemaker implantation, myocardial biopsy, myocardial necrosis, and atrial fibrillation during cardiac arrhythmia. These procedures are examples of various procedures that require information about the structure and location of the heart.

According to the present invention, in various kinds of cardiac / cardiovascular intervention procedures, appropriate guidance guidance information is provided as images according to the type and type of the operation, so that the guidance / navigation of the operation can be effectively performed. According to the present invention, if the position and contour information of the 4-chamber (2-chamber, 2-chamber) of the heart is required, the 4-chamber information of the heart is overlaid. If information about the thickness of the outer wall of the ventricle or atrium is required, Or an image of the outer wall of the atrium, and when the information about the position of the myocardial muscle is required, the operator can overlay the image of the myocardial muscle. Therefore, the practitioner can efficiently confirm the information necessary for the operation together with the real-time image information of the subject.

1 is a flowchart illustrating an operation of a medical image overlay method according to an embodiment of the present invention.
FIG. 2 shows an operational flow diagram of an embodiment of step S150 shown in FIG.
FIG. 3 illustrates an example of a process for extracting 2D cardiovascular information.
FIG. 4 illustrates an example of a process for extracting three-dimensional cardiac information.
FIG. 5 illustrates an example of a process of extracting transformation matrix information using 2D cardiovascular information and 3D cardiovascular information.
6 is a view illustrating an example of a screen in which a first medical image is overlapped with a second medical image.
FIG. 7 illustrates a configuration of a medical image overlay apparatus according to an embodiment of the present invention.

Other objects and features of the present invention will become apparent from the following description of embodiments with reference to the accompanying drawings.

Preferred embodiments of the present invention will be described in detail with reference to the accompanying drawings. In the following description of the present invention, a detailed description of known functions and configurations incorporated herein will be omitted when it may make the subject matter of the present invention rather unclear.

Hereinafter, a method and an apparatus for overlaying a medical image including a heart region according to an embodiment of the present invention will be described in detail with reference to FIGS. 1 to 7.

The present invention overcomes the 3D cardiac information including the cardiac periphery information on the X-ray angiography image including the heart region of the subject, thereby enabling the operator to perform a variety of cardiac procedures requiring the structure and position information of the heart, I would like to help you find your location. Herein, the term 'subject' expressed in the present specification is generally a term including the concept of 'patient'. The 'subject' does not necessarily have to be suffering from the disease and should be interpreted as a term referring to the subject of the intervention.

1 is a flowchart illustrating an operation of a medical image overlay method according to an embodiment of the present invention.

Referring to FIG. 1, a method according to the present invention receives a first medical image including a heart region of a subject (S110).

Here, the first medical image may be a real-time medical image of the subject photographed by, for example, X-ray angiography. Of course, the first medical image is not limited to the medical image photographed by the X-ray angiography, but may include a medical imaging device that can photograph the cardiovascular body of the subject in real time or a medical image photographed by the imaging method. Such real-time medical imaging is sometimes referred to as fluoroscopy in the art.

Of course, the first medical image received in step S110 may be a real-time moving image or an image of any one of the photographed images.

Then, the second medical image including the heart portion of the subject is received (S120).

Here, the second medical image is an image having a resolution and a rich information enough to form a three-dimensional model, and is a computer tomography (CT) apparatus, a magnetic resonance imaging (MRI) Or the like.

The second medical image received in step S120 may be a 3D medical image reconstructed by photographing a heart part of the subject. Such a 3D medical image may be previously photographed and stored in advance in a system such as a medical image storage transmission system (PACS) Lt; / RTI >

Of course, the 3D medical image may be a medical image photographed at the time of photographing determined by the user, that is, the operator, based on the time of the patient's operation.

When the first medical image and the second medical image are received in steps S110 and S120, 2D cardiovascular information is extracted from the received first medical image and 3D card information is extracted from the second medical image (S130).

Here, the 3D cardiac information extracted in step S130 may include 3D cardiovascular information, 4-chamber information including the atrium and the ventricle, cardiac outline information, cardiac information, and the like. In addition, information on the outer wall of the heart and the thickness of the outer wall Information.

Although it is illustrated in FIG. 1 that step S110 in which the first medical image is received and step S120 in which the second medical image is received are sequentially executed, steps S110 and S120 may be executed as independent events, none. In addition, steps S110 and S120 do not refer to a process in which each of the first medical image and the second medical image is generated, and each of the first medical image and the second medical image may be generated in advance or is currently being generated , And steps S110 and S120 indicate the process of receiving each medical image. According to various embodiments of the present invention, the second medical image is an image set having precise and rich information such as CT or MRI images, and may be generated before the patient's procedure. In step S130, The portion for extracting the 3D cardiac information from the image may take a long time and may be executed before step S110.

The first medical image may be generated during the procedure of the subject, and the process of extracting 2D cardiovascular information from the first medical image in step S130 may be continuously updated and replayed in real time.

A process of extracting 2D cardiovascular information from the first medical image will be described with reference to FIG.

As shown in FIG. 3A, when the first medical image photographed by the X-ray angiography is received, the received first medical image is converted using the binarization or thresholding, as shown in FIG. 3B do.

Here, the binarization method is a method of classifying a pixel value constituting a medical image into 0 and 1 values based on a specific threshold value or a boundary value.

As shown in FIG. 3C, the noise is removed from the first medical image converted by the binarization technique, and the noise-eliminated image is subjected to thinning Thereby extracting 2D cardiovascular information on the cardiovascular included in the first medical image.

Similarly, the process of extracting 3D cardiac information from the second medical image will be described with reference to FIG.

As shown in FIG. 4A, when the 3D medical image 410 of the subject photographed and reconstructed from the MRI apparatus and the CT apparatus is received, as shown in FIG. 4B, a feature point or a seed for a cardiovascular a region is grown based on input minutiae or seeds, an end point search and a mincost path process are performed, and then a thinning process is applied to the subject, The 3D cardiovascular information 420 is extracted from the 3D medical image.

At this time, the extracted 3D cardiovascular information is projected on a specific surface as shown in FIG. 4B, so that cardiovascular information matched with the 2D cardiovascular information extracted in FIG. 3 can be extracted. 3D cardiovascular information can be represented as a voxel with x, y, z coordinates. At this time, the 3D cardiovascular information is a 3D model, and in order to match with the 2D cardiovascular information extracted in FIG. 3, the 3D cardiovascular information can be projected on the reference surface with a plane perpendicular to the direction in which the first medical image is captured as a reference plane .

As shown in FIG. 4C, the 4-chamber or 4-chamber information including the ventricle is extracted from the 3D medical image of the subject through a registration method using an Atlas model for a 4-chamber or a heart, The 4-chamber information 430 can be extracted by extracting 4-chamber information or inputting a minutia or seed in the 3D medical image of the subject and performing region growing based on the input minutia or seed. have.

Similarly, as shown in FIG. 4D, in order to extract cardiac periphery information, cardiac circumference information is extracted from a 3D medical image of the subject through a registration method using an Atlas model of the heart or the heart, By using the propagation technique through the user input to the heart periphery, or using the delivery technique using the 4-chamber and the seed, the heart periphery information 440 of the subject can be extracted.

Although not shown in FIG. 4, it is possible to extract the thickness information about the atrium and the atrium, the atrium and the ventricle, the boundary information between the atrium / ventricle and the perimeter, or the thickness of the outer wall of the heart from the 3D medical image of the subject, can do.

Cardiac periphery information or cardiac information may be implemented in the form of a 3D model obtained from the second medical image. In this case, in order for the 3D model to be overlapped with the 2D cardiovascular information on the first medical image, the 3D cardiac model may be projected on the reference surface with a plane perpendicular to the direction in which the first medical image is captured as a reference plane.

In step S130, if 2D cardiovascular information and 3D cardiac information are extracted, transformation matrix information between the first medical information and the second medical information is extracted using the matching of the 3D cardiac information among the extracted 2D cardiovascular information and 3D card information (S140) .

Here, the step S140 may extract the transformation matrix information between the first medical information and the second medical information by using the non-rigid matching of the 2D cardiovascular information and the 3D cardiovascular information.

The process of extracting the transformation matrix information through cardiovascular matching in step S140 will be described with reference to FIG.

As shown in FIG. 5A, 2D cardiovascular information and 3D cardiovascular information extracted in step S130 are received as input information, and 2D cardiovascular information and 3D cardiovascular information are obtained through non-rigid body matching of two cardiovascular information as shown in FIG. And extracts transformation matrix information in which information is matched.

The input information for extracting the transformation matrix information includes not only the cardiovascular information shown in FIG. 5A but also a distance from the X-ray source to the detector for detec- tion of the first medical image / a 2D FOV indicating the distance from the object to the detector ), And angle information.

In the matching process using the input information, the features are extracted from the 2D center line from the 2D cardiogram information, and left and right are determined. Only one of the left and right blood vessels is selected from the 3D center line from the 3D cardiovascular information and rotated by the angle information By performing scaling with FOV information and non-rigid body matching, the transformation matrix information between the first medical image and the second medical image can be extracted. Of course, the process of extracting the transformation matrix information between the first medical image and the second medical image through the matching of the 2D cardiovascular information and the 3D cardiovascular information is not limited to the above-described contents, and various methods known in the art can be used .

The process of extracting the transformation matrix information between the first medical image and the second medical image will be described in more detail. A) In order to match the 2D cardiovascular information of the first medical image, the 3D cardiac model of the second medical image Position) is projected in a 2D direction in a direction coinciding with the direction in which the first medical image is captured. In other words, the 3D cardiovascular information of the second medical image can be converted to the 2D cardiovascular information (or the 2D projection version of the 3D cardiovascular information) of the second medical image by the 2D projection.

B) The 2D cardiovascular information of the first medical image and the 2D cardiovascular information (the 2D projection version of the 3D cardiovascular information) of the second medical image can be matched in 2D which is the same dimension. C) In this matching process, the transformation matrix information between 2D cardiovascular information can be extracted. That is, the transformation matrix information can be extracted through the matching between the 2D projection version of the 3D cardiovascular information of the second medical image and the 2D cardiovascular information of the first medical image.

When the transformation matrix information is extracted in step S140, the extracted transformation matrix information is applied to the 3D cardiac information, and the 3D cardiac information to which the transformation matrix information is applied is overlaid on the first medical image (S150).

That is, the transformation matrix information is applied to the 3D cardiac information by multiplying the transformation matrix information by 3D cardiovascular information, 4-chamber information, and cardiac edge information constituting 3D cardiac information.

More specifically, D) 3D cardiac information (including the structure and position information of the atrium, ventricle, myocardium, and cardiac circumference, etc. ) of the second medical image extracted in step S130 is compared with the direction in which the first medical image is captured And can be 2D projected in a matching direction.

D-1) Display forms of cardiac information that are projected and displayed in 2D include outer boundary line images projected with 2D heart outline boundary, maximum intensity projection (MIP) image, minimum intensity projection image (minIP, minimum intensity projection) images.

E) The 2D transformation matrix information may be applied by multiplying the 2D outline image, the 2D MIP image, and the 2D minIP image by the 2D transformation matrix information extracted in step S140. That is, the transformation matrix information may be applied to the 2D projection version of the 3D cardiac information of the second medical image.

The heart structure information of the second medical image to which the transformation matrix information between the first medical image and the second medical image is applied is in a state of being aligned with the first medical image through the transformation matrix information. The heart structure information of the second medical image that has been positionally matched as described above can be displayed overlaid on the first medical image. That is, the 3D heart structure information of the second medical image may be overlaid on the first medical image with the transformation matrix information applied to the 2D projection version.

At this time, the step S150 may overlay the thickness information on the outer wall of the heart when the 3D heart information includes the outer wall information of the heart.

The first medical image overlaid in step S150 and the 3D cardiac information to which the transformation matrix information is applied are displayed on the screen of the user or the practitioner. However, the 3D cardiac information may be displayed and stored as a medical image file in a PACS or the like. That is, the overlay step of step S150 may include not only displaying the overlaid image on the screen, but also storing the overlaid image in a system such as PACS.

For example, as shown in FIG. 6, the present invention displays an image in which a first medical image 610 and 3D cardiac information 620 are overlaid on a partial area of a screen, and separately displays 3D cardiac information in another area And when a specific point 630 is selected from any one of the displayed images, the position matched to the selected specific point can be displayed together with the other images. Here, the 3D cardiac information displayed in the other display area may vary depending on the type of treatment or the user's selection.

In addition, the 3D cardiac information overlaid on the first medical image in step S150 can be adjusted in display form and transparency through user's input using a user interface provided in advance. That is, in the medical image overlay device according to an embodiment of the present invention, a display form of 3D cardiac information overlaid on the first medical image and a selection menu may be provided as a user interface so that the user can adjust transparency. The transparency of each of the image elements constituting the 3D cardiac information, i.e., the outer boundary of the heart, the partition information of the 4-chamber, the image representing the thickness of the outer wall of the heart, the myocardial image, etc., Or the opacity may be adjusted. Here, the transparency to be adjusted may be controlled by a numerical value, but transparency may provide a predetermined item, for example, a maximum / intermediate / projection item, and transparency may be adjusted by user selection. The display form may also refer to an image processing / display form such as an outline boundary, a maximum intensity projection, a minimum intensity projection, etc. of the image in which the 3D cardiac information is projected in 2D.

In addition, the first medical image overlaying 3D cardiac information may include a moving image, and blood vessels in a non-angiographic frame of a moving image may be generated by interpolation using an angiographic front and back frames .

As described above, the medical image overlay method according to the present invention not only overlays 3D cardiovascular information to which the transformation matrix information is applied in the X-ray angiography image, but also includes 4-chamber information, cardiac outline information, cardiac wall thickness, Overlaying allows the practitioner to assist in finding the exact structure and location of the atria, ventricles, and myocardium as well as the vessels during cardiac surgery.

In addition, the method according to the present invention adaptively determines the 3D cardiac information overlaid on the first medical image according to the type of cardiac surgery, so that 3D cardiac information corresponding to the type of cardiac procedure, for example, 3D cardiovascular information and cardiac outline information The overlayed image can provide more help to the practitioner of the heart procedure.

Here, cardiac procedures that can help the practitioner by overlaying cardiac outline information include Pericardiocentesis, Permanent pacemaker Insertion, Myocadial Biopsy, Septal Ablation (Myocardial Necrosis) ), And atrial fibrillation during cardiac arrhythmia, and the emerging technology of various structural heart diseases that require cardiac structure and location information.

FIG. 2 is a flowchart illustrating an operation of step S150 shown in FIG. 1. FIG. 2 is a flowchart illustrating an operation for adaptively determining 3D cardiac information overlaid on a first medical image according to a cardiac procedure type.

Referring to FIG. 2, the overlaying step S150 includes checking at least one type of the subject's procedure to be performed, selecting at least one image corresponding to the confirmed type of the subject's procedure from the 3D cardiac information extracted at step S130, (S210, S220).

At least one or more images selected in step S220 may basically select 3D cardiovascular information, and may further select only cardiac periphery information according to a procedure type, or may additionally select only 4-chamber information. Of course, you can choose both cardiac outline information and 4-chamber information, and you can choose only myocardial and cardiac outline information. The display form that displays the selected 3D image information in 2D may be basically an outer boundary line image of the 2D projection image. In addition, various methods such as a maximum intensity projection image and a minimum intensity projection image may be additionally provided. A user interface capable of selecting a display form can be provided.

If at least one image according to the type of procedure is selected in step S220, the transformation matrix information extracted in step S140 is applied to at least one or more images selected, and at least one image to which the transformation matrix information is applied is compared with the identified procedure type (S230, S240) in consideration of a predetermined arrangement, structure, and image format with respect to the first medical image.

Here, the arrangement, structure, and image type according to the type of treatment vary depending on the type of treatment selected for the image, or when the 4-chamber is selected, the area of the chamber overlaid among the 4- Overlay only the atrium, overlay only the ventricle, and overlay the outer wall thickness of the heart and the outer wall thickness information according to the type of procedure. The arrangement, structure, and image type according to the type of procedure can be predetermined, and even after the information is overlaid on the first medical image, transparency and the like can be adjusted through the user interface provided to the operator.

For example, if the position and contour information of the 4-chamber (2-chamber, 2-chamber) of the heart are required depending on the type of procedure, the 4-chamber information of the heart is overlaid and information about the thickness of the outer wall of the atrium or atrium If necessary, the image of the outer wall of the ventricle or atrium is overlaid, and when the information about the position of the myocardial muscle is required, the image of the myocardial muscle can be overlaid. Therefore, the practitioner can efficiently check the information necessary for the operation, .

FIG. 7 illustrates a configuration of a medical image overlay apparatus according to an embodiment of the present invention.

7, the apparatus 700 includes a receiving unit 710, an image information extracting unit 720, a transform matrix extracting unit 730, and an image processing unit 740. The medical image overlay device 700 may execute the methods shown in Figures 1 and 2 by means of instructions / commands provided in the form of a software program or code. The medical image overlay device 700 may include at least one processor (not shown) that executes instructions / commands provided in the form of a program or code. The receiving unit 710, the image information extracting unit 720, the transform matrix extracting unit 730, and the image processing unit 740 shown in FIG. 7 may be included as components in the processor in the apparatus 700.

The receiving unit 710 receives the first medical image and the second medical image including the image of the subject's heart region.

At this time, the receiving unit 710 can receive the first medical image of the subject as the real-time medical image photographed by the X-ray angiography, and can receive the second medical image of the subject as a 3D medical image It can be received by the image.

The image information extracting unit 720 extracts 2D cardiovascular information from the first medical image received by the receiving unit 710 and extracts 3D cardiac information from the second medical image.

Here, the 3D cardiac information may include 3D cardiovascular information, 4-chamber information including the atrium and the ventricle, cardiac periphery information, and myocardial information, and information about the outer wall of the heart and thickness information of the outer wall .

3, the image information extracting unit 720 can extract 2D cardiovascular information from the first medical image using a binarization technique, a noise cancellation technique, and a thinning technique, and similarly, the atlas model, The 3D cardiac information can be extracted from the second medical image by using the area expansion method using the first medical image, the thinning technique, and the like.

The transformation matrix extraction unit 730 extracts transformation matrix information between the first medical image and the second medical image using the matching of the 2D cardiovascular information and the 3D cardiovascular information extracted by the image information extraction unit 720.

At this time, the transformation matrix extracting unit 730 can extract the transformation matrix information between the first medical image and the second medical image using the non-rigid matching of the 2D cardiovascular information and the 3D cardiovascular information, The distance between the X-ray source and the detector for detecting the first medical image / the 2D FOV (field of view) and the angle information indicating the distance from the object to the detector, The transformation matrix information between the medical images can be extracted.

The image processing unit 740 applies the transform matrix information extracted by the transform matrix extracting unit 730 to the 3D cardiac information, and overlays the 3D cardiac information to which the transform matrix information is applied on the first medical image.

At this time, the image processing unit 740 can overlay thickness information on the outer wall of the heart when the outer wall of the heart is included in the 3D image information, and by providing the user interface to the user, The transparency of the 3D cardiac information overlaid on the first medical image based on the input or the transparency of each of the images included in the 3D cardiac information may be adjusted. In addition, the image processor 740 may overlay various types of images such as an outer borderline image, a maximum intensity projection image, and a minimum intensity projection image of the 2D projection image according to the display format of the 3D image information.

Further, the image processing unit 740 confirms the type of the subject's procedure, selects or determines at least one image corresponding to the confirmed type of the plurality of images included in the 3D cardiac information, The at least one image selected in the first medical image can be overlaid on at least one of the predetermined arrangement, the structure, and the shape of the at least one image.

The medical image overlay method according to an exemplary embodiment of the present invention may be implemented in the form of program instructions that can be executed through various computer means and recorded on a computer readable medium. The computer-readable medium may include program instructions, data files, data structures, and the like, alone or in combination. The program instructions recorded on the medium may be those specially designed and configured for the present invention or may be available to those skilled in the art of computer software. Examples of computer-readable media include magnetic media such as hard disks, floppy disks and magnetic tape; optical media such as CD-ROMs and DVDs; magnetic media such as floppy disks; Magneto-optical media, and hardware devices specifically configured to store and execute program instructions such as ROM, RAM, flash memory, and the like. Examples of program instructions include machine language code such as those produced by a compiler, as well as high-level language code that can be executed by a computer using an interpreter or the like. The hardware devices described above may be configured to operate as one or more software modules to perform the operations of the present invention, and vice versa.

As described above, the present invention has been described with reference to particular embodiments, such as specific elements, and specific embodiments and drawings. However, it should be understood that the present invention is not limited to the above- And various modifications and changes may be made thereto by those skilled in the art to which the present invention pertains.

Accordingly, the spirit of the present invention should not be construed as being limited to the embodiments described, and all of the equivalents or equivalents of the claims, as well as the following claims, belong to the scope of the present invention .

Claims (12)

Receiving a first medical image including an image of a heart portion of the subject;
Receiving a second medical image including an image of the heart region of the subject;
Extracting two-dimensional cardiovascular information from the received first medical image and extracting three-dimensional cardiac information from the second medical image;
Extracting transformation matrix information between the first medical image and the second medical image using the extracted two-dimensional cardiovascular information and matching of three-dimensional cardiovascular information included in the three-dimensional cardiac information; And
Applying the extracted transformation matrix information to the 3D cardiac information, and overlaying the 3D cardiac information to which the transformation matrix information is applied on the first medical image
And a medical image overlay method. The method according to claim 1,
The overlaying step
Selecting at least one image corresponding to the type of treatment of the subject from among a plurality of images included in the 3D cardiac information, selecting at least one of the at least one image selected from among a predetermined arrangement, structure, Wherein at least one of the at least one type of image is overlaid on the first medical image in consideration of at least one of the types of the at least one image. The method according to claim 1,
The overlaying step
Wherein when the outer wall of the heart is included in the overlayed 3D image information, thickness information on the outer wall of the heart is overlaid together. The method according to claim 1,
The three-dimensional cardiac information
3-dimensional cardiovascular information, 4-chamber information including an atrium and a ventricle, cardiac periphery information, and myocardial information. The method according to claim 1,
The step of extracting the transform matrix information
Wherein the transformation matrix information is extracted using non-rigid matching of the 2D cardiovascular information and the 3D cardiovascular information. The method according to claim 1,
Wherein the 3D cardiac information overlaid on the first medical image
Wherein the transparency is adjusted by a user's input using a user interface. A receiving unit for receiving a first medical image and a second medical image including an image of a heart portion of the subject;
An image information extracting unit for extracting two-dimensional cardiovascular information from the received first medical image and extracting three-dimensional cardiac information from the second medical image;
A transformation matrix extraction unit for extracting transformation matrix information between the first medical image and the second medical image using the extracted two-dimensional cardiovascular information and the matching of three-dimensional cardiovascular information included in the three-dimensional cardiac information; And
Applying the extracted transformation matrix information to the 3D cardiac information and overlaying the 3D cardiac information to which the transformation matrix information is applied on the first medical image,
And a medical image overlay device. 8. The method of claim 7,
The image processing unit
Selecting at least one image corresponding to the type of treatment of the subject from among a plurality of images included in the 3D cardiac information, selecting at least one of the at least one image selected from among a predetermined arrangement, structure, And overlaying the at least one image on the first medical image in consideration of at least one of the types of the at least one image. 8. The method of claim 7,
The image processing unit
And overlays the thickness information on the outer wall of the heart when the outer wall of the heart is included in the overlayed 3D image information. 8. The method of claim 7,
The image information extracting unit
Dimensional cardiac information including three-dimensional cardiovascular information, four-chamber information including an atrium and a ventricle, cardiac periphery information, and myocardial information. 8. The method of claim 7,
The transformation matrix extracting unit
Wherein the transformation matrix information is extracted using non-rigid matching of the 2D cardiovascular information and the 3D cardiovascular information. 8. The method of claim 7,
Wherein the 3D cardiac information overlaid on the first medical image
Wherein the transparency is adjusted by a user's input using a user interface.

Images