KR101876643B1 - Therapy guiding system and method based on matching between 2D fluoroscopy image and 3D CT image - Google Patents

Abstract

A medical guiding system and method based on 2D fluoroscopic image and 3D CT image registration is disclosed. According to an embodiment of the present invention, there is provided a method of guiding the image matching-based therapy, comprising: obtaining a two-dimensional fluoroscopic image and a three-dimensional CT image for each of a plurality of breaths for a predetermined ROI of a patient; Extracting feature points for each of the obtained three-dimensional CT images; Forming a 4-dimensional CT image by matching the 3-dimensional CT images using the extracted feature points; Selecting a two-dimensional projection image most similar to the two-dimensional fluoroscopic image through matching of two-dimensional projection images of the formed four-dimensional CT image with the two-dimensional fluoroscopic image; And providing information about a guiding target of the ROI based on the selected two-dimensional projection image.

Description

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to a fluoroscopic guiding system and method based on 2D fluoroscopic image and 3D CT image matching,

The present invention relates to a therapeutic guiding technique based on a 2D fluoroscopic image and a 3D CT image matching, more particularly, to a system capable of automatically finding a 3D CT image matching a respiratory state of a 2D fluoroscopic image and using the same as a guide image And methods.

In the field of image guidance procedures such as fine needle aspiration biopsy, radiofrequency ablation (RFA), and high intensity focused ultrasound (HIFU), real-time medical images, such as 2D fluoroscopy images, . However, since the 2D fluoroscopic image is a projection image, the depth information can not be known, and in some cases, a small tumor is not visible, so that it is often difficult to accurately grasp the position of the tumor in real time.

The C-arm-based fluoroscopy system, which is currently in use, acquires a 3D cone-beam CT image, which has accurate information on the location of the tumor including depth information, on the same system and overlaps the 2D fluoroscopy image, To solve the above problems.

However, 3D cone beam CT can not guide the accurate location of the tumor when patient breathing and real - time 2D fluoroscopic image acquisition of the patient 's respiration are different when the 3D cone beam CT is acquired.

In order to acquire the accurate cardiovascular position information regardless of the cardiac movements, many researches have been conducted to deal with the matching between the 2D fluoroscopic image and the 3D CT image. Matching study.

In many cases, cardiac gated CT images are acquired and used to apply rigid registration or to represent non-rigid movements of the heart. In general, cardiac and respiratory movements are performed simultaneously In the case of an existing lung area, both of the movements must be considered. In the present study, respiratory gated CT as well as cardiac CT should be used, unless a simple rigid body alignment is used. Therefore, there is a problem of high complexity and large amount of exposure.

Therefore, there is a need for a system or method capable of matching without reducing the heart gate CT and respiratory gate CT images, reducing the amount of patient exposure and reducing the complexity.

Embodiments of the present invention provide a therapeutic guiding system and method that can automatically find and use a 3D CT image matching a breathing state of a 2D fluoroscopic image based on a 2D fluoroscopic image and a 3D CT image matching as a guide image .

In particular, embodiments of the present invention provide a therapeutic guiding system and method that can monitor the position of an accurate tumor in real time during a procedure through matching between a 2D fluoroscopic image and a 3D CT image.

According to an embodiment of the present invention, there is provided a method of guiding image matching-based therapy, comprising: acquiring a two-dimensional fluoroscopic image and a three-dimensional CT image for each of a plurality of breaths with respect to a predetermined ROI of a patient; Extracting feature points for each of the obtained three-dimensional CT images; Forming a 4-dimensional CT image by matching the 3-dimensional CT images using the extracted feature points; Selecting a two-dimensional projection image most similar to the two-dimensional fluoroscopic image through matching of two-dimensional projection images of the formed four-dimensional CT image with the two-dimensional fluoroscopic image; And providing information about a guiding target of the ROI based on the selected two-dimensional projection image.

Wherein the step of selecting the most similar two-dimensional projection image comprises: if the position of the guiding object is a region where breathing movement and heart movement are simultaneously performed, the first weighting of the matching by the movement of the heart and the movement of the breathing The most similar two-dimensional projection image can be selected by reflecting the second weight of the matching and controlling the reflected first weight and the second weight.

Wherein the forming of the 4-dimensional CT image comprises: arranging the 3-dimensional CT images in the respiration order of the patient and deriving a movement pattern for the guiding object through matching between adjacent 3-dimensional CT images; And forming the 4-dimensional CT image based on the derived motion pattern.

Wherein the step of selecting the most similar two-dimensional projection image comprises the steps of: transforming the four-dimensional CT image based on a viewing angle and a moving coordinate of the two-dimensional fluoroscopic image; Dimensional projection image candidates, and the most similar two-dimensional projection image can be selected through the matching of the two-dimensional projection images of the formed two-dimensional projection image candidate group and the two-dimensional fluoroscopic vision image .

Wherein the extracting of the feature points comprises: separating organs different from organs having the same repetitive pattern of movement as the related region of the guiding object; And extracting feature points for each of the obtained three-dimensional CT images from the same organs of repeated patterns of motion.

The image matching-based therapy guiding system according to an embodiment of the present invention includes an acquiring unit for acquiring a two-dimensional fluoroscopic image for a ROI of a patient and a three-dimensional CT image for each of a plurality of breaths; An extraction unit for extracting feature points for each of the obtained three-dimensional CT images; A forming unit for forming a four-dimensional CT image by matching the three-dimensional CT images using the extracted feature points; A selection unit for selecting a two-dimensional projection image most similar to the two-dimensional fluoroscopic image through matching of two-dimensional projection images of the formed four-dimensional CT image with the two-dimensional fluoroscopic image; And a providing unit for providing information about a guiding target of the ROI based on the selected two-dimensional projection image.

Wherein the selecting unit reflects the first weight of the matching due to the motion of the heart and the second weight of the matching due to the motion of the respiration when the position of the guiding object is a region in which breathing movement and heart movement are simultaneously present, By controlling the reflected first weight and the second weight, the most similar two-dimensional projection image can be selected.

Wherein the forming unit arranges the three-dimensional CT images in the respiration order of the patient, derives a motion pattern for the guiding object through matching between adjacent three-dimensional CT images, CT images can be formed.

Wherein the selecting unit converts the four-dimensional CT image based on a viewing angle and a moving coordinate of the two-dimensional fluoroscopic image, and projects the converted four-dimensional CT image at a viewing angle of the two-dimensional fluoroscopic image, Dimensional projected image, and the most similar two-dimensional projection image can be selected through the matching of the two-dimensional projection images of the formed two-dimensional projection image candidate group and the two-dimensional fluoroscopic vision image.

Wherein the extracting unit separates the organs different from the organs having the same repetitive pattern of motion with the related region of the guiding object and extracts the feature points for each of the obtained three- can do.

According to embodiments of the present invention, a 3D image matching a breathing state of a 2D fluoroscopic image based on a 2D fluoroscopic image and a 3D CT image matching is automatically found and used as a guide image, Can be monitored.

According to the embodiments of the present invention, it is possible to perform matching even in a situation where both movement of both heart and breath are not controlled, thereby reducing the complexity of the algorithm and reducing the dose of the operator.

FIG. 1 is a flowchart illustrating an operation of the image matching-based treatment guiding method according to an exemplary embodiment of the present invention.
FIG. 2 shows an operational flow chart of an embodiment of step S130 shown in FIG.
FIG. 3 shows an operational flow diagram of an embodiment of step S140 shown in FIG.
FIG. 4 is a diagram illustrating an example of matching between a real-time 2D fluoroscopic image and a 3D CT image.
FIG. 5 illustrates a configuration of a therapeutic matching guiding system based on an image matching according to an embodiment of the present invention.

Hereinafter, embodiments according to the present invention will be described in detail with reference to the accompanying drawings. However, the present invention is not limited to or limited by the embodiments. In addition, the same reference numerals shown in the drawings denote the same members.

Embodiments of the present invention can automatically detect and monitor the position of a tumor in real time during a procedure by automatically searching for a 3D image matching a breathing state of a 2D fluoroscopic image based on the 2D fluoroscopic image and the 3D CT image matching, In addition, it is possible to reduce the complexity of the algorithm and reduce the dose of the operator by making it possible to perform matching even when both movements are not controlled in the area where heart movement and respiration movement are simultaneously present.

FIG. 1 is a flowchart illustrating an operation of the image matching-based treatment guiding method according to an exemplary embodiment of the present invention.

As shown in FIG. 1, the image matching-based therapy guiding method according to the embodiment of the present invention includes a predetermined region of interest (ROI) of a patient, that is, a region including a guiding object for tumor treatment A 3D fluoroscopic image and a plurality of 3D CT images are acquired for each of the plurality of breaths (S110).

Here, the 2D fluoroscopic image can be obtained by photographing in real time the fluoroscopy apparatus with the region of interest to be guided, and the 3D CT image can be obtained by being photographed by the CT apparatus.

When the 2D fluoroscopic image for the region of interest and the 3D CT images for the breaths are obtained in Step S110, the feature points for the obtained 3D CT images are extracted, and the matching between the 3D CT images using the extracted feature points To form a 4D CT image (S120, S130).

A process of extracting the minutiae point of step S120 will be described below.

The feature points of 3D CT images refer to specific organs to be separated for individual non-rigid body matching, or specific organs to be used for matching, and specific organs to be separated for individual non-rigid body matching are the adjacent regions (or related And the repeated pattern of movement is not the same. Generally, the guiding object is located near the internal organs of the body. Therefore, the rib and the spine, which may have different movement patterns, .

The specific organs to be used for matching (or matching) can be selected among organs with the same repetitive patterns of motion and motion, and can be clearly identified in two images of 2D fluoroscopy and 3D CT images, It can mean a organ, usually a vessel or the like can be used.

The above-described various feature points can be segmented in advance using intensity or gradient information of the image.

The process of forming the 4D image in step S130 will be described with reference to FIG.

In step S130 of forming the 4D image, the 3D CT images are arranged in the respiration order of the patient, and the movement pattern for the guiding object is derived through matching between the adjacent 3D CT images. Based on the derived motion pattern, (S210 to S230).

Since the area around the guiding target moves with a specific pattern according to the respiration, several 3D CT images are acquired so as to have different motion states. For example, a plurality of 3D CT images may be sequentially acquired for each respiratory state, one at a maximum expiratory state, or one at a maximum inhalation state, and each acquired image may be acquired by a low dose photographing.

If a motion pattern is derived through non-rigid body matching between adjacent 3D CT images in step S220, an additional image between breathing and breathing may be generated by performing image interpolation using the derived motion pattern.

Here, in the process of forming the 4D image in step S130, when extracting the feature points of the 3D CT image, if there is a specific organ to be used for matching, the specific organ is extracted from each 3D CT image, Pattern interpolation can be performed to form a plurality of 3D feature points.

The 4D CT image may refer to the above-described 3D CT images.

Referring again to FIG. 1, when a 4D CT image is formed in step S130, 2D projection images of the formed 4D CT image are generated, and the 2D projection image, which is most similar to the 2D projection fluoroscopic image, A projection image is selected (S140, S150).

Here, step S140 is a step of acquiring a viewing angle and a moving coordinate of the 2D fluoroscopic image from the equipment, for example, a 2D fluoroscopic apparatus as shown in FIG. 3, and based on the obtained viewing angle and moving coordinates 4D CT image, and the converted 4D CT image is projected at a viewing angle of the 2D fluoroscopic image to form a 2D projection image candidate group (S310, S320). In step S150, a 2D projection image of the formed 2D projection image candidate group And a 2D projection image can be selected by matching the 2D fluoroscopic image with the 2D fluoroscopic image.

Step S140 and step S150 correspond to the step of matching the 2D fluoroscopic image. In step S140, the transformed 4D CT image is digitally projected through a 2D fluoroscopic image The 2D projection image candidate group having the same viewing angle as that of the 2D projection image candidate group may be formed, and the formed 2D projection image candidate group may reflect the morphological information of the organ according to different breathing phases.

Step S150 is to compare a 2D fluorescence projection image obtained in the current real-time from the 2D projection image candidate group to select one 2D projection image that best matches the 2D fluorescence projection image.

Here, when extracting feature points from the 3D CT image, if there are specific organs to be used for matching, matching of the 2D projection images of the 2D fluoroscopic image and the 2D projection image candidates using the specific organs is performed, Can be selected.

In the case where the position of the guiding object is the area where the motion of the respiration and the movement of the heart are simultaneously present, matching is difficult. In the 4D CT image formed in step S130, The shape of the organs that exactly match the shape of the object to be matched in the 2D fluoroscopic image captured and acquired at the moment of movement of the heart may not be present in the 2D projection image candidates.

In the present invention, the above-mentioned problem can be solved by using a method of reducing the weight of the matching measure for the motion of the heart and increasing the weight of the matching scale for the breathing motion. ≫

[Equation 1]

As shown in FIG. 4, I _{F, k} denotes a k-th frame of a 2D fluoroscopic image, which is a target image, as shown in FIG. 4. P _{v , n} denotes an n-th candidate image , ∇σ means a gradient of a Gaussian blurred image, and N (I _{F, k} , P _{v , n} ) can mean a general NMI measurement.

에 호흡에 의한 움직임과 타겟 오브젝트의 그래디언트의 일치 여부에 따른 가중치 항목

을 추가한 형태로서, 폐 영역에서 호흡의 움직임 방향과 심장의 움직임 방향이 다름을 이용하여 호흡의 움직임의 방향 따라 민감하게 반응 할 수 있는 영역의 가중치만 남기고 나머지 가중치를 낮추는 형태로 구성될 수 있다.

에서, v _resp는 2D 도메인에서 호흡의 방향 벡터를 의미할 수 있다. 이와 같은 형태의 가중치는 GNMI가 아니라 다른 정합 척도에도 적용하여 사용할 수 있다.G ( I _{F, k} , P _{v , n} ) is a weighted item of the matching scale, and is a weight of the existing GNMI

A weight item according to whether the movement by the respiration and the gradient of the target object coincide with each other

, It can be composed of a form in which only the weight of the region that can react sensitively to the direction of the respiratory movement is left and the remaining weight is lowered by using the difference between the direction of movement of the respiration and the direction of movement of the heart in the lung region .

, V _resp can be the direction vector of respiration in the 2D domain. This type of weighting can be applied to other matching scales, not GNMI.

As described above, the step S150 for selecting the 2D projection image most similar to the 2D fluoroscopic image is performed when the position of the guiding object is the area where the movement of the breath and the movement of the heart are simultaneously present. For example, by controlling the reflected first weight and the second weight so that the first weight is made lower and the second weight is made higher, a 2D projection image which is most similar to the 2D fluoroscopic image can be obtained by controlling the reflected first weight and the second weight, Can be selected.

If the 2D projection image most similar to the 2D fluoroscopic image is selected in step S150, matching information-based guiding information is provided based on the selected 2D projection image (S160).

Here, in step S160, guiding information for each application to be applied, for example, outputting the guiding position may be applied differently. For example, if the 3D position information of the guiding object, which is previously confirmed from the 3D CT image, exists, the 3D position information of the matched image can be used as the current representative position information after transforming it to the 4D coordinate. Alternatively, a method may be used in which a cross section of a corresponding position is overlapped with a 2D fluoroscopic image input in real time from a 3D CT image.

As described above, according to the embodiment of the present invention, the 3D image matching the breathing state of the 2D fluoroscopic image based on the 2D fluoroscopic image and the 3D CT image matching is automatically found and used as the guide image, The location of the tumor can be monitored.

In addition, the method according to the embodiment of the present invention enables matching even in a case where both movement of both heart and breath are not controlled, thereby lowering the complexity of the algorithm and lowering the dose of the subject .

In addition, the method according to embodiments of the present invention can accurately monitor the position of the tumor in real time during the procedure through matching between the 2D fluoroscopic image and the 3D CT image.

FIG. 5 illustrates a configuration of a video matching-based therapeutic guiding system according to an embodiment of the present invention, and shows a configuration of a system for performing the methods of FIGS. 1 to 4 described above.

5, a system 500 according to an embodiment of the present invention includes an acquiring unit 510, an extracting unit 520, a forming unit 530, a selecting unit 540, and a providing unit 550 .

The acquisition unit 510 acquires a 2D fluoroscopic image for a predetermined ROI of the patient and a 3D CT image for each of a plurality of breaths.

The extraction unit 520 extracts feature points for each of the obtained 3D CT images.

At this time, the extracting unit 520 separates the organs and the organs having the same repetitive pattern of motion with the related region (or the neighboring region) of the guiding object, extracts 3D CT images The minutiae points for each of them can be extracted.

The forming unit 530 forms a 4D CT image through matching between 3D CT images using the extracted feature points.

At this time, the forming unit 530 arranges the 3D CT images in the respiration order of the patient, and then derives the movement pattern for the guiding object through the matching between the adjacent 3D CT images. Based on the derived motion pattern, Can be formed.

The selection unit 540 selects the 2D projection image most similar to the 2D fluoroscopic image through matching of the 2D projection images with the 2D projection images of the formed 4D CT image.

At this time, the selector 540 reflects the weight of the matching scale due to the motion of the heart and the weight of the matching scale due to the movement of the respiration when the position of the guiding object is the area where the movement of the breath and the movement of the heart are simultaneously present For example, the selector may reduce the weight of the matching scale for cardiac movements and increase the weight of the matching scale by breathing movements, By minimizing the effect of cardiac movements, the 2D fluoroscopic image, which is the target image, can be selected most similar to the 2D fluoroscopic image.

Further, the selecting unit 540 converts the 4D CT image based on the viewing angle and the moving coordinate of the 2D fluoroscopic image, and projects the converted 4D CT image to the viewing angle of the 2D fluoroscopic image to form a 2D projection image candidate group And a 2D projection image, which is most similar to the 2D fluoroscopic image, can be selected through the matching of the 2D projection images of the 2D projection image candidate group and the 2D fluoroscopic image.

The providing unit 550 provides information on the guiding object based on the matching result based on the 2D projection image selected by the selecting unit 540. [

The apparatus according to the embodiment of the present invention may include not only the contents described in FIG. 5 but also the contents of FIGS. 1 to 4 described above.

The system or apparatus described above may be implemented as a hardware component, a software component, and / or a combination of hardware components and software components. For example, the systems, devices, and components described in the embodiments may be implemented in various forms such as, for example, a processor, a controller, an arithmetic logic unit (ALU), a digital signal processor, a microcomputer, a field programmable array ), A programmable logic unit (PLU), a microprocessor, or any other device capable of executing and responding to instructions. The processing device may execute an operating system (OS) and one or more software applications running on the operating system. The processing device may also access, store, manipulate, process, and generate data in response to execution of the software. For ease of understanding, the processing apparatus may be described as being used singly, but those skilled in the art will recognize that the processing apparatus may have a plurality of processing elements and / As shown in FIG. For example, the processing unit may comprise a plurality of processors or one processor and one controller. Other processing configurations are also possible, such as a parallel processor.

The software may include a computer program, code, instructions, or a combination of one or more of the foregoing, and may be configured to configure the processing device to operate as desired or to process it collectively or collectively Device can be commanded. The software and / or data may be in the form of any type of machine, component, physical device, virtual equipment, computer storage media, or device , Or may be permanently or temporarily embodied in a transmitted signal wave. The software may be distributed over a networked computer system and stored or executed in a distributed manner. The software and data may be stored on one or more computer readable recording media.

The method according to embodiments may be implemented in the form of a program instruction that may be executed through various computer means and recorded in a computer-readable medium. The computer-readable medium may include program instructions, data files, data structures, and the like, alone or in combination. The program instructions to be recorded on the medium may be those specially designed and configured for the embodiments or may be available to those skilled in the art of computer software. Examples of computer-readable media include magnetic media such as hard disks, floppy disks and magnetic tape; optical media such as CD-ROMs and DVDs; magnetic media such as floppy disks; Magneto-optical media, and hardware devices specifically configured to store and execute program instructions such as ROM, RAM, flash memory, and the like. Examples of program instructions include machine language code such as those produced by a compiler, as well as high-level language code that can be executed by a computer using an interpreter or the like. The hardware devices described above may be configured to operate as one or more software modules to perform the operations of the embodiments, and vice versa.

While the present invention has been particularly shown and described with reference to exemplary embodiments thereof, it is to be understood that the invention is not limited to the disclosed exemplary embodiments. For example, it is to be understood that the techniques described may be performed in a different order than the described methods, and / or that components of the described systems, structures, devices, circuits, Lt; / RTI > or equivalents, even if it is replaced or replaced.

Therefore, other implementations, other embodiments, and equivalents to the claims are also within the scope of the following claims.

Claims (10)

Acquiring a two-dimensional fluoroscopic image and a three-dimensional CT image for each of a plurality of breaths with respect to a predetermined ROI of a patient in an acquiring unit;
Extracting feature points for each of the obtained three-dimensional CT images in an extraction unit;
Forming a four-dimensional CT image by matching the three-dimensional CT images using the extracted feature points in a forming unit;
Selecting a two-dimensional projection image most similar to the two-dimensional fluoroscopic image through matching of two-dimensional projection images of the formed four-dimensional CT image with the two-dimensional fluoroscopic image; And
Providing information on an object to be guided in the ROI based on the selected two-dimensional projection image in the provisioning step
Lt; / RTI >
The step of selecting the most similar two-dimensional projection image
The first weight of the matching due to the motion of the heart and the second weight of the matching due to the motion of the respiration when the position of the guiding object is a region in which breathing movement and heart movement are simultaneously present, 1 < / RTI > weighted image and the second weighted value, thereby selecting the most similar two-dimensional projection image.
delete The method according to claim 1,
The step of forming the 4-dimensional CT image
Arranging the three-dimensional CT images in a breathing order of the patient, and deriving a movement pattern for the guiding object through matching between adjacent three-dimensional CT images; And
Forming the 4-dimensional CT image based on the derived motion pattern
Wherein the image matching based therapy guiding method comprises:
The method according to claim 1,
The step of selecting the most similar two-dimensional projection image
Dimensional fluoroscopic image based on a viewing angle and a moving coordinate of the two-dimensional fluoroscopic image, and projecting the converted four-dimensional CT image to a viewing angle of the two-dimensional fluoroscopic image to obtain a two- Dimensional projection images, and selecting the most similar two-dimensional projection image through matching of the two-dimensional projection images of the formed two-dimensional projection image candidate group and the two-dimensional fluoroscopic vision image.
The method according to claim 1,
The step of extracting the feature points
Separating organs different from organs having the same repetitive pattern of motion as the related region of the guiding object; And
Extracting feature points for each of the obtained three-dimensional CT images from the same organs having the repeated patterns of motion
Wherein the image matching based therapy guiding method comprises:
An acquiring unit acquiring a two-dimensional fluoroscopic image for a predetermined ROI of a patient and a three-dimensional CT image for each of a plurality of breaths;
An extraction unit for extracting feature points for each of the obtained three-dimensional CT images;
A forming unit for forming a four-dimensional CT image by matching the three-dimensional CT images using the extracted feature points;
A selection unit for selecting a two-dimensional projection image most similar to the two-dimensional fluoroscopic image through matching of two-dimensional projection images of the formed four-dimensional CT image with the two-dimensional fluoroscopic image; And
And providing information on an object to be guided in the ROI based on the selected two-
Lt; / RTI >
The selection unit
The first weight of the matching due to the motion of the heart and the second weight of the matching due to the motion of the respiration when the position of the guiding object is a region in which breathing movement and heart movement are simultaneously present, 1 < / RTI > weighting and second weighting, thereby selecting the most similar two-dimensional projection image.
delete The method according to claim 6,
The forming portion
Dimensional CT images are arranged in the respiration order of the patient, and a movement pattern for the guiding object is derived through matching between adjacent three-dimensional CT images, and the 4-dimensional CT image is generated based on the derived movement pattern Wherein the image-matching-based therapy guiding system comprises:
The method according to claim 6,
The selection unit
Dimensional fluoroscopic image based on a viewing angle and a moving coordinate of the two-dimensional fluoroscopic image, and projecting the converted four-dimensional CT image to a viewing angle of the two-dimensional fluoroscopic image to obtain a two- Dimensional projected image, and selects the most similar two-dimensional projection image through matching of the two-dimensional projection images of the formed two-dimensional projection image candidate group and the two-dimensional fluoroscopic vision image.
The method according to claim 6,
The extracting unit
Dimensional CT images are extracted from the same organs in which the repetitive patterns of motion are separated from the organs having the same repetition pattern of movement as the related region of the guiding object and different organs different from each other, Based matching guiding system.

Images