JP7019758B2 - Image display device and its operation method - Google Patents

Description

The present invention relates to detecting the position of a catheter for generating a tomographic image of a living tissue.

Endovascular treatment is performed using high-performance catheters such as balloon catheters and stents. An ultrasonic tomographic image diagnostic apparatus (IVUS: Intravascular ultrasound) is used for the diagnosis before the operation or for the progress confirmation after the operation. In addition, an optical coherence tomography (OCT: Optical Coherence Tomography) is used instead of IVUS, and an optical coherence tomography (SS-OCT: Swept-source Optical coherence Tomography) using wavelength sweep is used as an improved version of OCT. Has been done. Intravascular diagnostic devices that can acquire tomographic images such as IVUS and OCT have more detailed information on lesion sites confirmed by X-ray equipment, such as stenosis rate in blood vessels, presence of plaques in branches, distribution of calcification, etc. Used to get.

When the doctor determines that treatment is necessary, the doctor determines the details of treatment such as where to position the stent edge by observing the vascular tomographic image obtained by the above-mentioned intravascular diagnostic apparatus. When treating the determined treatment site, the doctor performs treatment such as placement of a balloon or a stent while observing an X-ray image (angiography) obtained by an X-ray apparatus. Therefore, it is a very important factor in treatment to understand which position on the X-ray image corresponds to the placement position of the balloon or stent determined by confirming the vascular tomographic image.

U.S. Pat. No. 7,93014

As described above, in endovascular treatment and the like, it is important to grasp the positional relationship between the obtained vascular tomographic image and the X-ray image. However, since the intravascular diagnostic device and the X-ray device are configured as different modalities, doctors rely on landmarks such as branch positions to determine the position on the X-ray image corresponding to the confirmed vascular tomographic image. It is necessary to guess and treat.

In order to improve the estimation accuracy of the position on the X-ray image corresponding to the blood vessel tomographic image as described above, the X-ray image at the time of acquiring the blood vessel tomographic image is captured and the X-ray image is displayed in synchronization with the blood vessel tomographic image. There is a vascular tomography device. Generally, an X-ray opaque marker is installed in the vicinity of the sensor portion of the catheter connected to the vascular tomographic image device, and the above estimation accuracy is improved by displaying the vascular tomographic image and the X-ray image in synchronization. I'm letting you. By using this function, it is possible to visualize the vascular tomographic image and the X-ray opaque marker position on the X-ray image in a one-to-one correspondence.

Further, in recent years, a technique for further improving visibility by automatically detecting an X-ray opaque marker on an X-ray image and highlighting it has been developed (Patent Document 1). However, the position of the blood vessels displayed on the X-ray image changes with the passage of time due to the influence of pulsation and breathing, and the number of branches and the composition of the branches of human blood vessels vary greatly among individuals. The automatic detection rate of the X-ray opaque marker in the X-ray image is not satisfactory because there are often many branches around the blood vessel.

The present invention has been made in view of the above problems, and an object of the present invention is to improve the detection rate of an X-ray opaque marker in an X-ray image acquired during acquisition of a tomographic image.

The X-ray marker detection device according to one aspect of the present invention that achieves the above object has the following configuration. That is,
An image display device that displays a plurality of tomographic images acquired while moving the probe in the axial direction of the catheter and a plurality of X-ray images taken during the movement of the probe.
An extraction means for extracting a line along the probe from an X-ray image taken during a predetermined period before the probe starts to move, and an extraction means.
Using the lines extracted by the extraction means, a detection means for detecting the position of an X-ray opaque marker provided on the probe in each of the plurality of X-ray images taken while the probe is moving.
A display control means for synchronously displaying the plurality of X-ray images and the plurality of tomographic images and clearly indicating the position of the X-ray opaque marker detected by the detection means in the display of the X-ray image in the synchronous display. , Equipped with.

Further, the method for detecting an X-ray opaque marker according to another aspect of the present invention is:
An X-ray opaque marker detection method that detects an X-ray opaque marker provided on the probe in a plurality of X-ray images taken while the probe is moving in the axial direction for acquiring a tomographic image. There,
An extraction step of extracting a line along the probe from an X-ray image taken during a predetermined period before starting the movement of the probe, and an extraction step.
Using the lines extracted in the extraction step, a detection step of detecting the position of an X-ray opaque marker provided on the probe in each of the plurality of X-ray images taken while the probe is moving, and a detection step. Have.

According to the present invention, by using the X-ray image before the tomographic image acquisition, the detection rate of the X-ray opaque marker existing in the X-ray image acquired during the tomographic image acquisition can be improved.

It is a figure explaining the structure of the display system by embodiment. It is a figure explaining the catheter system for acquiring a vascular tomographic image. It is a flowchart which shows the synchronous display process by the display system of embodiment. It is a figure explaining the collection timing of a vascular tomographic image and an X-ray image. It is a flowchart explaining the blood vessel model generation process. It is a figure explaining the blood vessel model generation processing by an embodiment. It is a flowchart explaining the marker detection process. It is a figure explaining the selection of the blood vessel model in the marker detection by an embodiment. It is a figure which shows the example of the synchronous display by an embodiment.

Hereinafter, an example of a preferred embodiment of the present invention will be described with reference to the accompanying drawings.

FIG. 1 is a diagram showing a configuration example of a display system that realizes synchronous display of a blood vessel tomographic image and an X-ray image (angio image) according to the present embodiment. In FIG. 1, the intravascular diagnostic apparatus 100 as a tomographic image acquisition unit acquires an intravascular tomographic image by an intravascular ultrasonic method (IVUS), an optical coherence tomography method (OCT), or the like. The catheter system 111 is inserted, for example, from the base of the foot or the like through an artery to near the heart in order to capture a tomographic image in a blood vessel around the heart of patient 10. In the case of the optical interference diagnostic device, the intravascular diagnostic device 100 obtains a tomographic image by emitting measurement light and incident reflected light through the probe of the catheter system 111, and X-rays as an X-ray image acquisition unit. The X-ray image (angio image) taken by the photographing apparatus 200 is acquired via the cable 103. On the other hand, in the case of an ultrasonic tomography apparatus, the intravascular diagnostic apparatus 100 obtains a tomographic image by outputting an ultrasonic signal and inputting a reflected signal thereof via a probe of the catheter system 111, and also obtains an X-ray imaging apparatus. The X-ray image taken by the 200 is acquired via the cable 103.

FIG. 2 is a diagram illustrating the catheter system 111. As shown in FIG. 2, the catheter system 111 includes a guiding catheter 112, a catheter 113 containing a probe 115 for tomography, and a guide wire 114. The guiding catheter 112 has a hollow for inserting the guide wire 114 and the catheter 113. For example, when taking a tomographic image of a coronary artery, the doctor inserts the guiding catheter 112 into the vicinity of the coronary artery and then sends the guide wire 114 through the guiding catheter 112 to the imaging region of the coronary artery. Then, the doctor sends the imaging core 117 of the catheter 113 to the imaging region of the coronary artery by feeding the catheter 113 along the guide wire 114. The catheter 113 is provided with a guide wire lumen 119, through which the guide wire 114 is passed so that the catheter 113 can travel along the guide wire.

The probe 115 for tomographic imaging includes a metal shaft 116, an imaging core 117, and an X-ray opaque marker 118. In the case of tomography by OCT, the imaging core 117 is connected to the tip of an optical fiber passing through the metal shaft 116. The imaging core 117 includes an optical component that transmits and receives measurement light from the tip of the optical fiber. On the other hand, in the case of tomography by IVUS, a signal line passing through the metal shaft 116 and an imaging core 117 including an ultrasonic transducer that transmits and receives ultrasonic signals are connected. The metal shaft 116 moves in the axial direction of the catheter (movement in the direction of arrow 121) while being rotationally driven (rotation in the direction of arrow 120) (hereinafter referred to as pullback). The imaging core 117 also rotates with the metal shaft 116 and moves in the axial direction of the catheter. Since a tomographic image is obtained by one rotation of the imaging core, the tomographic image acquired during the movement of the rotating imaging core 117 becomes a plurality of vascular tomographic images along the vascular path. An X-ray opaque marker 118 for recognizing the position of the imaging core 117 on the X-ray image is provided between the imaging core 117 and the metal shaft 116. The X-ray opaque marker 118 may be provided on the distal end side of the imaging core 117.

Returning to FIG. 1, the tomographic image obtained by IVUS or OCT (vascular tomographic image in this embodiment) is stored in the vascular tomographic image storage unit 101. The frame rate of the tomographic image by the intravascular diagnostic apparatus 100 is about 160 to 180 Hz. The intravascular diagnostic apparatus 100 and the X-ray imaging apparatus 200 are connected by a cable 103, and the X-ray image acquired by the X-ray imaging apparatus 200 is transmitted to the intravascular diagnostic apparatus 100. In the present embodiment, communication between the intravascular diagnostic device 100 and the X-ray imaging device is performed via a cable, but the present invention is not limited to this, and wireless communication or the like may be used.

The X-ray imaging apparatus 200 drives the X-ray source 211 to irradiate the patient 10 with X-rays, and detects transmitted X-rays with the X-ray sensor 212 to obtain an X-ray image (for example, an angio image). The obtained X-ray image is transmitted to the intravascular diagnostic apparatus 100 via the cable 103 and stored in the X-ray image storage unit 102. The frame rate of the X-ray image in the X-ray photographing apparatus 200 is, for example, about 7 to 30 Hz. In addition, the intravascular diagnostic apparatus 100 has a pre-pullback X-ray image taken before the start of the pullback (start of imaging of the blood vessel tomographic image) described above in the X-ray image storage unit 102, and a pullback during pullback (of the blood vessel tomographic image). The X-ray image during pullback taken during (during shooting) is distinguished and saved.

The image display device 300 synchronizes the X-ray image and the blood vessel tomographic image, and highlights the position of the X-ray opaque marker 118 on the X-ray image. In the image display device 300, the blood vessel model generation unit 301 generates a plurality of blood vessel models based on a plurality of pre-pullback X-ray images read from the X-ray image storage unit 102 by the pre-pullback X-ray image reading unit 302. do. The generated blood vessel model group is stored in the blood vessel model storage unit 303. Details on the generation of the blood vessel model will be described later.

The marker detection unit 304 detects the position of the X-ray opaque marker 118 from the X-ray image during pullback read from the X-ray image storage unit 102 by the X-ray image reading unit 305 during pullback. The marker detection unit 304 of the present embodiment estimates the position of the X-ray opaque marker 118 on the image by fitting the vascular model generated by the vascular model generation unit 301 to the X-ray image during pullback, and the X-ray opaque marker 118. By setting the search range of the marker 118 in the vicinity of the estimated position, the detection accuracy of the X-ray opaque marker 118 is improved.

The blood vessel tomographic image reading unit 306 reads the blood vessel tomographic image stored in the blood vessel tomographic image storage unit 101 and provides it to the synchronous display unit 307. The synchronous display unit 307 controls the display so that the X-ray image during pullback read by the X-ray image reading unit 305 during pullback and the vascular tomographic image read by the vascular tomographic image reading unit 306 are displayed synchronously on the display 308. .. In the synchronized display, a pullback X-ray image and a vascular tomographic image whose imaging timings correspond to each other are displayed. Further, the synchronous display unit 307 emphasizes the position of the X-ray opaque marker 118 on the X-ray image during pullback detected by the marker detection unit 304 in this synchronous display.

The operation of the image display device 300 of the present embodiment having the above configuration will be described. FIG. 3 is a flowchart showing a process for synchronously displaying a blood vessel tomographic image and an X-ray image (angio image) according to the present embodiment.

First, in step S301, the intravascular diagnostic apparatus 100 collects a vascular tomographic image and an X-ray image. The image collection in step S301 will be described with reference to FIG.

FIG. 4 is a diagram illustrating the timing of collecting X-ray images and vascular tomographic images according to the present embodiment. In an operation panel (not shown) of the intravascular diagnostic device 100, when the user instructs to start scanning, the intravascular diagnostic device 100 starts low-speed rotation of the metal shaft 116 (and the imaging core 117) in the catheter 113. After that, when the user instructs the start of pullback ready on the operation panel of the intravascular diagnostic device 100, the intravascular diagnostic device 100 starts high-speed rotation of the metal shaft 116 (and the imaging core 117). When the imaging core 117 is rotated at high speed, it is possible to start taking a tomographic image while pulling back.

After the start of the pullback ready, the user instructs the start of X-ray imaging from an operation panel (not shown) of the X-ray imaging apparatus 200. When the start of X-ray imaging is instructed, the X-ray imaging apparatus 200 irradiates X-rays from the X-ray source 211 and captures an X-ray image by the X-ray sensor 212. The obtained X-ray image is transmitted to the intravascular diagnostic device 100 via the cable 103. The intravascular diagnostic apparatus 100 stores the X-ray image received until the start of pullback is instructed in the X-ray image storage unit 102 as a pre-pullback X-ray image. In this way, the pre-pullback X-ray image group 401 is stored in the X-ray image storage unit 102. It should be noted that only one heartbeat may be saved from the start of X-ray imaging.

The user instructs the start of X-ray photography and then starts the flash. In the flash, a contrast medium is injected into the blood vessel. Therefore, the X-ray image group captured by the X-ray imaging apparatus 200 during the period 421 from the start of the X-ray imaging to the start of the flash does not contain a contrast medium, and immediately before the pullback is performed. The entire catheter is shown. In the present embodiment, a blood vessel model described later is created using an image group taken during the period 421 of the pre-pullback X-ray image group 401.

When the user instructs to start pullback, the intravascular diagnostic apparatus 100 starts pullback of the imaging core 117 and also starts generation of a vascular tomographic image. The vascular tomographic image group 411 during the pullback operation is stored in the vascular tomographic image storage unit 101. Further, the X-ray image group acquired by the X-ray imaging apparatus 200 during this pullback operation is stored in the X-ray image storage unit 102 as the X-ray image group 402 during pullback. In the X-ray image during pullback, the region of interest of the blood vessel being pulled back is shown with a contrast medium, and the X-ray opaque marker 118 moving with the pullback of the imaging core 117 is shown.

When the collection of the blood vessel tomographic image (vascular tomographic image group 411) and the X-ray image (pre-pullback X-ray image group 401 and pullback medium X-ray image group 402) is completed as described above, the process proceeds to step S302. In steps S302 to S303, the X-ray opaque marker detection process by the image display device 300 of the present embodiment is executed. First, in step S302, the blood vessel model generation unit 301 of the image display device 300 generates a blood vessel model using the image of the pre-pullback X-ray image group 401 stored in the X-ray image storage unit 102. Hereinafter, the blood vessel model generation process by the blood vessel model generation unit 301 will be described in more detail with reference to the flowchart of FIG.

First, in step S501, the pre-pullback X-ray image reading unit 302 is the pre-pullback X-ray corresponding to a predetermined period within the period 421 of the pre-pullback X-ray image group 401 stored in the X-ray image storage unit 102. Load the image. In the present embodiment, in order to prepare a blood vessel model for at least one heartbeat, the above predetermined period is set to at least one heartbeat of the period 421 from the start of X-ray photography to the start of flash. The period set to include (for example, 1 second).

The predetermined period may be a period in which the catheter is indwelled and before the start of pullback and flush, and the predetermined period is set as follows, for example.
-Specify a predetermined period based on the timing at which the user instruction to start X-ray photography is detected. For example, the predetermined period is 1 second from 1 second after the instruction to start X-ray photography is detected, or 1 second from 1 second to 2 seconds after the instruction to start X-ray photography is detected.
-Based on the timing when the start of the flash is detected, a period before that (for example, 3 seconds to 2 seconds before the detection timing of the start of the flash) is used as the predetermined period. The start of the flash can be detected by detecting a change in the X-ray image (for example, detecting an increase in a dark region).

In the above example, the X-ray image was started to be saved in response to the user's instruction to start the X-ray imaging, and the pre-pullback X-ray image corresponding to the period 421 was read out in order to generate the blood vessel model. Not limited. For example, X-ray images are collected from before the acquisition of the tomographic image (pullback operation) to after the acquisition of multiple tomographic images (after the pullback operation is completed), and X-rays are collected based on the collected X-ray images. The timing of starting shooting may be determined, and an X-ray image for a predetermined period may be acquired from the determined timing. The automatic determination of the timing of starting X-ray imaging based on the X-ray image can be performed by the presence or absence of a specific comment such as "during imaging" included in the X-ray image, detection of darkening of the X-ray image at the start of imaging, and the like.

Next, in step S502, the blood vessel model generation unit 301 is a line along the catheter 113 (probe 115) from the read pre-pullback X-ray image (the shape of the catheter arranged in the blood vessel, and is an X-ray opaque marker. (Corresponding to the trajectory that 118 will move) is detected. The probe 115 in the catheter 113 used in the intravascular diagnostic device 100 has a metal shaft 116 to rotate the imaging core 117. The metal shaft 116 has a low transmittance of X-rays as compared with other biological tissues, and is represented as a black linear object 601 on the X-ray image as shown in FIG. 6A. Further, as described above with reference to FIG. 2, an X-ray opaque marker 118 is arranged at the tip of the probe 115 of the catheter 113 so that its position can be easily understood on the X-ray image. The X-ray opaque marker 118 is represented as a black dot 602 on the X-ray screen.

The catheter tip (black dot 602) can be automatically recognized by using a general image processing technique such as a pattern matching technique or a gray search, based on the above-mentioned characteristics. Further, since the entire catheter 113 is represented by a black line from the tip of the catheter, it can be automatically recognized by applying an adaptive binarization filter or the like and then using a general pathfinding method. Can be done. Since the contrast medium is not added to the pre-pullback X-ray image corresponding to the predetermined period within the period 421, the black linear object 601 and the black dot 602 are relatively clear, and it is possible to extract them accurately. can.

Next, in step S503, the blood vessel model generation unit 301 detects the tip end portion (hereinafter referred to as GC entrance) of the guiding catheter 112 in the pre-pullback X-ray image. As described above, in the procedure using the catheter, the guiding catheter 112, which is a hollow tube, is inserted in order to insert the necessary device into the blood vessel, and the observation target site (pullback) by the intravascular diagnostic device 100 is inserted. The acquisition range of the vascular tomographic image by the above) is from the probe tip detected in step S502 to the GC inlet. Therefore, in order to determine the observation target site, the position of the GC entrance in the pre-pullback X-ray image is obtained. In the absence of contrast agent, the shape of the GC entrance site has a cylindrical shape regardless of the type of guiding catheter used. Therefore, the GC entrance can also be detected automatically and accurately by a general pattern matching method or the like.

The user may specify the GC entrance portion by directly instructing the GC entrance portion on the pre-pullback X-ray image. Alternatively, the GC entrance portion may be automatically detected for another one heartbeat X-ray image group by using the information around the GC entrance portion instructed by the user in a specific pre-pullback X-ray image. ..

Since the frame rate of radiography by the X-ray imaging apparatus 200 is about 7 to 30 Hz, the number of X-ray images for one second is about 7 to 30. It may be detected. Further, the user may instruct the catheter tip in a single frame and automatically detect the catheter tip position in the X-ray image group for one heartbeat by using the feature amount of the instructed peripheral image.

In step S504, the blood vessel model generation unit 301 defines the coordinates of the locus from the tip of the catheter 113 detected in steps S502 and S503 to the GC entrance as the blood vessel model. For example, the blood vessel model generation unit 301 puts a black linear object 601 from the position of the GC entrance 603 to the position of the black point 602 shown in FIG. 6A along the catheter 113 in the acquisition range of the blood vessel tomographic image. Extract as a line. Then, as shown in FIG. 6B, the blood vessel model generation unit 301 divides the locus (extracted line) from the tip of the catheter 113 to the GC inlet into p-1 pieces and divides the coordinates of the tip into p-1 pieces. (X _{1, j} , y _{1, j} ), the x, y coordinates of each position are obtained with the coordinates of the GC entrance as (x _{p, j} , y _{p, j} ), and the blood vessel model is used. The blood vessel model generated as described above is stored in the blood vessel model storage unit 303. Note that j is a blood vessel model number, and if the number of pre-pullback X-ray images acquired from a predetermined period is J, j takes a value of 1 to J.

In step S505, it is determined whether or not the final X-ray image (Jth X-ray image, hereinafter referred to as the final frame) of the pre-pullback X-ray images within the above-mentioned predetermined period has been processed. If the final frame has not been processed, the process returns to step S501, and the above-mentioned process is executed for the next pre-pullback X-ray image in a predetermined period.

As described above, when the blood vessel model is generated from all the pre-pullback X-ray images within the predetermined period, the process proceeds from step S505 to step S506. In step S506, the blood vessel model generation unit 301 stores the blood vessel model group (J blood vessel models in this embodiment) generated by steps S501 to S505 in the blood vessel model storage unit 303. As described above, a blood vessel model group including a blood vessel model for one heartbeat is generated and held. The blood vessel model group represents the position of the catheter 113 (the position of the blood vessel) according to the pulsation. In step S504, the coordinates are held as a blood vessel model, but the present invention is not limited to this. For example, the trajectory of the catheter 113 from the detected catheter tip to the GC inlet is white (or black), and the others are black (or white) -like image data (for example, an image as shown in FIG. 6 (b)). It may be retained as a group of models.

When the blood vessel model is generated by the blood vessel model generation unit 301 as described above, the process proceeds to step S303 in FIG. In step S303, the marker detection unit 304 detects the position of the X-ray opaque marker 118 for each X-ray image of the X-ray image group 402 during pullback using the blood vessel model group generated in step S302. Hereinafter, the marker detection process by the marker detection unit 304 will be described with reference to the flowchart of FIG. 7.

In step S701, the pullback X-ray image reading unit 305 reads the pullback X-ray image from the X-ray image storage unit 102. Next, in step S702, the marker detection unit 304 corrects the deviation between the respiratory state of the subject in the X-ray image read in step S701 and the respiratory state at the time of acquisition of each blood vessel model, that is, the influence of breathing. (Hereafter referred to as breathing correction).

Breathing is a vibrating movement linked to the movement of the diaphragm. Since the procedure is controlled by natural breathing during the procedure using a catheter, vibrational motion of about 0.3 times / frame is generally generated. Changes due to the beating of the heart result in translation, rotation, and deformation of blood vessels, whereas changes due to respiration mainly result in translation of blood vessels. Therefore, in the respiratory correction, the movement (parallel movement) of the entire blood vessel on the X-ray image in the vertical and horizontal directions is corrected. In the respiratory correction of the present embodiment, for example, the position of the GC entrance portion in the X-ray image during pullback is detected by a pattern matching method, and the blood vessel model is such that the GC entrance portion of the blood vessel model coincides with the detected position of the GC entrance portion. Move in parallel. Since it may be difficult to detect the GC entrance from the entire X-ray image, for example, the user specifies the GC entrance only in the first frame, and the GC entrance position of the previous frame is tracked for the subsequent frames. You may do it.

Another method is to detect the movement of the diaphragm existing in the X-ray image and determine the amount of movement due to respiration. The diaphragm is usually confirmed as a low-luminance uniform image in an X-ray image. The brightness of the entire X-ray image is (1) the blood vessel image in which the contrast medium is injected, which is the lowest brightness region, and (2) the diaphragm region, which is the next lowest brightness region (the region where the brightness value is intermediate). (3) It can be classified into three areas other than the above (1) and (2). Therefore, for example, by tracing the image center of the brightness classified in (2) above when classified into three using a clustering method, it is possible to detect the amount of movement in the vertical and horizontal directions due to respiration. .. When the user specifies the position of the GC entrance for the first frame, the position of the GC entrance for subsequent frames can be determined by the amount of movement obtained based on the displacement of the diaphragm described above.

As yet another method, instead of detecting the movement of the GC inlet described above, the movement of an X-ray opaque marker (hereinafter referred to as a tip marker) (hereinafter, tip marker) provided on the distal end side of the catheter 113 may be detected. good. Generally, the tip marker has a larger size than the X-ray opaque marker 118 arranged in the vicinity of the imaging core 117, and can be detected as a clearer black spot.

Since the displacement due to respiration is mainly in the vertical direction, the correction may be performed only in the vertical direction.

The respiratory correction of the blood vessel model will be described more specifically. When the coordinate group of the jth frame of the blood vessel model is (Xj, Yj), it is expressed as follows [Equation 1].

Here, there are p coordinates of the blood vessel model, and the indexes are stored so as to increase from the Distal side (X-ray opaque marker side) to the Proximal side (GC inlet side). Further, x is the coordinate in the left-right direction of the X-ray image, and y is the coordinate in the vertical direction on the X-ray image.

Assuming that the coordinates of the GC entrance in the X-ray image during pullback to be processed are (x _GC , y _GC ) and the coordinates of the GC entrance of the blood vessel model are (x _{p, j} , y _{p, j} ), after respiratory correction The coordinate group ( _X'j , _Y'j ) of the blood vessel model is as shown in [Equation 2] below.

In step S702, the above respiratory correction is performed on all the blood vessel models stored in the blood vessel model storage unit 303, and the respiratory corrected blood vessel model group is generated.

Next, in step S703, the marker detection unit 304 selects a blood vessel model corresponding to the blood vessel to be examined by using the blood vessel model after respiratory correction. The blood vessel model generated by the blood vessel model generation unit 301 is locus information from the GC entrance to the catheter tip in each heartbeat. The marker detection unit 304 detects a respiratory-corrected blood vessel model (that is, a blood vessel model whose heartbeat cycle corresponds to the X-ray image during pullback to be processed) that is adapted to the X-ray image during pullback during processing. The processing will be described below.

First, the X-ray image during pullback is binarized using an adaptive binarization filter. Using the luminance value corresponding to the coordinates of the blood vessel model after respiration correction for the obtained binarized image, for example, the correlation value Rj for the _jth blood vessel model is calculated by the following [Equation 3].

Here, I (x'i _{, j} , y'i _{, j} ) is a pixel value at the coordinates (x'i _{, j} , y'i _{, j} ) of the X-ray image during the pullback after binarization. In the present embodiment, the contrast medium is charged into the blood vessel, and the above-mentioned binarization makes the blood vessel image in which the contrast medium is present black and the other regions white. Therefore, if the coordinates ( _{x'i, j} , y'i _{, j} ) are on the blood vessel image in which the contrast medium is present, the value of I ( _{x'i, j} , y'i _{, j} ) is 1, and other than that. If it is in the area of, the value of I ( _{x'i, j} , y'i _{, j} ) becomes 0. Further, k is a weight for considering the continuity of the blood vessel, and is a weight when the position (coordinates ( _{x'i, j} , y'i _{, j} )) of the blood vessel model is continuously present on the blood vessel image. I am trying to increase the value. Needless to say, it is not necessary to use weighting. When no weighting is used, ki = 1.

According to the above calculation, for example, in the blood vessel model 801 having many overlapping portions with the blood vessel image 811 as shown in FIG. 8A, the calculated correlation value R becomes large and is shown in FIG. 8B. As described above, the correlation value R calculated in the blood vessel model 802 deviating from the blood vessel image 811 becomes small. In this embodiment, since J blood vessel models are generated, the correlation value is calculated for each blood vessel model of j = 1 to J, and the blood vessel model showing the largest correlation (where R _j is maximum) is the blood vessel model. It is adopted as a blood vessel model for the X-ray image during pullback to be processed.

Next, in step S704, the marker detection unit 304 estimates the position of the X-ray opaque marker 118 in the pullback medium X-ray image using the blood vessel model selected in step S703. The catheter 113 of the intravascular diagnostic apparatus 100 should be present on the blood vessel model (catheter trajectory) selected in the above process. Furthermore, it can be assumed that the probe (imaging core 117 and X-ray opaque marker 118) is moving at a constant velocity from the tip position of the catheter toward the GC inlet on the blood vessel model.

のように表される。 The vascular model selected in step S703 is a locus from the probe tip to the GC inlet before pullback, and is selected considering that X-ray images during pullback are collected in synchronization with the pullback operation by the intravascular diagnostic apparatus 100. The distance obtained by equally dividing the created blood vessel model by the number of frames (the number of blood vessel tomographic image groups 411) from the start of pullback to the detection of GC on the blood vessel tomographic image is the distance on the X-ray image by the X-ray opaque marker. The unit distance to move is Δl. The GC is depicted on the vascular tomographic image as a vibrant, substantially circular circumference, which is very characteristic of normal vascular images. Therefore, GC can be easily detected from the vascular tomographic image. Assuming that the number of blood vessel tomographic image groups 411 from the start of pullback to the display of GC is N and the total length of the selected blood vessel model is L, the unit distance Δl is

It is expressed as.

However, X-ray images are usually collected at sampling rates of 7, 15, and 30 Hz, whereas vascular tomographic images are collected at 30 Hz and above (for example, 160, 180 Hz). Therefore, it is necessary to consider the difference in the frame rate for the single movement distance of the X-ray opaque marker on the X-ray image that can be actually acquired. Therefore, the predicted position of the X-ray opaque marker 118 in the m-th pullback medium X-ray image is only the distance L _m indicated by the following [Equation 5] from the tip (pullback start position) on the selected blood vessel model. It exists in a distant position.

Here, _{f 0} is the sampling rate (Hz) of the blood vessel tomographic image, and fa is the sampling rate (Hz) of the X-ray image.

The length l _i from the tip (x _{1, j} , y _{1, j} ) corresponding to the coordinates of the i-index eye of the selected blood vessel model is expressed as the following [Equation 6].

となる血管モデルのｉインデックス目（または、ｉ＋１インデックス目）に該当する座標位置を、Ｘ線不透過マーカ１１８の予測位置とする。または、ｉインデックス目に該当する座標位置とｉ＋１インデックス目に該当する座標位置を用いて直線補間を行った結果を予測位置としてもよい。この場合、ｉインデックス目に該当する座標を（Ｘ_i，Ｙ_i+1）、ｉ＋１インデックス目に該当する座標を（Ｘ_i+1，Ｙ_i+1）とすると、
予測位置＝（ｋ＊Ｘ_i＋（１－ｋ）＊Ｘ_i+1，ｋ＊Ｙ_i＋（１－ｋ）＊Ｙ_i）
ｋ=（ｌ_i+1－Ｌ_ｍ）／（ｌ_i+1－ｌ_i）
となる。なお、当該予測位置に前フレームでのＸ線不透過マーカ１１８の位置をフィードバックして予測位置を求めることで予測位置の精度を高めても良い。 therefore,

The coordinate position corresponding to the i-index (or i + 1) index of the blood vessel model is the predicted position of the X-ray opaque marker 118. Alternatively, the result of linear interpolation using the coordinate position corresponding to the i-index and the coordinate position corresponding to the i + 1 index may be used as the predicted position. In this case, if the coordinates corresponding to the i-index are (X _i , Y _{i + 1} ) and the coordinates corresponding to the i + 1 index are (X _{i + 1} , Y _{i + 1} ),
Predicted position = (k * X _i + (1-k) * X _{i + 1} , k * Y _i + (1-k) * Y _i )
k = (l _{i + 1} -L _m ) / (l _{i + 1} -l _i )
Will be. The accuracy of the predicted position may be improved by feeding back the position of the X-ray opaque marker 118 in the previous frame to the predicted position to obtain the predicted position.

As described above, when the position of the X-ray opaque marker 118 in the pullback X-ray image is estimated based on the blood vessel model, the process proceeds to step S705. In step S705, the marker detection unit 304 searches for the X-ray opaque marker 118 in the vicinity of the position estimated in step S704, and determines the position of the X-ray opaque marker.

As described above, the X-ray opaque marker 118 is displayed as a black dot on the X-ray image. Therefore, the marker detection unit 304 performs a process of emphasizing the black spot with respect to a specific range (for example, in the vicinity of the predicted position) based on the predicted position of the X-ray opaque marker 118 estimated in step S704. Examples of such processing include the use of a filter that emphasizes black spots (for example, a convolution filter, frequency analysis (Wavelet, FFT)).

Since a plurality of candidate points are generally found as a result of the filtering process, the marker detection unit 304 obtains a correlation with the X-ray opaque marker 118 and selects a candidate point having a high correlation as the X-ray opaque marker 118. For example, the following correlation values can be used.
<Distance correlation value>
The squared value of the distance between the estimated position and the detected position, or the squared value of the distance between the blood vessel model and the detected position <shape correlation value>
As described above, the X-ray opaque metal shaft 116 exists behind the X-ray opaque marker. Further, due to the structure of the catheter 113, the inside of the catheter 113 on the distal end side of the X-ray opaque marker 118 is filled with an X-ray permeable component such as air or raw food. Therefore, in terms of shape, the luminance value in the base direction of the blood vessel model is low and the luminance value in the end direction is high from the position of the detected X-ray opaque marker 118. A one-dimensional filter that imitates such a shape is applied from the detection position of the black spot toward the blood vessel model to determine the correlation.

Next, in step S706, the marker detection unit 304 determines whether or not the above-mentioned processing has been completed for all the X-ray images during pullback. If there is an unprocessed X-ray image during pullback, the process returns to step S701, and the processes of steps S701 to S705 are performed for the next X-ray image during pullback. In this way, when the marker is detected for all the X-ray images during pullback, the marker detection process ends.

Returning to FIG. 3, in step S304, the synchronous display unit 307 synchronously displays the vascular tomographic image and the pullback X-ray image on the display 308. Assuming that N blood vessel tomographic images and M X-ray images during pullback are obtained during the pullback operation, the synchronous display unit 307 is the nth blood vessel tomographic image and m = fa / _{f 0} * nth. It is controlled so that the X-ray image during the pullback of is displayed at the same time. Note that _{f 0} is the sampling rate (Hz) of the blood vessel tomographic image, and fa is the sampling rate (Hz) of the X-ray image. Since the position of the X-ray opaque marker 118 is detected for each of the M pullback X-ray images by the marker detection process, the synchronous display unit 307 displays the X-ray opaque marker on the pullback X-ray image being displayed. Make a highlight to emphasize the position of 118. As a method of emphasizing the position of the X-ray opaque marker 118, for example, a figure having a predetermined luminance (or color) is displayed at a position detected by the marker detection unit 304, the figure is blinked, or detected. For example, displaying an arrow figure indicating a position.

FIG. 9 is a diagram showing an example of synchronous display according to an embodiment. The synchronous display unit 307 controls the display of the display 308, and synchronously displays the X-ray image 901 (angio image) and the blood vessel tomographic image 902 as shown in FIG. In the synchronous display, the X-ray image 901 taken at the same timing as the acquisition timing of the displayed blood vessel tomographic image 902 is displayed. Further, in FIG. 9, as a display example for clearly indicating the X-ray opaque marker position, a state in which the highlight mark 911 is superimposed on the X-ray opaque marker position and a state in which the X-ray opaque marker position is indicated by the arrow figure 912 are shown. Has been done. The shapes and colors of the highlight mark 911 and the arrow figure 912 are not limited to those shown in the drawings, and blinking display may be performed. Further, the display of the highlight mark 911 and the arrow figure 912 may be switched on and off by the user operation.

As described above, according to the image display device 300 of the above embodiment, the position of the X-ray opaque marker 118 on the X-ray image (angio image) can be detected with high reliability. Further, the blood vessel tomographic image and the X-ray image (X-ray image during pullback) are displayed synchronously on the display 308, and the position of the X-ray opaque marker 118 is clearly shown on the X-ray image. Therefore, the user can easily and immediately grasp the position of the X-ray opaque marker 118 in the X-ray image (angio image) at the time when the displayed blood vessel tomographic image is taken.

In the above embodiment, an example is shown in which the image display device is realized by an information processing device (for example, a general-purpose PC) independent of the intravascular diagnostic device 100 and the X-ray imaging device 200, but the present invention is not limited to this. do not have. For example, the function of the image display device 300 may be incorporated into the intravascular diagnostic device 100 or the X-ray imaging device 200. Further, in the above embodiment, the intravascular diagnostic device 100 stores the X-ray image from the X-ray imaging device 200 in the X-ray image storage unit 102, but the X-ray imaging device 200 may store the X-ray image. good. In this case, the pre-pullback X-ray image group 401 and the pullback medium X-ray image group are received from the X-ray imaging device 200 and the signal indicating the pullback start operation and the pullback end from the intravascular diagnostic device 100 via the cable 103. The 402 can be stored separately.

In the image display device 300, the selection result of the blood vessel model in step S703 may be changed according to the user operation. For example, J images of the respiratory-corrected blood vessel model are presented on the display 308 so that the user can select the desired blood vessel model. The image display device 300 superimposes and displays the blood vessel model selected by the user on the X-ray image during pullback, and the user determines the suitability of the blood vessel model by looking at the superposed display. Then, according to a predetermined determination operation from the user, the blood vessel model currently selected by the user is determined as the blood vessel model corresponding to the X-ray image during the pullback.

Further, the position of the X-ray opaque marker detected in step S705 may be corrected according to the user operation. For example, the highlighting on the X-ray image during pullback displayed during synchronous display can be moved according to the user operation such as mouse operation, and when there is a user operation to move the highlighting, the highlighting is displayed. Correct the position of the X-ray opaque marker to the position after moving.

In the synchronous display, the user may be able to set whether or not to execute the process of clearly indicating the position of the X-ray opaque marker. Further, each functional unit shown for the image display device 300 may be realized by a computer executing a predetermined program, or a part or all of them may be realized by hardware.

100: Intravascular diagnostic device, 101: Vascular tomographic image storage unit, 102: X-ray image storage unit, 103: Cable, 200: X-ray imaging device, 211: X-ray source, 212: X-ray sensor, 300: Image display Device, 301: Vascular model generation unit, 302: Pullback X-ray image reading unit, 303: Vascular model storage unit, 304: Marker detection unit, 305: Pullback medium X-ray image reading unit, 306: Vascular tomographic image reading unit, 307 : Synchronous display, 308: Display

Claims (12)

A plurality of tomographic images obtained by emitting measurement light through the probe while moving the probe in the axial direction of the catheter in the blood vessel to which the contrast medium is administered and incident the reflected light, and the movement of the probe. It is an image display device that displays a plurality of X-ray images taken in between.
For a predetermined period set to include at least one beat of the heart, from the start of radiography to the start of flash to administer the contrast medium intravascularly, prior to the start of movement of the probe. An extraction means for extracting a line along the probe from the captured X-ray image, and
An X-ray opaque marker provided on the probe in each of the plurality of X-ray images taken while moving the probe during the pullback period provided in the flash using the line extracted by the extraction means. Detection means to detect the position of
A display control means for synchronously displaying the plurality of X-ray images and the plurality of tomographic images and clearly indicating the position of the X-ray opaque marker detected by the detection means in the display of the X-ray image in the synchronous display. , Equipped with
The detection means
An estimation means for estimating the position of the X-ray opaque marker in the X-ray image taken during the movement based on the line extracted by the extraction means, and an estimation means.
An image display device comprising: a search means for searching an image of the X-ray opaque marker in the vicinity of a position estimated by the estimation means in the X-ray image taken during the movement. .. A selection means for selecting a line that most closely matches a blood vessel image in the X-ray image taken during the movement from a plurality of lines extracted from a plurality of X-ray images taken in the predetermined period by the extraction means. Further prepared,
The image display device according to claim 1 , wherein the estimation means estimates the position of the X-ray opaque marker based on a line selected by the selection means. The image display device according to claim 2 , further comprising a changing means for changing the line selection result by the selection means according to a user operation. One of claims 1 to 3 , wherein the extraction means extracts a line corresponding to a portion of the probe between the position of the X-ray opaque marker and the end of the guiding catheter. The image display device described in the section. The line is drawn so that the position of the end of the guiding catheter in the X-ray image taken during the movement of the probe coincides with the position of the line corresponding to the end of the guiding catheter. Further equipped with a means of transportation that moves in parallel,
The image display device according to claim 4 , wherein the estimation means uses a line moved by the moving means. The image display device according to any one of claims 1 to 3 , wherein the predetermined period is specified based on a timing at which a user instruction for starting X-ray imaging is detected. A collecting means for collecting X-ray images from before the acquisition of the plurality of tomographic images is started to after the acquisition of the plurality of tomographic images.
A claim characterized by further comprising an acquisition means for determining the timing of starting X-ray imaging based on the X-ray image collected by the collecting means and acquiring the X-ray image for the predetermined period from the determined timing. Item 6. The image display device according to any one of Items 1 to 5 . The image display device according to any one of claims 1 to 7 , further comprising a correction means for correcting the position of the X-ray opaque marker detected by the detection means according to a user operation. .. The image display device according to any one of claims 1 to 8 , further comprising a setting means for setting whether or not to execute a process for specifying the position of the X-ray opaque marker. A tomographic image acquisition means that emits measurement light through the probe and incidents the reflected light to obtain a tomographic image.
An X-ray image acquisition means for acquiring an X-ray image taken by an X-ray imaging device, and
An optical interference diagnostic device comprising the image display device according to any one of claims 1 to 9 . A plurality of tomographic images obtained by emitting measurement light through the probe while moving the probe in the axial direction of the catheter in the blood vessel to which the contrast medium is administered and incident the reflected light, and the movement of the probe. It is a method of operating an image display device that displays a plurality of X-ray images taken in between.
For a predetermined period set to include at least one beat of the heart, from the start of radiography to the start of flash to administer the contrast medium intravascularly, prior to the start of movement of the probe. An extraction step of extracting a line along the probe from the captured X-ray image, and
The X-ray opaque marker provided on the probe in each of the plurality of X-ray images taken while moving the probe during the pullback period provided in the flash using the line extracted in the extraction step. The detection process to detect the position of
A display control step of synchronously displaying the plurality of X-ray images and the plurality of tomographic images and clearly indicating the position of the X-ray opaque marker detected in the detection step in the display of the X-ray image in the synchronous display. Have,
The detection step is
An estimation step of estimating the position of the X-ray opaque marker in the X-ray image taken during the movement based on the line extracted in the extraction step, and an estimation step.
An image display device comprising a search step of searching for an image of the X-ray opaque marker in the vicinity of a position estimated by the estimation step in the X-ray image taken during the movement . How it works. A program for causing a computer to execute each step of the operation method of the image display device according to claim 11 .

Images