WO2023189308A1 - コンピュータプログラム、画像処理方法及び画像処理装置 - Google Patents

コンピュータプログラム、画像処理方法及び画像処理装置 Download PDF

Info

Publication number
WO2023189308A1
WO2023189308A1 PCT/JP2023/008701 JP2023008701W WO2023189308A1 WO 2023189308 A1 WO2023189308 A1 WO 2023189308A1 JP 2023008701 W JP2023008701 W JP 2023008701W WO 2023189308 A1 WO2023189308 A1 WO 2023189308A1
Authority
WO
WIPO (PCT)
Prior art keywords
treatment
image
tomographic
images
tomographic image
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Ceased
Application number
PCT/JP2023/008701
Other languages
English (en)
French (fr)
Japanese (ja)
Inventor
雄紀 坂口
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Terumo Corp
Original Assignee
Terumo Corp
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Terumo Corp filed Critical Terumo Corp
Priority to JP2024511607A priority Critical patent/JPWO2023189308A1/ja
Publication of WO2023189308A1 publication Critical patent/WO2023189308A1/ja
Anticipated expiration legal-status Critical
Ceased legal-status Critical Current

Links

Images

Classifications

    • A—HUMAN NECESSITIES
    • A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
    • A61B—DIAGNOSIS; SURGERY; IDENTIFICATION
    • A61B1/00—Instruments for performing medical examinations of the interior of cavities or tubes of the body by visual or photographical inspection, e.g. endoscopes; Illuminating arrangements therefor
    • A—HUMAN NECESSITIES
    • A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
    • A61B—DIAGNOSIS; SURGERY; IDENTIFICATION
    • A61B1/00—Instruments for performing medical examinations of the interior of cavities or tubes of the body by visual or photographical inspection, e.g. endoscopes; Illuminating arrangements therefor
    • A61B1/005—Flexible endoscopes
    • A61B1/01—Guiding arrangements therefore
    • A—HUMAN NECESSITIES
    • A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
    • A61B—DIAGNOSIS; SURGERY; IDENTIFICATION
    • A61B1/00—Instruments for performing medical examinations of the interior of cavities or tubes of the body by visual or photographical inspection, e.g. endoscopes; Illuminating arrangements therefor
    • A61B1/04—Instruments for performing medical examinations of the interior of cavities or tubes of the body by visual or photographical inspection, e.g. endoscopes; Illuminating arrangements therefor combined with photographic or television appliances
    • A61B1/045—Control thereof
    • A—HUMAN NECESSITIES
    • A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
    • A61B—DIAGNOSIS; SURGERY; IDENTIFICATION
    • A61B1/00—Instruments for performing medical examinations of the interior of cavities or tubes of the body by visual or photographical inspection, e.g. endoscopes; Illuminating arrangements therefor
    • A61B1/313—Instruments for performing medical examinations of the interior of cavities or tubes of the body by visual or photographical inspection, e.g. endoscopes; Illuminating arrangements therefor for introducing through surgical openings, e.g. laparoscopes
    • A—HUMAN NECESSITIES
    • A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
    • A61B—DIAGNOSIS; SURGERY; IDENTIFICATION
    • A61B8/00—Diagnosis using ultrasonic, sonic or infrasonic waves
    • A61B8/12—Diagnosis using ultrasonic, sonic or infrasonic waves in body cavities or body tracts, e.g. by using catheters

Definitions

  • the present invention relates to a computer program, an image processing method, and an image processing device.
  • IVUS IntraVascular Ultra Sound
  • OFDI near-infrared optical coherence tomography
  • IVUS IntraVascular Ultra Sound
  • OFDI near-infrared optical coherence tomography
  • a computer program capable of displaying tomographic images of a hollow organ obtained before and after treatment of the hollow organ with the longitudinal position of the hollow organ aligned;
  • An object of the present invention is to provide an image processing method and an image processing device.
  • a computer program acquires a plurality of tomographic images of a hollow organ while moving a sensor unit for scanning the hollow organ along a running direction of the hollow organ, Select at least one tomographic image related to the target site where the treatment for the hollow organ is to be performed from among the plurality of tomographic images acquired before the treatment of the hollow organ, and select an image of the plurality of tomographic images acquired after the treatment.
  • a tomographic image after the treatment corresponding to the selected tomographic image before the treatment is identified, and based on the identification result of the tomographic image, the tomographic image acquired before the treatment and the tomographic image acquired after the treatment are identified.
  • the computer executes the process of displaying the images in association with each other.
  • An image processing method provides a plurality of tomographic images of a hollow organ obtained by moving a sensor unit for scanning the hollow organ along a running direction of the hollow organ. , select at least one tomographic image related to the target site where the treatment for the hollow organ is to be performed from among the plurality of tomographic images acquired before the treatment for the hollow organ, and select the plurality of tomographic images acquired after the treatment.
  • a post-treatment tomographic image corresponding to the selected tomographic image before treatment is identified through image recognition processing of the tomographic image, and based on the identification result of the tomographic image, a tomographic image acquired before the treatment is identified;
  • the tomographic images acquired after the treatment are displayed in association with each other.
  • An image processing device acquires a plurality of tomographic images of a hollow organ while moving a sensor unit for scanning the hollow organ along a running direction of the hollow organ. and a processing unit that executes a process of displaying the tomographic image acquired by the acquisition unit on a display device, and the processing unit is configured to select one of the plurality of tomographic images acquired before the treatment of the hollow organ.
  • At least one tomographic image related to the target site where the treatment for the hollow organ is to be performed is selected, and the one tomographic image before the selected treatment is processed by image recognition processing of the plurality of tomographic images acquired after the treatment.
  • the corresponding tomographic image after the treatment is identified, and the tomographic image acquired before the treatment and the tomographic image acquired after the treatment are displayed in association with each other based on the identification result of the tomographic image.
  • the longitudinal position of the hollow organ when displaying tomographic images of a hollow organ acquired before and after treatment of the hollow organ, the longitudinal position of the hollow organ can be aligned and displayed.
  • FIG. 2 is an explanatory diagram showing a configuration example of an image diagnostic apparatus. It is an explanatory view showing an example of composition of a catheter for image diagnosis.
  • FIG. 2 is an explanatory diagram schematically showing a cross section of a blood vessel through which a sensor section is inserted.
  • FIG. 2 is an explanatory diagram of a tomographic image.
  • FIG. 2 is an explanatory diagram of a tomographic image.
  • 1 is a block diagram showing a configuration example of an image processing device.
  • FIG. FIG. 2 is a block diagram showing a configuration example of an IVUS image recognition learning model.
  • FIG. 2 is a block diagram showing a configuration example of an IVUS image recognition learning model.
  • 3 is a flowchart showing an image processing procedure according to the first embodiment.
  • FIG. 3 is a flowchart showing an image processing procedure according to the first embodiment.
  • This is a display example of an IVUS image and a treatment plan screen before treatment.
  • This is a display example of IVUS images before and after treatment.
  • 7 is a flowchart showing an image processing procedure according to the second embodiment.
  • FIG. 7 is an explanatory diagram showing an image processing method according to the second embodiment.
  • FIG. 7 is an explanatory diagram showing an image processing method according to the second embodiment.
  • FIG. 7 is an explanatory diagram showing an image processing method according to the second embodiment.
  • 12 is a flowchart showing an image processing procedure according to the third embodiment.
  • 12 is a flowchart showing an image processing procedure according to the fourth embodiment.
  • , is an explanatory diagram showing a plurality of pre-treatment and post-treatment IVUS images P1 arranged in the longitudinal direction of a blood vessel.
  • 12 is a flowchart showing an image processing procedure according to the fifth embodiment.
  • cardiac catheterization which is endovascular treatment
  • luminal organs targeted for catheterization are not limited to blood vessels, and include other organs such as bile ducts, pancreatic ducts, bronchi, and intestines. It may also be a hollow organ.
  • present invention is not limited to these examples, but is indicated by the scope of the claims, and is intended to include all changes within the meaning and scope equivalent to the scope of the claims. At least some of the embodiments described below may be combined arbitrarily.
  • an image diagnostic apparatus using a dual-type catheter that has the functions of both intravascular ultrasound diagnosis (IVUS) and optical coherence tomography diagnosis (OFDI) will be described.
  • a dual-type catheter has a mode in which ultrasonic tomographic images are acquired only by IVUS, a mode in which optical coherence tomographic images are acquired only in OFDI, and a mode in which both tomographic images are acquired by IVUS and OFDI. , you can switch between these modes.
  • the ultrasound tomographic image and the optical coherence tomographic image will be referred to as an IVUS image and an OFDI image, respectively.
  • IVUS images and OFDI images are collectively referred to as tomographic images.
  • FIG. 1 is an explanatory diagram showing a configuration example of an image diagnostic apparatus 100.
  • the image diagnostic apparatus 100 of the first embodiment includes an intravascular examination apparatus 101, an angiography apparatus 102, an image processing apparatus 3, a display apparatus 4, and an input apparatus 5.
  • the intravascular examination apparatus 101 includes an image diagnosis catheter 1 and an MDU (Motor Drive Unit) 2.
  • the diagnostic imaging catheter 1 is connected to an image processing device 3 via an MDU 2.
  • a display device 4 and an input device 5 are connected to the image processing device 3.
  • the display device 4 is, for example, a liquid crystal display or an organic EL display
  • the input device 5 is, for example, a keyboard, a mouse, a trackball, a microphone, or the like.
  • the display device 4 and the input device 5 may be integrally stacked to form a touch panel. Furthermore, the input device 5 and the image processing device 3 may be configured as one unit. Furthermore, the input device 5 may be a sensor that accepts gesture input, gaze input, or the like.
  • the angiography device 102 is connected to the image processing device 3.
  • the angiography apparatus 102 is an angiography apparatus that images the blood vessel using X-rays from outside the patient's body while injecting a contrast medium into the patient's blood vessel, and obtains an angiography image that is a transparent image of the blood vessel.
  • the angiography apparatus 102 includes an X-ray source and an X-ray sensor, and the X-ray sensor receives X-rays emitted from the X-ray source to image a patient's X-ray fluoroscopic image.
  • the angiography apparatus 102 outputs the obtained angiography image to the image processing apparatus 3 and displays it on the display apparatus 4 via the image processing apparatus 3.
  • FIG. 2 is an explanatory diagram showing an example of the configuration of the diagnostic imaging catheter 1. Note that the area surrounded by the upper dashed line in FIG. 2 is an enlarged area of the area surrounded by the lower dashed line.
  • the diagnostic imaging catheter 1 includes a probe 11 and a connector section 15 disposed at an end of the probe 11.
  • the probe 11 is connected to the MDU 2 via the connector section 15.
  • the side of the diagnostic imaging catheter 1 far from the connector portion 15 will be referred to as the distal end side, and the side of the connector portion 15 will be referred to as the proximal end side.
  • the probe 11 includes a catheter sheath 11a, and a guide wire insertion portion 14 through which a guide wire can be inserted is provided at the distal end thereof.
  • the guide wire insertion section 14 constitutes a guide wire lumen, and is used to receive a guide wire that has been inserted into a blood vessel in advance, and to guide the probe 11 to the affected area using the guide wire.
  • the catheter sheath 11a forms a continuous tube section extending from the guide wire insertion section 14 to the connector section 15.
  • a shaft 13 is inserted into the interior of the catheter sheath 11a, and a sensor section 12 is connected to the distal end side of the shaft 13.
  • the sensor section 12 has a housing 12c, and the distal end side of the housing 12c is formed into a hemispherical shape to suppress friction and catch with the inner surface of the catheter sheath 11a.
  • Inside the housing 12c there is an ultrasonic transmitting/receiving unit 12a that transmits ultrasonic waves into the blood vessel and receives reflected waves from within the blood vessel, and an ultrasonic transmitting/receiving unit 12a that transmits near-infrared light into the blood vessel and receives reflected light from within the blood vessel.
  • An optical transmitting/receiving section 12b is arranged. In the example shown in FIG.
  • an ultrasonic transmitter/receiver 12a is provided on the distal end side of the probe 11, and an optical transmitter/receiver 12b is provided on the proximal end side. That is, the ultrasonic transmitter/receiver 12a and the optical transmitter/receiver 12b are arranged in the housing 12c at a predetermined length apart along the axial direction on the central axis of the shaft 13 (on the chain double-dashed line in FIG. 2). .
  • the ultrasonic transmitting/receiving section 12a and the optical transmitting/receiving section 12b are arranged so that the direction of transmitting and receiving the ultrasonic waves and near-infrared light is approximately 90 degrees with respect to the axial direction of the shaft 13 (radial direction of the shaft 13). has been done. Note that it is desirable that the ultrasonic transmitter/receiver 12a and the optical transmitter/receiver 12b be installed slightly offset from the radial direction so as not to receive reflected waves and reflected light on the inner surface of the catheter sheath 11a. In the first embodiment, for example, as shown by arrows in FIG.
  • the ultrasonic transmitter/receiver 12a emits ultrasound in a direction inclined toward the proximal end with respect to the radial direction
  • the optical transmitter/receiver 12b The near-infrared light is irradiated in a direction that is inclined toward the tip side with respect to the direction.
  • An electrical signal cable (not shown) connected to the ultrasonic transmitting/receiving section 12a and an optical fiber cable (not shown) connected to the optical transmitting/receiving section 12b are inserted into the shaft 13.
  • the probe 11 is inserted into the blood vessel from the distal end side.
  • the sensor section 12 and the shaft 13 can move forward and backward inside the catheter sheath 11a, and can also rotate in the circumferential direction.
  • the sensor unit 12 and the shaft 13 rotate about the central axis of the shaft 13 as a rotation axis.
  • the connector part 15 of the diagnostic imaging catheter 1 is detachably attached to the MDU 2, and the MDU 2 drives a built-in motor according to the operation of the user (medical staff) to control the operation of the diagnostic imaging catheter 1 inserted into the blood vessel. It is a driving device to control.
  • the MDU 2 performs a pullback operation in which the sensor section 12 and shaft 13 inserted into the probe 11 are pulled toward the MDU 2 at a constant speed and rotated in the circumferential direction.
  • the sensor section 12 scans the inside of the blood vessel continuously at predetermined time intervals while rotating while moving from the distal end side to the proximal end side by a pullback operation, thereby generating a plurality of transverse layer images approximately perpendicular to the probe 11. are taken continuously at predetermined intervals.
  • the MDU 2 outputs the reflected ultrasound signal received by the ultrasound transmitter/receiver 12 a and the reflected light signal received by the optical transmitter/receiver 12 b to the image processing device 3 .
  • the image processing device 3 acquires the reflected wave signal of the ultrasound output from the ultrasound transmitting/receiving unit 12a via the MDU 2 as reflected wave data, and generates ultrasound line data based on the acquired reflected wave data.
  • the ultrasound line data is data indicating the reflected intensity of ultrasound in the depth direction of the blood vessel as seen from the ultrasound transmitter/receiver 12a.
  • the image processing device 3 constructs an IVUS image P1 (see FIGS. 4A and 4B) representing a cross-section of a blood vessel based on the generated ultrasound line data.
  • the image processing device 3 acquires interference light data by causing the reflected light signal output from the optical transmitter/receiver 12b via the MDU 2 to interfere with the reference light obtained by separating the light from the light source. Then, optical line data is generated based on the acquired interference light data.
  • the optical line data is data indicating the reflection intensity of reflected light in the depth direction of the blood vessel as seen from the optical transmitter/receiver 12b.
  • the image processing device 3 constructs an OFDI image P2 (see FIGS. 4A and 4B) representing a cross-sectional layer of a blood vessel based on the generated optical line data.
  • the ultrasound line data and optical line data obtained by the ultrasound transmitting/receiving unit 12a and the optical transmitting/receiving unit 12b, and the IVUS image P1 and OFDI image P2 constructed from the ultrasound line data and optical line data will be described.
  • FIG. 3 is an explanatory diagram schematically showing a cross section of a blood vessel through which the sensor section 12 is inserted
  • FIGS. 4A and 4B are explanatory diagrams of tomographic images.
  • the sensor section 12 When capturing a tomographic image is started with the sensor section 12 and shaft 13 inserted into the blood vessel, the sensor section 12 rotates about the central axis of the shaft 13 in the direction shown by the arrow. At this time, the ultrasonic transmitter/receiver 12a transmits and receives ultrasonic waves at each rotation angle. Lines 1, 2, . . . 512 indicate the transmission and reception directions of ultrasonic waves at each rotation angle. In the first embodiment, the ultrasound transmitting/receiving unit 12a intermittently transmits and receives ultrasound 512 times while rotating 360 degrees (one rotation) within the blood vessel.
  • the ultrasonic transmitter/receiver 12a acquires one line of data in the transmitting/receiving direction by transmitting and receiving ultrasonic waves one time, so it obtains data on 512 ultrasonic lines extending radially from the rotation center during one rotation. be able to.
  • the 512 ultrasonic line data are dense near the rotation center, but become sparse as they move away from the rotation center. Therefore, the image processing device 3 can construct a two-dimensional IVUS image P1 as shown in FIG. 4A by generating pixels in the empty space of each line using well-known interpolation processing.
  • the optical transmitter/receiver 12b also transmits and receives near-infrared light (measurement light) at each rotation angle. Since the optical transmitter/receiver 12b also transmits and receives measurement light 512 times while rotating 360 degrees within the blood vessel, it is possible to obtain data on 512 optical lines extending radially from the center of rotation during one rotation. I can do it.
  • the image processing device 3 can construct the two-dimensional OFDI image P2 shown in FIG. 4A by generating pixels in the empty space of each line using well-known interpolation processing.
  • a two-dimensional tomographic image constructed from a plurality of ultrasound line data in this way is called one frame of IVUS image P1.
  • a two-dimensional tomographic image constructed from data of a plurality of optical lines is referred to as one frame OFDI image P2. Since the sensor unit 12 scans while moving within the blood vessel, one frame of IVUS image P1 or OFDI image P2 is acquired at each position rotated once within the movement range. That is, since one frame of IVUS image P1 or OFDI image P2 is acquired at each position from the distal end to the proximal end of the probe 11 within the moving range, multiple frames of IVUS images within the moving range are obtained. Image P1 or OFDI image P2 is acquired.
  • the number of times of transmission and reception of ultrasonic waves and light in one rotation is an example, and the number of times of transmission and reception is not limited to 512 times. Furthermore, the number of times ultrasonic waves are transmitted and received and the number of times that light is transmitted and received may be the same or different.
  • FIG. 5 is a block diagram showing an example of the configuration of the image processing device 3.
  • the image processing device 3 is a computer and includes a processing section 31, a storage section 32, an ultrasound line data generation section 33, an optical line data generation section 34, an input/output I/F 35, and a reading section 36.
  • the processing unit 31 includes one or more CPUs (Central Processing Unit), MPU (Micro-Processing Unit), GPU (Graphics Processing Unit), GPGPU (General-purpose computing on graphics processing units), TPU (Tensor Processing Unit), etc. It is constructed using arithmetic processing units.
  • the processing section 31 is connected to each hardware section making up the image processing device 3 via a bus.
  • the storage section 32 includes, for example, a main storage section and an auxiliary storage section.
  • the main storage unit is a temporary storage area such as SRAM (Static Random Access Memory), DRAM (Dynamic Random Access Memory), flash memory, etc., and temporarily stores data necessary for the processing unit 31 to perform arithmetic processing. do.
  • the auxiliary storage unit is a storage device such as a hard disk, an EEPROM (Electrically Erasable Programmable ROM), or a flash memory.
  • the auxiliary storage unit stores a computer program (program product) P executed by the processing unit 31, an IVUS image recognition learning model 61, an OFDI image recognition learning model 62, and various data necessary for other processing.
  • the storage unit 32 functions as a tomographic image DB 63.
  • the tomographic image DB 63 stores ID (identification information), imaging date and time, imaged blood vessel name, lesion name, and treatment-related information in association with the plurality of IVUS images P1 acquired by the image processing device 3. Further, the tomographic image DB 63 stores ID (identification information), imaging date and time, imaged blood vessel name, lesion name, and treatment-related information in association with the plurality of OFDI images P2 acquired by the image processing device 3.
  • the treatment-related information is information indicating the type of treatment for a blood vessel, such as stent placement, balloon expansion, and expansion of an placed stent, and information related to the treatment.
  • the treatment-related information is information that serves as a clue for identifying a predetermined pre-treatment tomographic image as a tomographic image related to a post-treatment tomographic image among the plurality of IVUS images P1 acquired at different timings.
  • the content of the treatment-related information is not particularly limited as long as it is possible to identify a post-treatment tomographic image that corresponds to a prescribed pre-treatment tomographic image.
  • the auxiliary storage unit may be an external storage device connected to the image processing device 3.
  • the computer program P may be written into the auxiliary storage unit at the manufacturing stage of the image processing device 3, or the image processing device 3 may obtain the program distributed by a remote server device through communication and store it in the auxiliary storage unit. It's okay.
  • the computer program P may be readably recorded on a recording medium 30 such as a magnetic disk, an optical disk, or a semiconductor memory, or the reading section 36 may read it from the recording medium 30 and store it in an auxiliary storage section. .
  • the ultrasound line data generation unit 33 acquires the reflected wave signal of the ultrasound output from the ultrasound transmitting/receiving unit 12a of the intravascular examination apparatus 101 as reflected wave data, and generates ultrasound line data based on the acquired reflected wave data. generate.
  • the optical line data generation unit 34 generates interference light data by interfering the reflected light signal output from the optical transmitting/receiving unit 12b of the intravascular inspection device 101 with the reference light obtained by separating the light from the light source. and generate optical line data based on the acquired interference light data.
  • the input/output I/F 35 is an interface to which the intravascular examination device 101, the angiography device 102, the display device 4, and the input device 5 are connected.
  • the processing unit 31 acquires an angio image from the angiography apparatus 102 via the input/output I/F 35. Furthermore, the processing unit 31 displays the medical image on the display device 4 by outputting the medical image signal of the IVUS image P1, OFDI image P2, or angio image to the display device 4 via the input/output I/F 35. . Further, the processing unit 31 receives information input to the input device 5 via the input/output I/F 35.
  • the image processing device 3 may be a multicomputer including multiple computers. Further, the image processing device 3 may be a server client system, a cloud server, or a virtual machine virtually constructed using software. In the following description, it is assumed that the image processing device 3 is one computer.
  • the processing unit 31 of the image processing device 3 generates ultrasound line data in the ultrasound line data generation unit 33 by reading and executing the computer program P stored in the storage unit 32, and generates the generated ultrasound line data. A process for constructing an IVUS image P1 is executed based on the data. Further, the processing unit 31 generates optical line data in the optical line data generation unit 34 by reading out and executing the computer program P stored in the storage unit 32, and generates an OFDI image based on the generated optical line data. Execute processing to construct P2.
  • the processing unit 31 of the image processing device 3 reads out and executes the computer program P stored in the storage unit 32, thereby processing a plurality of IVUS images acquired before a predetermined treatment such as stent placement in a blood vessel.
  • a process of aligning P1 and a plurality of IVUS images P1 acquired after the treatment in the longitudinal direction and displaying them is executed (see FIG. 10).
  • the user can confirm the condition of the blood vessel and the treatment effect by comparing the pre-treatment and post-treatment IVUS images P1 displayed with their longitudinal positions aligned.
  • the IVUS image recognition learning model 61 is a model that recognizes a predetermined object image included in the IVUS image P1.
  • the IVUS image recognition learning model 61 can classify objects into classes on a pixel basis by using image recognition technology using semantic segmentation, and can recognize various objects included in the IVUS image P1. can do.
  • the IVUS image recognition learning model 61 recognizes the lumen image in the IVUS image P1, as shown in FIGS. 6A and 6B.
  • the IVUS image recognition learning model 61 recognizes stent images, plaque images, blood vessel wall images, etc., as shown in FIG. 6A, for example.
  • the blood vessel wall is the tunica media, more specifically the external elastic membrane (EEM).
  • EEM external elastic membrane
  • the plaque image shown in FIG. 6 is an example of an object image, and the IVUS image recognition learning model 61 may be configured to recognize side branches, epicardium, veins, and the like.
  • the IVUS image recognition learning model 61 is, for example, a convolutional neural network (CNN) trained by deep learning.
  • the IVUS image recognition learning model 61 includes an input layer 61a into which the IVUS image P1 is input, an intermediate layer 61b that extracts and restores the feature amount of the IVUS image P1, and an object that indicates object images included in the IVUS image P1 in pixel units. It has an output layer 61c that outputs the extracted IVUS image P1'.
  • the IVUS image recognition learning model 61 is, for example, U-Net, FCN (Fully Convolution Network), SegNet, or the like.
  • the input layer 61a of the IVUS image recognition learning model 61 has a plurality of neurons that accept the input of the pixel value of each pixel constituting the IVUS image P1, that is, the IVUS image P1, and the input pixel value is received in the intermediate layer 61b. hand over.
  • the intermediate layer 61b includes a plurality of convolution layers (CONV layers) and a plurality of deconvolution layers (DECONV layers).
  • CONV layers convolution layers
  • DECONV layers deconvolution layers
  • the convolution layer is a layer that compresses the dimensions of the IVUS image P1.
  • the feature amount of the object image is extracted by dimensional compression.
  • the deconvolution layer performs deconvolution processing to restore the original dimensions.
  • the restoration process in the deconvolution layer generates an object extracted IVUS image P1' in which each pixel has a pixel value (class data) corresponding to the class of the object.
  • the output layer 61c has a plurality of neurons that output the object extracted IVUS image P1'.
  • the object extraction IVUS image P1' is divided into classes according to the object type, such as a lumen image, a stent image, a blood vessel wall image, a plaque image, a side branch, an epicardium, or a vein. It is a color-coded image.
  • the IVUS image recognition learning model 61 includes an IVUS image P1 obtained by the ultrasound transmitting/receiving unit 12a and an object extraction IVUS in which each pixel of the IVUS image P1 is annotated with class data according to the type of the corresponding object. It can be generated by preparing training data having the image P1' and subjecting an untrained neural network to machine learning using the training data. Specifically, the IVUS image P1 of the training data is adjusted so that the difference between the object extraction IVUS image P1' that is output when the training data IVUS image P1 is input to an untrained neural network and the image annotated as the training data is small.
  • the parameter is, for example, a weight (coupling coefficient) between nodes.
  • the method for optimizing parameters is not particularly limited, for example, the processing unit 31 optimizes various parameters using a steepest descent method or the like.
  • the IVUS image recognition learning model 61 trained in this way, by inputting the IVUS image P1 obtained by imaging a blood vessel in which a stent is placed into the IVUS image recognition learning model 61 as shown in FIG. An object extracted IVUS image P1' is obtained in which each part is classified in pixel units according to the object type, such as a blood vessel lumen image, a blood vessel wall image, a plaque image, or a stent.
  • the IVUS image P1 obtained by imaging a blood vessel without a stent to the IVUS image recognition learning model 61, images such as a blood vessel lumen image, a blood vessel wall image, a plaque image, etc.
  • An object extracted IVUS image P1' is obtained in which each part is classified in pixel units according to the type of object.
  • the OFDI image recognition learning model 62 has the same configuration as the IVUS image recognition learning model 61, and recognizes the lumen image and other object images in the OFDI image P2.
  • objects are extracted in which each part is classified into classes on a pixel basis according to the object type, such as a blood vessel lumen image, stent image, blood vessel wall image, plaque image, etc. An OFDI image is obtained.
  • ⁇ Image processing procedure> 7 and 8 are flowcharts showing the image processing procedure according to the first embodiment. Below, the image processing procedure for the IVUS image P1 will be mainly described.
  • the processing unit 31 of the image processing device 3 uses the ultrasound line data generation unit 33 to acquire a plurality of IVUS images P1 before treatment such as stent placement (step S111). Specifically, the ultrasound line data generation unit 33 of the image processing device 3 generates a plurality of ultrasound line data based on the reflected wave data output from the ultrasound transmission/reception unit 12a. Line numbers are assigned to the plurality of ultrasound line data, for example, in chronological order of observation time. Line numbers correspond to observation time points. In other words, the line number corresponds to the observation position and observation direction. The processing unit 31 constructs a plurality of frames of IVUS images P1 based on the ultrasound line data. The multiple frames of IVUS images P1 are assigned frame numbers, for example, in chronological order of observation time.
  • the frame number corresponds to the observation position.
  • the multiple frames of IVUS images P1 correspond to images obtained by observing blood vessels at multiple observation positions ranging from the distal end to the proximal end of the probe 11.
  • the processing unit 31 that executes the process in step S111 functions as an acquisition unit that acquires a plurality of IVUS images P1 of the blood vessel while moving the sensor unit 12 along the running direction of the blood vessel.
  • the processing unit 31 stores the plurality of IVUS images P1 acquired before the treatment on the blood vessel in the storage unit 32 in association with the ID, imaging date and time, blood vessel name, lesion name, and treatment-related information (step S112).
  • the processing unit 31 creates treatment plan information for stent placement and stores it in the storage unit 32 (step S113).
  • FIG. 9 is a display example of the IVUS image P1 before treatment and the treatment plan screen.
  • the processing unit 31 Based on the plurality of IVUS images P1 acquired before the treatment, the processing unit 31 generates, for example, an IVUS image P1 which is a cross-sectional image substantially perpendicular to the long axis of the blood vessel, and a longitudinal cross-sectional image substantially parallel to the central axis of the blood vessel. Display them vertically.
  • IVUS images P1 at a distal reference area (Ref.Distal), a proximal reference area (Ref.Proximal), and a minimum lumen area (MLA) are displayed.
  • the user uses the vertical cross-sectional image and the cross-sectional image (IVUS image P1) of the blood vessel displayed on the display device 4 to confirm the narrowest part, and plans the size, position, etc. of the stent to be placed.
  • the user specifies the stent placement position and size using the input device 5, and the image processing device 3 receives various specifying operations using the input device 5, and creates treatment plan information including the stent placement position and size. , is stored in the storage unit 32.
  • the white arrows shown in FIG. 9 indicate treatment plan information, and the treatment plan information includes, for example, information indicating frames of the IVUS image P1 corresponding to both ends and the center position in the longitudinal direction of the stent to be placed.
  • the processing unit 31 uses the ultrasound line data generation unit 33 to acquire a plurality of post-treatment IVUS images P1 (step S114).
  • the processing unit 31 that executes the process in step S114 functions as an acquisition unit that acquires a plurality of IVUS images P1 of the blood vessel while moving the sensor unit 12 along the running direction of the blood vessel.
  • the processing unit 31 stores the plurality of IVUS images P1 acquired after the treatment on the blood vessel in the storage unit 32 in association with the ID, imaging date and time, blood vessel name, lesion name, and treatment-related information (step S115).
  • the processing unit 31 reads out the pre-treatment IVUS image P1 to be compared (step S116).
  • the user can select the pre-treatment IVUS image P1 to be compared using the input device 5, and the processing unit 31 reads the selected IVUS image P1 as the pre-treatment IVUS image P1 to be compared.
  • the processing unit 31 reads treatment plan information associated with the pre-treatment IVUS image P1 from the storage unit 32 (step S117). Based on the read treatment plan information, the processing unit 31 identifies a representative frame image of the pre-treatment IVUS image P1 related to the stent placement position (step S118). Specifically, the processing unit 31 identifies frame images of the IVUS image P1 corresponding to the positions of both ends and the center of the stent to be placed.
  • the frame number of the IVUS image P1 of the stent end on the distal side of the blood vessel is n
  • the frame number of the IVUS image P1 of the stent end on the proximal side of the blood vessel is (n+ns)
  • the frame number of the IVUS image P1 of the stent end on the proximal side of the blood vessel is (n+ns).
  • the processing unit 31 performs image recognition processing on the IVUS image P1 by inputting the plurality of IVUS images P1 acquired after the treatment to the IVUS image recognition learning model 61, and recognizes the stent image (step S119).
  • the processing unit 31 can recognize the plurality of post-treatment IVUS images P1 by distinguishing them into frame images that include a stent image and frame images that do not include an obtained stent image.
  • the processing unit 31 identifies a representative frame of the post-treatment IVUS image P1 related to the stent placement position (step S120). Specifically, the processing unit 31 identifies frame images of the IVUS image P1 corresponding to the positions of both ends and the center of the stent to be placed. Frame images including stent images are continuous. For example, if a stent image is included in the frame images from frame number N to frame number N+Ns, the IVUS image P1 with frame number N, the IVUS image P1 with frame number (N+Ns), and the frame number (N+Ns/ 2) IVUS image P1 is identified as the representative frame image.
  • the IVUS image P1 with frame number N is a frame image of the stent end on the distal side of the blood vessel
  • the IVUS image P1 with frame number (N+Ns) is a frame image of the stent end on the proximal side of the blood vessel.
  • the processing unit 31 associates the plurality of pre-treatment and post-treatment IVUS images P1, respectively, based on the specified pre-treatment representative frame image and post-treatment representative frame image (step S121). Specifically, the IVUS image P1 of frame number n, which corresponds to one end of the stent before treatment, is associated with the IVUS image P1 of frame number N, after treatment. Similarly, the IVUS image P1 with frame number ns, which corresponds to the other end of the stent before treatment, and the IVUS image P1 after treatment with frame number Ns are associated. Furthermore, the IVUS image P1 of frame number (n+ns/2) corresponding to the central part of the stent before treatment is associated with the IVUS image P1 of frame number (N+Ns/2) after treatment.
  • the IVUS image P1 before the treatment and the longitudinal scale of the IVUS image P1 acquired after the treatment do not completely match, the IVUS image P1 before the treatment or after the treatment
  • the pre-treatment representative frame image and the post-treatment representative frame image may be associated with each other by interpolating or thinning out as necessary.
  • the processing unit 31 displays the pre-treatment and post-treatment IVUS longitudinal cross-sectional images while aligning the longitudinal positions (step S122).
  • FIG. 10 is a display example of the IVUS images P1 and the like before and after the treatment.
  • the processing unit 31 displays a vertical cross-sectional image of a blood vessel before treatment and a vertical cross-sectional image of a blood vessel after treatment vertically side by side. Specifically, the processing unit 31 generates a first vertical cross-sectional image substantially parallel to the central axis of the blood vessel, based on the plurality of IVUS images P1 acquired before the treatment, and displays it on the display device 4. Similarly, the processing unit 31 generates a second longitudinal section image substantially parallel to the central axis of the blood vessel based on the plurality of IVUS images P1 acquired after the treatment, and displays it on the display device 4. In the example shown in FIG.
  • the processing unit 31 displays the first longitudinal tomographic image before the treatment on the upper side and the second longitudinal tomographic image after the treatment on the lower side.
  • the display of the distal reference area (Ref. Distal), the proximal reference area (Ref. Proximal), and the minimum lumen area (MLA) is the same as in FIG. 9 .
  • the processing unit 31 sets and displays the section designation bar B in the distal frame where the pre-treatment and post-treatment IVUS images P1 are located (step S123).
  • the section designation bar B is a line image that specifies an arbitrary IVUS image P1 in the pre-treatment IVUS longitudinal section image and an IVUS image P1 corresponding to that one IVUS image P1 in the post-treatment IVUS longitudinal section image.
  • the section designation bar B is a thick line image that vertically passes through the IVUS longitudinal tomographic image before the treatment and the IVUS longitudinal tomographic image after the treatment.
  • the processing unit 31 displays a section designation bar B in the peripheral frame (the leftmost frame image in FIG. 10) in which both the pre-treatment and post-treatment IVUS images P1 are present.
  • step S124 determines whether a change in the position of the cross-section designation bar B has been accepted. If it is determined that the position change has been accepted (step S124: YES), the processing unit 31 changes the position of the section designation bar B and displays it (step S125).
  • step S125 When the process of step S125 is finished or when it is determined that the change of the cross-section designation bar B is not accepted (step S124: NO), the processing unit 31 performs the processing before and after the treatment corresponding to the position of the cross-section designation bar B.
  • the IVUS image P1 cross-sectional image
  • step S126 The IVUS image P1 (cross-sectional image) is displayed (step S126).
  • the pre-treatment and post-treatment IVUS images P1 displayed by the above processing correspond in longitudinal position, but do not necessarily match in circumferential direction.
  • the user can rotate the IVUS image P1 by operating the input device 5, and the processing unit 31 accepts circumferential position correction in the IVUS image P1 (step S127). Then, the processing unit 31 rotates the IVUS image P1 and corrects its circumferential orientation based on the content of the received position correction, that is, the angle information for rotating the IVUS image P1 (step S128).
  • the processing unit 31 displays an automatic playback button, and the user can operate the automatic playback button by operating the input device 5.
  • the processing unit 31 determines whether an automatic playback instruction has been received (step S129). If it is determined that the automatic playback instruction has not been received (step S129: NO), the processing unit 31 returns the process to step S123.
  • step S129 If it is determined that the automatic reproduction instruction has been received (step S129: YES), the processing unit 31 converts the distal side IVUS image P1 to the proximal side IVUS image P1 for each of the plurality of pre-treatment and post-treatment IVUS images P1.
  • An automatic playback display is executed to sequentially display up to (step S130), and the process returns to step S123.
  • the image processing device 3 and the like when displaying the IVUS images P1 of the blood vessel acquired before and after the treatment on the blood vessel, the longitudinal position of the blood vessel is aligned and the images before and after the treatment are displayed. IVUS images P1 can be displayed in association with each other.
  • the IVUS images P1 of the ends and the center of the stent to be placed and the IVUS images of the ends and the center of the stent identified by image recognition of the IVUS image P1 after the treatment.
  • the image P1 By associating the image P1 with the image P1, it is possible to associate the plurality of IVUS images P1 acquired before and after the treatment.
  • the pre-treatment and post-treatment IVUS images P1 After matching the observation positions of the pre-treatment and post-treatment IVUS images P1 in the long axis direction, it becomes possible to compare the pre-treatment and post-treatment IVUS images P1 and extract the difference. For example, when a calcified lesion is scraped off, the three-dimensional volume scraped off can be calculated using a comparison image of the IVUS image P1.
  • Embodiment 1 an example was explained in which the pre-treatment and post-treatment IVUS images P1 are displayed by aligning the longitudinal positions of the blood vessels, but the OFDI image P2 is also aligned in the longitudinal direction of the blood vessels by the same process. It may be configured to display the OFDI images P2 before and after the treatment while aligning the positions.
  • the IVUS image recognition learning model 61 that can recognize a stent image by classifying the IVUS image P1 pixel by pixel using semantic segmentation has been described.
  • the image processing device 3 may be configured using a learning model that determines the presence or absence of a stent.
  • the image diagnostic apparatus 100 according to the second embodiment is implemented in that the pre-treatment and post-treatment IVUS images P1 are displayed in association with each other by aligning the longitudinal position of the blood vessel based on the feature amount on the outside of the balloon expansion site.
  • Different from form 1 The rest of the configuration of the image processing device 3 is the same as that of the image processing device 3 according to the first embodiment, so similar parts are given the same reference numerals and detailed explanations are omitted.
  • FIG. 11 is a flowchart showing an image processing procedure according to the second embodiment
  • FIGS. 12A, 2B, and 12C are explanatory diagrams showing the image processing method according to the second embodiment.
  • the processing unit 31 of the image processing device 3 acquires a plurality of IVUS images P1 (step S211) and stores them in the storage unit 32 (step S212), as in the first embodiment.
  • the user uses the vertical cross-sectional image and the cross-sectional image (IVUS image P1) of the blood vessel displayed on the display device 4 to confirm the narrowest part and plan the position for balloon expansion.
  • the user specifies the target region for balloon expansion using the input device 5, and the image processing device 3 receives various designation operations using the input device 5, and as shown in FIG. 12A, the target region for balloon expansion (planned expansion range) ) etc. is created and stored in the storage unit 32 (step S213).
  • the processing unit 31 uses the ultrasound line data generation unit 33 to acquire a plurality of post-treatment IVUS images P1 (step S214), and stores them in the storage unit 32 (step S215).
  • the processing unit 31 reads out the pre-treatment IVUS image P1 to be compared (step S216). Furthermore, treatment plan information associated with the pre-treatment IVUS image P1 is read from the storage unit 32 (step S217).
  • the processing unit 31 performs image recognition processing on the IVUS image P1 by inputting the plurality of IVUS images P1 acquired before and after the treatment to the IVUS image recognition learning model 61, and recognizes the lumen image or the external elastic membrane. (Step S218).
  • the processing unit 31 calculates the lumen diameter or blood vessel diameter outside the target area before the treatment where balloon expansion is to be performed, based on the image recognition result in step S218 and the treatment plan information (step S219). . Specifically, as shown in FIG. 12B, the processing unit 31 can calculate the lumen diameter based on the lumen image obtained by image recognition of the IVUS image P1 outside the target region. Furthermore, the processing unit 31 can calculate the blood vessel diameter based on the external elastic membrane image obtained by image recognition of the IVUS image P1 outside the target region.
  • the processing unit 31 calculates the lumen diameter or blood vessel diameter outside the target region after the balloon expansion procedure (step S220). Since the longitudinal positions of the IVUS image P1 acquired before the treatment and the IVUS image P1 acquired after the treatment roughly correspond, the processing unit 31, as shown in FIG. 12C, based on the treatment plan information, It is possible to specify the IVUS image P1 of the outside of the target region where balloon expansion has been performed, and calculate the lumen diameter based on the lumen image obtained by image recognition of the IVUS image P1. A similar processing unit 31 can calculate the blood vessel diameter. Note that the processing unit 31 may be configured to calculate the lumen diameter or blood vessel diameter for each of the plurality of IVUS images P1 after treatment.
  • the processing unit 31 associates the pre-treatment and post-treatment IVUS images P1 so that the lumen diameter or blood vessel diameter outside the expansion range matches (step S221). Since the lumen diameter or blood vessel diameter does not necessarily match completely before and after the treatment due to the influence of pulsation, etc., the processing unit 31 adjusts the diameter so that the lumen diameter or blood vessel diameter outside the expansion range approximately matches. A process of associating the pre-treatment and post-treatment IVUS images P1 (so that they substantially match) is executed.
  • the processing unit 31 calculates the difference between the lumen diameter or vascular diameter based on the pre-treatment IVUS image P1 and the lumen diameter or vascular diameter based on the corresponding post-treatment IVUS image P1, and A correspondence relationship between the pre-treatment IVUS image P1 and the post-treatment IVUS image P1 is identified such that the sum of the differences based on the above is minimized. That is, the processing unit 31 associates both the IVUS images P1 so that the observation position of the IVUS image P1 acquired before the treatment matches the observation position of the IVUS image P1 acquired after the treatment. By associating the IVUS image P1 before the treatment with the IVUS image P1 after the treatment so that the sum of the above-mentioned differences is minimized, the lumen diameter or blood vessel diameter outside the expansion range is made to match substantially. becomes possible.
  • n is an integer of 1 or more.
  • the processing unit 31 calculates, for example, a difference in the lumen diameter or blood vessel diameter based on the object extraction IVUS image P1' of the corresponding frame number.
  • the processing unit 31 calculates the difference in lumen diameter or blood vessel diameter for a predetermined number of object extraction IVUS images P1'.
  • the processing unit 31 calculates the difference between the lumen diameter or blood vessel diameter based on the object extraction IVUS image P1' of the (n+ ⁇ )th frame and the lumen diameter or blood vessel diameter based on the object extraction IVUS image P1' of the nth frame. is calculated in the same way.
  • ⁇ is an integer and corresponds to the amount of translation of the three-dimensional lumen image.
  • the processing unit 31 increments the variable ⁇ by 1 while verifying the correspondence between the pre-treatment and post-treatment object extraction IVUS images P1'. Then, the variable ⁇ that minimizes the sum of the differences, for example, the sum of squares of the differences, is specified.
  • step S221 After completing the process of step S221, the processing unit 31 executes the same process as steps S122 to S130 of the first embodiment.
  • the lumen diameter or the blood vessel diameter outside the target area before the balloon expansion is to be performed, and the lumen diameter or the blood vessel diameter of the blood vessel after the treatment.
  • the image diagnostic apparatus 100 When stent expansion (optimization) is performed, the image diagnostic apparatus 100 according to the third embodiment aligns the longitudinal position of the blood vessel based on the stent images before and after the procedure, and performs IVUS before and after the procedure.
  • This embodiment differs from the first embodiment in that images P1 are displayed in association with each other.
  • the rest of the configuration of the image processing device 3 is the same as that of the image processing device 3 according to the first embodiment, so similar parts are given the same reference numerals and detailed explanations are omitted.
  • FIG. 13 is a flowchart showing the image processing procedure according to the third embodiment.
  • the processing unit 31 of the image processing device 3 executes the same processing as steps S111 to S117 in the first embodiment (steps S311 to S317).
  • the processing unit 31 After completing the process in step S317, the processing unit 31 inputs the plurality of IVUS images P1 acquired before and after the treatment into the IVUS image recognition learning model 61, thereby performing image recognition processing on the IVUS image P1. and recognizes the stent image (step S318).
  • the processing unit 31 identifies a representative frame related to the stent placement position before expansion (step S319). Specifically, the processing unit 31 identifies frame images of the IVUS image P1 corresponding to the positions of both ends and the center of the stent from the plurality of IVUS images P1 acquired before the treatment.
  • the processing unit 31 identifies a representative frame related to the stent placement position after expansion (step S320). Specifically, the processing unit 31 identifies frame images of the IVUS image P1 corresponding to the positions of both ends and the center of the stent from the plurality of IVUS images P1 acquired after the treatment.
  • the processing unit 31 calculates pre-treatment (before optimizing the stent placement state) and post-treatment (stent placement state) based on the specified pre-treatment representative frame image and post-treatment representative frame image.
  • the plurality of IVUS images P1 (after optimization) are associated with each other (step S321). Specifically, the IVUS image P1 corresponding to one end of the stent before treatment is associated with the IVUS image P1. Similarly, the IVUS image P1 corresponding to the other end of the stent before treatment is associated with the IVUS image P1 after treatment. Furthermore, the IVUS image P1 corresponding to the central portion of the stent before treatment is associated with the IVUS image P1 after treatment.
  • step S321 After completing the process of step S321, the processing unit 31 executes the same process as steps S122 to S130 of the first embodiment.
  • the plurality of IVUS images P1 obtained before and after the treatment are matched by aligning the positions of the stent images.
  • the IVUS images P1 before and after the treatment can be displayed in association with each other by aligning the longitudinal positions of the blood vessels.
  • the image diagnostic apparatus 100 according to the fourth embodiment has the advantage that the pre-treatment and post-treatment IVUS images P1 are displayed in association with each other based on the image of the calcified site in the IVUS image P1, with the blood vessels aligned in the longitudinal direction. This is different from the first embodiment.
  • the rest of the configuration of the image processing device 3 is the same as that of the image processing device 3 according to the first embodiment, so similar parts are given the same reference numerals and detailed explanations are omitted.
  • FIG. 14 is a flowchart showing the image processing procedure according to the fourth embodiment.
  • the processing unit 31 of the image processing device 3 executes the same processing as steps S111 to S121 of the first embodiment. By the processing in step S121, the positions of the pre-treatment IVUS image P1 and the post-treatment IVUS image P1 in the longitudinal direction of the blood vessel can be approximately matched.
  • the processing unit 31 according to the fourth embodiment finely corrects the correspondence between the pre-treatment and post-treatment IVUS images P1 by executing the following process.
  • the processing unit 31 executes image recognition processing of the IVUS image P1 by inputting the plurality of IVUS images P1 acquired before and after the treatment to the IVUS image recognition learning model 61, and detects calcification.
  • the part image is recognized (step S421).
  • the processing unit 31 identifies a frame image of the IVUS image P1 including the calcified site image. Furthermore, the processing unit 31 identifies the central angle of the calcified site image in the IVUS image P1.
  • FIG. 15 is an explanatory diagram showing a plurality of pre-treatment and post-treatment IVUS images P1 arranged in the longitudinal direction of the blood vessel.
  • the above figure shows multiple pre-treatment IVUS images P1 lined up in the longitudinal direction of the blood vessel, the white part shows the IVUS image P1 that does not include calcified site images, and the hatched part shows the calcified site.
  • An IVUS image P1 including images is shown. "20°”, “70°”, “120°”, etc. indicate the central angle of the calcified site in the IVUS image P1 including the calcified site image, that is, the circumferential position of the calcified site in the IVUS image P1. ing.
  • the figure below shows a plurality of post-treatment IVUS images P1 arranged in the longitudinal direction of the blood vessel.
  • the processing unit 31 associates the pre-treatment and post-treatment IVUS images P1 so that the longitudinal positions of the plurality of calcified sites match (step S422). For example, when a calcified site image is included in a plurality of consecutive IVUS images P1, the processing unit 31 recognizes the plurality of IVUS images P1 as one group, and selects an IVUS image P1 at the center in the longitudinal direction of the blood vessel. Specify as a representative frame image. The processing unit 31 treats as a pair the representative frame image before treatment and the representative frame image after treatment whose distance is closest in the longitudinal direction, and calculates the distance between the representative frame images.
  • the processing unit 31 calculates the distance between each representative frame image, finely adjusts the longitudinal position of the IVUS image P1 so that the sum of the distances is the minimum, and takes appropriate action.
  • the previous IVUS image P1 and the post-treatment IVUS image P1 are associated.
  • the processing unit 31 corrects the rotation angle of the IVUS image P1 so that the center angles of the calcified site images in the IVUS image P1 match (step S423). For example, the processing unit 31 calculates the difference between the central angle of the calcified site image included in the representative frame image before processing and the central angle of the calcified site image included in the corresponding representative frame image after treatment. Then, the processing unit 31 similarly calculates the difference in center angle for the other representative frame images, calculates the correction amount of the rotation angle so that the sum of the differences is the minimum, and processes the IVUS image by the calculated correction amount. By rotating P1, the orientation of the IVUS image P1 is corrected. In the example shown in FIG.
  • the processing unit 31 displays the pre-treatment representative frame image whose longitudinal position has been finely adjusted and the center angle corrected in the processes of steps S422 and S423, and the post-treatment representative frame image in association with each other. (Step S424).
  • the longitudinal position of the blood vessel can be aligned more accurately and the IVUS images P1 before and after the treatment can be displayed in association with each other. I can do it. Furthermore, the rotation angles, that is, the orientations, of the IVUS images P1 before and after the treatment can be displayed together.
  • the pre-treatment and post-treatment IVUS images P1 are correlated based on the stent placement position, and the correspondence relationship and the IVUS image P1 are determined based on the position and center angle of the calcified site image.
  • the IVUS images P1 before and after the treatment may be displayed in association with each other using only the method of the fourth embodiment.
  • the image diagnostic apparatus 100 according to the fifth embodiment differs from the first embodiment in that the process of step S116 according to the first embodiment is automatically executed.
  • the rest of the configuration of the image processing device 3 is the same as that of the image processing device 3 according to the first embodiment, so similar parts are given the same reference numerals and detailed explanations are omitted.
  • FIG. 16 is a flowchart showing the image processing procedure according to the fifth embodiment.
  • the processing unit 31 executes the following process in step S116.
  • the processing unit 31 reads the imaging date and time, blood vessel name, lesion name, treatment-related information, etc. regarding the IVUS image P1 after the treatment from the storage unit 32 (step S551).
  • the processing unit 31 searches for an IVUS image P1 whose imaging date and time is earlier, whose blood vessel name, lesion name, etc.
  • step S552 match, and whose treatment details match (step S552), and searches for an IVUS image P1 that has an earlier imaging date and time, matches the blood vessel name, lesion name, etc., and matches the treatment details (step S552), and The IVUS image P1 is read out from the tomographic image DB 63 (step S553).
  • the image processing device 3 and the like according to the fifth embodiment when the IVUS image P1 is acquired after the treatment, the data of the IVUS image P1 before the treatment, which is the comparison target, is automatically read out from the storage unit 32. , the longitudinal tomographic image P1 of the blood vessel and the IVUS image P1 based on each IVUS image P1 can be displayed side by side on the display device 4.
  • Diagnostic imaging catheter 2 MDU 3 Image processing device 4
  • Display device 5 Input device 11
  • Probe 12 Sensor section 12a
  • Ultrasonic transceiver section 12b Optical transceiver section 12c Housing 13
  • Shaft 14 Guide wire insertion section 15
  • Connector section 30 Recording medium 31
  • Processing section 32 Storage section 33
  • Ultrasonic line data Generation unit 34 Optical line data generation unit 61
  • IVUS image recognition learning model 62 OFDI image recognition learning model P1 IVUS image P1' Object extraction IVUS image P2 OFDI image 100
  • Intravascular examination device 102 Angiography device

Landscapes

  • Health & Medical Sciences (AREA)
  • Life Sciences & Earth Sciences (AREA)
  • Surgery (AREA)
  • Nuclear Medicine, Radiotherapy & Molecular Imaging (AREA)
  • Biomedical Technology (AREA)
  • Veterinary Medicine (AREA)
  • Pathology (AREA)
  • Radiology & Medical Imaging (AREA)
  • Biophysics (AREA)
  • Engineering & Computer Science (AREA)
  • Physics & Mathematics (AREA)
  • Heart & Thoracic Surgery (AREA)
  • Medical Informatics (AREA)
  • Molecular Biology (AREA)
  • Animal Behavior & Ethology (AREA)
  • General Health & Medical Sciences (AREA)
  • Public Health (AREA)
  • Optics & Photonics (AREA)
  • Ultra Sonic Daignosis Equipment (AREA)
PCT/JP2023/008701 2022-03-28 2023-03-08 コンピュータプログラム、画像処理方法及び画像処理装置 Ceased WO2023189308A1 (ja)

Priority Applications (1)

Application Number Priority Date Filing Date Title
JP2024511607A JPWO2023189308A1 (https=) 2022-03-28 2023-03-08

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
JP2022-052274 2022-03-28
JP2022052274 2022-03-28

Publications (1)

Publication Number Publication Date
WO2023189308A1 true WO2023189308A1 (ja) 2023-10-05

Family

ID=88201447

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/JP2023/008701 Ceased WO2023189308A1 (ja) 2022-03-28 2023-03-08 コンピュータプログラム、画像処理方法及び画像処理装置

Country Status (2)

Country Link
JP (1) JPWO2023189308A1 (https=)
WO (1) WO2023189308A1 (https=)

Cited By (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20240382180A1 (en) * 2023-05-17 2024-11-21 Boston Scientific Scimed Inc. Alignment for multiple series of intravascular images
WO2025237821A1 (en) * 2024-05-15 2025-11-20 Koninklijke Philips N.V. Alignment of multiple longitudinal views of blood vessel from intravascular imaging and associated systems, devices, and methods

Citations (7)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JP2009240359A (ja) * 2008-03-28 2009-10-22 Kazunori Urasawa 画像解析装置及び画像解析方法
JP2012061086A (ja) * 2010-09-15 2012-03-29 Toshiba Corp 医用画像表示システム
JP2012200532A (ja) * 2011-03-28 2012-10-22 Terumo Corp 画像診断装置及び表示方法
JP2013056113A (ja) * 2011-09-09 2013-03-28 Toshiba Corp 画像表示装置
JP2016064073A (ja) * 2014-09-25 2016-04-28 テルモ株式会社 画像処理装置、画像処理装置の制御方法、プログラムおよび記憶媒体
JP2017527418A (ja) * 2014-07-24 2017-09-21 ライトラボ・イメージング・インコーポレーテッド ステント及び血管の可視化及び診断のシステム、装置、及び方法
JP2021517034A (ja) * 2018-03-15 2021-07-15 コーニンクレッカ フィリップス エヌ ヴェＫｏｎｉｎｋｌｉｊｋｅ Ｐｈｉｌｉｐｓ Ｎ．Ｖ． 管腔内病巣評価及び処置計画のための解剖学的標識の決定及び可視化

Patent Citations (7)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JP2009240359A (ja) * 2008-03-28 2009-10-22 Kazunori Urasawa 画像解析装置及び画像解析方法
JP2012061086A (ja) * 2010-09-15 2012-03-29 Toshiba Corp 医用画像表示システム
JP2012200532A (ja) * 2011-03-28 2012-10-22 Terumo Corp 画像診断装置及び表示方法
JP2013056113A (ja) * 2011-09-09 2013-03-28 Toshiba Corp 画像表示装置
JP2017527418A (ja) * 2014-07-24 2017-09-21 ライトラボ・イメージング・インコーポレーテッド ステント及び血管の可視化及び診断のシステム、装置、及び方法
JP2016064073A (ja) * 2014-09-25 2016-04-28 テルモ株式会社 画像処理装置、画像処理装置の制御方法、プログラムおよび記憶媒体
JP2021517034A (ja) * 2018-03-15 2021-07-15 コーニンクレッカ フィリップス エヌ ヴェＫｏｎｉｎｋｌｉｊｋｅ Ｐｈｉｌｉｐｓ Ｎ．Ｖ． 管腔内病巣評価及び処置計画のための解剖学的標識の決定及び可視化

Cited By (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20240382180A1 (en) * 2023-05-17 2024-11-21 Boston Scientific Scimed Inc. Alignment for multiple series of intravascular images
US20240382181A1 (en) * 2023-05-17 2024-11-21 Boston Scientific Scimed Inc. Dual view for multiple series of ivus images
WO2025237821A1 (en) * 2024-05-15 2025-11-20 Koninklijke Philips N.V. Alignment of multiple longitudinal views of blood vessel from intravascular imaging and associated systems, devices, and methods

Also Published As

Publication number Publication date
JPWO2023189308A1 (https=) 2023-10-05

Similar Documents

Publication Publication Date Title
US20250114150A1 (en) Artificial intelligence coregistration and marker detection, including machine learning and using results thereof
US20240013385A1 (en) Medical system, method for processing medical image, and medical image processing apparatus
WO2023189308A1 (ja) コンピュータプログラム、画像処理方法及び画像処理装置
US12518382B2 (en) Medical system, method for processing medical image, and medical image processing apparatus
WO2015044984A1 (ja) 画像診断装置及びその制御方法
JP2022055170A (ja) コンピュータプログラム、画像処理方法及び画像処理装置
JP6794226B2 (ja) 画像診断装置、画像診断装置の作動方法およびプログラム
JP7843748B2 (ja) プログラム、情報処理方法及び情報処理装置
WO2023054467A1 (ja) モデル生成方法、学習モデル、コンピュータプログラム、情報処理方法及び情報処理装置
US20250134387A1 (en) Tissue characterization in one or more images, such as in intravascular images, using artificial intelligence
JP7607482B2 (ja) コンピュータプログラム、画質改善学習モデル、学習モデル生成方法、画像処理方法及び画像処理装置
WO2023132332A1 (ja) コンピュータプログラム、画像処理方法及び画像処理装置
US20240016460A1 (en) Program, image processing method, and image processing device
US20240008849A1 (en) Medical system, method for processing medical image, and medical image processing apparatus
JP7623175B2 (ja) コンピュータプログラム、画像処理装置の作動方法及び画像処理装置
JP7548852B2 (ja) コンピュータプログラム、画像処理方法及び画像処理装置
JP7813276B2 (ja) コンピュータプログラム、情報処理方法及び情報処理装置
US20260029380A1 (en) Imaging system, method, and image processing apparatus
JP2024139509A (ja) プログラム、画像処理方法、画像処理装置及びモデル生成方法
US20250248664A1 (en) Image diagnostic system and method
US20260011131A1 (en) Automatic calibration of intravascular imaging catheter
JP2024142137A (ja) プログラム、画像処理方法及び画像処理装置
US20250221624A1 (en) Image diagnostic system, image diagnostic method, and storage medium
WO2023100838A1 (ja) コンピュータプログラム、情報処理装置、情報処理方法及び学習モデル生成方法
JP2024140944A (ja) プログラム、画像処理方法、画像処理装置及びモデル生成方法

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 23779335

Country of ref document: EP

Kind code of ref document: A1

ENP Entry into the national phase

Ref document number: 2024511607

Country of ref document: JP

Kind code of ref document: A

NENP Non-entry into the national phase

Ref country code: DE

122 Ep: pct application non-entry in european phase

Ref document number: 23779335

Country of ref document: EP

Kind code of ref document: A1