JP7297507B2 - Image processing device, X-ray diagnostic device and program - Google Patents

Description

TECHNICAL FIELD Embodiments of the present invention relate to an image processing apparatus, an X-ray diagnostic apparatus, and a program.

2. Description of the Related Art In recent years, various treatment methods have been known in which a device is inserted into a subject, manipulated, and placed in the subject. For example, in stent graft insertion, an operator such as a doctor prepares a stent graft according to the shape of the subject's aorta, and plans the placement position of the stent graft. The operator then inserts the stent graft into the subject and deploys the stent graft according to the preoperative plan.

Japanese translation of PCT publication No. 2014-514082

The problem to be solved by the present invention is to facilitate grasping of the position of the device in the subject.

A medical image processing apparatus according to an embodiment includes an identifying unit and a superimposed image generating unit. The specifying unit specifies the positions of the feature points of the device in the first X-ray image. A superimposed image generator generates a superimposed image in which a 3D model representing the device is superimposed on the first X-ray image or a second X-ray image acquired after the first X-ray image. The superimposed image generator superimposes the 3D model on the first X-ray image or the second X-ray image at a position based on the position of the identified feature point.

FIG. 1 is a block diagram showing an example of the configuration of a medical image processing system according to the first embodiment. FIG. 2 is a block diagram showing an example configuration of the X-ray diagnostic apparatus according to the first embodiment. FIG. 3 is a diagram showing an example of a stent graft according to the first embodiment; FIG. FIG. 4A is a diagram showing an example of a stent graft marker according to the first embodiment. FIG. 4B is a diagram showing an example of markers of the stent graft according to the first embodiment; FIG. 5 is a diagram for explaining generation of a superimposed image according to the first embodiment. FIG. 6 is a diagram for explaining generation of a superimposed image according to the first embodiment. FIG. 7 is a flowchart for explaining a series of processing flows of the image processing apparatus according to the first embodiment. FIG. 8 is a diagram for explaining generation of a superimposed image according to the second embodiment. FIG. 9 is a diagram for explaining generation of a superimposed image according to the second embodiment. FIG. 10 is a diagram for explaining generation of a superimposed image according to the third embodiment. FIG. 11 is a diagram showing an example of a blood vessel region image according to the fourth embodiment. FIG. 12 is a block diagram showing an example configuration of an X-ray diagnostic apparatus according to the fourth embodiment.

Hereinafter, embodiments of an image processing apparatus, an X-ray diagnostic apparatus, and a program will be described in detail with reference to the drawings.

(First embodiment)
First, the first embodiment will be described. In the first embodiment, a medical image processing system including an image processing device will be described as an example. Also, in the first embodiment, a stent graft will be described as an example of the device.

As shown in FIG. 1, the medical image processing system 1 according to the first embodiment includes an X-ray diagnostic apparatus 10, an image storage apparatus 20, and an image processing apparatus 30. FIG. Note that FIG. 1 is a block diagram showing an example of the configuration of the medical image processing system 1 according to the first embodiment. As shown in FIG. 1, the X-ray diagnostic apparatus 10, the image storage apparatus 20 and the image processing apparatus 30 are interconnected via a network.

The X-ray diagnostic apparatus 10 is an apparatus that acquires an X-ray image from a subject P. As shown in FIG. An X-ray image processed as data is also referred to as X-ray image data. For example, the X-ray diagnostic apparatus 10 acquires X-ray image data from a subject P into which a stent graft has been inserted, and transmits the acquired X-ray image data to the image storage device 20 and the image processing device 30 . The configuration of the X-ray diagnostic apparatus 10 will be described later.

The image storage device 20 is a device that stores X-ray image data acquired by the X-ray diagnostic apparatus 10 . For example, the image storage device 20 acquires X-ray image data from the X-ray diagnostic device 10 via a network, and stores the acquired X-ray image data in a memory provided inside or outside the device. For example, the image storage device 20 is realized by computer equipment such as a server device.

The image processing apparatus 30 acquires X-ray image data via a network, and uses the acquired X-ray image data to perform various processes. For example, the image processing device 30 acquires X-ray image data from the image archiving device 20 . Alternatively, the image processing device 30 acquires X-ray image data from the X-ray diagnostic device 10 without going through the image storage device 20 . In addition, the image processing device 30 identifies the positions of characteristic points of the stent graft in the acquired X-ray image data. The image processing device 30 also generates a superimposed image in which a 3D model representing the stent graft is superimposed on the X-ray image data based on the positions of the identified feature points. The processing performed by the image processing device 30 will be described in detail later. For example, the image processing device 30 is realized by computer equipment such as a workstation.

Note that the location where the X-ray diagnosis apparatus 10, the image storage apparatus 20, and the image processing apparatus 30 are installed is arbitrary as long as they can be connected via a network. For example, the image processing device 30 may be installed in a hospital different from the X-ray diagnostic device 10 . Although one X-ray diagnostic apparatus 10 is shown in FIG. 1 , the medical image processing system 1 may include multiple X-ray diagnostic apparatuses 10 .

As shown in FIG. 1, the image processing device 30 has an input interface 31, a display 32, a memory 33, and a processing circuit .

The input interface 31 receives various input operations from the operator, converts the received input operations into electrical signals, and outputs the electrical signals to the processing circuit 34 . For example, the input interface 31 includes a mouse, a keyboard, a trackball, a switch, a button, a joystick, a touch pad that performs input operations by touching an operation surface, a touch screen that integrates a display screen and a touch pad, and an optical sensor. It is realized by the used non-contact input circuit, voice input circuit, or the like. Note that the input interface 31 may be composed of a tablet terminal or the like capable of wireless communication with the main body of the image processing apparatus 30 . Also, the input interface 31 is not limited to those having physical operation parts such as a mouse and a keyboard. For example, an electrical signal processing circuit that receives an electrical signal corresponding to an input operation from an external input device provided separately from the image processing apparatus 30 and outputs the electrical signal to the processing circuit 34 is an example of the input interface 31. include.

The display 32 displays various information. For example, the display 32 displays X-ray images acquired by the X-ray diagnostic apparatus 10 and superimposed images generated by the processing circuit 34 under the control of the processing circuit 34 . Also, the display 32 displays a GUI (Graphical User Interface) for receiving various instructions and various settings from the operator via the input interface 31 . For example, the display 32 is a liquid crystal display or a CRT (Cathode Ray Tube) display. The display 32 may be of a desktop type, or may be configured by a tablet terminal or the like capable of wireless communication with the main body of the image processing apparatus 30 .

The memory 33 is implemented by, for example, a RAM (Random Access Memory), a semiconductor memory device such as a flash memory, a hard disk, an optical disk, or the like. For example, the memory 33 stores X-ray image data acquired from the X-ray diagnostic apparatus 10 or the image storage apparatus 20 . Also, for example, the memory 33 stores the superimposed image generated by the processing circuit 34 . Further, for example, the memory 33 stores a program for the circuit included in the image processing device 30 to implement its function. Note that the memory 33 may be realized by a server group (cloud) connected to the image processing device 30 via a network.

The processing circuit 34 controls the overall operation of the image processing device 30 by executing a specific function 34a, a superimposed image generation function 34b, a transformation processing function 34c, and an output function 34d. Here, the specific function 34a is an example of a specific part. Also, the superimposed image generation function 34b is an example of a superimposed image generation unit. Also, the transformation processing function 34c is an example of a transformation processing unit.

For example, the processing circuit 34 reads out from the memory 33 and executes a program corresponding to the specifying function 34a to specify the positions of characteristic points of the stent graft in the X-ray image data. Further, for example, the processing circuit 34 reads a program corresponding to the superimposed image generation function 34b from the memory 33 and executes it, thereby creating a 3D model showing the stent graft on the X-ray image data at a position based on the position of the feature point. Generate a superimposed superimposed image. Further, for example, the processing circuit 34 reads a program corresponding to the deformation processing function 34c from the memory 33 and executes it, thereby deforming the 3D model showing the stent graft based on the X-ray image data. Further, for example, the processing circuit 34 outputs a superimposed image by reading out a program corresponding to the output function 34d from the memory 33 and executing the program.

In the image processing apparatus 30 shown in FIG. 1, each processing function is stored in the memory 33 in the form of a computer-executable program. The processing circuit 34 is a processor that implements a function corresponding to each program by reading the program from the memory 33 and executing the program. In other words, the processing circuit 34 in a state where each program is read has a function corresponding to the read program. In FIG. 1, it is assumed that the single processing circuit 34 realizes the specific function 34a, the superimposed image generating function 34b, the transformation processing function 34c, and the output function 34d. The functions may be implemented by configuring the processing circuit 34 and having each processor execute a program. Further, each processing function of the processing circuit 34 may be appropriately distributed or integrated in a single or a plurality of processing circuits and implemented.

Next, the X-ray diagnostic apparatus 10 that acquires X-ray image data will be described with reference to FIG. FIG. 2 is a block diagram showing an example configuration of the X-ray diagnostic apparatus 10 according to the first embodiment. As shown in FIG. 2, the X-ray diagnostic apparatus 10 includes an X-ray high voltage device 101, an X-ray tube 102, an X-ray restrictor 103, a top plate 104, a C-arm 105, and an X-ray detector 106. , a memory 107 , a display 108 , an input interface 109 and a processing circuit 110 .

X-ray high voltage device 101 supplies high voltage to X-ray tube 102 under the control of processing circuitry 110 . For example, the X-ray high-voltage device 101 has electric circuits such as a transformer and a rectifier, and includes a high-voltage generator that generates a high voltage to be applied to the X-ray tube 102 and a and an X-ray control device for controlling an output voltage according to X-rays. The high voltage generator may be of a transformer type or an inverter type.

The X-ray tube 102 is a vacuum tube having a cathode (filament) that generates thermoelectrons and an anode (target) that generates X-rays upon collision with thermoelectrons. The X-ray tube 102 uses a high voltage supplied from the X-ray high-voltage device 101 to irradiate thermoelectrons from the cathode to the anode, thereby generating X-rays.

The X-ray restrictor 103 has a collimator that narrows down the irradiation range of X-rays generated by the X-ray tube 102 and a filter that adjusts the X-rays emitted from the X-ray tube 102 .

A collimator in the X-ray diaphragm 103 has, for example, four slidable diaphragm blades. The collimator narrows down the X-rays generated by the X-ray tube 102 and irradiates the subject P with the X-rays generated by the X-ray tube 102 by sliding the diaphragm blades. Here, the diaphragm blade is a plate-like member made of lead or the like, and is provided near the X-ray irradiation port of the X-ray tube 102 in order to adjust the X-ray irradiation range.

The filter in the X-ray restrictor 103 changes the quality of the transmitted X-rays depending on its material and thickness for the purpose of reducing the exposure dose to the subject P and improving the image quality of the X-ray image data. It reduces soft ray components that are likely to be depleted, and reduces high-energy components that cause deterioration in the contrast of X-ray image data. In addition, the filter changes the X-ray dose and irradiation range depending on its material, thickness, position, etc., and X-rays are arranged so that the X-rays irradiated from the X-ray tube 102 to the subject P have a predetermined distribution. attenuate

For example, the X-ray diaphragm 103 has a drive mechanism such as a motor and an actuator, and controls X-ray irradiation by operating the drive mechanism under the control of a processing circuit 110 described later. For example, the X-ray diaphragm 103 applies a driving voltage to the driving mechanism according to the control signal received from the processing circuit 110, thereby adjusting the opening of the diaphragm blades of the collimator and irradiating the subject P with the control the irradiation range of X-rays. Further, for example, the X-ray diaphragm 103 adjusts the position of the filter by applying a driving voltage to the driving mechanism according to the control signal received from the processing circuit 110, thereby irradiating the subject P. control the distribution of the X-ray dose.

The top board 104 is a bed on which the subject P is placed, and is arranged on a bed (not shown). Note that the subject P is not included in the X-ray diagnostic apparatus 10 . For example, the bed has a driving mechanism such as a motor and an actuator, and controls the movement and inclination of the tabletop 104 by operating the driving mechanism under the control of the processing circuit 110, which will be described later. For example, the bed moves or tilts the tabletop 104 by applying a drive voltage to the drive mechanism in accordance with the control signal received from the processing circuit 110 .

The C-arm 105 holds the X-ray tube 102, the X-ray restrictor 103, and the X-ray detector 106 so as to face each other with the subject P interposed therebetween. For example, the C-arm 105 has a driving mechanism such as a motor and an actuator, and rotates or moves by operating the driving mechanism under the control of the processing circuit 110 described later. For example, the C-arm 105 applies a drive voltage to the drive mechanism according to a control signal received from the processing circuit 110, thereby moving the X-ray tube 102, the X-ray diaphragm 103, and the X-ray detector 106 to the subject. It is rotated and moved with respect to P to control the X-ray irradiation position and irradiation angle. In FIG. 2, the case where the X-ray diagnostic apparatus 10 is a single plane is described as an example, but the embodiment is not limited to this, and may be a biplane.

The X-ray detector 106 is, for example, an X-ray flat panel detector (FPD) having detection elements arranged in a matrix. The X-ray detector 106 detects X-rays emitted from the X-ray tube 102 and transmitted through the subject P, and outputs a detection signal corresponding to the detected X-ray dose to the processing circuit 110 . The X-ray detector 106 may be an indirect conversion type detector having a grid, a scintillator array, and a photosensor array, or a direct conversion type detector having a semiconductor element that converts incident X-rays into electrical signals. It may be a detector.

The memory 107 is implemented by, for example, a RAM, a semiconductor memory device such as a flash memory, a hard disk, an optical disk, or the like. For example, memory 107 receives and stores x-ray image data acquired by processing circuitry 110 . The memory 107 also stores programs corresponding to various functions read and executed by the processing circuit 110 . Note that the memory 107 may be realized by a server group (cloud) connected to the X-ray diagnostic apparatus 10 via a network.

The display 108 displays various information. For example, the display 108 displays a GUI for accepting operator's instructions and various X-ray images under the control of the processing circuit 110 . For example, display 108 is a liquid crystal display or a CRT display. Note that the display 108 may be of a desktop type, or may be configured by a tablet terminal or the like capable of wireless communication with the processing circuit 110 .

The input interface 109 receives various input operations from the operator, converts the received input operations into electrical signals, and outputs the electrical signals to the processing circuit 110 . For example, the input interface 109 includes a mouse, a keyboard, a trackball, a switch, a button, a joystick, a touch pad that performs input operations by touching an operation surface, a touch screen that integrates a display screen and a touch pad, and an optical sensor. It is realized by the used non-contact input circuit, voice input circuit, or the like. Note that the input interface 109 may be configured by a tablet terminal or the like capable of wireless communication with the processing circuit 110 . Also, the input interface 109 is not limited to having physical operation parts such as a mouse and a keyboard. For example, an electrical signal processing circuit that receives an electrical signal corresponding to an input operation from an external input device provided separately from the X-ray diagnostic apparatus 10 and outputs the electrical signal to the processing circuit 110 is also included in the input interface 109. included in the example.

The processing circuitry 110 controls the overall operation of the X-ray diagnostic apparatus 10 by executing an acquisition function 110a, an output function 110b, and a control function 110c.

For example, the processing circuit 110 acquires X-ray image data by reading out a program corresponding to the acquisition function 110a from the memory 107 and executing the program. For example, the acquisition function 110a controls the X-ray high-voltage device 101 and adjusts the voltage supplied to the X-ray tube 102, thereby controlling the X-ray dose irradiated to the subject P and ON/OFF.

Further, for example, the acquisition function 110a controls the operation of the X-ray diaphragm 103 and adjusts the opening of the diaphragm blades of the collimator, thereby controlling the irradiation range of the X-rays with which the subject P is irradiated. do. The acquisition function 110a also controls the X-ray dose distribution by controlling the operation of the X-ray restrictor 103 and adjusting the position of the filter. Also, the collection function 110a controls the operation of the C-arm 105 to rotate or move the C-arm 105 . Also, for example, the collecting function 110a moves or tilts the tabletop 104 by controlling the motion of the bed.

The acquisition function 110 a also generates X-ray image data based on the detection signal received from the X-ray detector 106 and stores the generated X-ray image data in the memory 107 . The acquisition function 110a may also perform various image processing on the X-ray image data stored in the memory 107. FIG. For example, the acquisition function 110a performs noise reduction processing using an image processing filter and scattered radiation correction on X-ray image data.

In addition, the processing circuit 110 reads a program corresponding to the output function 110b from the memory 107 and executes it, thereby causing the display 108 to display a GUI and an X-ray image. The output function 110 b also outputs the X-ray image data to the image storage device 20 and the image processing device 30 . Further, the processing circuit 110 reads out a program corresponding to the control function 110c from the memory 107 and executes it, thereby controlling various functions of the processing circuit 110 based on an input operation received from the operator via the input interface 109. do.

In the X-ray diagnostic apparatus 10 shown in FIG. 2, each processing function is stored in the memory 107 in the form of a computer-executable program. The processing circuit 110 is a processor that implements a function corresponding to each program by reading the program from the memory 107 and executing the program. In other words, the processing circuit 110 having read each program has a function corresponding to the read program. Although FIG. 2 shows a case where the processing functions of the collection function 110a, the output function 110b, and the control function 110c are implemented by a single processing circuit 110, the embodiment is not limited to this. For example, the processing circuit 110 may be configured by combining a plurality of independent processors, and each processor may implement each processing function by executing each program. Moreover, each processing function of the processing circuit 110 may be appropriately distributed or integrated in a single or a plurality of processing circuits and implemented.

The term "processor" used in the above description is, for example, a CPU (Central Processing Unit), a GPU (Graphics Processing Unit), or an application specific integrated circuit (ASIC), a programmable logic device (e.g., Circuits such as Simple Programmable Logic Device (SPLD), Complex Programmable Logic Device (CPLD), and Field Programmable Gate Array (FPGA)). The processor implements its functions by reading and executing programs stored in the memory 33 or memory 107 .

1 and 2, it is assumed that the single memory 33 or memory 107 stores programs corresponding to each processing function. However, a configuration may be adopted in which a plurality of memories 33 are distributed and the processing circuit 34 reads corresponding programs from the individual memories 33 . Similarly, a plurality of memories 107 may be arranged in a distributed manner, and the processing circuit 110 may be configured to read corresponding programs from individual memories 107 . Also, instead of storing the program in the memory 33 and the memory 107, the program may be configured to be directly embedded in the circuit of the processor. In this case, the processor realizes its function by reading and executing the program embedded in the circuit.

Also, the processing circuit 34 and the processing circuit 110 may implement their functions using a processor of an external device connected via a network. For example, the processing circuit 34 reads and executes a program corresponding to each function from the memory 33, and uses a server group (cloud) connected to the image processing apparatus 30 via a network as computational resources. Each function shown in 1 is realized.

The medical image processing system 1 including the image processing apparatus 30 has been described above. With such a configuration, the image processing apparatus 30 in the medical image processing system 1 facilitates understanding of the position of the device inside the subject through processing by the processing circuit 34 .

A stent graft will now be described as an example of a device. For example, a stent graft is manufactured according to the shape of the blood vessel of the subject P to be treated. For example, in a stent graft insertion operation for an aortic aneurysm, three-dimensional image data representing the blood vessel shape of a subject P is acquired in a preoperative treatment plan, and a stent graft is fabricated based on the three-dimensional image data.

For example, the acquisition function 110a in the X-ray diagnostic apparatus 10 performs rotational imaging to acquire projection data at a predetermined frame rate while rotating the C-arm 105, and shows the blood vessel shape of the subject P from the acquired projection data. 3D image data is reconstructed. For example, the acquisition function 110a executes rotational imaging of a subject P whose blood vessel is not injected with a contrast medium, and acquires mask images at a predetermined frame rate. In addition, the acquisition function 110a performs rotational imaging of the subject P in which a contrast agent is injected into the blood vessel, and acquires contrast images at a predetermined frame rate. Next, the acquisition function 110a reconstructs three-dimensional image data representing the blood vessel shape of the subject P using difference image data obtained by subtracting the mask image and the contrast image as projection data. To give another example, the acquisition function 110a reconstructs three-dimensional image data using the mask image as projection data. The acquisition function 110a also reconstructs three-dimensional image data using the contrast image as projection data. Then, the acquisition function 110a generates three-dimensional image data representing the blood vessel shape of the subject P by subtracting the two reconstructed three-dimensional image data.

Note that the three-dimensional image data representing the blood vessel shape of the subject P may be collected by the X-ray diagnostic apparatus 10, by an X-ray diagnostic apparatus other than the X-ray diagnostic apparatus 10, or by an X-ray diagnostic apparatus other than the X-ray diagnostic apparatus 10. A medical image diagnostic apparatus (for example, X-ray CT (Computed Tomography), etc.) other than the radiological diagnostic apparatus may collect.

For example, a stent graft has a main body and multiple branches, as shown in FIG. The trunk shown in FIG. 3 is produced based on the shape of the aorta of the subject P, for example. Also, for example, the plurality of branch portions shown in FIG. 3 are produced based on the shapes of various arteries (branch blood vessels) branching from the aorta. FIG. 3 is a diagram showing an example of the stent graft according to the first embodiment.

For example, a stent graft is configured so that it can be divided into a trunk and a branch, and the trunk is provided with a plurality of holes for fitting the branch. In this case, in stent graft insertion, after the trunk of the stent graft is left in the aorta, the branch portion of the stent graft is left so as to fit into the hole in the trunk. As a result, blood flow to various arteries (branch vessels) branching from the aorta is maintained while reducing blood flow to the aortic aneurysm and preventing expansion of the aortic aneurysm.

After the stent-graft is fabricated, the operator performs stent-graft insertion according to the preoperative plan. For example, the operator first inserts the stent graft into the blood vessel of the subject P with the trunk of the stent graft housed in the catheter, and advances it to the position of the treatment target site (aortic aneurysm). At this time, the stem of the stent graft is accommodated in the catheter while being radially compressed.

Here, the acquisition function 110a acquires the X-ray image data of the treatment target site, and the output function 110b outputs the X-ray image data. For example, the acquisition function 110a acquires X-ray image data of the treatment target site in time series. The output function 110b also sequentially generates X-ray images for display based on the X-ray image data acquired in time series, and causes the display 108 to sequentially display the generated X-ray images. Alternatively, the output function 110 b sequentially outputs the X-ray image data acquired in time series to the image processing device 30 . In this case, the output function 34d in the image processing apparatus 30 sequentially generates X-ray images for display based on the X-ray image data acquired from the X-ray diagnostic apparatus 10, and sequentially displays the generated X-ray images on the display 32. Let In the following description, an X-ray image that is sequentially displayed in parallel with acquisition of X-ray image data or sequentially displayed in parallel with acquisition of X-ray image data from another device is also referred to as a fluoroscopic image. do.

The operator places the trunk of the stent graft at the preoperatively planned position while referring to the fluoroscopic image. Specifically, the operator pushes the trunk of the stent graft out of the catheter while the tip of the catheter is positioned at the treatment target site. Here, as shown in FIG. 3, the stent graft has a spring-like wire frame which is retracted and accommodated within the catheter. Therefore, the trunk of the stent graft pushed out from the catheter is deployed radially by the elastic force of the wire frame and left in the aorta in contact with the inner wall of the blood vessel.

Here, as described above, the trunk of the stent graft is provided with a plurality of holes for fitting the branch portions. If the position of the hole in the stem of the stent graft and the position of the inlet of the corresponding branch vessel are deviated from each other, the branch of the stent graft cannot be placed later, and the blood flow to the branch vessel is hindered. There is also a risk of hindering Accordingly, the operator places the stent-graft trunk such that the position of each of the plurality of holes in the stent-graft trunk coincides with the position of the entrance of the corresponding branch vessel. However, the stent graft is usually not clearly visualized in the fluoroscopic image, and it is not easy to grasp the position of the stent graft within the subject.

Therefore, the image processing apparatus 30 according to the first embodiment generates a superimposed image in which a 3D model representing the stent graft is superimposed on the X-ray image data, thereby facilitating grasping of the position of the stent graft within the subject. . Processing performed by the image processing apparatus 30 according to the first embodiment will be described in detail below.

The specifying function 34a acquires the X-ray image data collected for the subject P into which the stent graft has been inserted, and specifies the positions of the characteristic points of the stent graft in the acquired X-ray image data. Note that, hereinafter, the X-ray image data for specifying the position of the feature point by the specifying function 34a is referred to as X-ray image data I11. X-ray image data I11 is an example of a first X-ray image.

For example, the specifying function 34a sequentially acquires the X-ray image data I11 acquired in time series by the X-ray diagnostic apparatus 10, and sequentially specifies the positions of characteristic points of the stent graft in the acquired X-ray image data I11. . Note that the X-ray image data I11 may be X-ray image data collected from the subject P in which a contrast medium is injected and the vascular region of the subject P is contrasted, or may be X-ray image data in which the contrast medium is injected. It may be X-ray image data acquired from the subject P in a state in which it is not exposed.

Further, the X-ray image data I11 may be differential image data between the X-ray image data collected for the subject P into which the stent graft was inserted and the X-ray image data collected before the stent graft was inserted. good. In this case, in the X-ray image data I11, the background components corresponding to the bones, soft tissues, etc. of the subject P are removed, and only the stent graft is depicted. Further, when the X-ray image data I11 is differential image data based on the X-ray image data acquired from the subject P in which the contrast medium is injected, the X-ray image data I11 includes the stent graft and the contrast medium. A blood vessel region of the subject P imaged by the agent is depicted.

For example, the specifying function 34a specifies the position of the feature point by detecting the feature point of the stent graft in the X-ray image data I11. For example, the specifying function 34a detects markers attached to the stent graft in the X-ray image data I11 as feature points. Here, the marker is, for example, radiopaque metal attached to the stent graft.

For example, a stent graft has three markers (marker L11, marker L12 and marker L13) at the tip as shown in FIG. 4A. Also, for example, the stent graft has three markers (marker L14, marker L15 and marker L16) at the distal end and two markers (marker L21 and marker L22) at the midsection as shown in FIG. 4B. 4A and 4B are diagrams showing examples of markers of the stent graft according to the first embodiment. For example, the specifying function 34a performs pattern matching using the shape of the marker attached to the stent graft to the X-ray image data I11. Thereby, the specifying function 34a detects a plurality of markers in the X-ray image data I11.

For example, the operator pushes the tip portion of the stent graft out of the catheter while the tip of the catheter is positioned at the site to be treated. As a result, the stent graft is partially deployed as indicated by the dashed circular region in the X-ray image data I11 of FIG. Then, the specifying function 34a detects the markers attached to the tip of the deployed stent graft in the X-ray image data I11, and specifies the positions of the markers L11, L12, and L13. Note that FIG. 5 is a diagram for explaining generation of a superimposed image according to the first embodiment.

Also, the superimposed image generation function 34b acquires a 3D model M representing the stent graft. For example, the superimposed image generation function 34b acquires, as a 3D model M, 3D data (CAD (computer-aided design) data, etc.) used in fabricating the stent graft. Also, for example, the superimposed image generation function 34b acquires the 3D model M by imaging the manufactured stent graft from multiple directions using an optical camera, an X-ray CT apparatus, the X-ray diagnosis apparatus 10, or the like. In addition, below, the 3D model M acquired by the superimposed image generation function 34b is also described as a 3D model M1.

As shown in FIG. 5, the 3D model M1 includes a marker L11', a marker L12' and a marker L13'. For example, marker L11', marker L12', and marker L13' are information indicating the positions of three markers (marker L11, marker L12, and marker L13) attached to the stent graft. Furthermore, the marker L11', the marker L12', and the marker L13' may be information indicating the shape, size, etc. of each of the three markers (marker L11, marker L12, and marker L13) attached to the stent graft.

For example, the superimposed image generation function 34b acquires the 3D model M1 described above and stores it in the memory 33 before the stent graft insertion is started. Then, after stent graft interpolation is started, the superimposed image generation function 34b reads out the 3D model M1 from the memory 33 and superimposes it on the X-ray image data I11 to generate a superimposed image.

As shown in FIG. 5, the superimposed image generating function 34b superimposes the 3D model M1 on the positions based on the positions of the markers specified in the X-ray image data I11 by the specifying function 34a. Specifically, the superimposed image generation function 34b matches the position of the marker L11 specified in the X-ray image data I11 with the marker L11′ in the 3D model M1, and the position of the marker L12 specified in the X-ray image data I11. , and the marker L13' in the 3D model M1 matches the position of the marker L13 specified in the X-ray image data I11. is superimposed.

More specifically, the superimposed image generation function 34b first identifies the correspondence relationship between each marker identified in the X-ray image data I11 and each marker in the 3D model M1. Here, markers L11, L12 and L13 are placed on the stent graft at irregular intervals as shown in FIG. 4A. Therefore, the superimposed image generation function 34b can uniquely identify the correspondence relationship between the marker L11, the marker L12, and the marker L13, and the marker L11', the marker L12', and the marker L13'.

That is, when the 3D model M1 is superimposed on the X-ray image data I11 so that the marker L11' matches the position of the marker L12, the distance from the marker L12 to the marker L11 and the distance from the marker L11' to the marker L13' Since the interval is different, the marker L13' does not match the position of the marker L11 (similarly, the marker L13 and the marker L12' do not match). Similarly, when the 3D model M1 is superimposed on the X-ray image data I11 so that the marker L11' matches the position of the marker L13, the marker L12' does not match the position of the marker L11, and the marker L12 does not match the position of the marker L12. does not match the marker L13'.

On the other hand, when the 3D model M1 is superimposed on the X-ray image data I11 so that the marker L11' matches the position of the marker L11, the marker L12' matches the position of the marker L12 and the position of the marker L13. , the marker L13' matches. Therefore, the superimposed image generation function 34b can specify that the marker L11' corresponds to the marker L11, the marker L12' corresponds to the marker L12, and the marker L13' corresponds to the marker L13.

The method for uniquely identifying the correspondence between each marker identified in the X-ray image data I11 and each marker in the 3D model M1 is not limited to the example described above. For example, when the shapes and sizes of the markers attached to the stent graft are different, the superimposed image generation function 34b compares the shape and size of each marker detected in the X-ray image data I11 with the shape and size of each marker in the 3D model M1. By comparing , the correspondence can be uniquely identified.

Then, the superimposed image generation function 34b arranges the X-ray image so that the marker L11' matches the position of the marker L11, the marker L12' matches the position of the marker L12, and the marker L13' matches the position of the marker L13. The 3D model M1 is translated, enlarged or reduced, or rotated with respect to the data I11. As a result, the superimposed image generating function 34b superimposes the 3D model M1 on the X-ray image data I11 at the position and orientation based on the position of the marker specified by the specifying function 34a, and generates a superimposed image I21 shown in FIG. do.

Note that FIG. 5 describes a case where the specifying function 34a specifies the positions of three feature points in the X-ray image data I11. However, the number of feature points specified in the X-ray image data I11 by the specifying function 34a is not limited to three.

For example, as shown in FIG. 4B, three markers (marker L14, marker L15 and marker L16) are attached to the distal end of the stent graft and two markers (marker L21 and marker L22) are attached to the midsection. , the specifying function 34a specifies the positions of three or five feature points in the X-ray image data I11.

That is, when only the tip portion of the stent graft with markers L14, L15, and L16 is deployed, the specific function 34a uses the three markers attached to the tip of the stent graft in the X-ray image data I11. is detected and located. Further, in the case where the mid-abdominal portion to which the marker L21 and the marker L22 are attached in addition to the distal end portion of the stent graft is deployed, the specific function 34a determines the distal end and mid-abdominal portion of the deployed stent graft in the X-ray image data I11. The five markers attached to are detected and their positions are specified. Furthermore, if a portion other than the distal end or middle portion of the stent graft (for example, a branch portion) is marked, and the portion marked with such a marker is deployed, the specific function 34a is an X-ray image. In the data I11, a marker attached to the expanded portion is detected as a feature point, and its position is specified. As the identifying function 34a identifies the positions of more feature points, the superimposed image generating function 34b superimposes the 3D model M1 on the X-ray image data I11 at a position and orientation based on the positions of the feature points. It is possible to improve the accuracy when

Also, the case where the specifying function 34a specifies the positions of three or more feature points has been described so far. However, the number of feature point positions specified in the X-ray image data I11 by the specifying function 34a may be less than three. For example, when two markers are attached to the stent graft, the specifying function 34a specifies the positions of the two markers in the X-ray image data I11. Here, when the marker attached to the stent graft indicates a different shape depending on its orientation (for example, when the marker is plate-shaped or rod-shaped), the superimposed image generation function 34b generates two images in the X-ray image data I11. The 3D model M1 is superimposed on the X-ray image data I11 so that each marker and each of the two markers in the 3D model M1 match in position, shape, size, and the like. That is, the superimposed image generation function 34b superimposes the 3D model M1 on the X-ray image data I11 at the position and orientation based on the positions of the two identified feature points.

The output function 34d outputs the superimposed image I21 generated by the superimposed image generating function 34b. For example, the output function 34d causes the display 32 to display the generated superimposed image I21. Also, for example, the output function 34 d outputs the superimposed image I<b>21 to the X-ray diagnostic apparatus 10 . In this case, the output function 110b in the X-ray diagnostic apparatus 10 causes the display 108 to display the superimposed image I21 acquired from the image processing apparatus 30. FIG.

Note that the superimposed image I21 may be displayed instead of the X-ray image for display based on the X-ray image data I11, or may be displayed together with the X-ray image for display based on the X-ray image data I11. good. For example, the output function 110b may display the superimposed image I21 and the X-ray image for display based on the X-ray image data I11 side by side on one display, or may display them on separate displays.

When the X-ray image data I11 is acquired in time series, the superimposed image generation function 34b sequentially generates the superimposed image I21 in parallel with the acquisition of the X-ray image data I11, and the output function 34d outputs the generated superimposed The image I21 is sequentially output. Here, when the stent graft moves within the blood vessel of the subject P due to an operator's operation or the like, the 3D model M1 also moves or rotates in the superimposed image I21 so as to follow the movement of the marker. Therefore, the operator can easily grasp the position and orientation (orientation) of the stent graft to be operated in the blood vessel of the subject P.

Note that the superimposed image generation function 34b may translucent the 3D model M1 and superimpose it on the X-ray image data I11 to generate a superimposed image I21. For example, in FIG. 5, part of the catheter depicted in the X-ray image data I11 is hidden by the 3D model M1. Here, the superimposed image generation function 34b makes the 3D model M1 translucent, thereby facilitating the understanding of the position and orientation of the stent graft in the subject, and the area where the 3D model M overlaps in the X-ray image data I11. It is possible to visually recognize about

In FIG. 5, the case where the 3D model M1 is superimposed on the X-ray image data I11 to generate the superimposed image I21 has been described. However, embodiments are not so limited. For example, the deformation processing function 34c deforms the 3D model M such as the 3D model M1 based on the X-ray image data I11, and the superimposed image generation function 34b transforms the 3D model M after deformation to the X-ray image data I11. may be superimposed on each other.

For example, the deformation processing function 34c obtains the blood vessel shape of the subject P with the stent graft inserted based on the X-ray image data I11, and deforms the 3D model M according to the obtained blood vessel shape. In addition, below, the 3D model M deform|transformed according to the blood-vessel shape is described as 3D model M2.

For example, the deformation processing function 34c performs X-rays collected in a state in which a contrast medium is injected into the blood vessel of the subject P including the treatment target site after the stent graft is inserted to the treatment target site (aortic aneurysm, etc.). A blood vessel shape is obtained from the image data I11. For example, the deformation processing function 34c selects an area having a pixel value larger than a threshold from the X-ray image data I11 acquired with a positive contrast agent (such as an iodine-based contrast agent) injected as a blood vessel area. The outline of the extracted blood vessel region is obtained as a blood vessel shape.

Then, the deformation processing function 34c generates a 3D model M2 by deforming the 3D model M according to the acquired blood vessel shape. For example, the deformation processing function 34c generates the 3D model M2 by radially compressing or expanding the 3D model M1 or curving it along the core line according to the acquired blood vessel shape. In this case, the 3D model M2 approximates the shape of the stent graft after placement.

That is, since the stent graft is placed in contact with the inner wall of the blood vessel, the shape of the stent graft follows the shape of the blood vessel of the subject P after placement. Here, even if the stent graft is fabricated based on three-dimensional image data or the like indicating the blood vessel shape of the subject P, the time when the three-dimensional image data is collected (at the time of preoperative planning) and the inside of the stent graft The blood vessel shape of the subject P is not always the same as when the insertion is performed. Therefore, the 3D model M1 representing the shape of the stent graft at the time of preoperative planning does not necessarily match the shape of the stent graft after placement. On the other hand, the deformation processing function 34c obtains the blood vessel shape of the subject P into which the stent graft is inserted based on the X-ray image data I11, and deforms the 3D model M1 based on the obtained blood vessel shape. The 3D model M1 can be approximated to the shape of the stent graft after placement.

Further, the superimposed image generation function 34b superimposes the deformed 3D model M2 on the X-ray image data I11 to generate a superimposed image I22. Also, the output function 34d outputs the superimposed image I22 generated by the superimposed image generating function 34b. For example, when the X-ray image data I11 is acquired in time series, the superimposed image generation function 34b sequentially generates the superimposed image I22 in parallel with the acquisition of the X-ray image data I11, and the output function 34d The superimposed image I22 is sequentially output. This allows the operator to easily grasp the position and orientation of the stent graft to be operated in the subject P's blood vessel. Furthermore, the operator can refer to the 3D model M2, which is close to the shape of the stent-graft after placement, to determine whether or not to place the stent-graft at the current position.

To give another example, the deformation processing function 34c acquires the deployment status of the stent graft outside the catheter based on the X-ray image data I11, and generates a 3D model M1, a 3D model M2, or the like according to the acquired deployment status. Deform the 3D model M. In the following description, the 3D model M deformed according to the state of deployment of the stent graft is referred to as a 3D model M3.

For example, the identifying function 34a identifies the positions of three markers (marker L11, marker L12, and marker L13) attached to the tip of the stent graft in the X-ray image data I11, as shown in FIG. The specifying function 34a also specifies the tip position of the catheter in the X-ray image data I11, as shown in FIG. Note that FIG. 6 is a diagram for explaining generation of a superimposed image according to the first embodiment.

Next, the deformation processing function 34c transforms the X-ray image so that the marker L11' matches the position of the marker L11, the marker L12' matches the position of the marker L12, and the marker L13' matches the position of the marker L13. The 3D model M1 is translated, enlarged or reduced, or rotated with respect to the data I11. Thereby, the deformation processing function 34c identifies a position corresponding to the tip position of the catheter in the 3D model M1.

Here, in the 3D model M1, the portion from the marker L11′, marker L12′, and marker L13′ to the tip position of the catheter (hereinafter referred to as the first portion) corresponds to the portion of the stent graft that is deployed outside the catheter. . On the other hand, in the 3D model M1, the portion farther from the tip position of the catheter (hereinafter referred to as the second portion) from the marker L11', the marker L12', and the marker L13' is the portion of the stent graft housed inside the catheter. corresponds to

Therefore, the transformation processing function 34c generates a 3D model M31 by transforming the 3D model M1 so as to omit the second part. As a result, the 3D model M31 approximates the shape of the deployed portion of the stent graft. In addition, as shown in FIG. 6, the deformation processing function 34c deforms the 3D model M1 so as to omit the second portion, and removes one end of the first portion, which is close to the tip position of the catheter, from the catheter. A 3D model M31 may be generated by deforming so as to taper toward the tip position. Note that the 3D model M31 is an example of the 3D model M3.

Furthermore, as shown in FIG. 6, the superimposed image generation function 34b superimposes the deformed 3D model M31 on the X-ray image data I11 to generate a superimposed image I23. Also, the output function 34d outputs the superimposed image I23 generated by the superimposed image generating function 34b. For example, when the X-ray image data I11 is collected in time series, the superimposed image generation function 34b sequentially generates the superimposed image I23 in parallel with the collection of the X-ray image data I11, and the output function 34d The superimposed image I23 is sequentially output. This allows the operator to easily grasp the position and orientation of the stent graft to be operated in the subject P's blood vessel. Furthermore, the operator can refer to the 3D model M31, which is close to the shape of the deployed portion of the stent graft, to grasp the current status of deployment of the stent graft.

Another example will be described for the 3D model M3. For example, the deformation processing function 34c acquires the deployment status of the stent graft outside the catheter based on the intervals between the positions of the plurality of feature points specified in the X-ray image data I11, and a 3D model is obtained according to the acquired deployment status. A 3D model M3 may be generated by deforming M1.

For example, when five markers (marker L14, marker L15, marker L16, marker L21 and marker L22) are attached to the stent graft as shown in FIG. It is detected in data I11 and its position is specified.

Next, the transformation processing function 34c transforms the X-ray image data I11 so that three of the five marker positions specified in the X-ray image data I11 correspond to the three corresponding markers in the 3D model M1. , the 3D model M1 is translated, enlarged or reduced, or rotated. Further, the deformation processing function 34c partially deforms the 3D model M1 so that the two corresponding markers in the 3D model M1 match the positions of the other two markers specified in the X-ray image data I11. generates a 3D model M32. Note that the 3D model M32 is an example of the 3D model M3.

That is, when the stent graft is expanded in the blood vessel of the subject P, it may not be expanded as planned before surgery. For example, when the vascular shape of the subject P is not the same at the time of preoperative planning and at the time of stent graft insertion, when the subject P's blood vessel is harder than expected, or when the blood vessel is soft may expand more than assumed at the time of preoperative planning, or may not expand as much as expected. Therefore, the 3D model M1 representing the shape of the stent graft at the time of preoperative planning does not necessarily match the shape of the stent graft deployed in the blood vessel of the subject P. On the other hand, the deformation processing function 34c acquires the deployment status of the stent graft outside the catheter based on the X-ray image data I11, and deforms the 3D model M1 according to the acquired deployment status, thereby transforming the 3D model M1 into can approximate the shape of the stent graft deployed in the blood vessel of the subject P.

Furthermore, the superimposed image generation function 34b superimposes the deformed 3D model M32 on the X-ray image data I11 to generate a superimposed image I24. The output function 34d also outputs the superimposed image I24 generated by the superimposed image generating function 34b. For example, when the X-ray image data I11 is acquired in time series, the superimposed image generation function 34b sequentially generates the superimposed image I24 in parallel with the acquisition of the X-ray image data I11, and the output function 34d The superimposed image I24 is sequentially output. This allows the operator to easily grasp the position and orientation of the stent graft to be operated in the subject P's blood vessel. Furthermore, the operator can refer to the 3D model M32, which is similar to the shape of the stent graft deployed in the blood vessel of the subject P, to grasp the current deployment status of the stent graft.

Next, an example of the procedure of processing by the image processing device 30 will be described with reference to FIG. FIG. 7 is a flowchart for explaining a series of processing flows of the image processing apparatus 30 according to the first embodiment. Steps S101, S102 and S111 are steps corresponding to the specific function 34a. Step S109 is a step corresponding to the superimposed image generation function 34b. Steps S103, S104, S105, S106, S107 and S108 are steps corresponding to the transformation processing function 34c. Step S110 is a step corresponding to the output function 34d.

First, the processing circuit 34 acquires X-ray image data I11 (step S101), and identifies the positions of characteristic points of the stent graft in the acquired X-ray image data I11 (step S102).

Next, the processing circuit 34 acquires the blood vessel shape of the subject P into which the stent graft is inserted based on the X-ray image data I11, and the acquired blood vessel shape is preoperative (for example, a three-dimensional shape for fabricating a stent graft). It is determined whether or not the blood vessel shape matches the blood vessel shape at the time when the image data was collected (step S103). Here, if the blood vessel shapes do not match (No at step S103), the processing circuit 34 deforms the 3D model M according to the blood vessel shape acquired based on the X-ray image data I11 (step S104).

If the blood vessel shape acquired based on the X-ray image data I11 matches the preoperative blood vessel shape (Yes at step S103), or after step S104, the processing circuit 34 determines whether the deployment of the stent graft has been completed. is determined (step S105). If the deployment of the stent graft has not been completed (No at step S105), the processing circuit 34 deforms the 3D model M according to the deployment situation (step S106). For example, the processing circuitry 34 identifies a position in the 3D model M corresponding to the tip position of the catheter, and transforms the 3D model M so as to omit the portion corresponding to the portion housed in the catheter.

Next, the processing circuit 34 determines whether or not the plurality of feature points in the 3D model M match the positions of the plurality of feature points specified in the X-ray image data I11 (step S107). Here, if the feature points do not match (No at step S107), the processing circuit 34 deforms the 3D model M according to the deployment situation (step S108). For example, the processing circuit 34 may apply to the X-ray image data I11 such that the corresponding feature points in the 3D model M match some of the positions of the plurality of feature points specified in the X-ray image data I11. , the 3D model M is translated, enlarged or reduced, or rotated. After that, the processing circuit 34 partially deforms the 3D model M so that the corresponding feature points in the 3D model M match the positions of the identified other feature points.

If the deployment of the stent graft is completed (Yes at step S105), if the marker and the feature point match (Yes at step S107), or after step S108, the processing circuit 34 performs the following on the X-ray image data I11: A superimposed image is generated by superimposing the 3D model M on the positions based on the positions of the identified feature points (step S109), and the generated superimposed image is output (step S110). Here, the processing circuit 34 further determines whether or not the X-ray image data I11 has been acquired (step S111). If the X-ray image data I11 is further acquired (Yes at step S111), the processing circuit 34 proceeds to step S102 again. On the other hand, if the X-ray image data I11 is not acquired (No at step S111), the processing circuit 34 terminates the process.

As described above, according to the first embodiment, the identifying function 34a identifies the positions of characteristic points of the stent graft in the X-ray image data I11. Also, the superimposed image generation function 34b generates a superimposed image in which the 3D model M representing the stent graft is superimposed on the X-ray image data I11. Here, the superimposed image generation function 34b superimposes the 3D model M on the X-ray image data I11 at positions based on the positions of the identified feature points. Therefore, the image processing apparatus 30 according to the first embodiment can easily grasp the position of the stent graft within the subject P. FIG.

Further, according to the first embodiment, the identifying function 34a identifies positions of a plurality of feature points in the X-ray image data I11. Also, the superimposed image generation function 34b superimposes the 3D model M at the position and orientation based on the position of the identified feature point. Therefore, the image processing apparatus 30 according to the first embodiment can easily grasp the position and orientation of the stent graft in the subject P. FIG.

(Second embodiment)
In the first embodiment described above, the 3D model M1, the 3D model M2, and the 3D model M3 representing the shape of the stent graft have been described as examples of the 3D model M representing the stent graft. On the other hand, in the second embodiment, as an example of the 3D model M representing the stent graft, a 3D model M4 representing the position and orientation of the branching portion of the stent graft will be described.

The image processing apparatus 30 according to the second embodiment has the same configuration as the image processing apparatus 30 shown in FIG. 1, and part of the processing by the superimposed image generation function 34b is different. Therefore, the same reference numerals as in FIG. 1 are assigned to the same configurations as those described in the first embodiment, and the description thereof is omitted.

For example, the superimposed image generation function 34b first acquires a 3D model M1 representing the shape of the stent graft. Next, the superimposed image generation function 34b identifies the positions of the branch portions of the stent graft in the 3D model M1. For example, the superimposed image generating function 34b identifies, as the location of the branch, the location of a hole provided in the stem of the stent graft for fitting the branch. Next, the superimposed image generation function 34b identifies the orientation of the branched portion of the stent graft in the 3D model M1. For example, the superimposed image generation function 34b identifies the direction perpendicular to the plane surrounded by the outline (edge) of the trunk hole as the orientation of the branch.

Then, the superimposed image generation function 34b generates a 3D model M4 based on the specified position and direction of the branched portion. For example, the superimposed image generation function 34b generates a 3D model M41 showing the positions and orientations of the branches of the stent graft, as shown in FIG. Note that FIG. 8 is a diagram for explaining generation of a superimposed image according to the second embodiment.

The 3D model M41 shown in FIG. 8 includes an ellipse that indicates the outline of the trunk hole (position of the branch). The 3D model M41 also includes a line segment passing through the center of the plane surrounded by the outline of the trunk hole and indicating a direction perpendicular to this plane (orientation of the branch portion). For example, as shown in FIG. 8, the superimposed image generation function 34b superimposes a 3D model M41 on the X-ray image data I11 at positions based on the positions of the feature points specified by the specifying function 34a, thereby generating a superimposed image I25. to generate Also, the output function 34d outputs the superimposed image I25 generated by the superimposed image generating function 34b.

To give another example, the superimposed image generation function 34b generates a 3D model M42 showing the position and orientation of the branch of the stent graft, as shown in FIG. Note that FIG. 9 is a diagram for explaining generation of a superimposed image according to the second embodiment.

The 3D model M42 shown in FIG. 9 includes an arrow passing through the center of the plane surrounded by the outline of the hole in the trunk and pointing in a direction perpendicular to this plane as a figure indicating the position and direction of the branch. For example, as shown in FIG. 9, the superimposed image generation function 34b superimposes a 3D model M42 on the X-ray image data I11 at positions based on the positions of the feature points specified by the specifying function 34a, thereby generating a superimposed image I26. to generate Also, the output function 34d outputs the superimposed image I26 generated by the superimposed image generating function 34b.

Although two examples of the 3D model M41 and the 3D model M42 are shown in FIGS. 8 and 9 as examples of the 3D model M4, the embodiment is not limited to this. In other words, the superimposed image generation function 34b can generate the 3D model M4 using arbitrary graphics, images, characters, etc. that can indicate the position and orientation of the branched portion of the stent graft. The superimposed image generation function 34b may also generate the 3D model M4 by indicating the orientation of the branched portion of the stent graft using colors and shading.

As described above, the superimposed image generation function 34b according to the second embodiment generates a superimposed image by superimposing the 3D model M4 indicating the position and orientation of the branch portion of the stent graft on the X-ray image data I11. Therefore, the image processing apparatus 30 according to the second embodiment can more easily grasp the position and orientation of the stent graft within the subject.

The superimposed image generation function 34b superimposes a 3D model M (such as a 3D model M1) representing the shape of the stent graft and a 3D model M4 representing the position and direction of the branched portion of the stent graft onto the X-ray image data I11. may be used to generate a superimposed image. In this case, the image processing apparatus 30 presents different types of 3D models M to the operator, thereby making it easier to grasp the position and orientation of the stent graft within the subject.

On the other hand, when only the 3D model M4 is superimposed on the X-ray image data I11, the image processing device 30 can reduce the area where the 3D model M overlaps in the X-ray image data I11. For example, in the superimposed image I21 shown in FIG. 5, the 3D model M1 overlaps a part of the catheter depicted in the X-ray image data I11. On the other hand, in the superimposed image I25 of FIG. 8 and the superimposed image I26 of FIG. 9, the entire catheter depicted in the X-ray image data I11 is shown. The image processing device 30 may display a plurality of the various superimposed images described above on one or a plurality of displays, or may switch and display the images according to an input operation or the like from the operator.

(Third embodiment)
In the first and second embodiments described above, the case where the 3D model M is superimposed on the X-ray image data I11 (first X-ray image) has been described. In contrast, in the third embodiment, a case will be described in which the 3D model M is superimposed on X-ray image data (second X-ray image) different from the X-ray image data I11.

The image processing apparatus 30 according to the third embodiment has the same configuration as the image processing apparatus 30 shown in FIG. 1, and a part of the processing by the superimposed image generation function 34b is different. Therefore, the same reference numerals as in FIG. 1 are assigned to the same configurations as those described in the first embodiment, and the description thereof is omitted.

The superimposed image generation function 34b acquires X-ray image data different from the X-ray image data I11. Note that the superimposed image generation function 34b may generate X-ray image data different from the X-ray image data I11 based on the X-ray image data I11, or generate X-ray image data different from the X-ray image data I11. You may acquire from other apparatuses (X-ray diagnostic apparatus 10 etc.).

For example, the acquisition function 110a acquires multiple pieces of X-ray image data in time series. In addition, the output function 110 b sequentially outputs the X-ray image data collected in time series to the image processing device 30 . The identifying function 34a identifies the positions of characteristic points of the stent graft in the X-ray image data I11 collected by the collecting function 110a and output from the output function 110b. For example, the specifying function 34a specifies the positions of the marker L11, the marker L12, and the marker L13 in the X-ray image data I11.

In addition, the superimposed image generation function 34b acquires the position of the feature point specified in the X-ray image data I11 in the X-ray image data I12 acquired by the acquisition function 110a after the X-ray image data I11 and output from the output function 110b. Identify the position corresponding to . Here, the X-ray image data I12 is an example of a second X-ray image.

For example, since the X-ray image data I11 and the X-ray image data I12 indicate substantially the same region in the subject P, the superimposed image generation function 34b uses markers L11, L12, and The position of the X-ray image data I12 corresponding to the position of the marker L13 is specified as the positions of the marker L11, the marker L12 and the marker L13 in the X-ray image data I12. Alternatively, the superimposed image generation function 34b aligns the X-ray image data I11 and the X-ray image data I12 so that the X-ray image data I12 corresponds to the position of the feature point specified in the X-ray image data I11. to identify the location.

Then, the superimposed image generating function 34b superimposes the 3D model M on the X-ray image data I12 at positions based on the positions of the feature points specified by the specifying function 34a to generate a superimposed image I27. Also, the output function 34d outputs the superimposed image I27 generated by the superimposed image generating function 34b. That is, when the X-ray image data is collected in time series, the superimposed image generating function 34b generates the X-ray image data I12 of a different frame from the X-ray image data I11 for which the positions of the feature points are specified by the specifying function 34a. , the 3D model M is superimposed.

As a result, the image processing apparatus 30 can shorten the time from acquisition of X-ray image data to output. For example, when superimposing the 3D model M on the X-ray image data I11, the image processing device 30 acquires the X-ray image data I11, performs processing for specifying the positions of feature points in the X-ray image data I11, and then A superimposed image of the X-ray image data I11 and the 3D model M is output. On the other hand, when superimposing the 3D model M on the X-ray image data I12, the image processing device 30 acquires feature points in the previously acquired X-ray image data I11 before acquiring the X-ray image data I12. All or part of the process of determining the position can be performed. After acquiring the X-ray image data I12, the image processing device 30 can output an image superimposed on the X-ray image data I12 and the 3D model M in a short time.

To give another example, when the X-ray image data I11 is the X-ray image data acquired for the subject P with the stent graft inserted, the superimposed image generation function 34b acquires the difference image data I13. That is, the superimposed image generation function 34b acquires the differential image data I13 between the X-ray image data I11 acquired for the subject P into which the stent graft was inserted and the X-ray image data acquired before the stent graft was inserted. do. In this case, in the differential image data I13, the background components corresponding to the bones, soft tissues, etc. of the subject P are removed, and only the stent graft is rendered. Further, when the X-ray image data I11 is the X-ray image data acquired from the subject P in which the contrast agent is injected, the difference image data I13 includes the stent graft and the object imaged with the contrast agent. A blood vessel region of the specimen P is rendered. Note that the differential image data I13 is an example of a second X-ray image. Also, since the differential image data I13 is generated based on the X-ray image data I11, it is acquired after the X-ray image data I11.

The specifying function 34a specifies the positions of characteristic points of the stent graft in the X-ray image data I11. For example, the specifying function 34a specifies the positions of the marker L11, the marker L12, and the marker L13 in the X-ray image data I11. Next, the superimposed image generation function 34b identifies positions in the difference image data I13 corresponding to the positions of the feature points identified in the X-ray image data I11. For example, since the X-ray image data I11 and the difference image data I13 indicate the same region in the subject P, the superimposed image generation function 34b uses the marker L11, the marker L12, and the marker L13 in the X-ray image data I11. The position of the difference image data I13 corresponding to the position is specified as the positions of the marker L11, the marker L12 and the marker L13 in the difference image data I13.

Then, the superimposed image generating function 34b superimposes the 3D model M on the differential image data I13 at positions based on the positions of the feature points specified by the specifying function 34a. For example, the superimposed image generation function 34b performs the difference so that the marker L11', the marker L12' and the marker L13' of the 3D model M1 match the positions of the marker L11, the marker L12 and the marker L13 specified in the difference image data I13. The 3D model M1 is superimposed on the image data I13. Thereby, the superimposed image generation function 34b generates a superimposed image I28. The output function 34d also outputs the superimposed image I28 generated by the superimposed image generating function 34b.

To give another example, the superimposed image generation function 34b acquires image data I14 showing blood vessels of the subject P. FIG. For example, as shown in FIG. 10, the superimposed image generation function 34b acquires image data I14 indicating the shape of blood vessels in the subject P and entrances of branched blood vessels. Note that the image data I14 is an example of a second X-ray image. FIG. 10 is a diagram for explaining generation of a superimposed image according to the third embodiment.

For example, the superimposed image generation function 34b first acquires X-ray image data acquired from the subject P in which a contrast agent is injected, and acquires the blood vessel shape of the subject P from the X-ray image data.

For example, the superimposed image generation function 34b and the deformation processing function 34c extract, as blood vessel regions, regions with pixel values greater than a threshold value from X-ray image data acquired with a positive contrast agent injected. , the contour of the extracted blood vessel region is obtained as the blood vessel shape.

Next, the superimposed image generation function 34b identifies the inlet of the branched blood vessel (boundary between the aorta and the branched blood vessel) in the obtained blood vessel shape of the subject P. For example, the superimposed image generating function 34b identifies the entrance of the branched blood vessel in the shape of the blood vessel based on the boundary between the trunk and the branch in the stent graft (the position of the hole provided in the trunk). Also, for example, the superimposed image generation function 34b identifies the entrance of the branched blood vessel in the blood vessel shape by receiving a designation operation from the operator.

Then, the superimposed image generating function 34b generates image data I14 indicating the blood vessel shape of the subject P and the inlets of the branched blood vessels. For example, as shown in FIG. 10, the superimposed image generation function 34b generates image data I14 in which the contour of the blood vessel region is indicated by a curved line as the shape of the blood vessel, and the contour (edge) of the entrance of the branched blood vessel is indicated by an ellipse.

The specifying function 34a specifies the positions of characteristic points of the stent graft in the X-ray image data I11. For example, the specifying function 34a specifies the positions of the marker L11, the marker L12, and the marker L13 in the X-ray image data I11. Next, the superimposed image generation function 34b identifies positions in the image data I14 corresponding to the positions of the feature points identified in the X-ray image data I11.

For example, the superimposed image generation function 34b aligns the X-ray image data I11 and the image data I14 as shown in FIG. For example, the superimposed image generation function 34b can align the X-ray image data I11 and the image data I14 based on the blood vessel shape of the subject P in each image data. Thereby, the superimposed image generation function 34b identifies the positions of the marker L11, the marker L12, and the marker L13 in the image data I14.

Then, the superimposed image generating function 34b superimposes the 3D model M on the image data I14 at positions based on the positions of the feature points specified by the specifying function 34a. For example, the superimposed image generation function 34b generates image data so that the markers L11′, L12′ and L13′ of the 3D model M1 match the positions of the markers L11, L12 and L13 specified in the image data I14. The 3D model M1 is superimposed on I14. As a result, the superimposed image generation function 34b generates a superimposed image I29 as shown in FIG.

Also, the output function 34d outputs the superimposed image I29 generated by the superimposed image generating function 34b. As a result, the image processing apparatus 30 makes it easier to grasp the position and orientation of the stent graft in the subject, and presents the operator with information about the blood vessels of the subject P, such as the position of the inlet of the branched blood vessel. , can assist in determining where to place the stent graft.

(Fourth embodiment)
Now, although the first to third embodiments have been described so far, various different modes other than the above-described first to third embodiments may be implemented.

In the above-described embodiments, the markers attached to the stent graft have been described as features of the stent graft. However, embodiments are not so limited.

For example, the specifying function 34a detects, as feature points, information corresponding to the shape of the stent graft in the X-ray image data I11. Here, the information corresponding to the shape of the stent graft is, for example, information indicating a characteristic portion of the contour, core line, or the like of the stent graft.

For example, the specifying function 34a detects, as a feature point, information indicating a portion where the curvature of the outline of the stent graft has a maximum value (the tip of the stent graft, the bifurcated portion, etc.) in the X-ray image data I11. Further, for example, the specifying function 34a may set the information indicating the portion where the curvature of the core line of the stent graft takes the maximum value (such as the bent portion provided according to the blood vessel shape of the subject P) in the X-ray image data I11 as a feature point. Detect as

Also, for example, the specifying function 34a detects information corresponding to the shape of the wire frame of the stent graft in the X-ray image data I11 as a feature point. Here, the information corresponding to the shape of the wire frame of the stent graft is, for example, information indicating a characteristic portion of the wire frame of the stent graft.

For example, a stent graft has a wavy wire frame as shown in FIG. In this case, the specific function 34a is, for example, a portion of the X-ray image data I11 in which the wavelength, amplitude, etc. of the wavy wire frame take predetermined values (for example, the wavelength, amplitude, etc. of the wire frame located at the tip of the stent graft). is detected as a feature point. Further, for example, the specifying function 34a detects, as a feature point, information indicating a portion of the X-ray image data I11 in which the wireframe interval takes a predetermined value (for example, the minimum value of the wireframe interval).

Also, in the above-described embodiments, a stent graft has been described as an example of a device. However, embodiments are not so limited. For example, the identifying function 34a identifies the positions of feature points of stents other than stent grafts in the first X-ray image.

For example, a stent used in a procedure for dilating a stenosed site of a blood vessel has a cylindrical wire frame and is inserted into the subject while being in close contact with the outside of the balloon portion of a catheter with a balloon. Then, the stent is placed at the site of the stenosis by inflating the balloon at the site of the stenosis. In this case, the specifying function 34a uses, as feature points, markers attached to the stent, information corresponding to the shape of the stent, and Information corresponding to the shape of the wire frame that has is detected.

Since a stent used in a procedure for dilating a stenosed portion of a blood vessel faces in a direction along the blood vessel in which the stent is located, it is often sufficient to grasp the position. Therefore, the specifying function 34a may specify only one position of the feature point of the stent in the first X-ray image. Then, the superimposed image generation function 34b superimposes a 3D model showing the stent on the first X-ray image or the second X-ray image at a position based on the position of the specified feature point to generate a superimposed image. do.

Also, for example, the identifying function 34a identifies the position of the feature point of the device inserted into the subject in the first X-ray image. For example, in a procedure for dilating a stenosed site in a blood vessel, a balloon-equipped catheter and a stent in close contact with the outside of the balloon portion of the balloon-equipped catheter are inserted into the subject. In this case, the identification function 34a identifies the positions of the feature points of at least one of the balloon catheter and the stent. For example, in the first X-ray image, the specifying function 34a uses, as feature points, markers attached to the stent, information corresponding to the shape of the stent, information corresponding to the shape of the wire frame of the stent, and Markers attached to the balloons, information corresponding to the shapes of the balloons, and the like are detected. Then, the superimposed image generation function 34b superimposes a 3D model showing the device at a position and orientation based on the position of the specified feature point on the first X-ray image or the second X-ray image, and superimposes Generate an image. Here, the superimposed image generation function 34b may superimpose a 3D model showing a part of the device on the first X-ray image or the second X-ray image. For example, the superimposed image generation function 34b superimposes a 3D model showing a stent among devices inserted into the subject on the first X-ray image or the second X-ray image to generate a superimposed image. Generate.

Also, for example, the identifying function 34a identifies the position of the feature point of the device other than the stent in the first X-ray image. Examples of devices other than stents include, for example, artificial valves placed at the position of the aortic valve in Transcatheter Aortic Valve Implantation (TAVI), and mitral valves in the treatment of mitral regurgitation. and a clip-shaped device that fastens the tip of the device.

For example, the prosthetic valve used in TAVI has a cylindrical wire frame. A prosthetic valve is housed in a catheter, for example, in a compressed state. The prosthetic valve is then deployed outside the catheter at the aortic valve and left in place at the aortic valve. In this case, the specifying function 34a uses, as feature points, markers attached to the artificial valve and information corresponding to the shape of the artificial valve in the first X-ray image acquired for the artificial valve inserted into the subject. , and information corresponding to the shape of the wire frame of the artificial valve.

Further, in the above-described embodiment, a case has been described in which the specifying function 34a specifies the position of the feature point by detecting the feature point in the first X-ray image. However, embodiments are not so limited. For example, the specifying function 34a specifies the position of the feature point in the first X-ray image by receiving a specifying operation from the operator. To give an example, first, the output function 34d causes the display 32 to display a first X-ray image. Then, the specifying function 34a accepts, via the input interface 31, a specifying operation of specifying the position of the feature point in the first X-ray image from the operator who referred to the first X-ray image, thereby obtaining the first X-ray image. to locate feature points in the X-ray image of .

Also, in the above-described embodiment, the case where the first X-ray images are acquired in time series has been described. However, embodiments are not so limited. For example, the X-ray diagnostic apparatus 10 acquires one or a plurality of first X-ray images triggered by receiving an image acquisition instruction from an operator. Next, the identification function 34a identifies the positions of the feature points that the device has in the acquired first X-ray image, and the superimposed image generation function 34b applies On the other hand, the superimposed image is generated by superimposing the 3D model on the positions based on the positions of the identified feature points. Then, the output function 34d outputs the generated superimposed image. In this case, the operator can easily grasp the position of the device inside the subject at the time when the first X-ray image is acquired (at the time when the image acquisition is instructed).

Further, in the above-described embodiment, the 3D model is enlarged or reduced so that the plurality of feature points identified in the first X-ray image and the plurality of feature points in the 3D model match each other. That is, in the above-described embodiment, the magnifying power of the 3D model is determined based on the intervals between the feature points specified in the first X-ray image. However, embodiments are not so limited.

For example, the superimposed image generation function 34b first identifies the position in the depth direction of the feature point identified in the first X-ray image based on the three-dimensional information indicating the running blood vessels of the subject P. Here, the three-dimensional information indicating the running of blood vessels of the subject P is, for example, a three-dimensional blood vessel image indicating the blood vessels of the subject P. FIG.

As an example, the acquisition function 110a in the X-ray diagnostic apparatus 10 executes rotational imaging of the subject P and reconstructs a three-dimensional blood vessel image from the acquired projection data. In this case, both the first X-ray image and the three-dimensional blood vessel image are acquired by the X-ray diagnostic apparatus 10, and their positional relationship is clear. Therefore, the superimposed image generating function 34b can use the three-dimensional blood vessel image as three-dimensional information indicating the running of blood vessels in the first X-ray image.

Alternatively, a medical image diagnostic apparatus other than the X-ray diagnostic apparatus may acquire a three-dimensional blood vessel image showing blood vessels of the subject P. FIG. For example, the three-dimensional blood vessel image may be CT image data acquired by an X-ray CT apparatus. In this case, the superimposed image generation function 34b specifies the mutual positional relationship by aligning the first X-ray image and the 3D blood vessel image, for example, and converts the 3D blood vessel image into the first X-ray image. It can be used as three-dimensional information indicating blood vessel running in the image.

Here, for example, when a stent such as a stent graft is inserted into the body of the subject P, the stent is located in one of the blood vessels of the subject P. That is, the feature points specified in the first X-ray image are also located inside the blood vessels of the subject P. FIG. As described above, the superimposed image generation function 34b can identify the positions in the depth direction of the feature points identified in the first X-ray image based on the three-dimensional information indicating the blood vessel running in the first X-ray image. can.

Then, the superimposed image generation function 34b superimposes the 3D model on the first X-ray image or the second X-ray image at an enlargement ratio based on the specified position in the depth direction to generate a superimposed image. Specifically, the superimposed image generation function 34b reduces the 3D model as the specified position is farther (that is, the farther the specified position is from the X-ray tube 102). Thereby, the superimposed image generation function 34b can appropriately adjust the size of the 3D model for the first X-ray image or the second X-ray image to generate a superimposed image. Further, when the magnification of the 3D model is determined based on the intervals between the feature points, the magnification of the 3D model may vary depending on whether the stent is housed in the catheter or deployed. On the other hand, when determining the magnifying power of the 3D model based on the three-dimensional information indicating the course of the blood vessel of the subject P, the magnifying power of the 3D model is determined regardless of whether the stent is housed in the catheter or deployed. can be properly determined.

Note that the three-dimensional information indicating the course of blood vessels may include information about the heart. For example, an artificial valve used in TAVI and a clip-shaped device used in the treatment of mitral regurgitation are positioned in the blood vessel of the subject P or inside the heart. That is, the feature points specified in the first X-ray image are also located inside blood vessels or inside the heart of the subject P. As described above, the superimposed image generation function 34b can identify the position in the depth direction of the feature point identified in the first X-ray image based on the three-dimensional information including the heart information. Then, the superimposed image generation function 34b superimposes the 3D model on the first X-ray image or the second X-ray image at an enlargement ratio based on the specified position in the depth direction to generate a superimposed image.

Moreover, the superimposed image generating function 34b may further superimpose three-dimensional information indicating the running of blood vessels of the subject P. FIG. For example, the superimposed image generating function 34b generates a two-dimensional blood vessel region image based on the three-dimensional blood vessel image acquired from the subject P, and further superimposes the generated blood vessel region image on the first X-ray image. , to generate a superimposed image.

For example, the superimposed image generation function 34b first generates a two-dimensional blood vessel region image based on the three-dimensional blood vessel image and the acquisition direction of the first X-ray image. For example, the superimposed image generation function 34b acquires information about the imaging system when the first X-ray image was acquired from the X-ray diagnostic apparatus 10, and generates the first X-ray image based on the acquired information. determine the collection direction of The imaging system information is, for example, information indicating the state of inclination and movement of the table top 104, the state of rotation and movement of the C-arm 105, and the like. Then, the superimposed image generation function 34b generates a rendered image obtained by rendering the three-dimensional blood vessel image according to the determined acquisition direction as a blood vessel region image. Then, the superimposed image generation function 34b superimposes the blood vessel region image and the 3D model on the first X-ray image to generate a superimposed image. Note that the process of superimposing a blood vessel region image on an X-ray image is also referred to as a three-dimensional roadmap. Volume rendering, surface rendering, Latham, MIP, MinIP, etc. can be used as a rendering method for generating a rendering image from a three-dimensional blood vessel image.

Here, landmarks may be added to the blood vessel region image. A landmark is, for example, a symbol corresponding to a characteristic portion on the device. For example, in stent-graft insertion, landmarks are added to each position of the blood vessel region image so as to correspond to the tip portion, branching portion, etc. of the stent-graft when placed at the target location. A three-dimensional roadmap using a blood vessel region image to which landmarks are added is also referred to as a three-dimensional roadmap with landmarks.

For example, the superimposed image generation function 34b generates a blood vessel region image I3 in which landmarks I32 are added to the blood vessel region I31, as shown in FIG. For example, the superimposed image generation function 34b receives an input operation from the operator via the input interface 31, and assigns a landmark I32 to a designated position on the blood vessel region I31. For example, the operator designates a position corresponding to the distal end portion and the branching portion of the stent graft when placed at the target position on the blood vessel region I31. Then, the superimposed image generation function 34b generates a blood vessel region image I3 by adding a landmark I32 to the position specified by the operator. Note that FIG. 11 is a diagram showing an example of a blood vessel region image according to the fourth embodiment. Then, the superimposed image generation function 34b superimposes the blood vessel region image I3 and the 3D model showing the stent graft on the first X-ray image to generate a superimposed image.

For example, when X-ray images are acquired in time series, the superimposed image generation function 34b sequentially generates blood vessel region images according to the acquisition direction of the newly acquired first X-ray image. Also, the superimposed image generation function 34b superimposes the newly generated blood vessel region image and the 3D model on the first X-ray image to sequentially generate superimposed images. The output function 34d sequentially outputs the newly generated superimposed images.

Although the case where the blood vessel region image is superimposed on the first X-ray image has been described as an example, the blood vessel region image may be superimposed on the second X-ray image. For example, when X-ray images are acquired in time series, the superimposed image generation function 34b sequentially generates blood vessel region images according to the acquisition direction of the newly acquired first X-ray image. Also, the superimposed image generation function 34b superimposes the newly generated blood vessel region image and the 3D model on the second X-ray image to sequentially generate superimposed images. The output function 34d sequentially outputs the newly generated superimposed images.

Further, in the above-described embodiment, the case where the image processing device 30 includes the processing circuit 34 having the specific function 34a, the superimposed image generation function 34b, and the deformation processing function 34c has been described. However, embodiments are not so limited. For example, the processing circuit 110 in the X-ray diagnostic apparatus 10 may have functions corresponding to the specific function 34a, superimposed image generation function 34b, and deformation processing function 34c.

For example, as shown in FIG. 12, the processing circuit 110 in the X-ray diagnostic apparatus 10 has an acquisition function 110a, an output function 110b, and a control function 110c, as well as a specific function 110d, a superimposed image generation function 110e, and a deformation processing function 110f. . Here, the specific function 110d is a function corresponding to the specific function 34a. Also, the superimposed image generation function 110e is a function corresponding to the superimposed image generation function 34b. Also, the transformation processing function 110f is a function corresponding to the transformation processing function 34c. Note that FIG. 12 is a block diagram showing an example of the configuration of the X-ray diagnostic apparatus 10 according to the fourth embodiment.

In the case shown in FIG. 12, acquisition function 110a acquires a first X-ray image from subject P into which the device has been inserted. The identifying function 110d also identifies the positions of feature points in the first X-ray image collected by the collecting function 110a. Here, the deformation processing function 110f may deform the 3D model showing the device based on the first X-ray image. In addition, the superimposed image generation function 110e superimposes a 3D model on the first X-ray image or the second X-ray image at a position based on the positions of the feature points specified by the specifying function 110d, and creates a superimposed image. Generate. The output function 110b then outputs the superimposed image. For example, the output function 110b displays the superimposed image on the display 108 or outputs it to the image storage device 20 or the image processing device 30. FIG.

Each component of each device according to the above-described embodiments is functionally conceptual, and does not necessarily need to be physically configured as illustrated. In other words, the specific form of distribution and integration of each device is not limited to the illustrated one, and all or part of them can be functionally or physically distributed and integrated in arbitrary units according to various loads and usage conditions. Can be integrated and configured. Furthermore, all or any part of each processing function performed by each device can be implemented by a CPU and a program analyzed and executed by the CPU, or implemented as hardware based on wired logic.

Further, the image processing method described in the above embodiments can be realized by executing a prepared program on a computer such as a personal computer or a workstation. This program can be distributed via a network such as the Internet. In addition, this program is recorded on a computer-readable non-transitory recording medium such as a hard disk, flexible disk (FD), CD-ROM, MO, DVD, etc., and is executed by being read from the recording medium by a computer. can also

According to at least one embodiment described above, it is possible to easily grasp the position of the device inside the subject.

While several embodiments of the invention have been described, these embodiments have been presented by way of example and are not intended to limit the scope of the invention. These embodiments can be implemented in various other forms, and various omissions, replacements, and modifications can be made without departing from the scope of the invention. These embodiments and their modifications are included in the scope and spirit of the invention, as well as the scope of the invention described in the claims and equivalents thereof.

1 medical image processing system 10 X-ray diagnostic apparatus 30 image processing apparatus 34 processing circuit 34a specific function 34b superimposed image generation function 34c deformation processing function 34d output function

Claims (17)

an identifying unit that identifies the positions of the feature points of the device in the first X-ray image;
a superimposed image generator that generates a superimposed image in which a 3D model representing the device is superimposed on the first X-ray image or a second X-ray image acquired after the first X-ray image;
a deformation processing unit that deforms the 3D model based on the first X-ray image;
with
The superimposed image generation unit superimposes the deformed 3D model on the first X-ray image or the second X-ray image at a position based on the position of the identified feature point. Device. The specifying unit specifies positions of the plurality of feature points in the first X-ray image,
The image processing apparatus according to claim 1, wherein the superimposed image generation unit superimposes the 3D model at a position and orientation based on the specified position of the feature point. The image processing apparatus according to claim 2, wherein the specifying unit specifies positions of three or more of the feature points in the first X-ray image. The image processing apparatus according to any one of claims 1 to 3, wherein the specifying unit specifies the position of the feature point by detecting the feature point in the first X-ray image. The image processing apparatus according to claim 4, wherein the specifying unit detects a marker attached to the device in the first X-ray image as the feature point. The image processing apparatus according to claim 5, wherein the specifying unit detects a marker attached to the tip of the device in the first X-ray image as the characteristic point. 7. The image processing apparatus according to claim 6, wherein said specifying unit further detects, as said feature point, a marker attached to the midsection of said device in said first X-ray image. the device is a stent graft having at least one branch;
8. The image processing apparatus according to claim 6, wherein said specifying unit further detects, as said feature point, a marker attached to said branched portion in said first X-ray image. The image processing apparatus according to any one of claims 4 to 8, wherein the identifying unit detects information corresponding to the shape of the device in the first X-ray image as the characteristic points. The image processing apparatus according to any one of claims 4 to 9, wherein the specifying unit detects, as the feature points, information corresponding to a shape of a wire frame of the device in the first X-ray image. . The device is a stent that is inserted into a blood vessel of a subject while being housed in a catheter,
The deformation processing unit acquires a deployment state of the stent outside the catheter based on the first X-ray image, and deforms the 3D model according to the deployment state. The image processing device according to any one of the items . Claims 1 to 11 , wherein the deformation processing unit acquires a blood vessel shape of a subject into which the device is inserted based on the first X-ray image, and deforms the 3D model according to the blood vessel shape. The image processing device according to any one of . The superimposed image generation unit generates a superimposed image by superimposing the 3D model indicating the shape of the device on the first X-ray image or the second X-ray image . 1. The image processing device according to claim 1. the device is a stent graft having at least one branch;
The superimposed image generating unit generates a superimposed image by superimposing the 3D model indicating the position and orientation of the branch portion of the stent graft on the first X-ray image or the second X-ray image. Item 13. The image processing device according to any one of items 1 to 12 . The superimposed image generating unit identifies the positions of the feature points in the depth direction based on the three-dimensional information indicating the running of blood vessels of the subject, and superimposes the 3D model at an enlargement ratio based on the identified positions in the depth direction. The image processing apparatus according to any one of claims 1 to 14 . an acquisition unit that acquires a first X-ray image;
a specifying unit that specifies the positions of the feature points of the device in the first X-ray image;
a superimposed image generating unit that generates a superimposed image in which a 3D model representing the device is superimposed on the first X-ray image or a second X-ray image different from the first X-ray image;
a deformation processing unit that deforms the 3D model based on the first X-ray image;
with
The superimposed image generation unit superimposes the deformed 3D model on the first X-ray image or the second X-ray image at a position based on the position of the identified feature point. diagnostic equipment. Locating feature points of the device in the first X-ray image;
deforming a 3D model representing the device based on the first X-ray image;
placing the deformed 3D model at a position based on the positions of the identified feature points with respect to the first X-ray image or a second X-ray image acquired after the first X-ray image; A program that causes a computer to execute each process that generates a superimposed superimposed image.

Images