JP2021171399A - X-ray diagnostic apparatus, medical image processing apparatus, medical image diagnostic apparatus and medical image processing program - Google Patents

Abstract

To support treatment report writing.SOLUTION: An X-ray diagnostic apparatus includes an information generation unit and a storage control unit. The information generation unit generates status information relating to a therapeutic procedure for a subject on the basis of a medical image which is acquired in the therapeutic procedure. The storage control unit executes processing relating to image storage on the basis of the status information.SELECTED DRAWING: Figure 2

Description

The embodiments disclosed in the present specification and the like relate to an X-ray diagnostic apparatus, a medical image processing apparatus, a medical image diagnostic apparatus, and a medical image processing program.

In Interventional Radiology (IVR), while observing the state of the body in real time using a medical image diagnostic device (X-ray fluoroscopy device, ultrasonic diagnostic device, X-ray computer tomographic imaging device, magnetic resonance imaging device, etc.), in the body A catheter or puncture needle is recommended, and the target area is treated. A treatment report is prepared after the treatment procedure in IVR is completed. In general, the image attached to the treatment report is ex post facto selected from a plurality of images acquired by radiography and fluoroscopy, for example, when an X-ray diagnostic apparatus is used.

However, in the conventional treatment report creation, for example, there are the following problems.

First, the user selects an image to be attached to the treatment report, describes points to be noted in the treatment, and the like while tracing his / her own memory after the treatment is completed. In addition, if there are many images saved during the treatment procedure, it is necessary to look back at those images and select an image for the treatment report. Therefore, a large amount of human work and work time are required, which imposes a heavy burden on the user.

In addition, no image data is left except for images that are automatically saved during treatment according to conventional criteria or images that are positively saved with the intention of leaving them as evidence. Therefore, it is not possible to secure sufficient images that can be used for treatment reports.

JP-A-2019-220014

One of the problems to be solved by the embodiments disclosed in the present specification and the like is to support the preparation of a treatment report.

The X-ray diagnostic apparatus according to the embodiment includes an information generation unit and a storage control unit. The information generation unit generates status information regarding the treatment procedure based on the medical image acquired in the treatment procedure for the subject. The storage control unit executes a process related to image storage based on the status information.

FIG. 1 is a diagram showing a configuration example of an X-ray diagnostic apparatus according to an embodiment. FIG. 2 is a diagram for explaining an example of situation information generated by the information generation function. FIG. 3 is a diagram for explaining an example of a plurality of reference information (or reference progress information) stored in the storage circuit. FIG. 4 is a diagram for explaining an example in which situation information is evaluated based on feature information and time information, and storage control processing is executed according to the result of the evaluation. FIG. 5 is a diagram for explaining another example in which situation information is evaluated based on feature information and time information, and storage control processing is executed according to the result of the evaluation. FIG. 6 is a diagram for explaining an example in which situation information is evaluated using reference information that does not include time information, and storage control processing is executed according to the result of the evaluation. FIG. 7 is a diagram for explaining an example in which situation information is evaluated using only the feature information of the medical image acquired during the treatment procedure and the storage control process is executed according to the result of the evaluation. FIG. 8 is a flowchart showing the flow of the treatment report creation support process for the fluoroscopic image. FIG. 9 is a diagram for explaining an example in which status information is evaluated based on the flag included in the feature information and the reference information, and the storage control process is executed according to the result of the evaluation. FIG. 10 is a diagram for explaining an example in which situation information is evaluated using reference information that does not include time information, and storage control processing is executed according to the result of the evaluation. FIG. 11 is a diagram for explaining an example of treatment report creation support processing of a photographed image when a fluoroscopic image is continuously acquired beyond a set threshold time. FIG. 12 is a diagram for explaining an example of treatment report creation support processing of a photographed image when a fluoroscopic image is continuously acquired beyond a set threshold time. FIG. 13 is a diagram for explaining another example of the treatment report creation support process of the captured image when the fluoroscopic image is continuously acquired beyond the set threshold time. FIG. 14 is a diagram for explaining another example of the treatment report creation support process of the captured image when the fluoroscopic image is continuously acquired beyond the set threshold time. FIG. 15 is a flowchart showing the flow of treatment report creation support processing for captured images. FIG. 16 is a flowchart showing the flow of the reference information creation process for the ultrasonic image.

(First Embodiment)
Hereinafter, embodiments will be described in detail with reference to the drawings. In the following description, the parts with the same reference numerals perform the same operation, and duplicate description will be omitted as appropriate.

FIG. 1 is a diagram showing an example of the configuration of the X-ray diagnostic apparatus 1 according to the first embodiment. In the following, in order to make the description concrete, the case where the X-ray diagnostic apparatus 1 is a cardiovascular X-ray diagnostic apparatus will be described as an example.

The X-ray diagnostic apparatus 1 inputs the imaging unit 3, the sleeper 5, the drive unit 7, the operation unit 9, the X-ray high voltage device 11, the processing circuit 21, the storage circuit 23, and the display unit 25. It includes an interface 27. The processing circuit 21, the storage circuit 23, and the input interface 27 are built in, for example, the console device 10. The imaging unit 3 includes an X-ray tube 13 that irradiates the subject P with X-rays, an X-ray detector 17 that detects X-rays, an X-ray diaphragm 15, and a holding device 19. The imaging unit 3 further includes a support arm. The bed 5 is provided with an image pickup unit 3 and an operation unit 9 for operating the bed 5. The drive unit 7 that drives the image pickup unit 3 and the bed 5 includes an image pickup system movement drive unit 71 and a top plate movement drive unit 73.

The X-ray high-voltage device 11 includes an electric circuit such as a transformer and a rectifier, a high-voltage generator, and an X-ray control device. The high voltage generator has a function of generating a high voltage applied to the X-ray tube 13 and a filament current supplied to the X-ray tube 13. The X-ray control device controls the output voltage according to the X-rays emitted by the X-ray tube 13. The high voltage generator may be of a transformer type or an inverter type. The X-ray high voltage device 11 may be provided in the holding device 19.

The X-ray tube 13 is a vacuum tube that generates X-rays by irradiating thermoelectrons from the cathode (filament) toward the anode (target) by applying a high voltage from the X-ray high voltage device 11 and supplying a filament current. Is. X-rays are generated when thermions collide with the target. The X-ray tube 13 includes, for example, a rotating anode type X-ray tube that generates X-rays by irradiating a rotating anode with thermoelectrons. The type of the X-ray tube 13 is not limited to the rotating anode type, and any type of X-ray tube can be applied.

The X-ray diaphragm 15 is provided in front of the X-ray emission window in the X-ray tube 13. The X-ray diaphragm 15 has four diaphragm blades made of, for example, a metal plate such as lead. The diaphragm blades are driven by a drive device (not shown) according to the region of interest input by the operator via the operation unit 9 or the input interface 27. The X-ray diaphragm 15 adjusts the area where X-rays are shielded to an arbitrary size by sliding these diaphragm blades with a driving device. With the adjusted diaphragm blades, the X-ray diaphragm 15 shields X-rays outside the aperture region. As a result, the X-ray diaphragm 15 narrows down the X-rays generated by the X-ray tube 13 so as to irradiate the region of interest of the subject P.

The X-ray detector 17 detects the X-rays generated by the X-ray tube 13. The X-ray detector 17 is, for example, a flat panel detector (hereinafter referred to as FPD). The FPD has a plurality of semiconductor detection elements. There are two types of semiconductor detection elements: a direct conversion type that directly converts X-rays into an electric signal, and an indirect conversion type that converts X-rays into light with a phosphor and converts the light into an electric signal. Any format may be used for the FPD. The electric signals generated by the plurality of semiconductor detection elements due to the incident of X-rays are output to an analog-digital converter (Analog to Digital converter: hereinafter referred to as an A / D converter) (not shown). The A / D converter converts an electrical signal into digital data. The A / D converter outputs digital data to the processing circuit 21. An image intensifier may be used as the X-ray detector 17.

The processing circuit 21 controls the operation of the entire X-ray diagnostic apparatus 1 according to the electric signal of the input operation output from the operation unit 9 or the input interface 27. For example, the processing circuit 21 uses a processor such as a CPU (Central Processing Unit), an MPU (Micro Processing Unit), a GPU (Graphics Processing Unit), a ROM (Read Only Memory), a RAM (Random Access), or the like as hardware resources. Has a memory of.

Various processing functions executed in the processing circuit 21 are stored in the storage circuit 23 in the form of a program that can be executed by a computer. The processing circuit 21 is a processor that realizes a function corresponding to each program by reading a program from the storage circuit 23 and executing the program. In other words, each circuit in the state where each program is read has a function corresponding to the read program.

Specifically, the processing circuit 21 executes the operation control function 211, the image generation function 212, the information generation function 213, and the storage control function 215 by the processor that executes the program expanded in the memory. Further, the processing circuit 21 that executes the information generation function 213 and the storage control function 215, respectively, corresponds to the information generation unit and the storage control unit. The operation control function 211, the image generation function 212, the information generation function 213, and the storage control function 215 are not limited to the case where they are realized by a single processing circuit. A processing circuit may be formed by combining a plurality of independent processors, and the operation control function 211, the image generation function 212, the information generation function 213, and the storage control function 215 may be realized by each processor executing a program. ..

Further, the processing circuit 21 includes an integrated circuit for a specific application (Application Specific Integrated Circuit: ASIC), a field programmable gate array (Field Programmable Gate Array: FPGA), and another composite programmable logic device (Complex Programmable). ), It may be realized by a processor such as a simple programmable logic device (SPLD).

The processing circuit 21 includes a drive unit 7, an X-ray high voltage device 11, an X-ray diaphragm 15, and a storage circuit based on an input operation received from the operator via the operation unit 9 or the input interface 27 by the operation control function 211. 23, the display unit 25, the image generation function 212, the information generation function 213, the storage control function 215, and the like are controlled. Specifically, the operation control function 211 reads out the control program stored in the storage circuit 23, expands it on the memory in the processing circuit 21, and controls each part of the X-ray diagnostic apparatus 1 according to the expanded control program. do.

The processing circuit 21 generates image data based on the output from the X-ray detector 17 by the image generation function 212. Specifically, the processing circuit 21 generates projection data from the X-ray detector 17 based on the output. Next, the processing circuit 21 receives an input signal from the operation unit 9 or the input interface 27, performs image processing such as filtering processing on the projection data, and generates image data. The image data corresponds to the data of a medical image including a fluoroscopic image and a photographed image relating to the subject P. The processing circuit 21 performs synthesis processing, subtraction processing, and the like using image data. The processing circuit 21 outputs the generated image data to the storage circuit 23 and the display unit 25.

The processing circuit 21 realizes the treatment report creation support processing described later by, for example, the information generation function 213 and the storage control function 215. Here, the treatment report is a report that describes the status of the treatment after the treatment procedure is completed, with representative images before and after the operation attached. The treatment report is created by a doctor in charge of treatment, and is stored and managed in an electronic medical record server or the like as a treatment record, for example. In the following, in order to make the explanation concrete, the process of supporting the creation of a treatment report related to a treatment procedure using a catheter will be taken as an example. However, this is only an example, and any treatment procedure that involves image diagnosis by the X-ray diagnostic apparatus 1 can be the target of the creation support process.

The information generation function 213 generates status information regarding the treatment procedure based on the medical image acquired in the treatment procedure for the subject.

Here, the situation information regarding the treatment procedure (hereinafter, simply referred to as "situation information") is information indicating the situation at each time point when the image is acquired by the X-ray diagnostic apparatus 1 during the treatment procedure. It is automatically generated every time an image is acquired by the X-ray diagnostic apparatus 1. The status information is, for example, information including at least feature information regarding a medical image acquired by the X-ray diagnostic apparatus 1 during a treatment procedure.

The feature information related to the medical image is information indicating what, where, and how it is reflected in the medical image. Feature information related to such a medical image can be acquired by using, for example, an object recognition process, a semantic segmentation process, or the like. That is, the information generation function 213 includes an AI (Artificial Intelligence) model such as a deep neural network that executes object recognition processing and semantic segmentation processing using medical images as input and outputs feature information.

In addition, the situation information corresponds to the time information (an example of the second time information) indicating the progress status from the reference time of the treatment procedure, the visual field size of the image corresponding to the feature information, and the feature information, if necessary. It is also possible to include the holding device angle information (an example of the first setting information) of the X-ray diagnostic device 1 that has acquired the image. In the following description, a plurality of status information arranged in the order of time information indicating the progress status is referred to as status progress information.

FIG. 2 is a diagram for explaining an example of situation information generated by the information generation function 213. As shown in FIG. 2, when the medical images IM1, IM2, IM3, IM4, and IM5 are acquired by the X-ray diagnostic apparatus 1 as the treatment procedure progresses, the corresponding status information IV1, IV2, IV3, IV3 -1, IV4, IV5 are produced. In FIG. 2, five status information IV1 to IV5 are illustrated, but the status information includes the corresponding status information in parallel while the medical image is acquired by the X-ray diagnostic apparatus 1 in the treatment procedure. It is automatically generated.

Each of the status information IV1 to IV5 includes feature information 30, time information 31 indicating the progress status from the reference time point t0, and holding device angle information 32. Here, the reference time point t0 is, for example, a time when the user presses the start button for starting time measurement, a time when X-ray irradiation for the first fluoroscopy is started, and the like. However, the time information 31 is not limited to the above example, and may be normal time information such as "○ month ○ day ○ hour ○ minute ○ second".

In the information generation function 213, in addition to the time information 31 (that is, absolute time information) indicating the progress status from the reference time point t0, the latest information (described later) and the status information are associated with each other by the feature information. The time difference from the time point to the present time point (that is, relative time information) shall also be measured.

In addition, the information generation function 213 evaluates the situation information. This evaluation process is automatically executed each time status information is generated (hence, each time a medical image is acquired by the X-ray diagnostic apparatus 1). Here, the evaluation of the situation information is a process of determining whether or not the target situation is a situation in which the medical image should be flagged and stored for use in the treatment report.

For example, the information generation function 213 evaluates the situation information using, for example, the feature information 30 included in each situation information. Further, the information generation function 213 evaluates the situation information by using, for example, the feature information 30 and the time information 31 included in each situation information.

Further, the information generation function 213 evaluates the situation information by using, for example, the feature information 40 included in the reference information and the feature information 30 included in the situation information. Further, the information generation function 213 uses, for example, the feature information 40 included in the reference information and the feature information 30 included in the status information, and the time information 41 included in the reference information and the time information 31 included in the status information. , Evaluate the situation information. The information generation function 213 evaluates the situation information by using, for example, the holding device angle information 42 included in the reference information and the holding device angle information 32 included in the situation information. Further, the information generation function 213 calculates, for example, the degree of similarity between the feature information and the reference information, and evaluates the situation information based on the degree of similarity. Further, the information generation function 213 calculates, for example, the similarity between the feature information and the reference information, calculates the time difference based on the time information 41 included in the reference information and the time information 31 included in the situation information, and calculates the similarity and the similarity and the time difference. Evaluate status information based on time lag.

Regarding the evaluation of the situation information executed by the information generation function 213, for example, an AI model that inputs a medical image and outputs the evaluation result, or an AI model that inputs the situation information, the situation information, and the basic information and outputs the evaluation result. Includes the AI model to be used. Further, a specific example of the evaluation of the situation information executed by the information generation function 213 will be described in detail later.

The storage control function 215 executes a process related to image storage (storage control process) based on the status information generated by the information generation function 213. Specifically, the storage control function 215 executes the storage control process according to the result of evaluation of the status information by the information generation function 213. Here, the "preservation control process" is a process related to flagging storage and erasing of a medical image using the storage for primary storage and the storage for long-term storage of the storage circuit 23. Here, in the present embodiment, flagging and saving the medical image includes saving the medical image with a bookmark or a tag. The storage for primary storage is a storage circuit that sequentially stores medical images acquired by photographing. The storage for long-term storage is a storage circuit that stores medical images for a long period of time as compared with the storage for primary storage. For example, in the storage control process, among a plurality of medical images stored in the storage for primary storage, the medical images determined to be the target of long-term storage are sequentially (in real time) or set in the storage for long-term storage. It can include a form of duplication (copying) after a lapse of time. Further, in the storage control process, for example, among a plurality of medical images stored in the storage for primary storage, the medical image determined to be deleted is deleted, for example, in real time or after a set time has elapsed. Can be included. Further, the storage control process can include, for example, a form of erasing the medical image determined to be the target of erasure among the plurality of medical images stored in the storage for long-term storage.

Further, for example, the storage control function 215 stores a plurality of medical images associated with each other based on the similarity of the feature information by the information generation function 213 in the storage circuit 23 with a flag. Further, for example, the storage control function 215 stores a plurality of medical images associated with each other based on the time information by the information generation function 213 in the storage circuit 23 with a flag.

A specific example of the storage control process executed by the storage control function 215 will be described in detail later.

The storage circuit (memory) 23 is a storage device such as an HDD (Hard disk Drive), an SSD (Solid State Drive), or an integrated circuit storage device that stores various information, or a circuit in which a plurality of these storage devices are combined. Is. The storage circuit 23 has, for example, a storage for primary storage and a storage for long-term storage. The medical images that are sequentially and primarily stored in the primary storage storage are updated, for example, at regular intervals. The storage for primary storage and the storage for long-term storage are examples of the first memory and the second memory, respectively. Further, the storage circuit 23 stores, for example, projection data, image data, and programs corresponding to various functions read and executed by the processing circuit 21. In addition to HDDs and SSDs, the storage circuit 23 includes portable storage media such as CDs (Compact Discs), DVDs (Digital Versailles Discs), and flash memories, and semiconductor memory elements such as RAMs (Random Access Memory). It may be a drive device that reads and writes various information between them. Further, the storage circuit 23 may be in an external storage device connected by a network. Further, when the storage circuit 23 includes a plurality of storage devices, a part of the storage circuit 23 may be a storage device connected via a network.

In addition, the storage circuit 23 stores reference information used for evaluating the status information of the treatment procedure. Here, the reference information is information that serves as a reference for the procedure in various situations of the treatment procedure, and is created in advance based on, for example, a guideline or the like. In addition, the reference information may be separately generated or revised for each treatment procedure using past achievements in the hospital concerned. The reference information is, for example, information including at least feature information generated from an image to be a model of the procedure.

In addition, the reference information includes time information (an example of the first time information) indicating the standard progress status from the reference time of the corresponding treatment procedure, the visual field size of the image corresponding to the feature information, and the feature, if necessary. It is also possible to include holding device angle information (an example of the second setting information) of the medical image diagnostic device (X-ray diagnostic device in this embodiment) that has acquired the image corresponding to the information. In the following description, a plurality of reference information arranged in the order of time information indicating the standard progress status is referred to as standard progress information (or standard procedure progress table).

FIG. 3 is a diagram for explaining an example of a plurality of reference information (or reference progress information) stored in the storage circuit 23. As shown in FIG. 3, each of the reference information IA1, IA2, IA3, IA3-1, IA4, IA5, IB4, IB5 ... The holding device angle information 42 is included. The reference information IA3-1 further includes a flag 44 indicating that a condition determination occurs in the treatment procedure and the subsequent reference information branches into the reference information IA4 and the reference information IA4. Further, the reference information IA3 further includes a recording target flag 43 for recording a medical image having a high degree of similarity to the feature information 40 included in the reference information IA3. These reference information are used as criteria for evaluating the situation information in the treatment report creation support process described later.

Further, the storage circuit 23 stores an image acquired by fluoroscopy or radiography of the X-ray diagnostic apparatus 1. Here, "perspective" is an imaging method in which an X-ray tube 13 is continuously irradiated with X-rays to acquire an image (typically a moving image) of the inside of a subject in real time. Further, "shooting" is a shooting method of irradiating X-rays having a higher intensity than fluoroscopy to obtain a more detailed image (still image) of the inside of the subject. In the present embodiment, the image acquired by fluoroscopy will be referred to as a "perspective image", and the image acquired by imaging will be referred to as a "captured image".

Further, the storage circuit 23 flags and saves the fluoroscopic image targeted for flagged storage by the information generation function 213 under the control of the storage control function 215. Further, the storage circuit 23 flags the captured images that have already been stored and are targeted for flagged storage by the information generation function 213, according to the control of the storage control function 215.

Returning to FIG. 1, the display unit 25 starts with a display 251 that displays a medical image g1 and the like, an internal circuit that supplies a display signal to the display 251 and peripheral circuits such as a connector and a cable that connect the display 251 and the internal circuit. It is configured. The internal circuit generates display data by superimposing incidental information such as subject information and projection data generation conditions on the image data. Next, the internal circuit performs D / A conversion and TV format conversion on the obtained display data. The internal circuit displays the display data on which these conversions have been performed on the display 251 as a medical image. In addition to this, the display unit 25 displays a GUI (Graphical User Interface) or the like for receiving various operations from the operator.

As the display 251, for example, a liquid crystal display (LCD: Liquid Crystal Display), a CRT (Casode Ray Tube) display, an organic EL display (OELD: Organic Electro Luminescence Display), a plasma display or any other display may be used as appropriate. It is possible. Further, the display 251 may be a desktop type, or may be composed of a tablet terminal or the like capable of wireless communication with the processing circuit 21.

The input interface 27 receives various input operations from the operator, converts the received input operations into electric signals, and outputs the received input operations to the processing circuit 21. For example, the input interface 27 operates an operation for operating at least one of the image pickup unit 3 and the bed 5, an X-ray condition for generating X-rays, a condition for image processing executed by the image generation function 212, and the like. Accept from people. As the input interface 27, for example, a mouse, a keyboard, a trackball, a switch, a button, a joystick, a foot switch, a touch pad, a touch panel display, and the like can be appropriately used. The input interface 27 is mounted on, for example, a console device 10 installed in an operation room different from the examination room.

In the present embodiment, the input interface 27 is not limited to the one provided with physical operation parts such as a mouse, a keyboard, a trackball, a switch, a button, a joystick, a touch pad, and a touch panel display. For example, an example of the input interface 27 includes an electric signal processing circuit that receives an electric signal corresponding to an input operation from an external input device provided separately from the device and outputs the electric signal to the processing circuit 21. .. The input interface 27 may be composed of a tablet terminal or the like capable of wireless communication with the processing circuit 21.

[Treatment report creation support process]
Next, the treatment report creation support process executed by the X-ray diagnostic apparatus 1 will be described. The treatment report creation support process can be broadly classified into "treatment report creation support process for fluoroscopic images" and "treatment report creation support process for captured images".

In the actual treatment procedure, imaging is performed at an arbitrary timing while observing a live image by fluoroscopy. Therefore, each of the "treatment report creation support process for fluoroscopic images" and the "treatment report creation support process for captured images" described below is performed in parallel each time fluoroscopy and imaging are performed in the treatment procedure. Is executed.

(Treatment report creation support processing for fluoroscopic images)
First, the treatment report creation support process for fluoroscopic images will be described. In the present embodiment, all the fluoroscopic images acquired by the X-ray diagnostic apparatus 1 are temporarily stored in the storage circuit 23.

FIG. 4 is a diagram for explaining an example in which situation information is evaluated based on feature information and time information, and storage control processing is executed according to the result of the evaluation. In the example shown in FIG. 4, it is assumed that after taking time for the procedure at time t1, the procedure shifts to the procedure at time t2.

As shown by the broken line in FIG. 4, the information generation function 213 specifies the feature information 40 similar to the feature information 30 included in the situation information IV2 (for example, the similarity is equal to or higher than the threshold value) from the reference progress information. .. At this time, it can also be used that the degree of similarity between the holding device angle information 32 and the holding device angle information 42 is equal to or higher than the threshold value. The information generation function 213 refers to the time information 41 of the reference information IA2 including the similar feature information 40, and the time difference between the time information 31 (time t2) of the status information IV2 and the time information 41 (time ta2) of the reference information IA2. To calculate.

When the calculated time difference is equal to or greater than the threshold value, the information generation function 213 means that the medical image showing the situation up to the time t2 actually required a time exceeding the reference time. Evaluate that the situation should be saved with the "difficulty" flag.

The storage control function 215 means that, based on the evaluation result of the information generation function 213, the image IM1 corresponding to the time t1 in the previous stage of the time t2 actually required a time exceeding the reference time. The fluoroscopic image is stored in the storage circuit 23 with the “difficulty” flag. On the other hand, other fluoroscopic images that are not flagged by the storage control function 215 are erased from the storage circuit 23. The preservation of the flagged fluoroscopic image and the deletion of the unflagged fluoroscopic image by the preservation control function 215 may be in any form of the above-mentioned "preservation control process".

The information generation function 213 sequentially evaluates the situation information based on the feature information and the time information even after the time t2. At this time, in order not to include the delay of the treatment procedure up to the time t2 in the subsequent evaluation, the relative time information (that is, the reference information and the situation information are associated with each other by the feature information) after the time t2. The evaluation is performed using the time difference from the latest time point to the present time point).

FIG. 5 is a diagram for explaining another example in which situation information is evaluated based on feature information and time information, and storage control processing is executed according to the result of the evaluation. In the example shown in FIG. 5, it is assumed that a judgment branch is required in the procedure at time t4. Here, the judgment branch means a branch in which it is necessary to judge which method is to be selected from several options in the treatment procedure.

In such a case, the information generation function 213 specifies the feature information 40a similar to the feature information 30 included in the situation information IV4 from the reference progress information, as shown by the broken line in FIG. At this time, it can also be used that the degree of similarity between the holding device angle information 32 and the holding device angle information 42a is equal to or higher than the threshold value. The information generation function 213 means "branch determination" in the medical image IM5 acquired at time t4 because the reference information IA4 including the specified feature information 40a is one of the branches with the reference information IA5. Evaluate that the situation should be flagged and saved.

The storage control function 215 attaches a flag meaning "branch determination" to the image IM4 corresponding to the time t4, and stores the fluoroscopic image in the storage circuit 23. On the other hand, other fluoroscopic images that are not flagged by the storage control function 215 are erased from the storage circuit 23. The preservation of the flagged fluoroscopic image and the deletion of the unflagged fluoroscopic image by the preservation control function 215 may be in any form of the above-mentioned "preservation control process".

FIG. 6 is a diagram for explaining an example in which situation information is evaluated using reference information that does not include time information, and storage control processing is executed according to the result of the evaluation. In the example shown in FIG. 6, it is assumed that the reference information including only the feature information is used without including the time information. Since the plurality of reference information IA1 to IA4 and the like do not have time information, they do not form the reference progress information.

The information generation function 213 determines that the feature information 30 included in the status information IV4 is similar to the feature information 40 included in the reference information IA4. As shown by the broken line in FIG. 6, the information generation function 213 has the condition determination flag 45 attached to the specified feature information 40, which means "condition determination" in the medical image at time t4. Evaluate that the situation should be flagged and saved.

The storage control function 215 attaches a flag meaning "condition determination" to the image IM4 corresponding to the time t4, and stores the fluoroscopic image in the storage circuit 23. On the other hand, other fluoroscopic images that are not flagged by the storage control function 215 are erased from the storage circuit 23. The preservation of the flagged fluoroscopic image and the deletion of the unflagged fluoroscopic image by the preservation control function 215 may be in any form of the above-mentioned "preservation control process".

FIG. 7 is a diagram for explaining an example in which situation information is evaluated using only the feature information of the medical image acquired during the treatment procedure and the storage control process is executed according to the result of the evaluation. In the example shown in FIG. 7, it is assumed that the feature information of the medical image acquired at time t2 and t3 includes the information "with stent".

The information generation function 213 sequentially evaluates the feature information of the medical image acquired during the treatment procedure, for example, in real time. That is, as shown by the broken line in FIG. 7, the information generation function 213 includes the information that "there is a stent" in the feature information 30 included in the status information IV2 and IV3 acquired at the times t2 and t3. Judge that there is. The information generation function 213 evaluates that the medical image corresponding to the time t2 and t3 should be stored with a flag meaning "with stent".

The storage control function 215 adds a flag meaning "with stent" to the images IM2 and IM3 corresponding to the times t2 and t3, and stores each fluoroscopic image in the storage circuit 23. On the other hand, other fluoroscopic images that are not flagged by the storage control function 215 are erased from the storage circuit 23. The preservation of the flagged fluoroscopic image and the deletion of the unflagged fluoroscopic image by the preservation control function 215 may be in any form of the above-mentioned "preservation control process".

FIG. 8 is a flowchart showing the flow of the treatment report creation support process for the fluoroscopic image. As shown in FIG. 8, first, in the treatment procedure, the motion control function 211 starts acquiring a fluoroscopic image in response to an instruction from the input interface 27. Further, the information generation function 213 starts the time measurement (step S1).

The motion control function 211 continuously acquires fluoroscopic images. The acquired fluoroscopic image is temporarily stored in the storage circuit 23 (step S2).

The information generation function 213 generates status information including feature information, angle information, and time information for each of the perspective images continuously acquired, and temporarily stores the status information in the storage circuit 23 (step S3).

The information generation function 213 evaluates the situation information related to each medical image by using, for example, the generated situation information and a plurality of reference information (or reference progress information) stored in the storage circuit 23 (step S4). .. That is, in the step S4, the evaluation process described with reference to FIGS. 4 to 7 is executed.

The storage control function 215 executes a storage control process related to image storage for the fluoroscopic image according to the evaluation result by the information generation function 213 (step S5). That is, in the step S5, the storage control process described with reference to FIGS. 4 to 7 is executed.

(Treatment report creation support processing for captured images)
Next, the treatment report creation support process for the captured image will be described. In the present embodiment, all the captured images acquired by the X-ray diagnostic apparatus 1 are stored in, for example, the long-term storage storage of the storage circuit 23.

FIG. 9 is a diagram for explaining an example in which status information is evaluated based on the flag included in the feature information and the reference information, and the storage control process is executed according to the result of the evaluation. In the example shown in FIG. 9, it is assumed that the recording target flag 43 is included in the reference information corresponding to the time ta3.

The information generation function 213 sequentially generates status information corresponding to each medical image. For example, the information generation function 213 calculates the similarity with the feature information 40 of the reference information corresponding in time (or close in time) each time each feature information 30 is generated, and the similarity is set to be equal to or higher than the threshold value. Determine if it is. At this time, it can also be used that the similarity between the holding device angle information 32 is equal to or higher than the threshold value.

As shown by the broken line in FIG. 9, the feature information 40 of the reference information IA3 including the recording target flag 43 and the feature information 30 included in the situation information IV3 corresponding to the time t3 are similar. From this, the information generation function 213 evaluates that the medical image IM3 corresponding to the time t3 should be saved with a flag meaning "recording target".

The storage control function 215 adds a “recording target” flag (bookmark) to the captured image IM3 corresponding to the time t3 stored in the storage circuit 23.

FIG. 10 is a diagram for explaining an example in which situation information is evaluated using reference information that does not include time information, and storage control processing is executed according to the result of the evaluation. In the example shown in FIG. 10, similarly to the example shown in FIG. 6, reference information including only feature information without including time information is used. Further, in the example shown in FIG. 10, it is assumed that the recording target flag 43 is included in the reference information corresponding to the time ta3, as in the example shown in FIG.

The information generation function 213 sequentially generates status information including feature information corresponding to each medical image. For example, the information generation function 213 calculates the similarity with the feature information 40 of the reference information IA3 each time the feature information 30 is generated, and identifies the feature information 40 whose similarity is equal to or higher than the threshold value.

As shown by the broken line in FIG. 10, the feature information 40 of the reference information IA3 including the recording target flag 43 and the feature information 30 included in the situation information IV3 corresponding to the time t3 are similar. From this, the information generation function 213 evaluates that the information generation function 213 should save the medical image IM3 corresponding to the time t3 with a flag meaning "recording target".

The storage control function 215 adds a “recording target” flag (bookmark) to the captured image IM3 corresponding to the time t3 stored in the storage circuit 23.

In each of the examples shown in FIGS. 9 and 10, if the image IM3 corresponding to the time t3 is a perspective image, the storage control function 215 "records" the perspective image IM3 corresponding to the time t3. The perspective image IM3 is stored in the storage circuit 23 with a flag meaning. On the other hand, other fluoroscopic images that are not flagged by the storage control function 215 are erased from the storage circuit 23. The preservation of the flagged fluoroscopic image and the deletion of the unflagged fluoroscopic image by the preservation control function 215 may be in any form of the above-mentioned "preservation control process".

11 and 12 are diagrams for explaining an example of the treatment report creation support process of the captured image when the fluoroscopic image IM20 is continuously acquired beyond the set threshold time.

For example, it is assumed that the fluoroscopic image IM20 having the feature information 30 at the time t20 shown by the dotted line in FIG. 11 is continuously acquired beyond the set threshold time. In such a case, as shown in FIG. 12, the information generation function 213 selects the feature information 30 acquired before and after the time t20 whose similarity with the feature information 30 at the time t20 is equal to or higher than the threshold value. Identify. At this time, it can also be used that the similarity between the holding device angle information 32 is equal to or higher than the threshold value. The information generation function 213 evaluates that the captured image corresponding to the specified feature information 30 (photographed images IM11 and IM28 are illustrated in FIG. 12) should be saved with a flag (bookmark).

The storage control function 215 adds a flag (bookmark) to each of the captured image IM11 corresponding to the time t11 and the captured image IM28 corresponding to the time t28 stored in the storage circuit 23.

FIG. 13 is a diagram for explaining another example of the treatment report creation support process of the captured image when the fluoroscopic image IM20 is continuously acquired beyond the set threshold time. Compared with the example of FIG. 12, the example of FIG. 13 is different in that the time information is not used when evaluating the target of the flagged storage.

For example, in FIG. 13, it is assumed that the fluoroscopic image IM20 having the feature information 30 at the time t20 is continuously acquired beyond the time as the set threshold value. In such a case, as shown in FIG. 13, the information generation function 213 uses the feature information 30 at time t20 from the feature information 30 for all the captured images stored in the storage circuit 23 regardless of the time information. Identify those whose similarity is greater than or equal to the threshold. At this time, it can also be used that the similarity between the holding device angle information 32 is equal to or higher than the threshold value. The information generation function 213 evaluates that the captured image corresponding to the specified feature information 30 (photographed images IM11 and IM28 are illustrated in FIG. 13) should be saved with a flag (bookmark).

The storage control function 215 adds a flag (bookmark) to each of the captured image IM11 corresponding to the time t11 and the captured image IM28 corresponding to the time t28 stored in the storage circuit 23.

FIG. 14 is a diagram for explaining another example of the treatment report creation support process of the captured image when the fluoroscopic image IM20 is continuously acquired beyond the set threshold time. Compared with the examples of FIGS. 12 and 13, the example of FIG. 14 is different in that the feature information of other captured images is not used when evaluating the target of the flagged storage.

For example, in FIG. 14, it is assumed that the fluoroscopic image IM20 having the feature information 30 at the time t20 is continuously acquired beyond the time as the set threshold value. In such a case, as shown in FIG. 14, the information generation function 213 does not calculate the similarity of the feature information of other captured images, and the period set before and after the time t20 (in FIG. 14 in FIG. 14). It is evaluated that all the captured images acquired in the period T included in the broken line region of the time axis on the left side. Hereinafter referred to as “flagged storage target period T”) are subject to flagged storage.

The storage control function 215 adds a flag (bookmark) to all the captured images stored in the storage circuit 23 and included in the flagged storage target period T.

FIG. 15 is a flowchart showing the flow of treatment report creation support processing for captured images. As shown in FIG. 15, first, in the treatment procedure, the motion control function 211 starts acquiring a captured image in response to an instruction from the input interface 27. Further, the information generation function 213 starts the time measurement (step S11).

The operation control function 211 acquires a captured image. The acquired captured image is stored in the storage circuit 23 (step S12).

The information generation function 213 generates status information including feature information, angle information, and time information for the captured image and stores it in the storage circuit 23 (step S13).

The information generation function 213 evaluates the situation information related to each medical image by using, for example, the generated situation information and a plurality of reference information (or reference progress information) stored in the storage circuit 23 (step S14). .. That is, in the step S4, the evaluation process described with reference to FIGS. 9 to 14 is executed.

The storage control function 215 executes a storage control process related to image storage according to the evaluation result by the information generation function 213 (step S15). That is, in the step S15, the storage control process described with reference to FIGS. 9 to 14 is executed.

As described above, the X-ray diagnostic apparatus 1 according to the embodiment includes an information generation function 213 as an information generation unit and a storage control function 215 as a storage control unit. The information generation function 213 generates status information regarding the treatment procedure based on the medical image acquired in the treatment procedure for the subject. The storage control function 215 executes a process related to image storage based on the status information.

Specifically, the information generation function 213 generates status information including at least feature information related to the medical image, and evaluates the status information using the reference information and the feature information regarding the treatment procedure. The storage control function 215 executes a process related to image storage according to the evaluation result.

Therefore, the information generation function 213 uses, for example, characteristic information related to the treatment procedure, for example, "a situation in which it takes time to perform the treatment procedure", "a situation in which a branch judgment is made", and "a situation scheduled to be recorded in advance". From the situation information such as ", the medical image to be stored with a flag can be specified according to the situation. The storage control function 215 executes a storage control process for flagging and storing the medical image specified by the information generation function 213. Therefore, among the plurality of images recorded during the treatment procedure, the image presumed to be used at the time of report creation can be explicitly left in distinction from other images. In particular, useful images other than images that are automatically saved during treatment or images that are actively saved with the intention of leaving them as evidence can be saved without omission.

The user can create a treatment report for a flagged image among many fluoroscopic images and captured images acquired and saved during the treatment procedure. Therefore, it is possible to reduce the workload of the user in creating the treatment report. In addition, the situation information generated from each medical image is evaluated, and each medical image acquired in a situation presumed to be important during the treatment procedure is automatically saved as a flagged image. Therefore, it is possible to reduce the work load for image storage performed by the user during the treatment procedure.

Also, many of the images that are automatically flagged and saved in this way are images that have been acquired in situations that are presumed to be medically important. Therefore, a sufficient number of images that can be used for the treatment report can be secured, which can contribute to the improvement of the quality of the treatment report. In addition, these images have high educational and research value, and can be used as a database for conferences, reports to academic societies, and therapeutic procedures. Therefore, the utility value of the treatment report can be expanded, and it is possible to contribute to shortening the time required to create the treatment report. As a result, it is possible to assist in the preparation of treatment reports.

Further, the storage control function 215 executes the storage control process based on the evaluation of the information generation function 213. For example, the storage control function 215 stores the fluoroscopic image determined to be the target of long-term storage in the storage for long-term storage of the storage circuit 23 in real time among the plurality of fluoroscopic images stored in the storage for primary storage of the storage circuit 23. Copy to. Further, for example, the storage control function 215 erases the fluoroscopic image determined to be the target of erasure in real time from among the plurality of fluoroscopic images stored in the storage for primary storage of the storage circuit 23. Therefore, it is possible to reliably store an image effective in creating a treatment report without squeezing the memory area of the storage circuit 23.

(Modification example 1)
In the above embodiment, the case where the X-ray diagnostic apparatus 1 executes the above-mentioned treatment report support process has been illustrated. On the other hand, the above-mentioned treatment report support process may be executed by a medical image processing device such as a medical workstation. Such a medical image processing device can be used in any environment as long as it can communicate with the X-ray diagnostic device 1 and the medical image management server device via a network such as a hospital network or a cloud network. It is possible to do.

(Modification 2)
In the above embodiment, a case where the treatment report support process is executed in real time during the treatment procedure is taken as an example. On the other hand, for example, it is possible to save all the fluoroscopic images and captured images acquired during the treatment procedure, and use these to realize the treatment report support process after the treatment procedure is completed (ex post facto). .. In this case, the fluoroscopic image that is not the storage target can be deleted after the fact from the storage for long-term storage of the storage circuit 23, for example.

(Modification example 3)
In the above embodiment, the X-ray diagnostic apparatus for the circulatory system has been described as an example. On the other hand, the above-mentioned treatment report creation support function is applied to X-ray diagnostic devices other than those for the cardiovascular system, for example, X-ray TV devices used in lung examination treatment using a bronchoscope, digestive organ treatment, and the like. Is also applicable. Further, the present invention is not limited to the X-ray diagnostic apparatus, and can be applied to any medical diagnostic imaging apparatus used in a therapeutic procedure. Medical image diagnostic devices used in therapeutic procedures include ultrasonic diagnostic devices (for example, transesophageal ultrasonic diagnostic devices, intravascular ultrasonic diagnostic devices), endoscopic systems, X-ray computer tomographic imaging devices, and magnetic resonance imaging devices. And so on.

(Second Embodiment)
Next, the X-ray diagnostic apparatus 1 according to the second embodiment will be described. The second embodiment is an example in which, when a plurality of medical diagnostic imaging devices are used in combination in a treatment procedure, a treatment report creation support process straddling each device is applied. In order to make the explanation concrete, in the present embodiment, as a treatment technique, when performing the indwelling work of MitraClip (registered trademark) on the mitral valve, an X-ray diagnostic apparatus for circulatory organs and a transesophageal ultrasonic diagnostic apparatus are used. An example is the case where a medical diagnostic imaging system in which the above is used is used.

Treatment, including the indwelling operation of MitraClip®, is performed, for example, in three major steps: That is, as the first step, imaging (perspective, imaging) is performed by the X-ray diagnostic apparatus for the circulatory system. As a second step thereafter, an indwelling operation is performed by inserting a clip delivery system equipped with MitraClip (registered trademark) at the tip from the catheter, for example, under observation with a transesophageal ultrasonic diagnostic apparatus. As the third step, fluoroscopy and imaging of the mitral valve by an X-ray diagnostic apparatus are performed again for the purpose of observation after indwelling.

In such treatment, for example, using a plurality of ultrasonic images acquired in the second stage, situation information including feature information and time information is generated, and this is used as reference information in the first and third stages. The above-mentioned treatment report creation support process may be executed on the X-ray image.

Further, in the second stage, imaging by the cardiovascular X-ray diagnostic apparatus may be performed in parallel with the observation by the transesophageal ultrasonic diagnostic apparatus. Also in this case, the situation information including the feature information and the time information is generated by using the plurality of ultrasonic images acquired by the transesophageal ultrasonic diagnostic apparatus, and the situation information including the feature information and the time information is used as the reference information by the cardiovascular X-ray diagnostic apparatus. It is also possible to execute the above-mentioned treatment report creation support process on the acquired X-ray image.

FIG. 16 is a flowchart showing the flow of the reference information creation process for the ultrasonic image. As shown in FIG. 16, first, an ultrasonic image is acquired and saved by, for example, a transesophageal ultrasonic diagnostic apparatus (step S21).

The information generation function 213 generates and saves status information including feature information and time information for each of the acquired ultrasonic images (step S22). The saved status information is saved as reference information for the treatment report creation support process for the X-ray image in the first and third stages (step S23).

It is also possible to evaluate the situation information by using the feature information or the like generated by using the ultrasonic image. At this time, the feature information generated from the ultrasonic image by the object recognition process or the semantic segmentation process may be used, or another method may be adopted depending on the imaging method of the ultrasonic image.

For example, it generates feature information including reverse flow in the mitral valve calculated by the Doppler method. In such feature information, for the ultrasonic image corresponding to the feature information that the reverse flow rate does not fall below the threshold value over the threshold time, it is possible to adopt a situation information evaluation method such as targeting the flagged storage.

According to at least one embodiment described above, it is possible to support the preparation of a treatment report.

Although some embodiments of the present invention have been described, these embodiments are presented as examples and are not intended to limit the scope of the invention. These novel embodiments can be implemented in various other embodiments, and various omissions, replacements, and changes can be made without departing from the gist of the invention. These embodiments and modifications thereof are included in the scope and gist of the invention, and are included in the scope of the invention described in the claims and the equivalent scope thereof.

1 X-ray diagnostic device 2 X-ray TV device 3 Imaging unit 5 Sleeper 7 Drive unit 9 Operation unit 10 Console device 11 X-ray high voltage device 13 X-ray tube 15 X-ray filter 17 X-ray detector 19 Holding device 21 Processing circuit 23 Storage circuit (memory)
25 Display unit 27 Input interface 71 Imaging system movement drive unit 73 Top plate movement drive unit 211 Operation control function 212 Image generation function 213 Information generation function 215 Storage control function 251 Display

Claims (16)

An information generation unit that generates status information related to the treatment procedure based on the medical image acquired in the treatment procedure for the subject, and an information generation unit.
A storage control unit that executes processing related to image storage based on the status information,
X-ray diagnostic device equipped with. The information generation unit generates the situation information including at least the feature information about the medical image, evaluates the situation information using the feature information, and evaluates the situation information.
The storage control unit executes a process related to image storage according to the result of the evaluation.
The X-ray diagnostic apparatus according to claim 1. The information generation unit generates the situation information including the feature information associated with the first time information regarding the progress of the treatment procedure of the subject, and uses the first time information to generate the situation information. 2. The X-ray diagnostic apparatus according to claim 2. The X-ray diagnostic apparatus according to claim 2, wherein the information generation unit evaluates the situation information using the reference information regarding the treatment procedure and the feature information. The reference information is associated with a second time information regarding the progress of the treatment procedure.
The information generation unit generates the situation information including the feature information associated with the first time information regarding the progress of the treatment procedure of the subject, and the first time information and the second time. The X-ray diagnostic apparatus according to claim 4, wherein the situation information is evaluated using the information. The reference information is associated with the second setting information regarding the setting of the medical image diagnostic apparatus.
The information generation unit generates the situation information including the feature information associated with the first setting information regarding the setting of the medical image diagnostic apparatus used for acquiring the medical image, and the first setting information and the said Evaluate the situation information using the second setting information,
The X-ray diagnostic apparatus according to claim 5. The X-ray according to any one of claims 4 to 6, wherein the information generation unit calculates the similarity between the feature information and the reference information and evaluates the situation information based on the similarity. Diagnostic device. The information generation unit calculates the similarity between the feature information and the reference information, calculates the time difference based on the first time information and the second time information, and is based on the similarity and the time difference. The X-ray diagnostic apparatus according to claim 5 or 6, which evaluates the situation information. The X-ray diagnostic apparatus according to any one of claims 2 to 8, wherein the information generation unit generates the feature information by using an object recognition process or a semantic segmentation process. The information generation unit associates the medical image with at least one other medical image based on the feature information of the medical image.
The X-ray diagnostic apparatus according to any one of claims 2 to 9, wherein the storage control unit stores the medical image and the other medical image associated with the medical image in association with each other. The information generation unit associates the medical image with at least one other medical image based on the time information of the medical image.
The X-ray diagnostic apparatus according to any one of claims 2 to 10, wherein the storage control unit stores the medical image and the other medical image associated with the medical image in association with each other. In the evaluation, the information generation unit sequentially determines whether to save or delete each of the plurality of medical images to be sequentially acquired.
The X-ray diagnostic apparatus according to any one of claims 2 to 11, wherein the storage control unit sequentially erases an image determined to be erased among a plurality of the medical images stored in a storage circuit. The X-ray diagnosis according to claim 12, wherein the storage control unit sequentially stores the medical images determined to be stored among the plurality of medical images stored in the first storage circuit in the second storage circuit. Device. An information generation unit that generates status information related to the treatment procedure based on the medical image acquired in the treatment procedure for the subject, and an information generation unit.
A storage control unit that executes processing related to image storage based on the status information,
Medical image processing device equipped with. An information generation unit that generates status information related to the treatment procedure based on the medical image acquired in the treatment procedure for the subject, and an information generation unit.
A storage control unit that executes processing related to image storage based on the status information,
Medical diagnostic imaging equipment equipped with. On the computer
An information generation function that generates status information related to the treatment procedure based on the medical image acquired in the treatment procedure for the subject, and
A save control function that executes processing related to image save based on the status information, and
A medical image processing program that realizes.

Images