JP2021186257A - Medical image diagnostic apparatus and medical image processing apparatus - Google Patents

Abstract

To display a medical image with image quality which a user is used to see usually, in image analysis of a medical image by using a learned model.SOLUTION: A medical image diagnostic apparatus according to embodiment comprises an acquisition unit, a generation unit, and a display unit. The acquisition unit images an analyte and acquires image data concerning the analyte. The generation unit generates from the image data a first medical image on the basis of a first image quality condition which can be set by a user and generates a second medical image on the basis of a second image quality condition used for training of a learned model for performing image analysis. The display unit is capable of displaying the first medical image and the second medical image in association with each other.SELECTED DRAWING: Figure 1

Description

Embodiments of the present invention relate to a medical diagnostic imaging apparatus and a medical imaging apparatus.

The medical image diagnostic apparatus sets various image quality parameters such as gain, dynamic range, γ curve, and RGB with respect to the image data to predetermined values. The medical image diagnostic device generates and displays a medical image based on the image quality conditions defined by various image quality parameters.

Some medical image diagnostic devices have a built-in image analysis application for performing various image analyzes on a medical image (also referred to as an observation image) displayed on a display device, for example. Image analysis includes, for example, extracting a specific structure contained in a user-designated region of interest (ROI) on a medical image. Image analysis including such structure extraction can be realized, for example, by applying a trained model trained by deep learning to a medical image. Here, in order to guarantee the accuracy of image analysis by the trained model, it is necessary to train the training model using a large number of medical images fixed to predetermined image quality conditions as training data. Along with this, when the medical image to be analyzed (also called the image for analysis) is analyzed using the trained model trained using the above-mentioned training data, the image quality condition of the image for analysis is the same as the training data. It is necessary to fix the image quality conditions.

Therefore, in the above-mentioned medical diagnostic imaging apparatus, when image analysis is performed on the observation image (that is, when the observation image and the analysis image are the same), the observation image is a trained model used for image analysis. Since the image quality is fixed to the same as the image quality condition at the time of training, the image quality may be significantly different from the image quality that users such as medical workers are accustomed to. Further, when the user freely changes the image quality condition of the observation image to be analyzed, it is necessary to verify the accuracy of the image analysis by the trained model in each of the changed image quality conditions. Therefore, in order to save the workload required for accuracy verification, the above-mentioned medical image diagnostic apparatus often has specifications that the user cannot freely change the image quality condition of the observation image to be analyzed. As a result, for example, the user cannot observe the observation image under the desired image quality conditions, and it is difficult to properly evaluate the result of the image analysis displayed on the observation image.

Japanese Unexamined Patent Publication No. 2009-17991 JP-A-2015-8839

One of the problems to be solved by the embodiments disclosed in the present specification and the drawings is to enable the medical image to be displayed with the image quality that the user is accustomed to when performing image analysis on the medical image by the trained model. That is. However, the problems to be solved by the embodiments disclosed in the present specification and the drawings are not limited to the above problems. The problem corresponding to each effect by each configuration shown in the embodiment described later can be positioned as another problem.

The medical diagnostic imaging apparatus according to the present embodiment includes an acquisition unit, a generation unit, and a display unit. The acquisition unit photographs the subject and acquires image data related to the subject. The generation unit generates a first medical image from the image data based on the first image quality condition set by the user, and sets the second image quality condition used for training the trained model for image analysis. Based on this, a second medical image is generated. The display unit can display the first medical image and the second medical image in association with each other.

The figure which shows the structure of the ultrasonic diagnostic apparatus which concerns on 1st Embodiment. The figure which shows the operation of the ultrasonic diagnostic apparatus which concerns on 1st Embodiment. The figure which shows the display format of the observation image in a single display mode. The figure which shows the display format of the image for analysis in a single display mode. The figure which shows the display format of the observation image and the analysis image in a dual display mode. The figure which shows another operation of the ultrasonic diagnostic apparatus which concerns on 1st Embodiment. The figure which shows the display format of the observation image and the analysis image in a quad display mode. The figure which shows another display format of the observation image and the analysis image in a quad display mode. The figure which shows the structure of the medical image processing apparatus which concerns on 2nd Embodiment.

Hereinafter, the medical diagnostic imaging apparatus and the medical image processing apparatus according to the present embodiment will be described with reference to the drawings. In the following embodiments, the parts with the same reference numerals perform the same operation, and duplicate description will be omitted as appropriate. Hereinafter, an ultrasonic diagnostic apparatus will be described as an example of a medical diagnostic imaging apparatus, but the present invention is not limited to this. For example, the configuration according to this embodiment can be applied to various medical image diagnostic devices such as an X-ray CT (Computed Tomography) device, an MRI (Magnetic Resonance Imaging) device, a nuclear medicine diagnostic device, and an X-ray diagnostic device.

(First Embodiment)
The configuration of the ultrasonic diagnostic apparatus according to the first embodiment will be described with reference to FIG. FIG. 1 is a block diagram showing a configuration example of the ultrasonic diagnostic apparatus 1 according to the first embodiment. As shown in FIG. 1, the ultrasonic diagnostic apparatus 1 includes a main body apparatus 10 and an ultrasonic probe 30. The main body device 10 is connected to the external device 40 via the network 100. Further, the main body device 10 is connected to the display device 50 and the input device 60.

The ultrasonic probe 30 has a plurality of ultrasonic vibrators (hereinafter, also simply referred to as elements), a matching layer provided on the element, a backing material for preventing the propagation of ultrasonic waves from the element to the rear, and the like. The ultrasonic probe 30 is detachably connected to the main body device 10. The ultrasonic probe 30 is used in contact with the subject P.

The main body device 10 shown in FIG. 1 is a device that generates an ultrasonic image based on a reflected wave signal received by the ultrasonic probe 30. As shown in FIG. 1, the main body device 10 includes an ultrasonic transmission circuit 11, an ultrasonic reception circuit 12, a B mode processing circuit 13, a Doppler processing circuit 14, a three-dimensional processing circuit 15, a display processing circuit 16, and an internal storage circuit 17. , Image memory 18 (cine memory), image database 19, input interface circuit 20, communication interface circuit 21, and control circuit 22.

The ultrasonic transmission circuit 11 is a processor that supplies a drive signal to the ultrasonic probe 30. The ultrasonic transmission circuit 11 is realized by, for example, a trigger generation circuit, a delay circuit, a pulser circuit, or the like. The trigger generation circuit repeatedly generates rate pulses for forming transmitted ultrasonic waves at a predetermined rate frequency. The delay circuit sets the transmission delay time for each element required to focus the ultrasonic waves generated from the ultrasonic probe 30 in a beam shape and determine the transmission directivity for each rate pulse generated by the trigger generation circuit. give. The pulser circuit applies a drive signal (drive pulse) to the ultrasonic probe 30 at a timing based on the rate pulse. By changing the transmission delay time given to each rate pulse by the delay circuit, the transmission direction from the element surface can be arbitrarily adjusted.

The ultrasonic wave receiving circuit 12 is a processor that performs various processes on the reflected wave signal received by the ultrasonic probe 30 to generate a received signal. The ultrasonic wave receiving circuit 12 is realized by, for example, an amplifier circuit, an A / D converter, a reception delay circuit, an adder, and the like. The amplifier circuit amplifies the reflected wave signal received by the ultrasonic probe 30 for each channel and performs gain correction processing. The A / D converter converts the gain-corrected reflected wave signal into a digital signal. The reception delay circuit provides the digital signal with the reception delay time required to determine the reception directivity. The adder adds a plurality of digital signals with a given reception delay time. The addition process of the adder generates a received signal in which the reflection component from the direction corresponding to the reception directivity is emphasized.

The B-mode processing circuit 13 is a processor that generates B-mode data based on the received signal received from the ultrasonic wave receiving circuit 12. The B mode processing circuit 13 performs envelope detection processing, logarithmic amplification processing, and the like on the received signal received from the ultrasonic reception circuit 12, and the signal strength is expressed by the brightness of the luminance (hereinafter, B mode). Data) is generated. The generated B-mode data is stored in a RAW data memory (not shown) as B-mode RAW data on the ultrasonic scanning line. The B-mode RAW data may be stored in the internal storage circuit 17 described later.

The Doppler processing circuit 14 is a processor that generates Doppler waveforms and Doppler data based on the received signal received from the ultrasonic wave receiving circuit 12. The Doppler processing circuit 14 extracts a blood flow signal from a received signal, generates a Doppler waveform from the extracted blood flow signal, and extracts information such as average velocity, dispersion, and power from the blood flow signal at multiple points. (Hereinafter, Doppler data) is generated.

The three-dimensional processing circuit 15 is a processor capable of generating two-dimensional image data or three-dimensional image data (hereinafter, also referred to as volume data) based on the data generated by the B mode processing circuit 13 and the Doppler processing circuit 14. Is. The three-dimensional processing circuit 15 generates two-dimensional image data composed of pixels by executing RAW-pixel conversion.

Further, the three-dimensional processing circuit 15 executes RAW-voxel conversion including interpolation processing in which spatial position information is added to the B-mode RAW data stored in the RAW data memory, thereby performing voxels in a desired range. Generates volume data consisting of. The three-dimensional processing circuit 15 performs rendering processing on the generated volume data to generate rendered image data. Hereinafter, the B mode RAW data, the two-dimensional image data, the volume data, and the rendered image data are collectively referred to as ultrasonic data.

The display processing circuit 16 displays the image data by executing various processing such as gain, dynamic range, and γ curve correction, and RGB conversion on the various image data generated in the three-dimensional processing circuit 15. Convert to data (eg video signal). The display processing circuit 16 generates a medical image based on the display data, and causes the display device 50 to display the generated medical image. The display processing circuit 16 may generate a user interface (GUI: Graphical User Interface) for the operator to input various instructions by the input interface circuit 20, and display the GUI on the display device 50. As the display device 50, for example, a CRT display, a liquid crystal display, an organic EL display, an LED display, a plasma display, or any other display known in the art can be appropriately used.

The internal storage circuit 17 has, for example, a magnetic or optical recording medium, a recording medium that can be read by a processor such as a semiconductor memory, and the like. The internal storage circuit 17 stores a control program for realizing ultrasonic transmission / reception, a control program for performing image processing, a control program for performing display processing, and the like. Further, the internal storage circuit 17 presets, for example, a range of diagnostic information (for example, subject ID, doctor's findings, etc.), a diagnostic protocol, a body mark generation program, and color data used for visualization for each diagnostic site. Stores data groups such as conversion tables. Further, the internal storage circuit 17 may store an anatomical chart related to the structure of an organ in a living body, for example, an atlas.

In the present embodiment, the internal storage circuit 17 stores various trained models for performing various image analyzes and image quality conditions at the time of training associated with each trained model. The trained model and the image quality condition may be stored in association with each other in, for example, a LUT (Look Up Table) format. Image quality conditions are defined by various image quality parameters such as gain, dynamic range, γ curve, and RGB related to image data. The image quality parameter is not limited to a numerical value, but includes a color setting such as a color map. Further, it is assumed that the internal storage circuit 17 stores each program for realizing various display modes that define the layout when displaying the medical image on the display device 50.

Further, the internal storage circuit 17 stores the two-dimensional image data, the volume data, and the rendered image data generated by the three-dimensional processing circuit 15 according to the storage operation input via the input interface circuit 20. The internal storage circuit 17 performs operation order and operation time for the two-dimensional image data, volume data, and rendered image data generated by the three-dimensional processing circuit 15 according to the storage operation input via the input interface circuit 20. May be memorized including. The internal storage circuit 17 can also transfer the stored data to the external device 40 via the communication interface circuit 21.

The image memory 18 has, for example, a magnetic or optical recording medium, a recording medium that can be read by a processor such as a semiconductor memory, and the like. The image memory 18 stores image data corresponding to a plurality of frames immediately before the freeze operation input via the input interface circuit 20. The image data stored in the image memory 18 is, for example, continuously displayed (cine display).

The image database 19 stores image data transferred from the external device 40. For example, the image database 19 receives and stores past medical image data regarding the same subject acquired in the past examination stored in the external device 40. Past medical image data includes ultrasonic image data, CT image data, MR image data, PET (Positron Emission Tomography) -CT image data, PET-MR image data, and X-ray image data.

The image database 19 may store desired image data by reading image data recorded on a storage medium (media) such as MO, CD-R, or DVD.

The input interface circuit 20 receives various instructions from the user via the input device 60. The input device 60 is, for example, a mouse, a keyboard, a panel switch, a slider switch, a trackball, a rotary encoder, a touch command screen (TCS), an operation panel, and the like. The input interface circuit 20 is connected to the control circuit 22 via, for example, a bus, converts an operation instruction input from an operator into an electric signal, and outputs the electric signal to the control circuit 22. In the present embodiment, the input interface circuit 20 is not limited to those connected to physical operation parts such as a mouse and a keyboard. For example, processing of an electric signal that receives an electric signal corresponding to an operation instruction input from an external input device provided separately from the ultrasonic diagnostic device 1 as a radio signal and outputs this electric signal to the control circuit 22. The circuit is also included in the example of the input interface circuit 20. For example, it may be an external input device capable of transmitting an operation instruction corresponding to an instruction by an operator's gesture as a wireless signal.

The communication interface circuit 21 is connected to the external device 40 via the network 100, and performs data communication with the external device 40. The external device 40 is, for example, a database of PACS (Picture Archiving and Communication Systems), which is a system for managing data of various medical images, a database of an electronic medical record system for managing electronic medical records to which medical images are attached, and the like. Further, the external device 40 may be various medical image diagnostic devices other than the ultrasonic diagnostic device 1 according to the present embodiment, such as an X-ray CT device, an MRI device, a nuclear medicine diagnostic device, and an X-ray diagnostic device. The communication standard with the external device 40 may be any standard, and examples thereof include DICOM (Digital Imaging and COmmunications in Medicine).

The control circuit 22 is, for example, a processor that functions as the center of the ultrasonic diagnostic apparatus 1. The control circuit 22 realizes a function corresponding to the program by executing the control program stored in the internal storage circuit 17. Specifically, the control circuit 22 executes the acquisition function 101, the generation function 102, the analysis function 103, the calculation function 104, the determination function 105, and the display function 106.

By executing the acquisition function 101, the control circuit 22 photographs the subject and acquires image data related to the subject.
By executing the generation function 102, the control circuit 22 generates a first medical image from the image data based on the first image quality condition set by the user, and trains a trained model that performs image analysis. A second medical image is generated based on the second image quality condition used.
By executing the analysis function 103, the control circuit 22 applies the trained model trained under the second image quality condition to at least one of the first medical image and the second medical image. Image analysis is performed by this.
By executing the calculation function 104, the control circuit 22 performs calculation processing on the result of the image analysis.
By executing the determination function 105, the control circuit 22 determines the display mode regarding the format for displaying the first medical image and the second medical image from the plurality of display modes.
By executing the display function 106, the control circuit 22 can display the first medical image and the second medical image in association with each other. Further, the display function 106 superimposes and displays the result of the image analysis on the medical image on which the image analysis has been performed. Further, the display function 106 superimposes and displays the result of image analysis on one medical image on the other medical image that has not been image-analyzed.

The acquisition function 101, the generation function 102, the analysis function 103, the calculation function 104, the determination function 105, and the display function 106 may be incorporated as a control program, or may be incorporated in the control circuit 22 itself or the main unit 10. A dedicated hardware circuit capable of executing the function may be incorporated.

The control circuit 22 is an integrated circuit for specific use (ASIC) incorporating these dedicated hardware circuits, a field programmable logic device (FPGA), and other complex programmable logic devices. It may be realized by (Complex Programmable Logic Device: CPLD) or a simple programmable logic device (Simple Programmable Logic Device: SPLD).

The operation of the ultrasonic diagnostic apparatus according to the first embodiment will be described with reference to FIG. FIG. 2 is a flow chart showing an operation example of the ultrasonic diagnostic apparatus 1 according to the first embodiment. It should be noted that this operation example is an operation example for displaying an observation image and an analysis image in the single display mode and the dual display mode of FIGS. 3 to 5, which will be described later.

In step S201, the control circuit 22 acquires image data of the subject. Specifically, the control circuit 22 captures the subject and acquires image data related to the subject by executing the acquisition function 101. The control circuit 22 may acquire image data stored in the image database 19. The acquired image data is two-dimensional image data, volume data, or rendered image data generated by the three-dimensional processing circuit 15, and is stored in the internal storage circuit 17 in a predetermined image format. Image formats include, for example, JPEG, PNG, GIF, SVG, TIFF, DICOM and the like. The image data may be a plurality of frames acquired in real time or a single frame.

In step S202, the control circuit 22 generates an observation image and an analysis image. Specifically, the control circuit 22 generates an observation image from the image data based on the image quality condition set by the user by executing the generation function 102, and is used for training a trained model that performs image analysis. An image for analysis is generated based on the determined image quality conditions. The control circuit 22 may, for example, receive an instruction input by the user via an input device 60 such as an operation panel and set the image quality condition of the observation image. Further, the control circuit 22 may set the image quality condition of the image for analysis based on the image quality condition at the time of training associated with the trained model stored in the internal storage circuit 17. Further, some of the image quality parameters (for example, gain) that define the image quality conditions of the analysis image may be changed by user input. When the image quality condition of the observation image is updated by the user, a new observation image may be generated based on the updated image quality condition. The generated observation image and analysis image are stored in the internal storage circuit 17 in the same image format as in step S201. At this time, the image quality condition of the observation image and the image quality condition of the analysis image are stored in the internal storage circuit 17 in association with the observation image and the analysis image, respectively.

At least one observation image and at least one analysis image may be generated. For example, with respect to the observation image, when the user sets a plurality of different image quality conditions, a plurality of observation images based on the respective image quality conditions may be generated. Regarding the analysis image, a plurality of analysis images may be generated based on the image quality conditions at the time of training of each trained model, which are associated with a plurality of different trained models.

In step S202, the control circuit 22 generates an observation image from the acquired image data based on the image quality condition set by the user, but the present invention is not limited to this. For example, in step S201, the control circuit 22 may acquire image data for an observation image based on a parameter related to transmission / reception of ultrasonic waves set by the user. As a result, an observation image may be generated based on the image data. This makes it possible to adjust the image quality of the observation image, which is difficult to adjust the image quality parameters.

In step S203, the control circuit 22 performs image analysis. Specifically, the control circuit 22 executes the analysis function 103 to perform image analysis by applying the trained model based on the generation of the analysis image in step S202 to the analysis image. .. At this time, the control circuit 22 may apply the trained model only to the region of interest set by the user on the medical image via the input device 60 such as the operation panel. The image analysis may also be performed on the observation image. That is, the control circuit 22 may perform image analysis by applying the trained model to the observation image by executing the analysis function 103. The result of the image analysis is associated with the medical image on which the image analysis has been performed and stored in the internal storage circuit 17.

Here, since the image quality condition of the image for analysis is the same as the image quality condition used for training the trained model, the accuracy of the image analysis result for the image for analysis by the trained model is guaranteed. On the other hand, the image quality conditions of the observation image may differ from the image quality conditions used for training the trained model. However, by applying the trained model to the observation image, the user can compare the result of the image analysis on the observation image with the result of the image analysis on the analysis image.

Further, the type of image analysis may be an analysis for extracting a specific structure such as a lesion, a tissue, or an organ appearing on a medical image, but the analysis is not limited to this. For example, it may be an analysis for other purposes for medical images, such as measurement of the minor axis or major axis length of a specific structure, lesion determination processing such as cancer category determination, and the like.

In step S204, the control circuit 22 determines the display mode. Specifically, the control circuit 22 determines the display mode regarding the format for displaying the observation image and the analysis image from the plurality of display modes by executing the determination function 105. The plurality of display modes include, for example, a single display mode in which an observation image and a display image are switched and displayed, and a dual display mode and a quad display mode in which the observation image and the display image are displayed in parallel. In the single display mode, one medical image is displayed, in the dual display mode, a maximum of two medical images are displayed, and in the quad display mode, a maximum of four medical images are displayed.

The control circuit 22 may, for example, receive an instruction input by the user via an input device 60 such as an operation panel to determine the display mode, or the display mode set in advance as an initial setting (default). May be determined.

In step S205, the control circuit 22 displays an image for observation and an image for analysis. Specifically, the control circuit 22 can display the observation image and the analysis image in association with each other by executing the display function 106. At this time, the observation image and the analysis image are displayed on the display device 50 based on each layout defined for each display mode. Further, the result of the image analysis in step S203 is superimposed and displayed on the observation image and the analysis image. For example, the result of the image analysis may be superimposed and displayed on the medical image on which the image analysis has been performed. Further, the result of image analysis on one medical image may be superimposed and displayed on the other medical image that has not been image-analyzed.

Hereinafter, as a result of the ultrasonic diagnostic apparatus 1 executing the series of operations of steps S201 to S205, the display formats of the observation image and the analysis image in the single display mode and the dual display mode are shown in FIGS. 3 to 5. As a trained model to be applied, a trained model for extracting a specific structure will be described as an example. Further, the observation image 310 and the analysis image 320, which are examples of the observation image and the analysis image, have different image quality conditions based on the same image data acquired by ultrasonically photographing the same phantom. It was generated. In reality, the observation image 310 and the analysis image 320 are displayed with the frames updated in real time.

The display format of the observation image in the single display mode will be described with reference to FIG. FIG. 3 is a schematic diagram showing a display example of an observation image in the single display mode.
In FIG. 3, the observation image 310 is displayed on the display device 50. Specifically, the detection region 322, which is the result of applying the trained model in the region of interest 321 of the analysis image 320 of FIG. 4, is superimposed and displayed as the detection region 312 on the observation image 310 that has not been analyzed. .. The region of interest 311 of the observation image 310 and the region of interest 321 of the analysis image 320 are set to be the same. Here, the detection region 312 is the result of extracting a circular structure, and is represented by a frame surrounding the extracted structure. Further, as a part of the image quality parameters set by the user, for example, the gain "G: 80" and the dynamic range "DR: 70" are displayed in the vicinity of the observation image 310. Further, while the observation image 310 is being displayed, the user presses, for example, a switching button provided on the input device 60 such as an operation panel, so that the ultrasonic diagnostic apparatus 1 can display the analysis image 320 shown in FIG. Switch and display.

The display format of the analysis image in the single display mode will be described with reference to FIG. FIG. 4 is a schematic diagram showing a display example of an analysis image in the single display mode.
In FIG. 4, the analysis image 320 is displayed on the display device 50. Specifically, the detection region 322, which is the result of applying the trained model in the region of interest 321 of the analysis image 320, is superimposed and displayed on the analysis image 320 that has been analyzed. Further, the gain "G: 80" is set as a part of the image quality parameter set by the user, and the dynamic range "DR: 80" is set as a part of the image quality parameter automatically fixed for analysis. Displayed alongside the neighborhood. Further, while the analysis image 320 is being displayed, the user presses, for example, a switching button provided on the input device 60 such as an operation panel, so that the ultrasonic diagnostic apparatus 1 can display the observation image 310 shown in FIG. Switch and display.

As described above, in the single display mode, one of the medical images is displayed on the display device by switching between the observation image and the analysis image. Therefore, one medical image can be displayed using a relatively large area on the screen. Further, when the user has doubts about the analysis result, he / she can easily confirm how the analysis image is drawn by switching from the observation image to the analysis image and displaying the image. Further, the detection area displayed on the analysis image is an analysis result based on the analysis image with image quality conditions suitable for applying the trained model. Therefore, the highly accurate detection area detected on the analysis image can be superimposed and displayed on the observation image that is easy for the user to see. Furthermore, if the observation image and the analysis image are displayed at the same position on the screen with the same size, the difference in image quality between the observation image and the analysis image when switching, and the position and size of the detection area are displayed. Differences can be easily compared. Further, by displaying some image quality parameters that define the image quality conditions of the observation image and the analysis image as numerical values, it is possible to quantitatively compare the difference in image quality.

The display formats of the observation image and the analysis image in the dual display mode will be described with reference to FIG. FIG. 5 is a schematic diagram showing a display example of an observation image and an analysis image in the dual display mode.
In FIG. 5, the observation image 310 and the analysis image 320 are displayed on the display device 50. Specifically, the screen configuration including the observation image 310 shown in FIG. 3 is displayed in parallel on the left side, and the screen configuration including the analysis image 320 shown in FIG. 4 is displayed in parallel on the right side. In FIG. 5, the observation image 310 and the analysis image 320 are adjacent to each other on one side, and the accompanying information including the image quality parameter is arranged on the opposite side to the adjacent portion. The observation image 310 and the analysis image 320 may be interchanged on the left and right sides of each other, or may be displayed side by side in the vertical direction.

In FIG. 5, the detection area 312 of the observation image 310 is set to be hidden, but the present invention is not limited to this. For example, the region of interest 311 of the observation image 310 may be hidden.

As described above, in the dual display mode, the observation image and the analysis image are displayed in parallel, so that the two medical images are displayed on the display device. That is, two medical images can be displayed on the same screen. Further, it is possible to easily confirm how the analysis image and the analysis image are drawn on the same screen without switching between the observation image and the analysis image. Further, by setting the interest area and the detection area of the observation image to be hidden, the user can easily read the observation image. Further, it is possible to easily compare the difference in image quality between the observation image and the analysis image on the same screen, and the difference in the position and size of the detection area. Further, by displaying some image quality parameters that define the image quality conditions of the observation image and the analysis image as numerical values, it is possible to quantitatively compare the difference in image quality.

Another operation of the ultrasonic diagnostic apparatus according to the first embodiment will be described with reference to FIG.
FIG. 6 is a flow chart showing another operation example of the ultrasonic diagnostic apparatus according to the first embodiment. It should be noted that this operation example is an operation example for displaying an observation image and an analysis image in the quad display mode of FIGS. 7-8, which will be described later.

In step S601, the control circuit 22 acquires image data of the subject. The operation of step S601 is the same as the operation of step S201.

In step S602, the control circuit 22 generates an observation image and an analysis image. Specifically, the control circuit 22 generates a single observation image and a plurality of analysis images.

In step S603, the control circuit 22 performs image analysis. Specifically, the control circuit 22 performs image analysis on each of the plurality of analysis images.

In step S604, the control circuit 22 performs arithmetic processing. Specifically, the control circuit 22 executes the calculation function 104 to integrate the results of each image analysis. As the calculation process, for example, the logical product (AND) or the logical sum (OR) of the results of each image analysis may be calculated. Further, it may be calculated whether or not the analysis accuracy of the result of each image analysis is equal to or higher than the threshold value. The result of the arithmetic processing is stored in the internal storage circuit 17.

In step S605, the control circuit 22 determines the display mode. The operation of step S605 is the same as the operation of step S204.

In step S606, the control circuit 22 displays an image for observation and an image for analysis. Further, the control circuit 22 superimposes and displays the calculated result on a single observation image. Further, the control circuit 22 may superimpose and display the result of the image analysis whose analysis accuracy is equal to or higher than the threshold value on a single observation image.

The display formats of the observation image and the analysis image in the quad display mode will be described with reference to FIG. 7. FIG. 7 is a schematic diagram showing a display example of an observation image and an analysis image in the quad display mode. In the quad display mode shown in FIGS. 7-8, it is assumed that one observation image is generated and three analysis images are generated. In addition, the image quality conditions of each analysis image and the trained model to be applied shall be different for each analysis image. Even if the total number of the observation image and the analysis image is less than four, it can be displayed by the quad display mode, but in this case, nothing may be displayed in the area where the medical image does not exist.

In FIG. 7, the observation image 310 is displayed in the upper left, the analysis image 320 is displayed in the upper right, the analysis image 330 is displayed in the lower right, and the analysis image 340 is displayed in the lower left. In each analysis image, the detection area is superimposed and displayed as a result of performing different image analysis for extracting different structures. Specifically, in the analysis image 320, the result of applying the trained model for detecting the circular structure to the region of interest 321 is superimposed and displayed as the detection region 322. In the analysis image 330, the result of applying the trained model for detecting the rectangular structure to the region of interest 331 is superimposed and displayed as the detection region 332. In the analysis image 340, the result of applying the trained model for detecting the triangular structure to the region of interest 341 is superimposed and displayed as the detection region 342. The logical sum (OR) of each result of these different image analyzes is calculated and superimposed and displayed on the observation image 310. The region of interest 311 of the observation image 310, the region of interest 321 of the analysis image 320, the region of interest 331 of the analysis image 330, and the region of interest 341 of the analysis image 340 are all set to be the same.

The detection area 322, the detection area 332, and the detection area 342 may be displayed separately according to different colors or different border forms (for example, solid line, broken line, dotted line), and are superimposed on the observation image 310. It is only necessary to be able to identify when.

Another display format of the observation image and the analysis image in the quad display mode will be described with reference to FIG. FIG. 8 is a schematic diagram showing another display example of the observation image and the analysis image in the quad display mode. Note that, in FIG. 7, different trained models for extracting different structures are applied, but in FIG. 8, different trained models for extracting the same structure are applied.

In FIG. 8, the observation image 310 is displayed in the upper left, the analysis image 320 is displayed in the upper right, the analysis image 330 is displayed in the lower right, and the analysis image 340 is displayed in the lower left. In each analysis image, the detection area is superimposed and displayed as a result of performing different image analysis for extracting the same structure. Specifically, in the analysis image 320, the result of applying the trained model for detecting the circular structure to the region of interest 321 is superimposed and displayed as the detection region 322. In the analysis image 330, the result of applying the trained model for detecting the circular structure to the region of interest 331 is superimposed and displayed as the detection region 332. In the analysis image 340, the result of applying the trained model for detecting the circular structure to the region of interest 341 is superimposed and displayed as the detection region 342. The logical product (AND) of each result of these different image analyzes is calculated and displayed superimposed on the observation image 310 as the detection area 312.

The detection area 312 of the observation image 310 is an AND of the results of the three analysis methods, but is not limited to this. For example, among the overlapping regions of each detection region, the region in which the accuracy of including the structure in the results of all the analysis methods is equal to or higher than a predetermined threshold value may be defined as the AND region. The threshold may be selected from presets. Further, the method of taking AND may be softened by including a region exceeding the threshold value in two of the three analysis methods. Further, the user may arbitrarily select the analysis method to be ANDed from the three analysis methods while comparing the analysis results of the three analysis methods.

In this way, in the quad display mode, by displaying the observation image and the analysis image in parallel, four medical images are displayed on the display device. That is, four medical images can be displayed on the same screen. Further, it is possible to easily confirm how the analysis image and the analysis image are drawn on the same screen without switching between the observation image and the analysis image. Further, it is possible to easily compare the difference in image quality between the observation image and the analysis image on the same screen, and the difference in the position and size of the detection area. Further, by displaying some image quality parameters that define the image quality conditions of the observation image and the analysis image as numerical values, it is possible to quantitatively compare the difference in image quality. Further, the results of different analyzes performed on each of the plurality of analysis images can be calculated and displayed by superimposing them on one observation image. As a result, highly accurate calculation results can be superimposed and displayed on the observation image that is easy for the user to see.

(Other variants)
The display formats of the observation image and the analysis image in each display mode have been shown above, but the present invention is not limited to this. For example, as image quality parameters other than gain and dynamic range, a γ curve, RGB, and the like may be displayed along with the observation image and the analysis image. This allows the user to compare the difference in image quality conditions between the observation image and the analysis image on the screen of the display device together with the medical image.

Further, by displaying the observation image and the color bar corresponding to the analysis image incidentally, it is possible to compare the difference in image quality conditions regarding the color map of the observation image and the analysis image. Further, if some image quality parameters that are not displayed on the display device are displayed on the operation panel, the difference in image quality conditions between the observation image and the analysis image due to the image quality parameters can be compared on the operation panel.

The region of interest of the observation image and the region of interest of the analysis image are not the same and may be different. In this case, the region of interest of the analysis image may be set within a range including the region of interest of the observation image. As a result, it is possible to perform image analysis on the analysis image at least for the region of interest of the observation image set by the user.

It should be noted that each display mode including the single display mode, dual display mode, and quad display mode may be switched by pressing a switching button provided on an input device such as an operation panel, for example. Of course, a display mode for displaying more than four images may be set.

According to the first embodiment described above, the observation image can be observed under the image quality conditions and layout desired by the user. Further, by displaying the observation image and the analysis image in association with each other, it is possible to compare the difference in the image quality and the analysis result of each medical image. Further, since the image quality condition of the image for analysis is fixed to one type, the accuracy needs to be verified only for the image quality condition, and the workload related to the accuracy verification can be minimized. There may be a plurality of types of image quality conditions for the analysis image within the range in which the accuracy of the analysis can be verified.

(Second embodiment)
The configuration of the medical image processing apparatus according to the second embodiment will be described with reference to FIG. In the ultrasonic diagnostic apparatus according to the first embodiment, an observation image and an analysis image are simultaneously generated and displayed based on the image data obtained by ultrasonically photographing the subject. The medical image processing apparatus according to the second embodiment is different in that it does not have a mechanism for photographing a subject and acquires image data from an external device. The medical image processing apparatus according to the second embodiment is assumed to be included in an analysis apparatus such as a workstation, but may be mounted on another apparatus. Other than that, it is the same as the ultrasonic diagnostic apparatus 1.

FIG. 9 is a block diagram showing a configuration example of the medical image processing apparatus 9 according to the second embodiment. The medical image processing device 9 includes a control circuit 90, a memory 91, an input interface circuit 92, a communication interface circuit 93, a display device 94, an input device 95, a network 96, and an external device 97. It should be noted that each component of the medical image processing device 9 shall perform the same function as each component of the ultrasonic diagnostic device 1, and duplicated description will be omitted as appropriate.

The control circuit 90 is, for example, a processor that functions as the center of the medical image processing device 9. The control circuit 90 realizes a function corresponding to the control program stored in the memory 91 by executing the control program. Specifically, the control circuit 90 executes the acquisition function 901, the generation function 902, the analysis function 903, the calculation function 904, the determination function 905, and the display function 906.

By executing the acquisition function 901, the control circuit 22 acquires image data relating to the subject.
By executing the generation function 902 (corresponding to the generation function 102), the control circuit 22 generates a first medical image from the image data based on the first image quality condition set by the user, and performs image analysis. A second medical image is generated based on the second image quality condition used for training the trained model to be performed.
By executing the analysis function 903 (corresponding to the analysis function 103), the control circuit 22 is trained on at least one of the first medical image and the second medical image under the second image quality condition. Image analysis is performed by applying the trained model.
By executing the calculation function 904 (corresponding to the calculation function 104), the control circuit 22 performs calculation processing on the result of the image analysis.
By executing the determination function 905 (corresponding to the determination function 105), the control circuit 22 determines the display mode relating to the format for displaying the first medical image and the second medical image from the plurality of display modes.
By executing the display function 906 (corresponding to the display function 106), the control circuit 22 makes it possible to display the first medical image and the second medical image in association with each other. Further, the display function 906 superimposes and displays the result of the image analysis on the medical image on which the image analysis has been performed. Further, the display function 906 superimposes and displays the result of image analysis on one medical image on the other medical image that has not been image-analyzed.

The memory 91 (corresponding to the internal storage circuit 17) stores each program for realizing each function of the control circuit 90. Further, the image data acquired from the external device 97 (corresponding to the external device 40) via the network 96 (corresponding to the network 100) is stored.

The input interface circuit 92 (corresponding to the input interface circuit 20) receives various instructions from the user via the input device 95 (corresponding to the input device 60).

The communication interface circuit 93 (corresponding to the communication interface circuit 21) is connected to the external device 97 via the network 96 (corresponding to the network 100), and performs data communication with the external device 97. The external device 97 stores image data.

The display device 94 (corresponding to the display device 50) displays a medical image.

According to the second embodiment described above, an observation image and an analysis image can be simultaneously generated based on the image data obtained from the external device. In addition, the observation image can be observed under the image quality conditions and layout desired by the user. Further, by displaying the observation image and the analysis image in association with each other, it is possible to compare the difference in the image quality and the analysis result of each medical image. Further, since the image quality condition of the image for analysis is fixed to one type, the accuracy needs to be verified only for the image quality condition, and the workload related to the accuracy verification can be minimized. There may be a plurality of types of image quality conditions for the analysis image within the range in which the accuracy of the analysis can be verified.

According to at least one embodiment described above, when the image analysis by the trained model is performed on the medical image, the medical image can be displayed with the image quality that the user is accustomed to.

Although some embodiments have been described, these embodiments are presented as examples and are not intended to limit the scope of the invention. These embodiments can be implemented in various other embodiments, and various omissions, replacements, changes, and combinations of embodiments can be made without departing from the gist of the invention. These embodiments and variations thereof are included in the scope of the invention described in the claims and the equivalent scope thereof, as are included in the scope and gist of the invention.

1 ... Ultrasonic diagnostic device, 9 ... Medical image processing device, 10 ... Main device, 11 ... Ultrasonic transmission circuit, 12 ... Ultrasonic reception circuit, 13 ... B mode processing Circuit, 14 ... Doppler processing circuit, 15 ... Dimension processing circuit, 16 ... Display processing circuit, 17 ... Internal storage circuit, 18 ... Image memory, 19 ... Image database, 20, 92 ... Input interface circuit, 21,93 ... Communication interface circuit, 22, 90 ... Control circuit, 30 ... Ultrasonic probe, 40, 97 ... External device, 50, 94 ... Display device, 60, 95 ... Input device, 91 ... Memory, 96, 100 ... Network, 101, 901 ... Acquisition function, 102, 902 ... Generation function, 103, 903 ... Analysis function, 104,904 ... Calculation function, 105,905 ... Decision function, 106,906 ... Display function, 310 ... Observation image, 311, 3211, 331, 341 ... Area of interest , 312,322,332,342 ... Detection area, 320,330,340 ... Image for analysis

Claims (13)

An acquisition unit that photographs a subject and acquires image data related to the subject,
From the image data, a first medical image is generated based on a first image quality condition set by a user, and a second image quality condition is used for training a trained model for image analysis. And the generator that generates the medical image of
A display unit capable of displaying the first medical image and the second medical image in association with each other.
A medical diagnostic imaging device equipped with. The medical image according to claim 1, wherein the generation unit generates a new first medical image based on the updated first image quality condition when the first image quality condition is updated by the user. Diagnostic device. The medical image diagnostic apparatus according to claim 1 or 2, further comprising a determination unit for determining a display mode relating to a format for displaying the first medical image and the second medical image from a plurality of display modes. When the determination unit determines the first display mode in which the first medical image and the second medical image are switched and displayed among the plurality of display modes, the display unit determines.
The medical image diagnostic apparatus according to claim 3, wherein the first medical image and the second medical image are switched and displayed in response to an image switching instruction from the user. When the determination unit determines a second display mode for displaying the first medical image and the second medical image in parallel among the plurality of display modes, the display unit determines.
The medical image diagnostic apparatus according to claim 3, wherein the first medical image and the second medical image are displayed in parallel. An analysis unit that performs image analysis by applying the trained model trained under the second image quality condition to at least one of the first medical image and the second medical image. Further equipped,
The medical image diagnostic apparatus according to any one of claims 1 to 5, wherein the display unit superimposes and displays the result of the image analysis on the medical image on which the image analysis has been performed. Further, an analysis unit that performs image analysis by applying the trained model trained under the second image quality condition to either one of the first medical image and the second medical image. Equipped with
The medical image diagnostic apparatus according to any one of claims 1 to 5, wherein the display unit superimposes and displays the result of the image analysis on the other medical image on which the image analysis has not been performed. Further, it is provided with a calculation unit that performs calculation processing on the result of the image analysis.
The generation unit generates a single number of the first medical images and a plurality of the second medical images.
The analysis unit performs image analysis on each of the plurality of second medical images, and then performs image analysis.
The arithmetic unit performs arithmetic processing that integrates the results of each image analysis.
The medical image diagnostic apparatus according to claim 6 or 7, wherein the display unit superimposes and displays the result calculated by the calculation unit on the first medical image of the singular number. The display unit calculates and processes the logical product or the logical sum of the results of the respective image analysis by the arithmetic unit, and superimposes and displays the logical product or the logical sum on the unilateral first medical image. Item 8. The medical diagnostic imaging apparatus according to Item 8. The display unit calculates whether or not the analysis accuracy of the result of each of the image analyzes is equal to or higher than the threshold value by the calculation unit, and the result of the image analysis equal to or higher than the threshold value is the single first medical image. The medical image diagnostic apparatus according to claim 8, which is superimposed and displayed on the above. One of claims 1 to 10, wherein the first image quality condition and the second image quality condition are defined by an image quality parameter including at least one of gain, dynamic range, γ curve, and RGB. The medical diagnostic imaging apparatus described in the section. One of claims 1 to 11, wherein the first image quality condition and the second image quality condition are stored in association with the first medical image and the second medical image, respectively. The medical diagnostic imaging apparatus described in. The acquisition unit that acquires image data related to the subject, and
From the image data, a first medical image is generated based on a first image quality condition set by a user, and a second image quality condition is used for training a trained model for image analysis. And the generator that generates the medical image of
A display unit that displays at least one of the first medical image and the second medical image in association with each other.
A medical image processing device comprising.

Images