JP7304150B2 - Image analysis device, diagnostic imaging device, and ROI setting program - Google Patents

Description

Embodiments of the present invention relate to an image analysis device, an image diagnosis device, a model learning device, and a ROI setting program.

Ultrasound diagnostic apparatuses have a function of evaluating the state of at least one organ based on the relationship between feature amounts in regions of interest (ROI) set for a plurality of organs in an ultrasound image, for example. Specifically, in a B-mode image including a plurality of organs such as the liver, the degree of hepatitis, fatty liver, etc. is evaluated from the relationship between the luminance values in the ROI set for the plurality of organs including at least the liver. I have an application.

When executing the evaluation function, an operator such as a doctor needs to set an ROI with an appropriate size at an appropriate position of the organ included in the image. However, it takes time and effort to appropriately set the ROI for the organs included in the image each time the evaluation function is executed.

Japanese Patent No. 5619191

A problem to be solved by the invention is to reduce the trouble of setting a region of interest in a medical image.

According to an embodiment, an image analysis device includes a setting section. The setting unit inputs medical image data including a predetermined organ in a preset region to a trained model, and sets a region of interest for the organ in the medical image data based on an output result from the trained model. .

FIG. 1 is a block diagram showing the functional configuration of a system in a hospital provided with an image analysis apparatus according to the first embodiment. FIG. 2 is a block diagram showing the functional configuration of the image analysis apparatus shown in FIG. 1. As shown in FIG. FIG. 3 is a diagram showing the configuration of a medical information processing system that generates trained models. FIG. 4 is a diagram showing a medical image treated as an input by the model learning device shown in FIG. FIG. 5 is a diagram showing ROI position information treated as a correct answer output by the model learning apparatus shown in FIG. FIG. 6 is a flow chart showing the operation when the processing circuit shown in FIG. 2 executes the ROI setting process using the learned model. FIG. 7 is a diagram showing a B-mode image input to a trained model and points based on coordinate information output from the trained model. FIG. 8 is a diagram representing a B-mode image with two circular ROIs placed thereon. FIG. 9 is a diagram showing a modification of the flowchart shown in FIG. FIG. 10 is a diagram representing movement of a circular ROI by processing circuitry. FIG. 11 is a diagram showing ROI position information treated as correct output by the model learning device in the second embodiment. FIG. 12 is a flow chart showing the operation when the processing circuit according to the second embodiment executes the ROI setting process using the learned model. FIG. 13 is a diagram showing a B-mode image input to a trained model and points based on coordinate information output from the trained model. FIG. 14 is a diagram showing a B-mode image on which the first circular ROI is placed. FIG. 15 is a diagram representing a B-mode image with two second circular ROIs placed thereon. FIG. 16 is a diagram showing ROI position information treated as correct output by the model learning device in the third embodiment. FIG. 17 is a flow chart showing the operation when the processing circuit according to the third embodiment executes the ROI setting process using the learned model. FIG. 18 is a diagram showing points based on a B-mode image input to a trained model and coordinate information output from the trained model. FIG. 19 depicts a B-mode image with a circular ROI placed within the kidney. FIG. 20 is a diagram representing a B-mode image in which two circular ROIs are placed. FIG. 21 is a block diagram showing the functional configuration of an ultrasonic diagnostic apparatus as a medical image diagnostic apparatus.

Embodiments will be described below with reference to the drawings.

(First embodiment)
FIG. 1 is a block diagram showing an example of the functional configuration of a system in a hospital provided with an image analysis apparatus according to the first embodiment. The system shown in FIG. 1 includes an image analysis device 10 , a medical image diagnostic device 20 , a medical image archiving and communication system (PACS) 30 , and a communication terminal 40 . The image analysis apparatus 10, the medical image diagnostic apparatus 20, the medical image management system 30, and the communication terminal 40 are connected via an intra-hospital network such as a LAN (Local Area Network) so as to be capable of data communication. At this time, the connection to the intra-hospital network may be wired connection or wireless connection.

Although FIG. 1 shows an example in which the image analysis apparatus 10, the medical image diagnostic apparatus 20, the medical image management system 30, and the communication terminal 40 are connected to an intra-hospital network, the present invention is not limited to this. The line to be connected is not limited to the intra-hospital network as long as security is ensured. For example, it may be connected to a public communication line such as the Internet via a VPN (Virtual Private Network) or the like.

The medical image diagnostic apparatus 20 is an apparatus that generates medical image data by photographing a subject. The medical image diagnostic device 20 includes, for example, an X-ray diagnostic device, an X-ray CT device, an MRI device, an ultrasonic diagnostic device, a SPECT (Single Photon Emission Computed Tomography) device, a PET (Positron Emission computed Tomography) device, a SPECT device and an X A SPECT-CT device integrated with a line CT device, a PET-CT device integrated with a PET device and an X-ray CT device, a PET-MRI device integrated with a PET device and an MRI device, or these and the like.

Upon receiving the imaging order, the medical image diagnostic apparatus 20 images the patient based on the received imaging order and patient information about the patient to be imaged, and generates medical image data. The medical image data may be data after reconstruction, or may be raw data before reconstruction. The medical image diagnostic apparatus 20 converts the generated medical image data into an image file conforming to the DICOM (Digital Imaging and Communication Medicine) standard, for example. The image file includes medical image data and incidental information added to the medical image data. The incidental information is information for managing medical image data, and includes, for example, examination UID, examination date, examination time, patient ID, and the like. The study UID is an identifier that uniquely identifies the study.

The medical image management system 30 is a system for managing medical image data. The medical image data managed by the medical image management system 30 includes, for example, medical image data generated by the medical image diagnostic apparatus 20 and medical image data provided from the outside.

The communication terminal 40 is a terminal such as a viewer for medical staff to access systems and devices connected to the LAN. Although FIG. 1 shows a case where the communication terminal 40 does not belong to the medical image management system 30 and is not directly connected to the image analysis apparatus 10, the present invention is not limited to this. The communication terminal 40 may belong to the medical image management system 30 or may be directly connected to the image analysis apparatus 10 as long as it can be used by the medical staff.

The image analysis device 10 is a device that analyzes medical image data. For example, the image analysis apparatus 10 sets a region of interest (ROI) in medical image data generated by the medical image diagnostic apparatus 20 or medical image data managed by the medical image management system 30, and A patient's condition is evaluated based on the feature amount.

FIG. 2 is a block diagram showing an example of the functional configuration of the image analysis apparatus 10 shown in FIG. 1. As shown in FIG. The image analysis device 10 shown in FIG. 2 has a processing circuit 11 , a memory 12 and a communication interface 13 . The processing circuit 11, the memory 12, and the communication interface 13 are communicably connected to each other via a bus, for example.

The processing circuit 11 is a processor that functions as the core of the image analysis apparatus 10 . The processing circuit 11 implements a function corresponding to the program by executing the program stored in the memory 12 or the like.

The memory 12 is a storage device such as a ROM (Read Only Memory), a RAM (Random Access Memory), an HDD (Hard Disk Drive), an SSD (Solid State Drive), and an integrated circuit storage device that stores various information. Also, the memory 12 may be a drive device or the like that reads and writes various information from/to a portable storage medium such as a CD-ROM drive, a DVD drive, and a flash memory. Note that the memory 12 does not necessarily have to be realized by a single storage device. For example, the memory 12 may be realized by multiple storage devices. Alternatively, the memory 12 may reside in another computer connected to the image analysis apparatus 10 via a network.

The memory 12 stores an ROI setting program, a state evaluation program, and the like according to this embodiment. Note that these programs may be stored in advance in the memory 12, for example. Alternatively, for example, it may be stored in a non-transitory storage medium and distributed, read from the non-transitory storage medium and installed in the memory 12 .

The memory 12 also stores a learned model 121 as a discriminator generated by machine learning, for example. The learned model 121 is an example of a computational model. In this embodiment, a trained model represents a model generated by causing a machine learning model to perform machine learning according to a model learning program. The trained model 121 is, for example, a model for setting ROI based on medical images, and is generated as follows and stored in the memory 12 of the image analysis apparatus 10 .

FIG. 3 is a schematic diagram showing a configuration example of a medical information processing system that generates a trained model 121. As shown in FIG. The medical information processing system shown in FIG. 3 has an image analysis device 10 , a learning data storage device 50 and a model learning device 60 . The image analysis device 10, the learning data storage device 50, and the model learning device 60 are communicably connected via a cable or a communication network, for example.

The learning data storage device 50 stores learning data including multiple learning samples. For example, the learning data storage device 50 is a computer with a built-in mass storage device. The learning data storage device 50 may also be a mass storage device communicatively connected to a computer via a cable or communication network. As the storage device, an HDD (Hard Disk Drive), an SSD (Solid State Drive), an integrated circuit storage device, or the like can be used as appropriate.

The model learning device 60 generates a trained model 121 by causing the machine learning model to perform machine learning according to the model learning program based on the learning data stored in the learning data storage device 50 . In this embodiment, machine learning algorithms include discriminant analysis, logistic regression, support vector machines, neural networks, randomized trees, subspace methods, and the like. The model learning device 60 is, for example, a computer such as a workstation having processors such as a CPU (Central Processing Unit) and a GPU (Graphics Processing Unit).

The trained model 121 may be supplied to the image analysis device 10 at any time after the image analysis device 10 is manufactured. For example, it may be at any point between manufacturing and installation in a medical facility or the like, or during maintenance. The supplied trained model 121 is stored in the memory 12 of the image analysis device 10 .

The trained model 121 according to the present embodiment is, for example, a parameterized synthesis function obtained by synthesizing a plurality of functions for inputting medical image data including a region representing a plurality of organs and outputting a position for setting an ROI. be. A parameterized composite function is defined by a combination of multiple adjustable functions and parameters. The trained model 121 according to this embodiment may be any parameterized composite function that satisfies the above requirements.

A parameterized composite function changes its form as a function depending on how the parameters are chosen. For example, if the trained model 121 is a network that has a structure in which only adjacent layers arranged in layers are connected and information propagates in one direction from the input layer side to the output layer side, that is, a forward propagation type multi-layered network. , by appropriately setting the constituent parameters, it is possible to define a function capable of outputting favorable results from the output layer.

The parameters are set by performing learning using learning data and an error function. The learning data is, for _example , a set _D (n ₌ 1 _, . is. The error function is a function that expresses the closeness between the output from the multi-layered network to which _xn is input and the correct output _dn . For the parameter, a value that minimizes the error function, for example, is determined for each learning sample. In addition, error backpropagation may be used in order to reduce the amount of calculation when determining the parameters.

More specifically, the model learning device 60 according to the present embodiment receives, for example, medical image data including a predetermined organ in a preset region, and sets for the organ included in the medical image data. Machine learning is performed based on the learning data in which the positional information of the ROI is output as the correct answer.

FIG. 4 is a diagram showing an example of a medical image based on medical image data handled as input by the model learning device 60 shown in FIG. In the medical image shown in FIG. 4, the upper left area of the screen contains the liver, and the lower right area contains the kidneys. Note that the types and arrangement of organs included in the medical image are not limited to those described in FIG. 4 as long as they are common in the input of learning data. For example, liver and spleen, or liver, kidney, and spleen.

FIG. 5 is a diagram showing an example of ROI position information treated as correct output by the model learning device 60 shown in FIG. In FIG. 5, coordinates of eight points are set as ROI position information for the image shown in FIG. In FIG. 5, points P1-P4 are the vertices of a square inscribed by a circular ROI placed at appropriate positions in the intrahepatic region. Also, points P5 to P8 are vertices of a square inscribed by a circular ROI placed at an appropriate position in the intrarenal region. Note that the ROI position information is not limited to the coordinates of the vertices of the square inscribed by the circular ROI as long as the ROI position can be specified and the correct output of the learning data is common. For example, it may be the center coordinates and the radius of a circular ROI. Alternatively, the coordinates of each vertex of a square circumscribing the circular ROI may be used.

The model learning device 60 generates a trained model 121 that determines the position of the ROI based on the training data, for example, based on the input medical image data.

The communication interface 13 shown in FIG. 2 performs data communication with the medical image diagnostic apparatus 20 and the medical image management system 30 connected via the hospital network. The communication interface 13 performs data communication in accordance with, for example, a preset known standard such as DICOM.

Note that the image analysis apparatus 10 may have an input interface. The input interface receives various input operations from the user, converts the received input operations into electrical signals, and outputs the electrical signals to the processing circuit 11 . The input interface is connected to input devices such as a mouse, keyboard, trackball, switch, button, joystick, touch pad, and touch panel for inputting instructions by touching the operation surface. Also, the input device connected to the input interface may be an input device provided in another computer connected via a network or the like.

Further, the image analysis device 10 may have a display. The display displays various information according to instructions from the processing circuit 11 . Also, the display may display a GUI (Graphical User Interface) or the like for receiving various operations from the user. Any display such as a CRT (Cathode Ray Tube) display, a liquid crystal display, an organic EL display, an LED display, a plasma display, or the like can be appropriately used as the display.

The processing circuit 11 shown in FIG. 2 executes a ROI setting program, a state evaluation program, and the like stored in the memory 12, thereby realizing functions corresponding to the programs. For example, the processing circuitry 11 has an ROI setting function 111 by executing an ROI setting program. The processing circuit 11 also has an analysis function 112 and a display control function 113 by executing a state evaluation program. In this embodiment, a case where the ROI setting function 111, the analysis function 112, and the display control function 113 are realized by a single processor will be described, but the present invention is not limited to this. For example, the ROI setting function 111, the analysis function 112, and the display control function 113 may be realized by configuring a processing circuit by combining a plurality of independent processors and executing programs by each processor.

The ROI setting function 111 is a function of setting an ROI for medical image data using the learned model 121 with medical image data as an input, and is an example of a setting unit. Specifically, for example, in the ROI setting function 111 , the processing circuit 11 inputs medical image data specified by the operator to the learned model 121 stored in the memory 12 . Medical image data includes, for example, ultrasound image data, CT (Computed Tomography) image data, X-ray image data, MRI (Magnetic Resonance Imaging) image data, and PET (Positron Emission computed Tomography) image data. The processing circuit 11 sets the ROI for the medical image data based on the ROI position information for the medical image data output from the trained model 121 .

The analysis function 112 is a function for evaluating the state of at least one organ included in the medical image data from the relationship of feature amounts within the ROI.

The display control function 113 is a function for controlling the display of the evaluated organ states. Specifically, for example, the processing circuit 11 in the display control function 113 generates image data for displaying the state of the organ evaluated by the analysis function 112 . The processing circuit 11 also displays a GUI for instructing change of the reason for identification.

Next, the ROI setting operation by the image analysis apparatus 10 configured as described above will be described according to the processing procedure of the processing circuit 11. FIG.
FIG. 6 is a flow chart showing an example of the operation when the processing circuit 11 shown in FIG. 2 executes the ROI setting process using the trained model 121. FIG. Note that in the description of FIG. 6, a case where medical image data is B-mode image data generated by an ultrasonic diagnostic apparatus will be described as an example. At this time, the trained model 121 is assumed to be generated based on learning data that receives B-mode image data as input and correct output is position information that defines an ROI on a B-mode image based on this B-mode image data. do. The B-mode image used as input contains the liver in the upper left region and the kidney in the lower right region. Also, the positional information defining the ROI on the B-mode image is expressed as the coordinates of the vertices of the squares inscribed by the circular ROI placed at appropriate positions in the liver and kidney regions.

First, an operator who operates the communication terminal 40 designates B-mode image data, and inputs an instruction to activate an application for evaluating the state of the liver, for example, with respect to the designated B-mode image data. When the activation instruction is input, the processing circuit 11 of the image analysis apparatus 10 reads the ROI setting program from the memory 12 and executes the read ROI setting program. When the processing circuit 11 executes the ROI setting program, the processing shown in FIG. 6 is started.

In FIG. 6, processing circuitry 11 performs a ROI setting function 111 . When the ROI setting function 111 is executed, the processing circuit 11 reads the B-mode image data specified by the operator from the medical image diagnostic apparatus 20 or the medical image management system 30, and inputs it to the read-out learned model 121 (step S61). From the learned model 121, the coordinates of eight points based on the input B-mode image data are output as ROI position information (step S62).

FIG. 7 is a diagram showing an example of a B-mode image based on B-mode image data input to the trained model 121 and an example of points based on coordinate information output from the trained model 121. FIG. The left diagram of FIG. 7 represents a B-mode image acquired using, for example, a convex probe and input to the trained model 121 . In the B-mode image shown in FIG. 7, the upper left region contains the liver and the lower right region contains the kidney. The right diagram of FIG. 7 represents points P9 to P16 based on the coordinate information output from the trained model 121. FIG.

When coordinate information about points P9 to P16 is output from trained model 121, processing circuit 11 generates a circular ROI based on this coordinate information, and places the generated circular ROI on the B-mode image ( step S63). Specifically, for example, the processing circuit 11 regards the points P9 to P12 as one group, and generates a square having the points P9 to P12 as vertices based on the coordinate information of the points P9 to P12. The processing circuit 11 generates a circle inscribed in the generated square, and sets this circle as a circular ROIC1 for the liver. The processing circuit 11 places the generated circular ROIC1 at the corresponding position on the B-mode image.

The processing circuit 11 regards the points P13 to P16 as one group, and generates a square having the points P13 to P16 as vertices based on the coordinate information of the points P13 to P16. The processing circuit 11 generates a circle inscribed in the generated square, and defines this circle as the circular ROIC2 for the kidney. The processing circuit 11 places the generated circular ROIC2 at the corresponding position on the B-mode image.

FIG. 8 is a diagram showing an example of a B-mode image in which circular ROIC1 and C2 are placed. According to FIG. 8, a circular ROIC1 generated from points P9-P12 is placed on the liver, and a circular ROIC2 generated from points P13-P16 is placed on the kidney. Once circular ROIC1 has been placed in the liver and circular ROIC2 has been placed in the kidney, processing circuitry 11 terminates the process shown in FIG.

The processing circuit 11 executes the analysis function 112, for example, when the ROI setting process shown in FIG. 6 is finished. When executing the analysis function 112, the processing circuit 11 acquires the luminance value of each pixel within the circular ROIC1, C2. The processing circuit 11 evaluates the state of the liver included in the B-mode image, such as the degree of hepatitis and fatty liver, from the relationship between the luminance values within the circular ROIC1 and the luminance values within the circular ROIC2. For example, the processing circuit 11 evaluates the degree of hepatitis, fatty liver, etc. by comparing the sum of the luminance values of the pixels within the circular ROIC1 and the sum of the luminance values of the pixels within the circular ROIC2. .

Processing circuitry 11 executes display control function 113 upon assessing the state of the liver contained in the B-mode image. When executing the display control function 113, the processing circuit 11 generates image data for displaying the evaluation result. The processing circuit 11 outputs the generated image data to the communication terminal 40 .

As described above, in the present embodiment, the processing circuit 11 inputs the medical image data to the learned model 121 in the ROI setting function 111 so as to set the ROI on the organ included in the medical image data. there is As a result, in an actual examination, when an application for evaluating the state of an organ is started, a medical image is displayed with an ROI of an appropriate size placed at an appropriate position within the organ.

(Modification 1)
If the ROI placed inside the organ includes a structure such as a blood vessel, the organ may not be correctly evaluated by the analysis function 112 . This is because an accurate feature amount cannot be obtained from the ROI placed inside the organ. Therefore, in Modified Example 1, a case will be described in which the ROI is set in consideration of the structures inside the organ in the ROI setting process.

FIG. 9 represents a variation of the flow chart shown in FIG. In the description of FIG. 9, as in the description of FIG. 6, the case where medical image data is B-mode image data generated by an ultrasonic diagnostic apparatus will be described as an example.

In FIG. 9, the processing circuit 11 places circular ROIC1 and C2 on the B-mode image through the same processing as steps S61 to S63 shown in FIG. Subsequently, the processing circuit 11 determines whether or not the distribution of luminance values within the circular ROIC1 and the distribution of luminance values within the circular ROIC2 are uniform (step S91).

Specifically, for example, the processing circuit 11 acquires the brightness value of each pixel within the circular ROIC 1 placed on the liver, and based on the acquired brightness value of each pixel, represents the distribution of the brightness values within the circular ROIC 1. Calculate the index value. The processing circuit 11 determines whether predetermined luminance values are uniformly distributed within the circular ROIC1 based on the calculated index value.

Further, for example, the processing circuit 11 acquires the brightness value of each pixel within the circular ROIC 2 placed on the kidney, and based on the acquired brightness value of each pixel, calculates an index value representing the distribution of the brightness values within the circular ROIC 2. calculate. The processing circuit 11 determines whether predetermined luminance values are uniformly distributed within the circular ROIC2 based on the calculated index value. If the luminance value distribution is uniform both in the circular ROIC1 and in the circular ROIC2 (Yes in step S91), the processing circuit 11 terminates the process.

If the luminance value distribution is not uniform in at least one of the circular ROIC1 and the circular ROIC2 (No in step S91), the processing circuit 11 determines that the luminance value distribution is not uniform. is moved by a preset amount (step S92). At this time, the processing circuit 11 moves the circular ROI so that the depth from the body surface is maintained. This is because the attenuation of the reflected wave changes according to the depth from the body surface in the B-mode image.

FIG. 10 is a schematic diagram showing an example in which the processing circuitry 11 moves a circular ROI. FIG. 10, for example, represents movement of a circular ROI on a B-mode image acquired using a convex probe. In FIG. 10, the direction in which the circular ROI moves is the direction along the waveform of the arc-shaped plane wave. Note that when a B-mode image is acquired using, for example, a linear probe, the direction in which the circular ROI moves is along the waveform of the linear plane wave. That is, the direction in which the circular ROI moves is the horizontal direction.

Note that the operation for making the brightness distribution in the circular ROI uniform is not limited to movement of the circular ROI. The processing circuit 11 may make the circular ROI smaller.

In step S91, the processing circuit 11 repeats steps S91 and S92 until the luminance value distribution becomes uniform in both the circular ROIC1 and the circular ROIC2.

As described above, in Modification 1, the processing circuit 11 uses the ROI setting function 111 to finely adjust the positions of the circular ROIC1 and C2 until the brightness distribution in the circular ROIC1 and C2 becomes uniform. As a result, no structure is included in the circular ROIC1, C2 that is set, and the analysis function 112 can correctly evaluate the organ.

(Second embodiment)
In the first embodiment, a medical image in which the upper left region includes the liver and the lower right region includes the kidney is input, and the learning is generated based on the learning data in which the coordinate information of eight points is the correct output. The case of using the finished model 121 has been described as an example. In the second embodiment, a medical image in which the upper left region includes the liver and the lower right region includes the kidney is input, and the learning data generated based on the learning data in which the coordinate information of the four points is the correct output. Use the finished model 121.

FIG. 11 is a diagram showing an example of ROI position information treated as correct output by the model learning device 60 in the second embodiment. In FIG. 11, coordinates of four points are set as ROI position information for the medical image shown in FIG. 4 in which the liver is included in the upper left region and the kidney is included in the lower right region. In FIG. 11, points P17-P20 are the vertices of a square inscribed by a first circular ROI placed across two organs, namely liver and kidney. Note that the ROI position information is not limited to the coordinates of the vertices of the square inscribed in the first circular ROI as long as the correct output of the learning data is common. For example, it may be the center coordinates and the radius of the first circular ROI.

Based on the learning data, the model learning device 60 generates a trained model 121 that determines the position of the first circular ROI based on the input medical image data, for example.

The processing circuit 11 according to the second embodiment executes a ROI setting program, a state evaluation program, and the like stored in the memory 12, thereby realizing functions corresponding to the programs. For example, the processing circuit 11 has an ROI setting function 111, an analysis function 112, and a display control function 113 by executing a ROI setting program, a state evaluation program, and the like.

The ROI setting function 111 is a function of setting an ROI for medical image data using the learned model 121 with medical image data as an input, and is an example of a setting unit. Specifically, for example, in the ROI setting function 111 , the processing circuit 11 inputs medical image data specified by the operator to the learned model 121 stored in the memory 12 . The processing circuit 11 sets ROIs for a plurality of organs included in the medical image data based on the positional information of the first circular ROI for the medical image data output from the learned model 121 .

Next, the ROI setting operation by the image analysis apparatus 10 according to the second embodiment will be described according to the processing procedure of the processing circuit 11. FIG.
FIG. 12 is a flow chart showing an example of the operation when the processing circuit 11 according to the second embodiment executes the ROI setting process using the trained model 121. As shown in FIG. Note that in the description of FIG. 12, an example in which medical image data is B-mode image data generated by an ultrasonic diagnostic apparatus will be described. At this time, the trained model 121 is generated based on learning data that receives B-mode image data as an input and outputs position information that defines the first circular ROI on a B-mode image based on this B-mode image data as a correct answer. shall be assumed. The B-mode image used as input contains the liver in the upper left region and the kidney in the lower right region.

In FIG. 12, when the B-mode image data is input to the trained model 121 in the same procedure as step S61 shown in FIG. It is output as position information of one circular ROI (step S121).

FIG. 13 is a diagram showing an example of a B-mode image based on B-mode image data input to the trained model 121 and an example of points based on coordinate information output from the trained model 121. FIG. The left diagram of FIG. 13 represents a B-mode image acquired using, for example, a convex probe and input to the trained model 121 . In the B-mode image shown in FIG. 13, the upper left region contains the liver and the lower right region contains the kidney. The right diagram of FIG. 13 represents points P21 to P24 based on the coordinate information output from the trained model 121. FIG.

When coordinate information about points P21 to P24 is output from trained model 121, processing circuit 11 generates a first circular ROI based on this coordinate information, and displays the generated first circular ROI on the B-mode image. (step S122). Specifically, for example, the processing circuit 11 generates a square having the points P21 to P24 as vertices based on the coordinate information of the points P21 to P24. The processing circuit 11 generates a circle inscribed in the generated square, and defines this circle as the first circular ROIC3 spanning the liver and kidney. The processing circuit 11 places the generated first circular ROIC3 at the corresponding position on the B-mode image.

FIG. 14 is a diagram showing an example of a B-mode image in which the first circular ROIC3 is set. According to FIG. 14, a first circular ROIC3 generated from points P21 to P24 is placed across the liver and kidney.

When the first circular ROI3 is placed across the liver and kidney, the processing circuit 11 generates a second circular ROI based on the first circular ROI3 and places the generated second circular ROI on the image (step S123 ). Specifically, for example, the processing circuit 11 generates the second circular ROIC4, C5 so as to be inscribed in the first circular ROIC3 and satisfy the following conditions. The conditions are, for example, to be provided in the vicinity of the boundary region between the liver and kidney, to be equal in size, and to be provided at the same depth. The processing circuit 11 places the generated second circular ROIC4, C5 at corresponding positions on the B-mode image.

FIG. 15 is a diagram showing an example of a B-mode image in which the second circular ROIC4 and C5 are installed. According to FIG. 15, the second circular ROIC4 is inscribed in the first circular ROIC3 and is provided near the border region between the liver and the kidney in the liver. Also, the second circular ROIC5 is inscribed in the first circular ROIC3 and provided near the boundary region between the liver and the kidney in the kidney. Also, the second circular ROICs 4 and C5 are equal in size and placed in the same depth region.

It should be noted that if the B-mode image is acquired using, for example, a linear probe, the plane wave will be linear rather than arc-shaped. In this case, the second circular ROICs 4 and C5 are provided at the same depth considering this linear plane wave.

As described above, in the second embodiment, the processing circuit 11 inputs the medical image data to the learned model 121 in the ROI setting function 111, thereby obtaining the first circular ROI 3 that straddles organs included in the medical image data. to be installed. Then, the processing circuit 11 sets the second circular ROIC4, C5 for each organ based on the first circular ROIC3. As a result, even if it is difficult to estimate the ROI to be placed in the liver by machine learning, it is possible to place an ROI of an appropriate size at an appropriate position in the organ.

Note that the processing circuit 11 moves the positions of the second circular ROIC4 and C5 set in step S123 until the feature amounts in the second circular ROIC4 and C5 become uniform, as in steps S91 and S92 in FIG. , may be fine-tuned while maintaining the same depth.

(Third embodiment)
In the second embodiment, the input is a medical image in which the upper left region includes the liver and the lower right region includes the kidney, and the correct output is the coordinate information of four points that define the region spanning the liver and the kidney. The case of using the trained model 121 generated based on the learning data has been described as an example. In the third embodiment, a medical image in which the upper left region includes the liver and the lower right region includes the kidney is input, and the correct output is the coordinate information of four points in the kidney. The trained model 121 is used.

FIG. 16 is a diagram showing an example of ROI position information handled as correct output by the model learning device 60 in the third embodiment. In FIG. 16, the coordinates of four points are set as the position information of the ROI in the kidney for the medical image shown in FIG. 4 in which the liver is included in the upper left region and the kidney is included in the lower right region. In FIG. 16, points P25-P28 are the vertices of a square inscribed by a circular ROI placed on the kidney. Note that the ROI position information is not limited to the coordinates of the vertices of the square inscribed by the circular ROI as long as the correct output of the learning data is common. For example, it may be the center coordinates and the radius of a circular ROI.

The model learning device 60 generates a trained model 121 that determines the position of the ROI based on the training data, for example, based on the input medical image data.

The processing circuit 11 according to the third embodiment executes a ROI setting program, a state evaluation program, and the like stored in the memory 12, thereby realizing functions corresponding to the programs. For example, the processing circuit 11 has an ROI setting function 111, an analysis function 112, and a display control function 113 by executing a ROI setting program, a state evaluation program, and the like.

The ROI setting function 111 is a function of setting an ROI for medical image data using the learned model 121 with medical image data as an input, and is an example of a setting unit. Specifically, for example, in the ROI setting function 111 , the processing circuit 11 inputs medical image data specified by the operator to the learned model 121 stored in the memory 12 . The processing circuit 11 sets ROIs for multiple organs included in the medical image data based on the positional information of the ROIs in the kidney output from the trained model 121 .

Next, the ROI setting operation by the image analysis apparatus 10 according to the third embodiment will be described according to the processing procedure of the processing circuit 11. FIG.
FIG. 17 is a flow chart showing an example of the operation when the processing circuit 11 according to the third embodiment executes the ROI setting process using the trained model 121. As shown in FIG. Note that in the description of FIG. 17, an example in which medical image data is B-mode image data generated by an ultrasonic diagnostic apparatus will be described. At this time, the trained model 121 is generated based on learning data that receives B-mode image data as an input and correct output is position information that defines an ROI within the kidney on a B-mode image based on this B-mode image data. shall be assumed. The B-mode image used as input contains the liver in the upper left region and the kidney in the lower right region.

In FIG. 17, when the B-mode image data is input to the trained model 121 in the same procedure as in step S61 shown in FIG. ROI position information is output (step S171).

FIG. 18 is a diagram showing an example of a B-mode image based on B-mode image data input to the trained model 121 and an example of points based on coordinate information output from the trained model 121. FIG. The left diagram of FIG. 18 represents a B-mode image acquired using, for example, a convex probe and input to the trained model 121 . In the B-mode image shown in FIG. 18, the upper left region contains the liver and the lower right region contains the kidney. The right diagram of FIG. 18 represents points P29 to P32 based on the coordinate information output from the trained model 121. In FIG.

When coordinate information about points P29 to P32 is output from trained model 121, processing circuit 11 generates a circular ROI based on this coordinate information, and places the generated circular ROI on the B-mode image ( step S172). Specifically, for example, the processing circuit 11 generates a square having the points P29 to P32 as vertices based on the coordinate information of the points P29 to P32. The processing circuit 11 generates a circle inscribed in the generated square, and defines this circle as the circular ROIC6 of the kidney. The processing circuit 11 places the generated circular ROIC 6 at the corresponding position on the B-mode image.

FIG. 19 is a diagram showing an example of a B-mode image in which a circular ROIC 6 is installed. According to FIG. 19, a circular ROIC6 generated from points P29-P32 is placed in the kidney.

After setting the circular ROI 6, the processing circuitry 11 generates a circular ROI corresponding to the circular ROI 6, and sets the generated circular ROI on the B-mode image (step S173). Specifically, for example, the processing circuit 11 generates the circular ROIC 6 and the corresponding circular ROIC 7 by moving the circular ROIC 6 so as to satisfy the following conditions. The conditions are, for example, to be provided in the vicinity of the boundary region between the liver and kidney, to be equal in size, to be equal in depth, and to have uniform feature amounts within the circular ROI. The processing circuit 11 places the generated circular ROIC 7 at the corresponding position on the B-mode image.

FIG. 20 is a diagram showing an example of a B-mode image in which circular ROICs 6 and C7 are installed. According to FIG. 20, the circular ROIC 6 is provided near the boundary region between the liver and the kidney within the kidney. Moreover, circular ROIC7 is provided in the vicinity of the boundary region between the liver and the kidney in the liver. Also, the circular ROICs 6 and C7 are equal in size and placed in the same depth region.

It should be noted that if the B-mode image is acquired using, for example, a linear probe, the plane wave will be linear rather than arc-shaped. In this case, the circular ROIC 7 is provided at the same depth based on the circular ROIC 6 and considering this linear plane wave.

As described above, in the third embodiment, the processing circuit 11 inputs the medical image data to the learned model 121 in the ROI setting function 111, thereby setting the circular ROI 6 in the kidney included in the medical image data. do. Then, based on the circular ROIC 6, the processing circuit 11 sets the circular ROIC 7 in the liver. As a result, even if there is a structure in the liver, it is possible to place an ROI of an appropriate size at an appropriate position in the organ while avoiding the structure in the liver.

In the above embodiment, the medical image data is B-mode image data generated by an ultrasonic diagnostic apparatus. may be

Further, in the above embodiment, the medical image data is two-dimensional B-mode image data generated by an ultrasonic diagnostic apparatus. It may be B-mode image data or three-dimensional color Doppler image data.

Further, in the above embodiment, the case where machine learning is performed based on learning data in which the correct output is the position information of the ROI set for the organ included in the medical image data has been described as an example. However, the correct answer output is not limited to this. A medical image in which an appropriate ROI is set may be used as a correct output.

Further, in the above embodiment, the case of inputting medical image data including a predetermined organ in a preset region has been described as an example. However, it is not limited to this. For example, the processing circuit 11 may have a preprocessing function of selecting medical image data including a predetermined organ in a preset region from a plurality of pieces of medical image data. The preprocessing function is realized by using a trained model for preprocessing, which is stored in the memory 12, for example. A trained model for preprocessing is generated, for example, based on learning data in which a plurality of pieces of medical image data are input, and medical image data including a predetermined organ in a preset region is output as a correct answer.

By executing the preprocessing function, the processing circuit 11 inputs a plurality of pieces of medical image data to a trained model for preprocessing, and outputs a predetermined organ in a preset region output from the trained model. Acquire medical image data containing: That is, for example, the processing circuit 11 extracts medical image data including the liver in the upper left region and the kidney in the lower right region from a plurality of pieces of medical image data.

Further, in the above embodiment, the case where the image analysis apparatus 10 is configured as a separate apparatus from the medical image diagnostic apparatus 20 has been described as an example. However, it is not limited to this. The ROI setting function 111 and the analysis function 112 executed by the processing circuit 11 of the image analysis apparatus 10 may be executed by a medical image diagnostic apparatus.

FIG. 21 is a block diagram showing an example of a functional configuration of an ultrasonic diagnostic apparatus 20A as a medical image diagnostic apparatus including an ROI setting function 111 and an analysis function 112. As shown in FIG. According to FIG. 21, the ultrasonic diagnostic apparatus 20A has an apparatus main body 21 and an ultrasonic probe 22. As shown in FIG. The ultrasonic probe 22 radiates ultrasonic waves to the subject P and receives reflected waves of the radiated ultrasonic waves.

The device body 21 has an internal storage circuit 211, a processing circuit 212, and the like. The internal storage circuit 211 is a storage device such as ROM, RAM, HDD, SSD, and integrated circuit storage device that stores various information. The completed model 121 and the like are stored.

The processing circuit 212 is, for example, a processor that functions as the core of the ultrasonic diagnostic apparatus 20A. The processing circuit 212 generates ultrasonic image data based on reflected waves received by the ultrasonic probe 22 by executing a program stored in the internal storage circuit 211 . Also, the processing circuit 212 implements functions such as the ROI setting function 111 and the analysis function 112 by executing programs stored in the internal storage circuit 211 .

In the above embodiment, the model learning device 60 generates the trained model 121 by causing the machine learning model to perform machine learning according to the model learning program based on the learning data. However, it is not limited to this. The model generation function for generating the trained model 121 may be possessed by the processing circuit 11 of the image analysis apparatus 10 or may be possessed by the processing circuit 212 of the ultrasonic diagnostic apparatus 20A.

According to at least one embodiment described above, it is possible to reduce the time and effort required to determine the position of the ROI, and improve operability. In addition, it becomes possible to make a diagnosis that does not depend on the doctor's experience.

The word "processor" used in the description of the embodiments is, for example, a CPU (central processing unit), a GPU (Graphics Processing Unit), or an application specific integrated circuit (ASIC)), a programmable logic device (eg, Simple Programmable Logic Device (SPLD), Complex Programmable Logic Device (CPLD), and Field Programmable Gate Array (FPGA)). The processor realizes its functions by reading and executing the programs stored in the memory circuit. It should be noted that instead of storing the program in the memory circuit, the program may be directly installed in the circuit of the processor. In this case, the processor realizes its function by reading and executing the program embedded in the circuit. Note that each processor in each of the above embodiments is not limited to being configured as a single circuit for each processor, but may be configured as a single processor by combining a plurality of independent circuits to realize its function. good too. Furthermore, a plurality of components in each of the above-described embodiments may be integrated into one processor to realize its functions.

While several embodiments of the invention have been described, these embodiments have been presented by way of example and are not intended to limit the scope of the invention. These embodiments can be implemented in various other forms, and various omissions, replacements, and modifications can be made without departing from the scope of the invention. These embodiments and their modifications are included in the scope and spirit of the invention, as well as the scope of the invention described in the claims and equivalents thereof.

DESCRIPTION OF SYMBOLS 10... Image analysis apparatus 11... Processing circuit 12... Memory 13... Communication interface 20... Medical image diagnosis apparatus 20A... Ultrasound diagnosis apparatus 21... Apparatus body 22... Ultrasound probe 30... Medical image management system 40... Communication terminal 50... Learning Data storage device 60 Model learning device 111 ROI setting function 112 Analysis function 113 Display control function 121 Trained model 211 Internal storage circuit 212 Processing circuit

Claims (21)

Input first medical image data including a first organ and a second organ to a trained model, and define a first region of interest and a second region of interest in the first medical image data output from the trained model. a setting unit that sets the first region of interest for the first organ in the first medical image data and sets the second region of interest for the second organ based on position information;
and
The setting unit sets at least one of the first region of interest and the second region of interest so that the distribution of luminance values in the first region of interest and the distribution of luminance values in the second region of interest are uniform. adjust the position of
Image analysis device. The trained model receives as input second medical image data including the first organ and the second organ, and sets position information defining a plurality of regions of interest in the second medical image data as learning data as a correct output. generated based on
The image analysis apparatus according to claim 1. The setting unit sets the first region of interest and the second region of interest based on coordinate information output from the learned model as the position information.
The image analysis apparatus according to claim 1 . The setting unit, based on first coordinate information defining the first region of interest and second coordinate information defining the second region of interest, which are output from the learned model as the coordinate information, setting one region of interest and the second region of interest;
4. The image analysis apparatus according to claim 3. The setting unit adjusts the position in a direction that maintains the depth from the body surface of the subject in the first medical image data.
The image analysis apparatus according to claim 1 . Input first medical image data including a first organ and a second organ into a trained model, and output from the trained model, based on position information defining a first region of interest in the first medical image data, a setting unit that sets the first region of interest so as to straddle the first organ and the second organ in the first medical image data;
and
The setting unit is configured to apply a second organ to each of the first organ and the second organ within the first region of interest so that the depths from the body surface of the subject in the first medical image data are equal. setting the region of interest,
Image analysis device. The trained model is generated based on learning data that receives as input second medical image data including the first organ and the second organ and that correct output is position information that defines a region of interest in the second medical image data. to be
The image analysis apparatus according to claim 6 . The setting unit is configured to apply the first organ to each of the first organ and the second organ in the vicinity of a boundary region between the first organ and the second organ in the first medical image data and inside the first region of interest. 2 Setting a region of interest,
The image analysis apparatus according to claim 6 . The setting unit sets each of the second regions of interest such that the sizes of the second regions of interest are equal.
The image analysis apparatus according to claim 6 . The setting unit adjusts the position of each of the second regions of interest so that the distribution of luminance values in each of the second regions of interest is uniform.
The image analysis apparatus according to claim 6 . Input first medical image data including a first organ and a second organ into a trained model, and output from the trained model, based on position information defining a first region of interest in the first medical image data, a setting unit that sets the first region of interest to the first organ in the first medical image data;
and
The setting unit defines a second region of interest in the second organ in the first medical image data such that the depth from the body surface of the subject in the first medical image data is equal to the first region of interest. set,
Image analysis device. The trained model is generated based on learning data that receives as input second medical image data including the first organ and the second organ and that correct output is position information that defines a region of interest in the second medical image data. to be
The image analysis apparatus according to claim 11 . The setting unit sets the second region of interest in the second organ in the first medical image data in the vicinity of a boundary region between the first organ and the second organ in the first medical image data.
The image analysis apparatus according to claim 11 . The setting unit sets the first region of interest and the second region of interest such that the sizes of the first region of interest and the size of the second region of interest are equal.
The image analysis apparatus according to claim 11 . The setting unit sets at least one of the first region of interest and the second region of interest so that the distribution of luminance values in the first region of interest and the distribution of luminance values in the second region of interest are uniform. adjust the position of
The image analysis apparatus according to claim 11 . further comprising a preprocessing unit that selects the first medical image data from a plurality of medical image data;
The image analysis apparatus according to any one of claims 1 to 15 . Input medical image data including a first organ and a second organ into a trained model, and output from the trained model, based on position information defining a first region of interest and a second region of interest in the medical image data a setting unit that sets the first region of interest to the first organ in the medical image data and sets the second region of interest to the second organ;
and
The setting unit sets at least one of the first region of interest and the second region of interest so that the distribution of luminance values in the first region of interest and the distribution of luminance values in the second region of interest are uniform. adjust the position of
diagnostic imaging equipment. Input medical image data including a first organ and a second organ to a trained model, and output the medical image data based on position information defining a first region of interest in the medical image data output from the trained model. A setting unit that sets the first region of interest so as to straddle the first organ and the second organ in
and
The setting unit sets second regions of interest for each of the first organ and the second organ within the first region of interest so that the depths from the body surface of the subject in the medical image data are equal. to set the
diagnostic imaging equipment. further comprising an ultrasonic probe that emits ultrasonic waves and receives reflected waves of the emitted ultrasonic waves;
The medical image data is ultrasound image data generated based on the reflected waves,
19. The diagnostic imaging apparatus according to claim 17 or 18 . Input medical image data including a first organ and a second organ into a trained model, and output from the trained model, based on position information defining a first region of interest and a second region of interest in the medical image data causing a processor to perform processing for setting the first region of interest in the first organ and setting the second region of interest in the second organ in the medical image data;
At least one of the first region of interest and the second region of interest is processed so that the distribution of luminance values in the first region of interest and the distribution of luminance values in the second region of interest are uniform. adjust the position of
ROI setting program. Input medical image data including a first organ and a second organ into a trained model, and output from the trained model, based on position information defining a first region of interest in the medical image data, cause a processor to perform processing for setting the first region of interest so as to straddle the first organ and the second organ in the medical image data ;
The processing includes forming a second region of interest on each of the first organ and the second organ within the first region of interest so that depths from the body surface of the subject in the medical image data are equal. set,
ROI setting program.

Images