JP7346192B2 - Device, medical information processing device, and program - Google Patents

Description

Embodiments of the present invention relate to a device, a medical information processing device, and a program.

A wall motion analysis technique is known that analyzes the wall motion of the heart using time-series medical image data obtained by imaging the heart. For example, in ultrasound imaging, speckle tracking is performed on time-series ultrasound image data to calculate motion information from strain, displacement, etc. of the heart wall.

It is known that local wall motion abnormalities (hypokinesis) appear in ischemic conditions as one of the symptoms of heart disease. Therefore, in order to diagnose this wall motion abnormality, there are various techniques that define features obtained from strain waveform information as index values and display index images that map luminance values according to the index values. Proposed.

However, since the absolute value of strain varies from patient to patient (subject) or from region to heart region, it is not robust to identify whether wall motion has decreased based on the peak value. Further, since characteristics of waveform information observed in an ischemic state may also be observed in healthy individuals, index definitions based on these characteristics have low specificity. Therefore, with existing techniques, it is difficult to identify local wall motion abnormalities with high accuracy and robustness.

Japanese Patent Application Publication No. 2007-117611 Japanese Patent Application Publication No. 2009-448 Japanese Patent Application Publication No. 2009-165815

Ishii K, Suyama T, Imai M, Maenaka M, Yamanaka A, Makino Y, Seino Y, Shimada K, Yoshikawa J. “Abnormal regional left ventricular systolic and diastolic function in patients with coronary artery disease undergoing percutaneous coronary intervention: clinical significance of post-ischemic diastolic stunning.” J Am Coll Cardiol. 2009 Oct 20;54(17):1589-97. Perk G, Tunick PA, Kronzon I. “Non-Doppler two-dimensional strain imaging by echocardiography--from technical considerations to clinical applications.” J Am Soc Echocardiogr. 2007 Mar;20(3):234-43. Onishi T, Uematsu M, Nanto S, Morozumi T, Watanabe T, Awata M, Iida O, Sera F, Nagata S. “Detection of diastolic abnormality by dyssynchrony imaging: correlation with coronary artery disease in patients presenting with visibly normal wall motion. ” Circ J. 2009 Jan;73(1):125-31.

The problem to be solved by the present invention is to provide a device, a medical information processing device, and a program that can identify local wall motion abnormalities with high accuracy and robustness.

A medical image diagnostic apparatus according to an embodiment includes an acquisition section and a determination section. The acquisition unit acquires a plurality of pieces of first medical information representing time-series changes in wall motion parameters in each of the first local region and the first overall region, regarding the heart of the first subject. The determination unit determines an abnormality in the first local region based on the plurality of first medical information.

FIG. 1 is a block diagram showing a configuration example of an ultrasound diagnostic apparatus according to a first embodiment. FIG. 2 is a block diagram showing a configuration example of the medical information processing apparatus according to the first embodiment. FIG. 3 is a diagram showing processes during learning and operation performed by the ultrasound diagnostic apparatus and medical information processing apparatus according to the first embodiment. FIG. 4 is a diagram for explaining a waveform information group for machine learning according to the first embodiment. FIG. 5 is a diagram for explaining an example of a waveform information group according to the first embodiment. FIG. 6 is a diagram for explaining abnormality determination results for machine learning according to the first embodiment. FIG. 7 is a diagram for explaining an example of output by the output control function according to the first embodiment. FIG. 8 is a diagram for explaining an example of output by the output control function according to the first embodiment. FIG. 9 is a flowchart showing the processing procedure of the medical information processing apparatus according to the first embodiment. FIG. 10 is a flowchart showing the processing procedure of the ultrasound diagnostic apparatus according to the first embodiment. FIG. 11 is a block diagram showing a configuration example of an ultrasound diagnostic apparatus according to the second embodiment. FIG. 12 is a diagram for explaining index values calculated by the calculation function according to the second embodiment. FIG. 13 is a diagram for explaining an example of output by the output control function according to the second embodiment. FIG. 14 is a block diagram showing a configuration example of a medical information processing system according to another embodiment.

Hereinafter, an apparatus, a medical information processing apparatus, and a program according to an embodiment will be described with reference to the drawings. Note that the embodiments described below are not limited to the following description. The embodiment described below can be combined with other embodiments or conventional techniques as long as there is no contradiction in processing content.

(First embodiment)
An ultrasound diagnostic apparatus and a medical information processing apparatus according to a first embodiment will be described. Here, the ultrasonic diagnostic apparatus according to the first embodiment includes a trained model, and can identify local wall motion abnormalities with high accuracy and robustness using the trained model. Furthermore, the medical information processing apparatus according to the first embodiment constructs a learned model included in the ultrasound diagnostic apparatus. Note that in the first embodiment, a case will be described in which the ultrasonic diagnostic apparatus includes a learned model, but the embodiment is not limited to this. An embodiment in which a device other than an ultrasound diagnostic device includes a learned model will be described later.

The configurations of an ultrasound diagnostic apparatus and a medical information processing apparatus according to the first embodiment will be described using FIGS. 1 and 2. FIG. 1 is a block diagram showing a configuration example of an ultrasound diagnostic apparatus 100 according to the first embodiment. FIG. 2 is a block diagram showing a configuration example of the medical information processing apparatus 200 according to the first embodiment. Note that in the first embodiment, a case will be described in which the ultrasound diagnostic apparatus 100 and the medical information processing apparatus 200 are communicably connected via the network NW, but the present invention is not limited to this. For example, the ultrasound diagnostic apparatus 100 and the medical information processing apparatus 200 can exchange information via a storage medium, a removable external storage device, or the like, without using the network NW.

First, the configuration of the ultrasonic diagnostic apparatus 100 shown in FIG. 1 will be described. As shown in FIG. 1, the ultrasonic diagnostic apparatus 100 according to the first embodiment includes an ultrasonic probe 101, an input interface 102, a display 103, an electrocardiograph 104, and an apparatus main body 105. The ultrasound probe 101, input interface 102, display 103, and electrocardiograph 104 are communicably connected to the device main body 105.

The ultrasonic probe 101 has a plurality of transducers, and these plurality of transducers generate ultrasonic waves based on a drive signal supplied from a transmitting/receiving circuit 110 included in the apparatus main body 105. Further, the ultrasound probe 101 receives reflected waves from the subject X and converts them into electrical signals. Further, the ultrasonic probe 101 includes a matching layer provided on the transducer, a backing material, etc. that prevents propagation of ultrasonic waves backward from the transducer. Note that the ultrasonic probe 101 is detachably connected to the apparatus main body 105.

When ultrasonic waves are transmitted from the ultrasound probe 101 to the subject The signal is received by a plurality of transducers included in the device 101. The amplitude of the received reflected wave signal depends on the difference in acoustic impedance at the discontinuity surface from which the ultrasound wave is reflected. Note that when a transmitted ultrasound pulse is reflected from a moving bloodstream or a surface such as a heart wall, the reflected wave signal depends on the velocity component of the moving object in the ultrasound transmission direction due to the Doppler effect. and undergo a frequency shift.

The input interface 102 receives various instructions and information input operations from an operator. Specifically, the input interface 102 converts an input operation received from an operator into an electrical signal and outputs it to the processing circuit 170 of the apparatus main body 105. For example, the input interface 102 may include a trackball, a switch button, a mouse, a keyboard, a touchpad that performs input operations by touching the operation surface, a touchscreen that integrates a display screen and a touchpad, and a non-control device that uses an optical sensor. This is realized by a touch input circuit, a voice input circuit, etc. Note that the input interface 102 is not limited to one that includes physical operation components such as a mouse and a keyboard. For example, an example of the input interface 102 includes an electrical signal processing circuit that receives an electrical signal corresponding to an input operation from an external input device provided separately from the device and outputs this electrical signal to a control circuit.

Display 103 displays various information and images. Specifically, the display 103 converts information and image data sent from the processing circuit 170 into electrical signals for display, and outputs the electrical signals. For example, the display 103 is realized by a liquid crystal monitor, a CRT (cathode ray tube) monitor, a touch panel, or the like. Note that the output device included in the ultrasound diagnostic apparatus 100 is not limited to the display 103, and may include, for example, a speaker. For example, the speaker outputs a predetermined sound such as a beep sound in order to notify the operator of the processing status of the device main body 105.

The electrocardiograph 104 acquires an electrocardiogram (ECG) of the subject X as a biological signal of the subject X. The electrocardiograph 104 transmits the acquired electrocardiogram waveform to the main body 105 of the apparatus. Note that in this embodiment, a case will be described in which the electrocardiograph 104 is used as one of the means for acquiring information regarding the cardiac phase of the heart of the subject X, but the embodiment is not limited to this. . For example, the ultrasound diagnostic apparatus 100 obtains the time of sound II (second sound) of a phonocardiogram or the aortic valve close (AVC) time determined by measuring the ejection blood flow of the heart using spectrum Doppler. In this way, information regarding the cardiac phase of the heart of the subject X may be acquired.

The device main body 105 is a device that generates ultrasound image data based on the reflected wave signal received by the ultrasound probe 101. The device main body 105 shown in FIG. 1 is a device that can generate two-dimensional ultrasound image data based on two-dimensional reflected wave data received by the ultrasound probe 101. Further, the device main body 105 is a device that can generate three-dimensional ultrasound image data based on three-dimensional reflected wave data received by the ultrasound probe 101.

As shown in FIG. 1, the device main body 105 includes a transmission/reception circuit 110, a signal processing circuit 120, an image generation circuit 130, an image memory 140, a storage circuit 150, a network (NW) interface 160, and a processing circuit 170. The transmission/reception circuit 110, the signal processing circuit 120, the image generation circuit 130, the image memory 140, the storage circuit 150, the NW interface 160, and the processing circuit 170 are connected to each other so as to be able to communicate with each other.

The transmission/reception circuit 110 includes a pulse generator, a transmission delay section, a pulser, etc., and supplies a drive signal to the ultrasound probe 101. The pulse generator repeatedly generates rate pulses to form transmitted ultrasound waves at a predetermined rate frequency. Further, the transmission delay section focuses the ultrasound generated from the ultrasound probe 101 into a beam shape, and adjusts the delay time for each transducer necessary for determining the transmission directivity to each of the ultrasound waves generated by the pulse generator. Give for rate pulse. Further, the pulser applies a drive signal (drive pulse) to the ultrasound probe 101 at a timing based on the rate pulse. That is, the transmission delay section arbitrarily adjusts the transmission direction of the ultrasound transmitted from the transducer surface by changing the delay time given to each rate pulse.

Note that the transmitter/receiver circuit 110 has a function that can instantly change the transmission frequency, transmission drive voltage, etc. in order to execute a predetermined scan sequence based on instructions from the processing circuit 170. In particular, changing the transmission drive voltage is realized by a linear amplifier-type oscillation circuit that can instantly switch its value, or by a mechanism that electrically switches multiple power supply units.

The transmitter/receiver circuit 110 includes a preamplifier, an A/D (Analog/Digital) converter, a reception delay unit, an adder, etc., and performs various processing on the reflected wave signal received by the ultrasound probe 101 to reflect the reflected wave signal. Generate wave data. The preamplifier amplifies the reflected wave signal for each channel. The A/D converter performs A/D conversion on the amplified reflected wave signal. The reception delay section provides a delay time necessary to determine reception directivity. The adder performs addition processing of the reflected wave signals processed by the reception delay section to generate reflected wave data. By the addition process of the adder, the reflected component from the direction corresponding to the receiving directivity of the reflected wave signal is emphasized, and a comprehensive beam of ultrasonic transmission and reception is formed by the receiving directivity and the transmitting directivity.

Here, the format of the output signal from the transmitter/receiver circuit 110 may be in various formats, such as a signal containing phase information called an RF (Radio Frequency) signal, or amplitude information after envelope detection processing. Selectable.

The signal processing circuit 120 receives reflected wave data from the transmitting/receiving circuit 110, performs logarithmic amplification, envelope detection processing, etc., and generates data (B mode data) in which signal strength is expressed by brightness of luminance. In addition, the signal processing circuit 120 performs frequency analysis on velocity information from the reflected wave data received from the transmission/reception circuit 110, extracts blood flow, tissue, and contrast agent echo components due to the Doppler effect, and extracts the velocity, dispersion, power, etc. of the moving object. Generate data (Doppler data) in which information is extracted from multiple points.

Note that the signal processing circuit 120 illustrated in FIG. 1 is capable of processing both two-dimensional reflected wave data and three-dimensional reflected wave data. That is, the signal processing circuit 120 generates two-dimensional B-mode data from two-dimensional reflected wave data, and generates three-dimensional B-mode data from three-dimensional reflected wave data. Further, the signal processing circuit 120 generates two-dimensional Doppler data from two-dimensional reflected wave data, and generates three-dimensional Doppler data from three-dimensional reflected wave data.

The image generation circuit 130 generates ultrasound image data from the data generated by the signal processing circuit 120. That is, the image generation circuit 130 generates two-dimensional B-mode image data in which the intensity of the reflected wave is expressed by luminance from the two-dimensional B-mode data generated by the signal processing circuit 120. Further, the image generation circuit 130 generates two-dimensional Doppler image data representing moving object information from the two-dimensional Doppler data generated by the signal processing circuit 120. The two-dimensional Doppler image data is a velocity image, a dispersion image, a power image, or a combination thereof. The image generation circuit 130 can also generate M-mode image data from time-series data of B-mode data on one scanning line generated by the signal processing circuit 120. The image generation circuit 130 can also generate a Doppler waveform in which blood flow and tissue velocity information is plotted in time series from the Doppler data generated by the signal processing circuit 120.

Here, the image generation circuit 130 generally converts (scan convert) a scanning line signal sequence of ultrasonic scanning into a scanning line signal sequence of a video format typified by television etc. Generate image data. Specifically, the image generation circuit 130 generates ultrasound image data for display by performing coordinate transformation according to the scanning form of ultrasound by the ultrasound probe 101. In addition to scan conversion, the image generation circuit 130 performs various types of image processing, such as image processing (smoothing processing) for regenerating an average luminance image using a plurality of image frames after scan conversion; Performs image processing (edge enhancement processing), etc. using a differential filter within the image. Further, the image generation circuit 130 synthesizes text information of various parameters, scales, body marks, etc. to the ultrasound image data.

Further, the image generation circuit 130 performs rendering processing on the volume data in order to generate various two-dimensional image data for displaying the volume data on the display 103. The rendering process performed by the image generation circuit 130 includes a process of performing Multi Planer Reconstruction (MPR) to generate MPR image data from volume data. Furthermore, the rendering processing performed by the image generation circuit 130 includes processing that performs "Curved MPR" on volume data and processing that performs "Maximum Intensity Projection" on volume data. Furthermore, the rendering processing performed by the image generation circuit 130 includes volume rendering (VR) processing and surface rendering (SR) processing.

That is, the B-mode data and Doppler data are ultrasound image data before scan conversion processing, and the data generated by the image generation circuit 130 is ultrasound image data for display after scan conversion processing. Note that B-mode data and Doppler data are also called raw data. The image generation circuit 130 converts "2D B-mode data or 2D Doppler data", which is 2D ultrasound image data before scan conversion processing, into 2D ultrasound image data, which is 2D ultrasound image data for display. B-mode image data and two-dimensional Doppler image data.

The image memory 140 is a memory that stores image data for display generated by the image generation circuit 130. Further, the image memory 140 can also store data generated by the signal processing circuit 120. The B-mode data and Doppler data stored in the image memory 140 can be called up by the operator after diagnosis, for example, and become ultrasound image data for display via the image generation circuit 130.

Note that the image generation circuit 130 associates the ultrasound image data and the time of ultrasound scanning performed to generate the ultrasound image data with the electrocardiogram waveform transmitted from the electrocardiograph 104. The image is stored in the image memory 140. By referring to the data stored in the image memory 140, the processing circuit 170, which will be described later, can obtain the cardiac phase at the time of ultrasound scanning performed to generate ultrasound image data.

The storage circuit 150 stores various data. For example, the storage circuit 150 stores various data such as control programs for transmitting and receiving ultrasound waves, image processing, and display processing, diagnostic information (e.g., patient ID, doctor's findings, etc.), diagnostic protocols, and various body marks. Remember. Furthermore, the storage circuit 150 is also used for storing image data stored in the image memory 140, as needed. For example, the memory circuit 150 is realized by a semiconductor memory element such as a RAM (Random Access Memory) or a flash memory, a hard disk, an optical disk, or the like. Further, data stored in the storage circuit 150 can be transferred to an external device via the NW interface 160.

The NW interface 160 controls communication between the device main body 105 and external devices. Specifically, the NW interface 160 receives various information from external devices and outputs the received information to the processing circuit 170. For example, the NW interface 160 is realized by a network card, network adapter, NIC (Network Interface Controller), or the like.

The processing circuit 170 controls the entire processing of the ultrasound diagnostic apparatus 100. Specifically, the processing circuit 170 controls the transmission/reception circuit 110 and the signal processing circuit 120 based on various setting requests input from the operator via the input interface 102 and various control programs and various data read from the storage circuit 150. , and controls the processing of the image generation circuit 130. Further, the processing circuit 170 controls the display 103 to display the ultrasound image data for display stored in the image memory 140 and the storage circuit 150.

The processing circuit 170 also executes an acquisition function 171, a tracking function 172, a determination function 173, and an output control function 174. Here, the acquisition function 171 is an example of an acquisition unit. The tracking function 172 is an example of a tracking unit. The determination function 173 is an example of a determination unit. The output control function 174 is an example of an output control section. Note that the processing contents of the acquisition function 171, tracking function 172, determination function 173, and output control function 174 executed by the processing circuit 170 will be described later.

Here, for example, each processing function executed by the acquisition function 171, tracking function 172, determination function 173, and output control function 174, which are the components of the processing circuit 170 shown in FIG. 1, is in the form of a program executable by a computer. is recorded in the memory circuit 150. The processing circuit 170 is a processor that reads each program from the storage circuit 150 and executes it to realize a function corresponding to each program. In other words, the processing circuit 170 in a state where each program is read has each function shown in the processing circuit 170 of FIG.

In this embodiment, each processing function described below will be realized in a single processing circuit 170. However, a processing circuit is configured by combining a plurality of independent processors, and each processor The function may be implemented by executing a program.

Next, the configuration of the medical information processing apparatus 200 shown in FIG. 2 will be explained. As shown in FIG. 2, the medical information processing device 200 includes an input interface 201, a display 202, a NW interface 203, a storage circuit 204, and a processing circuit 210.

The input interface 201 receives various instructions and information input operations from an operator. Specifically, the input interface 201 converts an input operation received from an operator into an electrical signal and outputs it to the processing circuit 210. For example, the input interface 201 includes a trackball, a switch button, a mouse, a keyboard, a touchpad that performs input operations by touching the operation surface, a touchscreen that integrates a display screen and a touchpad, and a non-control device that uses an optical sensor. This is realized by a touch input circuit, a voice input circuit, etc. Note that the input interface 201 is not limited to one that includes physical operation parts such as a mouse and a keyboard. For example, examples of the input interface 201 include an electrical signal processing circuit that receives an electrical signal corresponding to an input operation from an external input device provided separately from the device and outputs this electrical signal to a control circuit.

Display 202 displays various information and images. Specifically, the display 202 converts information and image data sent from the processing circuit 210 into electrical signals for display, and outputs the electrical signals. For example, the display 202 is realized by a liquid crystal monitor, a CRT (cathode ray tube) monitor, a touch panel, or the like. Note that the output device included in the medical information processing apparatus 200 is not limited to the display 202, and may include, for example, a speaker. For example, the speaker outputs a predetermined sound such as a beep sound in order to notify the operator of the processing status of the medical information processing device 200.

The NW interface 203 is connected to the processing circuit 210 and controls communication between the medical information processing device 200 and external devices. Specifically, the NW interface 203 receives various types of information from external devices and outputs the received information to the processing circuit 210. For example, the NW interface 203 is realized by a network card, network adapter, NIC, or the like.

The storage circuit 204 is connected to the processing circuit 210 and stores various data. For example, the memory circuit 204 is realized by a semiconductor memory element such as a RAM or a flash memory, a hard disk, an optical disk, or the like.

The processing circuit 210 controls the operation of the medical information processing apparatus 200 according to input operations received from the operator via the input interface 201. For example, processing circuit 210 is implemented by a processor.

The processing circuit 210 also executes a learning function 211. Note that the processing content of the learning function 211 executed by the processing circuit 210 will be described later. The learning function 211 is an example of a learning section.

Here, if a case shows wall motion abnormality in all regions, it is reflected in the ejection fraction of the heart, so any medical worker can easily identify the case. On the other hand, it is difficult even for experts to detect local wall motion abnormalities associated with ischemic heart disease, which is one of the diseases with a high mortality rate.For example, multiple experts check the strain waveform information of the same patient. Opinions are often divided. Therefore, in order to accurately diagnose local wall motion abnormalities associated with ischemic heart disease, it is necessary to define feature quantities obtained from strain waveform information as index values, and to use index images that map the index values. Various display techniques have been proposed.

For example, PSI (Post-systolic Strain Index) is used as an index value for PSS (Post-Systolic Shortening), which is a sign specific to ischemic state (ischemic memory), contraction delay (tardokinesis), and diastolic delay (postischemic diastolic stunning). ) and SI-DI (Strain Imaging Diastolic Index) have been proposed. In addition, the use of peak strain has been proposed because the strain in areas exhibiting wall motion abnormalities is relatively small compared to healthy areas. Furthermore, in order to capture wall motion delay at the end of systole due to ischemia, the use of DI (Dysynchrony Imaging), which images the time when strain reaches its peak value, has been proposed. In addition, in order to capture the delay in contraction itself that occurs in the early stages of contraction due to ischemia and the contraction delay associated with pre-stretching, it has been proposed to use AI (Activation Imaging), which images the time when the strain reaches a predetermined threshold. .

However, in actual clinical situations, various existing indicators for ischemia diagnosis may not work well. For example, since the absolute value of strain varies from patient to patient (subject) or from region to region of the heart, it is not robust to identify whether or not the wall motion of the heart during activity is reduced based on the peak value. Furthermore, since signs similar to ischemic memory such as PSS may be observed in healthy subjects, PSI and SI-DI, which focus on abnormal timing of strain waveform information, also have low specificity. Furthermore, even in cases of ischemia, characteristic signs are not necessarily recognized for individual indicators, so identification accuracy is low. Therefore, it is difficult to accurately and robustly identify local wall motion abnormalities using strain-based classification using "a certain index."

Although it is difficult for even experts to identify local wall motion abnormalities, some experts (hereinafter abbreviated as "some experts") who have accumulated an extremely large amount of experience If so, wall motion abnormalities can be identified by visual inspection of the waveform information. At this time, it is thought that some experts (people) grasp the relative difference in the waveform information of the local strain with respect to the overall wall motion. Existing index values can be said to be an approach that captures this difference. However, as described above, existing approaches that define and identify signs of local wall motion abnormalities from strain waveform information have limitations. Note that in the above description, strain was cited as an example of the wall motion parameter, but the same applies when displacement is used.

Therefore, the medical information processing apparatus 200 according to the first embodiment uses a plurality of waveform information (waveform information group) of wall motion parameters in a plurality of cases as input data, and some experts use it as input data to determine whether wall motion is abnormal or not. A trained model is constructed using machine learning that uses the identified abnormality judgment results as correct data. The ultrasound diagnostic apparatus 100 according to the first embodiment inputs a group of waveform information of wall motion parameters of the person to be diagnosed (subject) to the trained model constructed by the medical information processing apparatus 200. The abnormality determination result for the person to be diagnosed is output.

Processing during learning and operation performed by the ultrasound diagnostic apparatus 100 and the medical information processing apparatus 200 according to the first embodiment will be described using FIG. 3. FIG. 3 is a diagram showing processes during learning and operation performed by the ultrasound diagnostic apparatus 100 and the medical information processing apparatus 200 according to the first embodiment.

As shown in the upper part of FIG. 3, during learning, the medical information processing apparatus 200 performs machine learning using, for example, the waveform information group of the subjects P1 to PN and the abnormality determination results. Here, the waveform information group for the subjects P1 to PN includes a plurality of pieces of waveform information representing time-series changes in wall motion parameters in each of a plurality of local regions and an overall region. Further, the abnormality determination results for the subjects P1 to PN include the abnormality determination results for each local area. Through this machine learning, the medical information processing apparatus 200 constructs a learned model that outputs an abnormality determination result for each local region by inputting the waveform information group of the subject. This learned model is delivered to the ultrasound diagnostic apparatus 100 and stored in the storage circuit 150.

As shown in the lower part of FIG. 3, during operation, the ultrasonic diagnostic apparatus 100 transmits the waveform information group of the subject X, who is the person to be diagnosed, to the trained model constructed by the medical information processing apparatus 200 By inputting the input, the learned model outputs the abnormality determination result of the subject X. Then, the ultrasound diagnostic apparatus 100 presents the abnormality determination result of the subject X output from the learned model to the operator.

Note that in this embodiment, a "local area" is also referred to as a "local area." Furthermore, the "overall area" is also referred to as the "overall area." Further, the subject X is also referred to as a "first subject." Further, the local area of the first subject is also referred to as a "first local area," and the entire area is also referred to as a "first overall area." Further, the subjects P1 to PN are also referred to as "second subjects." Further, the local area of the second subject is also referred to as a "second local area," and the entire area is also referred to as a "second overall area."

Each processing function of the ultrasound diagnostic apparatus 100 and the medical information processing apparatus 200 according to the first embodiment will be described below. First, the medical information processing apparatus 200 that generates a trained model will be described below, and then the ultrasound diagnostic apparatus 100 that uses the trained model to determine local wall motion abnormality will be described.

In the medical information processing apparatus 200, the storage circuit 204 stores strain waveform information groups in each of a plurality of local regions and the overall region, and abnormality determinations for the plurality of local regions, regarding the hearts of a plurality of subjects P1 to PN. Memorize the results. Note that the strain waveform information group is an example of a plurality of pieces of medical information representing time-series changes in wall motion parameters.

A waveform information group for machine learning according to the first embodiment will be described using FIG. 4. FIG. 4 is a diagram for explaining a waveform information group for machine learning according to the first embodiment.

As shown in FIG. 4, the waveform information group includes a plurality of pieces of waveform information. For example, the waveform information group includes different types of information on five axes, ie, the first to fifth axes. Here, it is preferable that the waveform information group includes at least regional differences in order to capture the relative difference in the waveform information of local strains with respect to the overall wall motion. Furthermore, the waveform information group includes different types of information in at least one of wall motion parameters, time phases, positions in the wall thickness direction, and heart motion directions.

The first axis is a region axis that represents the difference between regions. For example, the first axis is represented by a global (overall area) and a plurality of segments 1, 2, . . . N (local areas). Global typically refers to the entire region to be analyzed by speckle tracking, and corresponds to, for example, the entire left ventricle in a three-dimensional image or the entire region of interest in a two-dimensional cross-sectional image. In addition, a segment is a global divided into a predetermined division method, such as the 16-segment division method recommended by ASE (American Society of Echocardiography) or the 17-segment division recommended by AHA (American Heart Association). A dividing method is known. Note that the term "global" is not necessarily limited to the entire region of a certain region, such as the left ventricle, and may be wider than the local region.

Further, as waveform information of the entire region, waveform information representing a time-series change in lumen volume that reflects cardiac function (left ventricular pumping function) can also be used. The waveform information of the lumen volume can be calculated based on the intimal surface information of speckle tracking.

The second axis is a time phase axis that represents the difference in load or time phase before and after treatment. For example, when a stress echo test is performed, the second axis is represented by the pre-load period and the recovery period. The pre-load period is a time phase before an exercise load or a drug load is applied to the subject's heart, that is, a time phase during which the subject is at rest. Moreover, the recovery period is an arbitrary time phase after the load until the state returns to the resting state, and is, for example, 5 minutes or 10 minutes after the load.

Here, the recovery period is a time phase in which ischemic memory appears, and the resting period is a time phase to be compared with the recovery period, so it is preferable that the waveform information group includes these two time phases as a pair. It is. However, if a stress echo test is not performed, the waveform information group does not include the recovery period and is used for machine learning as information only for the resting period. Note that during the load phase, the subject's breathing becomes rough and the heart rate increases, making it difficult to maintain the image quality and tracking accuracy of the ultrasound image. For this reason, it is preferable that the time phase under load is not included as time phase axis information. Further, the second axis is not limited to the time phase before and after loading, but may be, for example, the time phase before and after treatment. In other words, the second axis represents the difference between the time phase before drug treatment and the time phase after drug treatment (several days to several months later, etc.), or the difference between the time phase before surgery and the time phase after surgery. It may also be a case where it is expressed.

The third axis is a wall thickness direction axis that represents the difference in position in the wall thickness direction. For example, the third axis is represented by an inner layer, a middle layer, and an outer layer. The inner layer is, for example, information on tracking points on the intima, and the outer layer is information on tracking points on the adventitia. Furthermore, the middle layer is information about the intermediate point between the tracking point on the intima and the tracking point on the adventitia.

Note that in two-dimensional speckle tracking (2DST), since the spatial resolution of an ultrasound image is sufficient, it is preferable that the waveform information group includes information on each of the inner layer, middle layer, and outer layer. On the other hand, in three-dimensional speckle tracking (3DST), since the spatial resolution of ultrasound images is not sufficient, the waveform information group only needs to include at least one of an inner layer, a middle layer, and an outer layer. However, in ischemic heart disease, it is known that necrosis and myocardial ischemia occur more easily in the intimal tissue than in the adventitial tissue, so the waveform information group includes at least information on the inner layer. is preferable.

Further, the third axis is not limited to the above example. For example, when the waveform information group is represented by wall thickness change rate, the third axis can include inner half, outer half, and total wall thickness. That is, the waveform information group includes information on at least one of the inner layer, middle layer, and outer layer, or information on at least one of the inner layer, the outer layer, and the wall thickness, as a position in the wall thickness direction.

The fourth axis is a parameter axis that represents differences in wall motion parameters. For example, the fourth axis is represented by strain and displacement. The waveform information group only needs to include information on at least one of strain and displacement as the fourth axis. However, in order to capture wall motion with high sensitivity, it is preferable that strain, which is an index of the rate of change based on the difference value (length) between two displacements, be included as the fourth axis.

In addition, although the displacement is not highly sensitive because it is easily affected by the movement (translation) of the whole heart and the movement (tethering) in the vicinity, it is robust against noise because it is not a differential value. Since RD (Radial Displacement) is the object of visual recognition when an expert visually recognizes the movement of the intimal surface, it is preferable to include it as the fourth axis. Further, since LD (Longitudinal Displacement) is suggested to have a possibility of contributing to the detection of ischemic heart disease in Non-Patent Document 3, it is preferable to include it as the fourth axis.

The fifth axis is a movement direction axis that represents the difference in the movement direction of the heart. For example, in 3DST, the fifth axis is represented by the area of the long axis direction, circumferential direction, radial direction (short axis direction) of the heart (myocardial muscle), and planes parallel to the endomyocardial surface. For example, when the wall motion parameter is strain, the fifth axis is represented by LS (Longitudinal Strain), CS (Circumferential Strain), and RS (Radial Strain). Further, the area of a plane parallel to the endomyocardial plane corresponds to AC (Area Change ratio) as its rate of change. Note that AC represents a change in a component that cannot be classified into any of the major axis direction, circumferential direction, and radial direction (including the major axis direction, circumferential direction, and oblique direction at the same time), so AC represents the change in the fifth axis. It is classified as follows.

Note that when the wall motion parameter is displacement, the fifth axis is preferably represented by the above-mentioned LD and RD. Further, in 2DST, it is preferable that the fifth axis is represented by a major axis direction and a minor axis direction.

An example of the waveform information group according to the first embodiment will be described using FIG. 5. FIG. 5 is a diagram for explaining an example of a waveform information group according to the first embodiment. The left diagram in FIG. 5 is waveform information before loading (at rest), and the right diagram is waveform information during the recovery period (5 minutes after loading). In FIG. 5, the vertical axis corresponds to strain, and the horizontal axis corresponds to time. The solid line in FIG. 5 is the strain of the segment corresponding to the anterior wall septum, and the dashed line is the strain of the segment corresponding to the inferolateral part.

As shown in the left diagram of FIG. 5, before loading, the anterior wall septum and the inferolateral portion contract and expand at approximately the same timing. On the other hand, as shown in the right diagram of FIG. 5, it can be seen that during the recovery period, the expansion of the inferolateral part is delayed compared to the anterior wall septum. This delayed expansion of the inferolateral region is one of ischemic memory. That is, in FIG. 5, ischemic wall motion abnormalities are observed in the inferolateral region.

Here, the waveform information group includes four pieces of waveform information illustrated in FIG. In addition, in FIG. 5, four pieces of waveform information are illustrated for convenience of illustration, but the waveform information group is not limited to this, and various waveform information having differences in the first to fifth axes described above can be used. This includes:

Note that all of the waveform information groups described above are information obtained by wall motion analysis (speckle tracking). Further, the waveform information group is stored in advance in the storage circuit 204. The waveform information group stored in the storage circuit 204 may be calculated by the ultrasound diagnostic apparatus 100, or may be calculated by the medical information processing apparatus 200 equipped with a function to perform wall motion analysis. Further, the waveform information group stored in the storage circuit 204 is not limited to a device in the same facility, but may be calculated by a device installed in another facility.

Anomaly determination results for machine learning according to the first embodiment will be described using FIG. 6. FIG. 6 is a diagram for explaining abnormality determination results for machine learning according to the first embodiment.

As shown in FIG. 6, the abnormality determination result is information indicating whether each segment is abnormal or normal in wall motion. This abnormality determination result is information identified by some experts and is stored in the storage circuit 204 in advance.

Although FIG. 6 describes a case in which the abnormality determination result is expressed by whether or not wall motion is abnormal, the embodiment is not limited to this. For example, the abnormality determination result may be expressed in multiple stages including an intermediate range. In this case, the abnormality determination result is expressed in a plurality of levels, with normality being ``0'' and wall motion abnormality being ``1 to 3'' (the larger the value, the more severe the abnormality).

In this way, the storage circuit 204 stores the waveform information group and the abnormality determination results. Note that the waveform information group and abnormality determination results described above are merely examples, and the embodiment is not limited thereto. For example, the waveform information group can include any type of information that can be obtained by known wall motion analysis.

In the medical information processing apparatus 200, the learning function 211 collects waveform information groups of wall motion parameters in each of the plurality of local regions and the overall region, and the waveform information about the plurality of local regions, regarding the hearts of the plurality of subjects P1 to PN. A trained model is constructed by performing machine learning using the abnormality determination results. The constructed trained model is stored in the storage circuit 204. Note that the waveform information group is an example of a plurality of pieces of medical information representing time-series changes in wall motion parameters.

For example, the learning function 211 reads out waveform information groups and abnormality determination results for the plurality of subjects P1 to PN from the storage circuit 204. The learning function 211 then normalizes the time direction of each waveform information included in the waveform information group of each subject. For example, the learning function 211 normalizes the time direction of each waveform information by setting one cardiac cycle interval (1RR interval) as 100%. At this time, the learning function 211 generates (interpolates) data (wall motion parameters) so that one cardiac cycle section is divided into 1 to 2% units. As a result, the learning function 211 can express each waveform information as a one-dimensional vector space in which about 100 pieces of data are arranged in one cardiac cycle section.

In addition, in the above explanation, the case where the time direction of each waveform information is normalized by setting one cardiac cycle interval as 100% was explained, but the embodiment is not limited to this, and any interval in the cardiac time phase is normalized. Normalization in the time direction can be performed based on . For example, since contraction after the end of acquisition, which is the focus of PSS and SI-DI, and abnormal wall motion in early diastole are characteristic signs of ischemic heart disease, the interval from the R wave to the end of systole It is also effective to normalize as 100% from the viewpoint of detecting ischemic heart disease. This end-diastolic phase can be given by the time when the lumen volume is at its minimum or the aortic valve closure (AVC) time. Here, if the expansion period is sufficiently long, it is preferable to cut off the time direction at an interval of about 200%.

The learning function 211 can be realized by a known machine learning engine, and for example, a support vector machine (SVM) is applied. In the case of SVM, the learning function 211 constructs a correlation matrix between waveform information for each of the first to fifth axes of the waveform information group that is input data. Then, the learning function 211 performs learning by determining the boundary plane (hyperplane) in the k eigenvector spaces that have the largest contribution to the discrimination between normal and abnormal through principal component analysis using the correlation matrix. Generate a completed model. This trained model applies a group of waveform information of the person to be diagnosed to a discriminator based on a hyperplane in the above-mentioned eigenvector space, and identifies whether the waveform information is normal or abnormal (wall motion abnormality).

Note that deep learning (neural network), which is another known machine learning engine, can be applied to the above machine learning. Here, SVM is designed to obtain a hyperplane that provides the maximum discrimination margin for various input data, and discrimination is possible even when the number of machine learning data is not large enough. Furthermore, as described above, since the principal component (upper eigenvalue) is used as a criterion for determining normality and abnormality, a determination process that is relatively easy for humans to understand can be provided. On the other hand, in the case of deep learning, even without preprocessing to construct a correlation matrix, learning can be performed automatically if input waveform information and correct abnormality determination result information are provided. However, the disadvantage is that it is difficult for humans to understand the criteria for determining normal and abnormal results (the automatic learning process, in other words, how the neurons in each layer were designed to be weighted). .

In machine learning (supervised learning) according to this embodiment, the abnormality determination results determined by some experts are used as correct data, so there are restrictions on increasing the number of data for machine learning. Therefore, SVM is suitable as the machine learning engine according to this embodiment. Note that if the amount of data for machine learning is sufficiently large, it is expected that deep learning will provide high identification performance.

Here, the actual number of data for machine learning will be explained by exemplifying 2DST and 3DST. Note that the number of data for machine learning is the number of waveform information (number of types) included in the waveform information group of one subject. The number of data for machine learning described below is just an example, and varies depending on various conditions of imaging and wall motion analysis.

First, a case where wall motion analysis results by 2DST are used will be described. In this case, the 16 segment division method recommended by ASE is common, and in this division method, there are 6 segments per cross section. Since globals are added to these six segments, the number of area axes is at least seven. From the perspective of covering the left ventricle extensively with multiple two-dimensional cross-sections, a combination of three long-axis images (A4C, A2C, and A3C) and one short-axis image (Mid level) is common; 7 parts x 4 cross sections = 28 parts. Note that in the case of the 17 segment division method recommended by AHA, there are 7 segments per cross section, so including the global, there are 8 sites x 3 + 7 = 31 sites.

Since the wall motion parameters used in the three long-axis images are two parameters, LS and RD, there are 7 regions x 3 cross sections x 2 parameters = 42 pieces. Furthermore, the number of wall motion parameters used in one short-axis image is three parameters, RS, CS, and RD, so the number is 21 (7 regions x 1 cross section x 3 parameters). Moreover, in each cross section, the number of positions in the wall thickness direction is three (inner layer, middle layer, and outer layer). Further, in each cross section, there are two time phases: a pre-load phase and a recovery phase. Therefore, when the wall motion analysis results by 2DST are used, the number of machine learning data is (42+21)×3×2=378 (types).

Next, a case will be described in which the results of wall motion analysis by 3DST are used. In this case, in the 16 segment division method recommended by ASE, there will be 17 parts (16 segments + 1 global). Note that in the 17 segment division method recommended by AHA, there will be 18 parts (17 segments + 1 global).

The wall motion parameters used for the 18 parts are six parameters: LS, LD, RS, CS, RD, and AC, and one type of volume curve. Further, the number of positions in the wall thickness direction is one in the inner layer. Further, the number of time phases is one time phase during rest, for example when the test is not a stress echo test. Therefore, when the wall motion analysis results by 3DST are used, the number of data for machine learning is 18×(6+1)×1×1=126 pieces (types).

In addition, since it is also possible to apply both 2DST and 3DST to the same case (subject), the number of data for machine learning is 378 + 126 = 504 pieces (types) if you add up the two examples above. .

In other words, the number of data for machine learning ranges from several hundred to a maximum of one thousand, depending on various conditions of imaging and wall motion analysis. Further, each piece of waveform information is expressed as a one-dimensional vector space in which about 100 pieces of data are arranged. As a result, the number of machine learning data according to this embodiment is configured as a vector space of about 1000×100 at most.

In this way, the learning function 211 performs machine learning based on the waveform information group of the data number configured as a vector space of about 1000 x 100 at most and the abnormality determination results determined by some experts. , build a trained model. The constructed trained model is delivered to the ultrasound diagnostic apparatus 100 and stored in the storage circuit 150.

Returning to the explanation of FIG. In the ultrasound diagnostic apparatus 100, the acquisition function 171 acquires a plurality of medical image data (medical image data group) in which the heart of the subject X is imaged in time series. For example, the ultrasound diagnostic apparatus 100 performs ultrasound scanning on a region (space) including the heart of the subject X, and collects reflected wave data. Then, the ultrasound diagnostic apparatus 100 generates a plurality of ultrasound image data (ultrasound image data group) arranged in time series based on the collected reflected wave data. Here, the plurality of generated ultrasound image data are stored in the image memory 140. The acquisition function 171 reads out a plurality of ultrasound image data in which the heart of the subject X is imaged from the image memory 140.

The tracking function 172 generates a plurality of pieces of waveform information representing time-series changes in wall motion parameters in each of a plurality of local regions and an overall region by performing a tracking process on a plurality of pieces of medical image data. For example, the tracking function 172 performs wall motion analysis on a plurality of ultrasound image data of the subject X to generate wall motion information.

For example, the tracking function 172 sets an initial contour in which a plurality of constituent points are set for the ultrasound image data of the initial time phase. Then, the tracking function 172 performs tracking processing including pattern matching using the ultrasound image data of the initial time phase and the ultrasound image data of the next time phase, so that multiple images included in the ultrasound image data group are track the positions of multiple constituent points in ultrasound image data. The tracking function 172 then generates wall motion information representing the motion of the heart wall of the subject X based on the positions of the plurality of constituent points in the plurality of ultrasound image data included in each ultrasound image data group.

Here, the wall motion information of the subject X generated by the tracking function 172 includes the same type of information as the waveform information group used to construct the learned model by the medical information processing apparatus 200. In other words, the tracking function 172 can generate a waveform information group containing different types of information on five axes, ie, the first to fifth axes. Note that, as the wall motion analysis technique, known techniques such as those described in Patent Documents 1 to 3 can be applied.

The determination function 173 generates an abnormality determination result for each local region of the subject X by inputting the waveform information group of the subject X to the learned model. For example, the determination function 173 reads out the trained model stored in the storage circuit 150. Then, the determination function 173 inputs the waveform information group generated by the tracking function 172 to the learned model that has been read out, thereby converting the abnormality determination results for each local region of the subject X into the learned model. Output. Thereafter, the determination function 173 sends the abnormality determination result output from the trained model to the output control function 174.

Note that the abnormality determination result output from the trained model includes the same type of information as the abnormality determination result used to construct the trained model. In other words, if the abnormality determination result used in machine learning is binary information indicating whether or not wall motion is abnormal, the abnormality determination result output from the trained model will be binary information. . Furthermore, if the abnormality determination result used in machine learning is information expressed in multiple stages, the abnormality determination result output from the trained model will be information expressed in multiple stages.

The output control function 174 outputs the abnormality determination result. For example, the output control function 174 causes the abnormality determination result to be displayed superimposed on at least one of the ultrasound image and the polar map.

An example of output by the output control function 174 according to the first embodiment will be explained using FIGS. 7 and 8. FIGS. 7 and 8 are diagrams for explaining examples of output by the output control function 174 according to the first embodiment. FIG. 7 illustrates a short-axis image of the left ventricle. FIG. 8 illustrates a polar map corresponding to the short-axis image of FIG. 7. In FIGS. 7 and 8, regions determined to have abnormal wall motion are shown in black, and regions determined to be normal are shown in shaded areas. In addition, in FIGS. 7 and 8, "Ant-Sept" indicates the anterior wall septum, "Ant" indicates the anterior wall, "Lat" indicates the side wall, and "Post" indicates the posterior wall. "Inf" indicates the inferior wall, and "Sept" indicates the septum. Further, in FIG. 7, a B-mode image serving as a background is shown in a simplified manner.

As shown in FIG. 7, the output control function 174 causes the abnormality determination result to be displayed superimposed on the ultrasound image. Specifically, the output control function 174 assigns and displays a brightness value according to the abnormality determination result to a position corresponding to each segment on the short-axis image of the left ventricle. In the example of FIG. 7, the anterior wall has wall motion abnormality, and the other segments are normal.

As shown in FIG. 8, the output control function 174 causes the abnormality determination result to be displayed superimposed on the polar map. Specifically, the output control function 174 assigns and displays a brightness value according to the abnormality determination result to a position corresponding to each segment on the polar map. In the example of FIG. 8, the anterior wall has wall motion abnormality, and the other segments are normal.

In this way, the output control function 174 displays the abnormality determination result on the display 103. Note that FIGS. 7 and 8 are just examples, and the embodiment is not limited thereto. For example, the output control function 174 can output the abnormality determination results for each segment as a table (see FIG. 6) or a simple character string.

Note that the output control function 174 is capable of outputting not only binary information indicating whether or not wall motion is abnormal, but also a continuous amount including an intermediate region. This continuous quantity is a value representing which one is closer between abnormal wall motion and normal wall motion, so it can be said to be a probabilistic output method (for example, referred to as "abnormal wall motion probability"). For example, when using a discriminator, there is an algorithm that votes on the identification results of each of the first to fifth axes of the waveform information group that is input data, and finally determines the one with the most votes as the determination result.

Therefore, by weighting the abnormality determination result by the total number of votes obtained for each waveform information, the abnormality determination result can be expressed as a continuous quantity. In addition, in the case of deep learning, by spatially averaging the determination results of "0 (normal) / 1 (abnormal wall motion)" determined for each segment or each position in the wall thickness direction, It is possible to output a distribution image with smooth continuous quantities. Further, even when the abnormality determination result is expressed in multiple stages, continuous quantitative output is possible.

Next, the processing procedure of the medical information processing apparatus 200 according to the first embodiment will be explained using FIG. 9. FIG. 9 is a flowchart showing the processing procedure of the medical information processing apparatus 200 according to the first embodiment. The process shown in FIG. 9 is started, for example, when an instruction to start machine learning is received from an operator.

As shown in FIG. 9, for example, in the medical information processing apparatus 200, the processing circuit 210 determines to start processing when receiving an instruction to start machine learning from the operator (step S101, Yes). . Note that the processing circuit 210 is in a standby state until the instruction is received (step S101, No).

Subsequently, the processing circuit 210 reads out the waveform information group and abnormality determination results for each of the subjects P1 to PN from the storage circuit 204 (step 102). Then, the processing circuit 210 generates a learned model by machine learning using the waveform information group and the abnormality determination result as learning data (step S103). Then, the processing circuit 210 stores the generated trained model in the storage circuit 204 (step S104), and ends the process.

The processing procedure of the ultrasonic diagnostic apparatus 100 according to the first embodiment will be explained using FIG. 10. FIG. 10 is a flowchart showing the processing procedure of the ultrasound diagnostic apparatus 100 according to the first embodiment. The process shown in FIG. 10 is started, for example, when an instruction to start determining local wall motion abnormality is received from the operator.

As shown in FIG. 10, for example, in the ultrasonic diagnostic apparatus 100, the processing circuit 170 determines to start processing when receiving an instruction from the operator to start determining local wall motion abnormality. (Step S201, Yes). Note that the processing circuit 170 is in a standby state until the instruction is received (step S201, No).

Subsequently, the processing circuit 170 reads out the medical image data group of the subject X from the image memory 140 (step S202). Then, the processing circuit 170 generates a waveform information group by performing tracking processing on the medical image data group (step S203). The processing circuit 170 generates an abnormality determination result by inputting the waveform information group to the trained model (step S204). The processing circuit 170 displays the abnormality determination result for the subject X (step S205), and ends the process.

As described above, in the ultrasound diagnostic apparatus 100 according to the first embodiment, the acquisition function 171 acquires a plurality of medical image data in which the heart of the first subject is imaged in time series. The tracking function 172 generates a plurality of pieces of first medical information representing wall motion parameters in each of a plurality of local regions and a global region by performing a tracking process on a plurality of pieces of medical image data. The determination function 173 includes a plurality of pieces of second medical information representing wall motion parameters in each of a plurality of local regions and a global region, and a plurality of pieces of second medical information representing each local area of the plurality of second medical information, regarding the hearts of a plurality of second subjects. A first abnormality determination result for each local region of the first subject is generated by inputting a plurality of first medical information to the learned model trained using the abnormality determination result for the region. . The output control function 174 outputs the first abnormality determination result. According to this, the ultrasound diagnostic apparatus 100 can identify local wall motion abnormalities with high precision and robustness.

Note that in the first embodiment, a case has been described in which the determination function 173 determines abnormality using waveform information of each of a plurality of local regions, but the embodiment is not limited to this. For example, the determination function 173 inputs the waveform information of at least one local region that is noticed by the doctor and the waveform information of the entire region among the plurality of local regions into the learned model, thereby determining the local region. It is also possible to determine abnormalities in a region. That is, the determination function includes a plurality of pieces of second medical information representing time-series changes in wall motion parameters in each of the second local region and the second whole region, and a plurality of pieces of second medical information regarding the hearts of the plurality of second subjects, and By inputting a plurality of first medical information about the first subject to the trained model trained using the abnormality determination result, an abnormality is determined about the first local region of the first subject. .

Furthermore, the acquisition function 171 may acquire a plurality of pieces of first medical information representing wall motion parameters in each of the first local region and the first whole region, regarding the heart of the first subject. In this case, the ultrasound diagnostic apparatus 100 does not necessarily have to include the tracking function 172. For example, the acquisition function 171 may acquire a plurality of pieces of first medical information generated by an external device (an ultrasound diagnostic device different from the ultrasound diagnostic device 100 or a medical information processing device) equipped with the tracking function 172. It may also be obtained from the device.

(Second embodiment)
In the second embodiment, a case will be described in which a known index value is presented to the operator along with the abnormality determination result from the trained model. This is because in a method using a trained model, the basis for making a determination is complicated, and it is difficult for the operator to understand the basis of the output result. For this reason, the ultrasonic diagnostic apparatus 100 according to the second embodiment provides known index values that are easy for the operator to understand as feedback of the abnormality determination results from the trained model, thereby providing reference information for the abnormality determination results. It is presented to the operator.

The configuration of an ultrasound diagnostic apparatus 100 according to the second embodiment will be described using FIG. 11. FIG. 11 is a block diagram showing a configuration example of an ultrasound diagnostic apparatus 100 according to the second embodiment. The ultrasonic diagnostic apparatus 100 according to the second embodiment has the same configuration as the ultrasonic diagnostic apparatus 100 illustrated in FIG. 1, except that the processing circuit 170 further executes a calculation function 175. Therefore, in the second embodiment, the points that are different from the first embodiment will be mainly explained, and the points having the same functions as the configuration explained in the first embodiment will be the same as those in FIG. Reference numerals are given and explanations are omitted.

The calculation function 175 according to the second embodiment calculates an index value regarding local wall motion abnormality. Specifically, the calculation function 175 calculates a plurality of types of index values for each waveform information included in the waveform information group of the subject X. For example, the calculation function 175 calculates SI-DI and PSI as index values.

The index value calculated by the calculation function 175 according to the second embodiment will be explained using FIG. 12. FIG. 12 is a diagram for explaining index values calculated by the calculation function 175 according to the second embodiment. In FIG. 12, the vertical axis corresponds to strain, and the horizontal axis corresponds to time.

As shown in FIG. 12, the calculation function 175 detects point A, point B, and point C from the waveform information. Here, point A corresponds to the strain in AVC time. Point B corresponds to the strain at a time 1/3 elapsed from the AVC time during the diastole. Point C corresponds to the peak strain. Note that Tε corresponds to the arrival time at which the peak strain is reached.

Then, the calculation function 175 calculates SI-DI and PSI based on the values (strain) at points A, B, and C shown in FIG. For example, the calculation function 175 calculates SI-DI using the following equation (1). Further, the calculation function 175 calculates PSI using the following equation (2).

For example, the calculation function 175 performs a process of testing various index values for abnormality determination results based on the learned model. For example, the calculation function 175 uses multivariate analysis to identify an index value that is significant for the abnormality determination result from among the plurality of types of index values. Alternatively, the calculation function 175 identifies the index value that is the largest principal component by principal component analysis. The calculation function 175 then sends information regarding the specified index value to the output control function 174.

Then, the output control function 174 outputs the index value specified for the abnormality determination result together with the abnormality determination result. For example, the output control function 174 outputs an index value that is significant for the abnormality determination result for each waveform information.

An example of output by the output control function 174 according to the first embodiment will be explained using FIG. 13. FIG. 13 is a diagram for explaining an example of output by the output control function 174 according to the first embodiment. Note that FIG. 13 illustrates a case where SI-DI is identified as a significant index value for the abnormality determination result.

As shown in FIG. 13, the output control function 174 outputs SI-DI in addition to the abnormality determination result shown in FIG. SI-DI is displayed for each segment. Note that the output control function 174 is not limited to index values, but can also display, for example, regression coefficients regarding a plurality of index values in multivariate analysis. Further, the output control function 174 may display index values that are not significant.

Note that FIG. 13 is just an example, and the embodiment is not limited thereto. For example, the output control function 174 can generate and display an index image to which a luminance value is assigned according to an index value such as SI-DI.

In this way, the ultrasonic diagnostic apparatus 100 according to the second embodiment identifies an index value that is significant for the abnormality determination result or is obtained as a main component, along with the abnormality determination result from the trained model. and present it to the operator as reference information. Thereby, the ultrasound diagnostic apparatus 100 can support the operator's interpretation of the abnormality determination results from the learned model.

In addition, although the above description describes the case where SI-DI and PSI are calculated as index values, the embodiment is not limited to this. For example, the calculation function 175 can calculate any index value that can serve as an index value for local wall motion abnormality. For example, the calculation function 175 calculates peak strain and DSR (diastolic strain ratio). Here, the DSR is a value obtained by calculating a ratio similar to SI-DI in equation (1) for each strain calculation point, and averaging the calculated value among the calculation points included in a segment.

(Other embodiments)
In addition to the embodiments described above, the present invention may be implemented in various different forms.

(Judgment processing by medical information processing system)
The local wall motion abnormality determination process using the above-described learned model may be executed by a medical information processing system. In this case, the determination process is executed on the network NW.

The configuration of a medical information processing system according to another embodiment will be described using FIG. 14. FIG. 14 is a block diagram showing a configuration example of a medical information processing system according to another embodiment.

As shown in FIG. 14, the medical information processing system includes an ultrasound diagnostic device 100 and a server device 300. The ultrasonic diagnostic apparatus 100 in FIG. 14 has basically the same configuration as the ultrasonic diagnostic apparatus 100 illustrated in FIG. 1, except that the processing circuit 170 executes a transmitting function 176 and a receiving function 177. Further, the server device 300 in FIG. 14 has basically the same configuration as the medical information processing device 200 illustrated in FIG. 2, except that the processing circuit 310 executes the determination function 311. Note that the server device 300 is an example of a medical information processing device. Ultrasonic diagnostic apparatus 100 and server device 300 are communicably connected via network NW.

In the ultrasonic diagnostic apparatus 100, the processing of the acquisition function 171 and the tracking function 172 is basically the same as the processing of the acquisition function 171 and the tracking function 172 shown in FIG. 1, so a description thereof will be omitted. Then, the transmission function 176 transmits the waveform information group of the subject X generated by the tracking function 172 to the server device 300.

In the server device 300, the determination function 311 receives the waveform information group transmitted by the transmission function 176. Then, the determination function 311 generates an abnormality determination result for the subject X by inputting the received waveform information group to the learned model. Note that the processing of the determination function 311 is basically the same as the processing of the determination function 173 shown in FIG. 1, so a description thereof will be omitted. Then, the determination function 311 transmits the generated abnormality determination result for the subject X to the ultrasound diagnostic apparatus 100.

In the ultrasound diagnostic apparatus 100, the reception function 177 receives the abnormality determination result of the subject X transmitted by the determination function 311. Then, the output control function 174 causes the abnormality determination result of the subject X received by the reception function 177 to be output. Note that the processing of the output control function 174 is basically the same as the processing of the determination function 173 shown in FIG. 1, so a description thereof will be omitted.

That is, in the medical information processing system, the acquisition function 171 acquires a plurality of pieces of first medical information representing time-series changes in wall motion parameters in each of a plurality of local regions and a global region, regarding the heart of the first subject. . The determination function 311 then generates a plurality of second medical information representing time-series changes in wall motion parameters in a plurality of local regions and a plurality of global regions, and a plurality of second medical information regarding the hearts of a plurality of second subjects. By inputting a plurality of pieces of first medical information to a learned model trained using the abnormality determination results for each local area of the information, the first medical information for each local area of the first subject is inputted. Generate abnormality determination results. Then, the output control function 174 outputs the first abnormality determination result. According to this, the medical information processing system can execute a process for determining a local wall motion abnormality using a trained model on the network NW.

(Determination processing by medical information processing device)
Further, the process for determining local wall motion abnormality using the learned model described above may be executed by the medical information processing apparatus 200. For example, the medical information processing device 200 corresponds to a terminal operated by a doctor, a console device of another medical image diagnostic device, and the like. Note that other medical image diagnostic apparatuses are medical image diagnostic apparatuses other than the ultrasound diagnostic apparatus 100, such as an MRI (Magnetic Resonance Imaging) apparatus and an X-ray CT (Computed Tomography) apparatus.

In this case, the medical information processing apparatus 200 preferably acquires the waveform information group of the subject X from the ultrasound diagnostic apparatus 100. That is, in the medical information processing apparatus 200, the processing circuit 210 acquires a plurality of pieces of first medical information representing time-series changes in wall motion parameters in each of a plurality of local regions and a global region, regarding the heart of the first subject. do. The processing circuit 210 then generates a plurality of second medical information representing time-series changes in wall motion parameters in each of a plurality of local regions and a global region, and a plurality of second medical information regarding the hearts of a plurality of second subjects. By inputting a plurality of pieces of first medical information to a learned model trained using the abnormality determination results for each local area of the information, the first medical information for each local area of the first subject is inputted. Generate abnormality determination results. Then, the processing circuit 210 outputs the first abnormality determination result.

Note that if the medical information processing apparatus 200 has a function similar to the tracking function 172, the medical information processing apparatus 200 can also generate a waveform information group of the subject X.

(Use of medical image data other than ultrasound image data)
Further, in the above embodiment, a case where the present invention is applied to a group of ultrasound image data has been described, but the present invention is not limited to this. For example, as long as a plurality of pieces of medical image data are arranged in time series, the present invention can be applied to image data captured by any medical image diagnostic apparatus, such as magnetic resonance image data or CT image data. In other words, if a plurality of pieces of medical image data are arranged in time series, it is possible to generate a waveform information group by wall motion analysis. Therefore, using this waveform information group, it becomes possible to perform machine learning and determination processing using a learned model. That is, the embodiments described above are applicable not only to ultrasound diagnostic apparatuses but also to other medical image diagnostic apparatuses.

(Example other than trained model)
Further, in the above description, a case has been described in which the above-described embodiment is realized by a trained model, but the embodiment is not limited to this. For example, the ultrasound diagnostic apparatus 100 can determine an abnormality based on the relative difference in waveform information of local strain with respect to the overall wall motion.

That is, in the ultrasound diagnostic apparatus 100, the acquisition function 171 acquires a plurality of pieces of first medical information representing time-series changes in wall motion parameters in each of the first local region and the first whole region, regarding the heart of the first subject. do. Then, the determination function 173 determines abnormality in the first local region based on the plurality of first medical information.

For example, the determination function 173 compares the waveform information of the entire area and the waveform information of the local area to detect a relative difference. Here, the relative difference is, for example, a difference value of an arbitrary parameter (numeric value) such as an integral value obtained from waveform information. The determination function 173 determines an abnormality in the local area based on the detected difference.

The term "processor" used in the above description refers to, for example, a CPU (Central Processing Unit), a GPU (Graphics Processing Unit), an Application Specific Integrated Circuit (ASIC), a programmable logic device (for example, Refers to circuits such as a simple programmable logic device (SPLD), a complex programmable logic device (CPLD), and a field programmable gate array (FPGA). The processor realizes its functions by reading and executing programs stored in the storage circuit 150. Note that instead of storing the program in the storage circuit 150, the program may be directly incorporated into the circuit of the processor. In this case, the processor realizes its functions by reading and executing a program built into the circuit. Note that each processor of this embodiment is not limited to being configured as a single circuit for each processor, but may also be configured as a single processor by combining multiple independent circuits to realize its functions. good. Furthermore, multiple components in each figure may be integrated into one processor to implement its functionality.

Furthermore, each component of each device shown in the drawings is functionally conceptual, and does not necessarily need to be physically configured as shown in the drawings. In other words, the specific form of distributing and integrating each device is not limited to what is shown in the diagram, and all or part of the devices can be functionally or physically distributed or integrated in arbitrary units depending on various loads, usage conditions, etc. Can be integrated and configured. Further, each processing function performed by each device may be realized in whole or in part by a CPU and a program that is analyzed and executed by the CPU, or may be realized as hardware using wired logic.

Further, among the processes described in the above embodiments, all or part of the processes described as being performed automatically can be performed manually, or the processes described as being performed manually All or part of this can also be performed automatically using known methods. In addition, information including processing procedures, control procedures, specific names, and various data and parameters shown in the above documents and drawings may be changed arbitrarily, unless otherwise specified.

Further, the medical image processing method described in the above embodiment can be realized by executing a medical image processing program prepared in advance on a computer such as a personal computer or a workstation. This medical image processing program can be distributed via a network such as the Internet. Furthermore, this medical image processing program may be recorded on a computer-readable recording medium such as a hard disk, flexible disk (FD), CD-ROM, MO, or DVD, and executed by being read from the recording medium by the computer. can.

According to at least one embodiment described above, local wall motion abnormalities can be identified with high accuracy and robustness.

Although several embodiments of the invention have been described, these embodiments are presented by way of example and are not intended to limit the scope of the invention. These embodiments can be implemented in various other forms, and various omissions, substitutions, and changes can be made without departing from the gist of the invention. These embodiments and their modifications are included within the scope and gist of the invention as well as within the scope of the invention described in the claims and its equivalents.

100 Ultrasonic diagnostic device 170 Processing circuit 171 Acquisition function 172 Tracking function 173 Judgment function 174 Output control function

Claims (22)

an acquisition unit that acquires a plurality of first medical information representing wall motion parameters in each of a first local region and a first overall region regarding the heart of a first subject;
a determination unit that determines an abnormality in the first local region based on the plurality of first medical information;
an output control unit that calculates a plurality of types of index values for each of the plurality of first medical information;
Equipped with
The output control unit outputs an index value specified for the abnormality determination result for the first local region among the plurality of types of index values, together with the abnormality determination result for the first local region. Output,
equipment . The determination unit includes a plurality of pieces of second medical information representing wall motion parameters in each of a second local region and a second whole region, and an abnormality determination result for the second local region, regarding the hearts of a plurality of second subjects. determining an abnormality in a first local region of the first subject by inputting the plurality of first medical information about the first subject to a trained model trained using
The device according to claim 1. The wall motion parameters include different types of information in at least one of the load or the time phase before and after treatment, the position in the wall thickness direction, and the heart motion direction,
The plurality of second medical information includes the same type of information as the plurality of first medical information,
3. The device according to claim 2. Each of the first entire region and the second entire region corresponds to the entire left ventricle in a three-dimensional image or the entire region of interest in a two-dimensional cross-sectional image,
The first local region corresponds to a plurality of regions obtained by dividing the first overall region by a predetermined division method,
The second local region corresponds to a plurality of regions obtained by dividing the second entire region using a predetermined division method.
4. The device according to claim 3. The wall motion parameter includes information on at least one of strain and displacement.
The device according to claim 3 or 4. The time phase includes information on at least one of a preload period and a recovery period.
Apparatus according to any one of claims 3 to 5. The position in the wall thickness direction includes information on at least one of the inner layer, middle layer, and outer layer, or information on at least one of the inner layer, the outer layer, and the wall thickness.
Apparatus according to any one of claims 3 to 6. The movement direction includes information on at least one of the longitudinal direction, the circumferential direction, the radial direction, and the area of a plane parallel to the endomyocardial surface.
Apparatus according to any one of claims 3 to 7. The information on the wall motion parameters of each of the first overall area and the second overall area further includes information representing a time-series change in volume change.
Apparatus according to any one of claims 3 to 8. The trained model is constructed based on a support vector machine or a neural network.
Apparatus according to any one of claims 2 to 9. The plurality of types of index values include at least SI-DI (Strain Imaging Diastolic Index) and PSI (Post-systolic Strain Index),
The device according to claim 1 . further comprising an output control unit that superimposes and displays the abnormality determination result for the first local region on an ultrasound image or a polar map;
Apparatus according to any one of claims 1 to 11 . The first medical information and the second medical information are information standardized in the time direction.
Device according to any one of claims 2 to 10. The abnormality determination result for the first local area is information indicating whether the first local area is an ischemic area.
Apparatus according to any one of claims 1 to 13 . The first medical information is waveform information representing time-series changes in wall motion parameters in each of the first local region and the first overall region,
The second medical information is waveform information representing time-series changes in wall motion parameters in each of the second local region and the second entire region.
Device according to any one of claims 2 to 10. The abnormality determination result for the second local area is information on which a person has identified whether or not there is a wall motion abnormality.
Device according to any one of claims 2 to 10. further comprising an output control unit that outputs the abnormality determination result for the first local area;
Apparatus according to any one of claims 1 to 16 . acquiring the plurality of first medical information from an ultrasound image;
Apparatus according to any one of claims 1 to 17 . acquiring a plurality of first medical information representing wall motion parameters in each of a first local region and a first overall region regarding the heart of a first subject;
determining an abnormality in the first local region based on the plurality of first medical information;
calculating a plurality of types of index values for each of the plurality of first medical information;
Outputting an index value identified for the abnormality determination result for the first local region among the plurality of types of index values together with the abnormality determination result for the first local region;
A program that causes a computer to perform various processes. For the hearts of a plurality of subjects, the time direction of first waveform information representing wall motion parameters in a local region and the time direction of second waveform information representing time-series changes in volume in the entire region are calculated based on cardiac time. Normalize based on an arbitrary interval in the phase,
Machine learning is performed using the standardized first waveform information, the standardized second waveform information, and the abnormality determination result regarding ischemic heart disease in the local region to create a trained model. A medical information processing device comprising a learning section to be constructed. an acquisition unit that acquires a plurality of first medical information representing wall motion parameters in each of a first local region and a first overall region regarding the heart of a first subject;
a determination unit that determines an abnormality in the first local region based on the plurality of first medical information;
an output control unit that calculates a plurality of types of index values for each of the plurality of first medical information;
Equipped with
The plurality of types of index values include at least SI-DI (Strain Imaging Diastolic Index) and PSI (Post-systolic Strain Index),
equipment . acquiring a plurality of first medical information representing wall motion parameters in each of a first local region and a first overall region regarding the heart of a first subject;
determining an abnormality in the first local region based on the plurality of first medical information;
calculating a plurality of types of index values for each of the plurality of first medical information;
A program that causes a computer to execute each process,
The plurality of types of index values include at least SI-DI (Strain Imaging Diastolic Index) and PSI (Post-systolic Strain Index),
program .

Images