JP2021164572A - Device and program - Google Patents

Abstract

To accurately identify a position of a tricuspid regurgitation maximum flow velocity (TR-Vmax) even when folding exists or the like, in a case where a Doppler waveform image includes much noise.SOLUTION: A device according to the embodiment comprises an acquisition part and a detection part. The acquisition part acquires first Doppler waveform data concerning tricuspid regurgitation. The detection part detects a position of the maximum value of a Doppler waveform in the first Doppler waveform data by using: a learned model learned with training data including at least the position of the maximum value of the Doppler waveform in each of a plurality of pieces of second Doppler waveform data concerning tricuspid regurgitation and the pieces of second Doppler waveform data; and the first Doppler waveform data.SELECTED DRAWING: Figure 1

Description

Embodiments disclosed herein and in the drawings relate to devices and programs.

For example, when diagnosing tricuspid valve regurgitation, the region including the tricuspid valve is imaged with a pulse Doppler, and the maximum tricuspid valve regurgitation flow velocity (TR- Vmax) is specified. Identification of TR-Vmax on this Doppler waveform image is executed by algorithm calculation or artificially.

However, when there is a lot of noise in the Doppler waveform image, or when there is wrapping, there is a possibility that the position of TR-Vmax cannot be accurately specified by the conventional method.

Japanese Unexamined Patent Publication No. 2018-187087

Christopher M. Bishop (Christopher M. Bishop), "Pattern recognition and machine learning", (USA), 1st Edition, Springer, 2006, P.M. 225-290

One of the problems to be solved by the embodiments disclosed in the present specification and the drawings is to accurately specify the position of TR-Vmax even when there is a lot of noise in the Doppler waveform image or there is wrapping. That is. However, the problems to be solved by the embodiments disclosed in the present specification and the drawings are not limited to the above problems. It is also possible to position the problem corresponding to each effect of each configuration shown in the embodiment described later as another problem.

The device according to the embodiment includes an acquisition unit and a detection unit. The acquisition unit acquires the first Doppler waveform data relating to tricuspid valve regurgitation. The detection unit has been trained by training data including at least the position of the maximum value of the Doppler waveform in each of the plurality of second Doppler waveform data relating to the tricuspid valve regurgitation and the plurality of second Doppler waveform data. Using the model and the first Doppler waveform data, the position of the maximum value of the Doppler waveform in the first Doppler waveform data is detected.

FIG. 1 is a diagram showing a configuration example of an ultrasonic diagnostic apparatus according to an embodiment. FIG. 2 is a diagram showing an example of the input / output relationship of the trained model during the operation of the TR-Vmax detection process. FIG. 3 is a flowchart showing an example of the flow of the TR-Vmax detection process. FIG. 4 is a diagram showing an example of a Doppler waveform image displayed on the display device as a result of the TR-Vmax detection process according to the embodiment. FIG. 5 is a diagram showing an example of data input / output for learning a multi-layered network in the model generation process according to the embodiment. FIG. 6 is a diagram showing an example of a Doppler mode display screen including multi-noise waveform data used in generating a trained model. FIG. 7 is a diagram showing an example of a Doppler mode display screen including the folded waveform data used in the generation of the trained model. FIG. 8 is a diagram showing an example of a Doppler mode display screen including normal waveform data used in generating a trained model.

Hereinafter, the apparatus and the program related to the present embodiment will be described with reference to the drawings. In order to make the explanation concrete, an ultrasonic diagnostic apparatus will be described as an example of the apparatus according to the present embodiment. In the following embodiments, the parts with the same reference numerals perform the same operation, and duplicate description will be omitted as appropriate.

(Embodiment)
FIG. 1 is a diagram showing a configuration example of the ultrasonic diagnostic apparatus 100 according to the present embodiment. As shown in FIG. 1, the ultrasonic diagnostic apparatus 100 includes an ultrasonic probe 1, an input device 3, a display device (display unit) 5, and a device main body 7.

The ultrasonic probe 1 has a plurality of piezoelectric vibrators, a matching layer provided on the piezoelectric vibrator, a backing material for preventing the propagation of ultrasonic waves from the piezoelectric vibrator to the rear, and the like. The ultrasonic probe 1 is detachably connected to the apparatus main body 7. The plurality of piezoelectric vibrators generate ultrasonic waves based on the drive signal supplied from the ultrasonic transmission circuit 71 in the apparatus main body 7. The ultrasonic probe 1 may be provided with a button that is pressed during various operations such as a freeze operation.

When ultrasonic waves are transmitted from the ultrasonic probe 1 to the subject P, the transmitted ultrasonic waves are reflected one after another at the discontinuity surface of the acoustic impedance in the body tissue of the subject P. The reflected ultrasonic waves are received as reflected wave signals (hereinafter referred to as echo signals) by a plurality of piezoelectric vibrators included in the ultrasonic probe 1. The amplitude of the received echo signal depends on the difference in acoustic impedance on the discontinuity where the ultrasonic waves are reflected. The echo signal when the transmitted ultrasonic pulse is reflected on the moving blood flow or the surface of the heart wall, etc., has a frequency depending on the velocity component with respect to the ultrasonic transmission direction of the moving body due to the Doppler effect. Receive a shift. The ultrasonic probe 1 receives an echo signal from the subject P and converts it into an electric signal. In the present embodiment, the ultrasonic probe 1 is, for example, a 1D array probe in which a plurality of piezoelectric vibrators are arranged along a predetermined direction, and a 2D array probe in which a plurality of piezoelectric vibrators are arranged in a two-dimensional matrix. Or, a mechanical 4D probe or the like capable of performing ultrasonic scanning while mechanically fanning the piezoelectric vibrator train in a direction orthogonal to the arrangement direction.

The input device 3 receives various input operations from the operator, converts the received input operations into electric signals, and outputs the received input operations to the processing circuit 87. The input device 3 includes, for example, a mouse, a keyboard, a trackball, a switch, a button, a joystick, a touch pad, a touch panel display, and the like. In the present embodiment, the input device 3 is not limited to the one provided with physical operation parts such as a mouse, a keyboard, a trackball, a switch, a button, a joystick, a touch pad, and a touch panel display. For example, an example of the input device 3 is an electric signal processing circuit that receives an electric signal corresponding to an input operation from an external input device provided separately from the input device 3 and outputs the electric signal to the processing circuit 87. included. Further, the input device 3 may be provided on the device main body 7. Further, the input device 3 may be composed of a tablet terminal or the like capable of wireless communication with the device main body 7.

For example, in response to pressing the end button or pressing the freeze button (hereinafter referred to as a freeze operation) in the input device 3, the ultrasonic diagnostic apparatus 100 interrupts the transmission / reception of ultrasonic waves and enters a paused state.

Further, the ultrasonic diagnostic apparatus 100 has a real-time display mode in which an ultrasonic image generated by transmitting and receiving ultrasonic waves is displayed in real time in response to a freeze operation from the input device 3 during scanning in the B mode. Then, the mode shifts to a cine display mode in which a plurality of ultrasonic images stored in the image memory 83 can be displayed in time series (hereinafter, referred to as cine display).

Further, the ultrasonic diagnostic apparatus 100 has a real-time display mode in which the Doppler waveform generated by transmitting and receiving ultrasonic waves is displayed in real time in response to a freeze operation from the input device 3 during scanning in the Doppler mode. , Shift to scroll display mode. Here, the scroll display mode is a mode in which a plurality of Doppler waveform images stored in the image memory 83 can be scrolled in the forward direction or the reverse direction in chronological order and displayed. For example, in the scroll display mode, when the operator rotates the trackball or the like, the ultrasonic diagnostic apparatus 100 determines the rotation direction and the amount of rotation of the trackball among the plurality of Doppler waveform images stored in the image memory 83. Read and display the corresponding Doppler waveform image. The rotation of the trackball corresponds to a scrolling operation in which the Doppler waveform image in the time series is scrolled in the forward or reverse direction in the time series after the input of the freeze operation.

The display device 5 is an arbitrary display such as a liquid crystal display (LCD: Liquid Crystal Display), a CRT (Cathode Ray Tube) display, an organic EL display (OELD: Organic Electro Luminescence Display), or a plasma display. The display device 5 may be incorporated in the device main body 7. Further, the display device 5 may be a desktop type, or may be composed of a tablet terminal or the like capable of wireless communication with the device main body 7. The display device 5 is an example of a display unit.

The display device 5 displays various types of information. For example, the display device 5 provides an ultrasonic image generated by the processing circuit 87 and the image generation circuit 79, a user interface for receiving various operations from the operator (hereinafter, referred to as a GUI (Graphical User Interface)), and the like. indicate. In the cine display mode, the display device 5 displays ultrasonic images in chronological order according to the instruction of the operator via the input device 3. In the scroll display mode, the display device 5 displays a time-series Doppler waveform image according to the instruction of the operator via the input device 3. Further, for example, in the Doppler mode, when the scroll operation is input after the freeze operation is input (scroll display), the display device 5 has a Doppler waveform corresponding to at least one heartbeat according to the direction and amount of the scroll operation. Is displayed.

The device main body 7 is a device that generates an ultrasonic image based on an echo signal received by the ultrasonic probe 1. As shown in FIG. 1, the apparatus main body 7 includes an ultrasonic transmission circuit 71, an ultrasonic reception circuit 73, a B mode processing circuit 75, a Doppler processing circuit 77, an image generation circuit 79, an internal storage circuit (storage unit) 81, and an image. It has a memory 83 (also referred to as a cine memory or cache), a communication interface 85, and a processing circuit 87.

The ultrasonic transmission circuit 71 is a processor that supplies a drive signal to the ultrasonic probe 1. The ultrasonic transmission circuit 71 includes, for example, a trigger generation circuit, a delay circuit, a pulsar circuit, and the like. The trigger generation circuit repeatedly generates rate pulses for forming transmitted ultrasonic waves at a predetermined rate frequency by the system control function 871 in the processing circuit 87. The delay circuit gives each rate pulse a delay time for each piezoelectric vibrator, which is necessary for focusing the ultrasonic waves generated from the ultrasonic probe 1 in a beam shape and determining the transmission directivity. The pulsar circuit applies a drive signal (drive pulse) to the ultrasonic probe 1 at a timing based on the rate pulse by the system control function 871. By changing the delay time given to each rate pulse by the delay circuit, the transmission direction from the piezoelectric vibrator surface can be arbitrarily adjusted.

The ultrasonic wave receiving circuit 73 is a processor that performs various processes on the echo signal received by the ultrasonic probe 1 to generate a received signal. The ultrasonic wave receiving circuit 73 includes, for example, an amplifier circuit, an A / D converter, a reception delay circuit, an adder, and the like. The amplifier circuit amplifies the echo signal received by the ultrasonic probe 1 for each channel and performs gain correction processing. The A / D converter converts the gain-corrected echo signal into a digital signal. The reception delay circuit provides the digital signal with the delay time required to determine the reception directivity. The adder adds a plurality of digital signals with a delay time. The addition process of the adder generates a received signal in which the reflection component from the direction corresponding to the reception directivity is emphasized.

The B-mode processing circuit 75 is a processor that generates B-mode data based on the received signal received from the ultrasonic wave receiving circuit 73. The B-mode processing circuit 75 performs envelope detection processing, logarithmic amplification processing, and the like on the received signal received from the ultrasonic reception circuit 73, and expresses the signal strength in terms of brightness (that is, by the B mode). The acquired data (hereinafter referred to as B mode data) is generated. The generated B-mode data is stored in a RAW data memory (not shown) as B-mode RAW data on a two-dimensional ultrasonic scanning line.

The Doppler processing circuit 77 is a processor that generates Doppler waveform data and Doppler data based on the received signal received from the ultrasonic wave receiving circuit 73. The Doppler processing circuit 77 extracts a blood flow signal from the received signal, generates Doppler waveform data from the extracted blood flow signal, and extracts information such as average velocity, dispersion, and power from the blood flow signal at multiple points. (That is, the data acquired by the Doppler mode. Hereinafter referred to as Doppler data) is generated. The generated Doppler data is stored in a RAW data memory (not shown) as Doppler RAW data on a two-dimensional ultrasonic scanning line.

The image generation circuit 79 generates a GUI for the operator to input various instructions via the input device 3. The image generation circuit 79 is a processor having a function (scan converter) of generating various ultrasonic image data based on the data generated by the B mode processing circuit 75 and the Doppler processing circuit 77. The image generation circuit 79 includes an internal memory (not shown). The image generation circuit 79 generates two-dimensional ultrasonic image data (B mode image data, color Doppler image data, Doppler waveform image data, etc.) composed of pixels by executing RAW-pixel conversion. The image generation circuit 79 stores the generated ultrasonic image data in the internal storage circuit 81. The image generation circuit 79 executes various image processing such as dynamic range, brightness (brightness), contrast, γ-curve correction, and RGB conversion on the generated ultrasonic image data.

The image generation circuit 79 can also generate volume data composed of voxels in a desired range by executing interpolation processing or the like in which spatial position information is added to B-mode image data or the like. .. The image generation circuit 79 generates volume data by executing RAW-voxel conversion including interpolation processing in which spatial position information is added to the B mode RAW data stored in the RAW data memory. You may. Further, the image generation circuit 79 may generate a rendered image or an MPR image by performing, for example, rendering processing or multi-section conversion reconstruction (hereinafter referred to as MPR (Multi Planar Reconnection)) processing on various volume data. good.

The internal storage circuit 81 is realized, for example, by a magnetic or optical storage medium, a storage medium that can be read by a processor such as an integrated circuit storage device, or the like. For example, the internal storage circuit 81 corresponds to an HDD (Hard disk Drive), an SSD (Solid State Drive), a semiconductor memory, or the like that stores various information. In addition to HDDs and SSDs, the internal storage circuit 17 includes portable storage media such as CDs (Compact Discs), DVDs (Digital Versailles Discs), and flash memories, semiconductor memory elements such as RAMs (Random Access Memory), and the like. It may be a drive device that reads and writes various information between the two.

The internal storage circuit 81 stores programs and the like for realizing various functions according to the present embodiment. The internal storage circuit 81 includes data such as diagnostic information (for example, patient ID, doctor's findings, etc.), a diagnostic protocol, a body mark generation program, and a conversion table in which the range of color data used for visualization is preset for each diagnostic site. Memorize the group. Various data stored in the internal storage circuit 81 can also be transferred to an external device via the communication interface 21 by the system control function 871. The internal storage circuit 81 stores the trained model. The trained model may be stored in the memory of the processing circuit 87 itself. The trained model is operated by the detection function 875 in the processing circuit 87 in the detection of the maximum flow velocity value (TR-Vmax) regarding the tricuspid valve regurgitation (hereinafter referred to as TR-Vmax detection processing).

FIG. 2 is a diagram showing an example of the input / output relationship of the trained model during the operation of the TR-Vmax detection process. As shown in FIG. 2, the trained model outputs the position of TR-Vmax on the Doppler waveform in the Doppler waveform image data as a result of the TR-Vmax detection process by inputting the Doppler waveform image data. That is, the trained model is trained to include at least the position of the maximum value of the Doppler waveform in each of the plurality of Doppler waveform image data (plurality of second Doppler waveform data) relating to the tricuspid valve regurgitation and the plurality of Doppler waveform image data. It is a model trained by data. Specifically, the trained model inputs Doppler waveform image data and outputs coordinates corresponding to the position of TR-Vmax on the Doppler waveform in the Doppler waveform image data.

The Doppler waveform image data input to the trained model may include Doppler waveforms of at least one heartbeat or more. When the Doppler waveform image data input to the trained model includes Doppler waveforms of multiple heartbeats, the trained model outputs, for example, the coordinates of TR-Vmax on each Doppler waveform as a result of TR-Vmax detection processing. .. However, not limited to this example, the coordinates of TR-Vmax on the Doppler waveform corresponding to at least one heartbeat among the Doppler waveforms of a plurality of heartbeats may be output as a result of the TR-Vmax detection process.

Further, the trained model is not limited by the image quality of the input Doppler waveform image data. For example, in the trained model, the input Doppler waveform image data may be any of image data containing a large amount of noise, image data including wrapping (aliasing) of the Doppler waveform, and image data having low brightness.

The trained model is a plurality of Doppler waveform image data including Doppler waveforms each corresponding to at least one heartbeat, and teacher data (correct answer) which is the coordinates of TR-Vmax on the Doppler waveform for each of the plurality of Doppler waveform image data. It is generated by performing machine learning on a multi-layered network, for example, using training data (learning data) composed of a combination with data). The multi-layered network is, for example, a machine learning model such as a deep neural network (hereinafter referred to as DNN) or a convolutional neural network (hereinafter referred to as CNN). Learning for a multi-layered network corresponds to adjusting multiple parameters in a multi-layered network. The model to be machine-learned is not limited to the multi-layered network, and any model can be used as long as the input / output relationship with the trained model can be maintained.

The teacher data included in the training data, that is, the coordinates of TR-Vmax on the Doppler waveform included in each Doppler waveform image data are generated, for example, using an existing algorithm or based on artificial processing by a doctor or a technician. be able to. The process related to the generation of the trained model (hereinafter referred to as the model generation process) will be described later.

The image memory 83 has, for example, a recording medium such as a semiconductor memory that can be read by a processor. The image memory 83 is realized by, for example, a cache memory. The image memory 83 stores data of various images acquired in a certain period retroactively from the freeze operation input via the input device 3. Specifically, the image memory 83 stores Doppler waveform image data for a predetermined past fixed period from the moment the freeze button is pressed so as not to be overwritten by other data in order to perform scroll display. The internal storage circuit 81 and the image memory 83 may be integrated as one storage device.

The communication interface 85 is connected to an external device via a network. The communication interface 85 performs data communication with an external device via a network. The external device is, for example, a medical image management system (PACS (Picture Archiving and Communication System)), which is a system for managing data of various medical images, an electronic medical record system for managing an electronic medical record to which a medical image is attached, and the like. The standard for communication with an external device may be any standard, and examples thereof include DICOM (Digital Imaging and Communications in System).

The processing circuit 87 is, for example, a processor that functions as the center of the ultrasonic diagnostic apparatus 100. The processing circuit 87 has a processor such as a CPU (Central Processing Unit) and an MPU (Micro Processing Unit) and a memory such as a ROM (Read Only Memory) and a RAM (Random Access Memory) as hardware resources. Further, the processing circuit 87 includes an integrated circuit for a specific application (Application Special Integrated Circuit: ASIC), a field programmable gate array (Field Programmable Gate Array: FPGA), and another composite programmable logic device (Complex Programmable). ), Or a simple programmable logic device (SPLD).

The processing circuit 87 has various functions such as a system control function 871, an acquisition function 873, a detection function 875, and a display control function 877. The processing circuit 87 expands various programs stored in the internal storage circuit 81 into its own memory and executes them, so that the system control function 871, the acquisition function 873, the detection function 875, and the display control corresponding to the programs are executed. Perform function 877. The program may be configured to incorporate the program directly into the circuit of the processor instead of being stored in the internal storage circuit 81. In this case, the processor realizes the above function by reading and executing a program incorporated in the circuit.

The processing circuit 87 that executes the system control function 871, the acquisition function 873, the detection function 875, and the display control function 877, respectively, corresponds to the system control unit, the acquisition unit, the detection unit, and the display control unit. The system control function 871, the acquisition function 873, the detection function 875, and the display control function 877 are not limited to those realized by a single processing circuit. A processing circuit may be formed by combining a plurality of independent processors, and the system control function 871, the acquisition function 873, the detection function 875, and the display control function 877 may be realized by executing the program by each processor. ..

The processing circuit 87 controls basic operations such as input / output of the ultrasonic diagnostic apparatus 100 by the system control function 871. When the system control function 871 is executed, the processing circuit 87 receives inputs of various scan modes, for example, via an input device 3. The processing circuit 87 executes various ultrasonic scans according to the received scan mode and generates various ultrasonic images. For example, when the scan mode is the pulse Doppler mode, the processing circuit 87 controls the ultrasonic transmission circuit 71, the ultrasonic reception circuit 73, the Doppler processing circuit 77, and the image generation circuit 79 to obtain time-series Doppler waveform image data. Generate.

The processing circuit 87 scans the blood flow in the vicinity of the tricuspid valve of the subject P in the pulse Doppler mode by the acquisition function 873, and acquires Doppler waveform image data regarding the tricuspid valve regurgitation. Specifically, when the freeze operation is input via the input device 3 in the real-time display mode, the processing circuit 87 acquires the Doppler waveform image data displayed on the display device 5. The processing circuit 87 acquires the trained model from the internal storage circuit 81, triggered by the input of the freeze operation. The processing circuit 87 is triggered by the input of the execution instruction of the TR-Vmax detection process (hereinafter referred to as the TR-Vmax detection instruction) instead of the input of the freeze operation, and the Doppler waveform image displayed on the display device 5. Data may be acquired. When the trained model is stored in the memory of the processing circuit 87 itself, the processing circuit 87 acquires the trained model from its own memory when the freeze operation is input or the like.

The processing circuit 87 inputs the Doppler waveform image data to the trained model by the detection function 875, and the trained model detects the coordinates of TR-Vmax on the Doppler waveform included in the Doppler waveform image data. Specifically, the processing circuit 87 inputs the Doppler waveform image data of the subject P as the first Doppler waveform data into the trained model to obtain TR-Vmax on the Doppler waveform included in the Doppler waveform image data. Detect coordinates. The processing circuit 87 calculates a flow velocity value or the like corresponding to TR-Vmax for each heartbeat based on the detected coordinates of TR-Vmax. The flow velocity value based on the detected TR-Vmax coordinates may be calculated from the graph on the Doppler waveform image data input to the trained model, or calculated from the Doppler data generated by the Doppler processing circuit 77. You can also do it.

Further, the trained model may output the flow velocity value of TR-Vmax for each heartbeat together with the coordinates of TR-Vmax for each heartbeat. Further, the processing circuit 87 associates the TR-Vmax coordinates for each heartbeat output from the trained model with the corresponding Doppler waveform included in the Doppler image data input to the trained model, and is an internal storage circuit. It may be stored in 81.

The processing circuit 87 maps the position of TR-Vmax for each heartbeat on the corresponding Doppler waveform based on the detected coordinates of TR-Vmax for each heartbeat by the display control function 877. The processing circuit 87 causes the display device 5 to display the Doppler waveform image data in which the TR-Vmax position (coordinates) for each heartbeat is superimposed on the corresponding Doppler waveform. Specifically, the processing circuit 87 displays Doppler waveform image data in which the position of TR-Vmax corresponding to each Doppler waveform is indicated by a guide based on the detected coordinates of TR-Vmax for each heartbeat. Displayed on the device 5.

(TR-Vmax detection process)
Next, the TR-Vmax detection process executed by the ultrasonic diagnostic apparatus 100 will be described. FIG. 3 is a flowchart showing an example of the flow of the TR-Vmax detection process. In order to make the explanation concrete, the trained model is input as Doppler waveform image data obtained by scanning the blood flow near the tricuspid valve of the subject P in the pulse Doppler mode, and is included in the Doppler waveform image data. It is assumed that the coordinates of TR-Vmax on the Doppler waveform for each heartbeat are output. In addition, the input of Doppler waveform image data to the trained model shall be executed triggered by the freeze operation in the real-time display mode.

(Step S301)
By transmitting and receiving ultrasonic waves to the subject P, the Doppler processing circuit 77 generates Doppler mode RAW data, and stores the generated Doppler mode RAW data in the RAW data memory. The image generation circuit 79 generates Doppler waveform image data based on the Doppler mode RAW data read from the RAW data memory. The display device 5 displays a Doppler waveform image. When the scroll display mode is executed, in this step, the display device 5 displays the Doppler waveform corresponding to at least one heartbeat corresponding to the scroll operation.

(Step S302)
When the freeze operation is executed via the input device 3 (Yes in step S302), the process of step S303 is executed. If the freeze operation is not executed via the input device 3, (No in step S302) and the process of step S301 are executed. When the scroll display mode is executed, when an execution instruction of the TR-Vmax detection process (for example, pressing a button for the TR-Vmax detection process) is input via the input device 3, the process is performed in step S303. Is executed. Further, when the scroll display mode is executed, if the execution instruction of the TR-Vmax detection process is not input via the input device 3 in this step, the scroll display described above is executed in step S301.

(Step S303)
The processing circuit 87 acquires the data corresponding to the Doppler waveform image displayed on the display device 5 from the internal storage circuit 81 or the image memory 83 by the acquisition function 873. The processing circuit 87 reads the trained model from the internal storage circuit 81. At this time, the Doppler waveform image displayed on the display device 5 may include a large amount of noise, the Doppler waveform may be folded back, or the brightness may be low. Further, the Doppler waveform image displayed on the display device 5 may include a Doppler waveform corresponding to at least one heartbeat.

(Step S304)
The processing circuit 87 inputs the acquired Doppler waveform image data to the trained model by the detection function 875. As a result, the processing circuit 87 outputs the TR-Vmax coordinates for the Doppler waveform of each heartbeat included in the input Doppler waveform image data by the trained model.

(Step S305)
The processing circuit 87 maps the position of TR-Vmax for each heartbeat on the corresponding Doppler waveform based on the coordinates of TR-Vmax for the Doppler waveform of each heartbeat by the display control function 877. As a result, Doppler waveform image data in which the TR-Vmax position (coordinates) for each heartbeat is superimposed on the corresponding Doppler waveform is generated.

(Step S306)
The processing circuit 87 causes the display device 5 to display Doppler waveform image data in which the position of TR-Vmax corresponding to each Doppler waveform is indicated by a guide based on the detected coordinates of TR-Vmax for each heartbeat. .. The display device 5 displays a Doppler waveform image in which the position of TR-Vmax is indicated by a guide.

FIG. 4 is a diagram showing an example of a Doppler waveform image displayed on the display device 5 as a result of the TR-Vmax detection process according to the embodiment. As shown in FIG. 4, the display device 5 includes a Doppler waveform image 40 of three heartbeats, a color Doppler image 41, TR-Vmax guides 42a, 42b, 42c, an ECG (Electrocardiography) waveform 43, and an index 44. It is displayed. That is, in the example of FIG. 4, the position of TR-Vmax in the Doppler waveform of each heartbeat of the Doppler waveform image 40 is indicated by the TR-Vmax guides 42a, 42b, and 42c superimposed on the Doppler waveform image 40. .. The broken line of the TR-Vmax guide indicates the position on the time axis, and the + mark indicates the position on the Doppler waveform. Further, in the index 44, the value of TR-Vmax or the like related to the Doppler waveform of the selected heartbeat is displayed.

(Step S307)
When the end instruction of the TR-Vmax detection process is input via the input device 3 (Yes in step S307), the TR-Vmax detection process ends. When the end instruction of the TR-Vmax detection process is not input via the input device 3 (No in step S307), the processes after step S301 are repeated.

(Model generation process)
Next, the model generation process related to the generation of the trained model will be described. This will be described with reference to FIG. FIG. 5 is a diagram showing an example of data input / output for learning a multi-layered network MLN in a model generation process. The trained model is generated, for example, by a learning device different from the ultrasonic diagnostic device 100. The learning device is realized by a stand-alone (independent) computer, a server provided on the network, or the like. Further, it is assumed that the learning data is stored in the memory or storage device mounted on the learning device or the training data storage device.

Hereinafter, for the sake of specificity, the plurality of second Doppler waveform data input to the multi-layered network MLN as training data include, for example, at least one of multi-noise waveform data and folded waveform data. Here, the multi-noise waveform data means Doppler waveform data including noise of a certain reference or more in addition to the Doppler waveform corresponding to at least one heartbeat. The folded waveform data means Doppler waveform data including a Doppler waveform corresponding to at least one heartbeat in which the folded waveform occurs. In the present embodiment, in order to make the explanation concrete, the plurality of second Doppler waveform data input to the multi-layered network MLN as training data are multi-noise waveform data, folded waveform data, and normal waveform data (noise is the reference). It is assumed that the Doppler waveform data including the Doppler waveform corresponding to at least one heartbeat, which is less than or equal to the value and does not cause wrapping, is included.

The second Doppler waveform data used for the training data includes Doppler waveforms corresponding to a plurality of types of heart rates. That is, the Doppler waveform included in the second Doppler waveform data used for the training data may be an arbitrary heart rate. Typically, a second Doppler waveform data including Doppler waveforms for 3 to 5 heartbeats can be used.

FIG. 6 is a diagram showing an example of a Doppler mode display screen including multi-noise waveform data used in generating a trained model. The Doppler mode display screen shown in FIG. 6 displays a Doppler waveform image 45 for three heartbeats, a color Doppler image 46, TR-Vmax guides 47, 48, 49, and an ECG waveform 50. Further, the Doppler waveform image 45 for three heartbeats as the multi-noise waveform data shown in FIG. 6 includes a multi-noise such as a mist (exemplified as a shaded area 51 in FIG. 6). The correct answer data corresponding to the multi-noise waveform data shown in FIG. 6 is the coordinates marked with + in the TR-Vmax guides 47, 48, and 49 shown in FIG.

FIG. 7 is a diagram showing an example of a Doppler mode display screen including the folded waveform data used in the generation of the trained model. The Doppler mode display screen shown in FIG. 7 displays a Doppler waveform image 55 for five heartbeats, a color Doppler image 56, TR-Vmax guides 57, 58, 59, 60, 61, and an ECG waveform 62. Further, the Doppler waveform image 55 for 5 heartbeats as the folded waveform data shown in FIG. 7 includes the folded waveform 63 in each heartbeat. The correct answer data corresponding to the folded waveform data shown in FIG. 7 is the coordinates marked with + of the TR-Vmax guides 57, 58, 59, 60, and 61 shown in FIG.

FIG. 8 is a diagram showing an example of a Doppler mode display screen including normal waveform data used in generating a trained model. The Doppler mode display screen shown in FIG. 8 displays Doppler waveform image 65 for three heartbeats, color Doppler image 66, TR-Vmax guides 67, 68, 69, and ECG waveform 70 as normal waveform data. In the Doppler mode display screen example of FIG. 8, an index 72 indicating the value of TR-Vmax for each heartbeat is further displayed. The correct answer data corresponding to the normal waveform data shown in FIG. 8 is the coordinates marked with + in the TR-Vmax guides 67, 68, 69d shown in FIG.

As shown in FIG. 5, the second Doppler waveform data as shown in FIGS. 6 to 8 is input to the multi-layered network MLN. The learning device differs between the output data from the multi-layered network MLN and the correct answer data corresponding to the second Doppler waveform data input to the multi-layered network MLN. The learning device adjusts a plurality of parameters in the multi-layered network MLN by, for example, an error back propagation method so that the difference (error) becomes a certain value or less. The learning device inputs a second Doppler waveform data different from the second Doppler waveform data used for the adjustment to the multi-layered network MLN in which a plurality of parameters are adjusted. Similarly, the learning device further adjusts a plurality of parameters in the multi-layered network MLN. As the learning process for the multi-layered network MLN, for example, the existing method described in Non-Patent Document 1 and the like can be appropriately used, and thus the description thereof will be omitted.

As described above, the ultrasonic diagnostic apparatus 100 according to the embodiment includes an acquisition function 873 as an acquisition unit and a detection function 875 as a detection unit. The acquisition function 873 acquires the first Doppler waveform data relating to tricuspid valve regurgitation. The detection function 875 is a trained model trained by training data including at least the position of the maximum value of the Doppler waveform in each of the plurality of second Doppler waveform data relating to the tricuspid valve regurgitation and the plurality of second Doppler waveform data. And the first Doppler waveform data are used to detect the position of the maximum value of the Doppler waveform in the first Doppler waveform data.

Specifically, the first ultrasonic image data and the second ultrasonic image data correspond to the Doppler waveform image data. According to the ultrasonic diagnostic apparatus 100 according to the embodiment, the trained model detects the position of TR-Vmax in the Doppler waveform for each heartbeat included in the first ultrasonic image data, and the detected TR-Vmax The position can be superimposed and displayed on the Doppler waveform imager corresponding to the first ultrasonic image data.

The trained model is, for example, a multi-layered network MLN trained using multi-noise waveform data, folded waveform data, low-luminance Doppler waveform data, and the like. Therefore, when the first ultrasonic image data input to the trained model contains noise and wrapping, the position of TR-Vmax can be accurately specified even when the Doppler waveform is thin image data. ..

The trained model is, for example, a multi-layered network MLN trained using data including Doppler waveforms corresponding to various heart rates. Therefore, the position of TR-Vmax can be accurately specified regardless of the number of Doppler waveforms included in the first ultrasonic image data input to the trained model.

From the above, according to the ultrasonic diagnostic apparatus 100 according to the embodiment, in the TR-Vmax detection process, when the Doppler waveform image contains noise, wrapping, etc., it is difficult to use the existing algorithm or artificial processing. Even if there is, the accurate TR-Vmax position can be automatically detected for the Doppler waveform for each heartbeat and can be superimposed and displayed on the Doppler waveform image. Thereby, according to this apparatus, it is possible to improve the throughput in the ultrasonic image diagnosis of heart failure.

(Modification example 1)
In the above embodiment, a trained model in which Doppler waveform image data is input and the coordinates of TR-Vmax on the Doppler waveform in the Doppler waveform image data are output has been described as an example. On the other hand, in addition to the Doppler waveform corresponding to at least one heartbeat, Doppler waveform image data including the ECG waveform acquired in synchronization with the Doppler waveform corresponding to at least one heartbeat is input, and the Doppler waveform image data is input. A trained model that outputs the coordinates of TR-Vmax on each Doppler waveform may be used.

The following training data is used to generate such a trained model. That is, in addition to the Doppler waveform corresponding to at least one heartbeat, the Doppler waveform image data including the ECG waveform acquired in synchronization with the Doppler waveform corresponding to at least one heartbeat is used as input data. Further, for each input data, the coordinates of TR-Vmax on the Doppler waveform are used as teacher data. It is generated by performing machine learning on a multi-layered network, for example, using training data consisting of a plurality of combinations of this input data and teacher data.

Further, if necessary, the image data of the expiratory waveform acquired in synchronization with the Doppler waveform included in the Doppler waveform image data is input together with the ECG waveform image data or in place of the ECG waveform image data. A trained model that outputs the coordinates of TR-Vmax on the Doppler waveform in the Doppler waveform image data may be used. In this case as well, the image data of the expiratory waveform may be included in the Doppler waveform image data, or may be data separate from the Doppler waveform image data.

For example, the following training data is used to generate a learning model for inputting image data of an expiratory waveform and an ECG waveform acquired in synchronization with the Doppler waveform included in the Doppler waveform image data. That is, in addition to the Doppler waveform corresponding to at least one heartbeat, the Doppler waveform image data including the ECG waveform and the expiratory waveform acquired in synchronization with the Doppler waveform corresponding to at least one heartbeat is used as input data. Further, for each input data, the coordinates of TR-Vmax on the Doppler waveform are used as teacher data. It is generated by performing machine learning on a multi-layered network, for example, using training data consisting of a plurality of combinations of this input data and teacher data.

(Modification 2)
In the above embodiment, a case where the Doppler waveform image data in which the TR-Vmax position (coordinates) for each heartbeat detected by the trained model is superimposed on the corresponding Doppler waveform is displayed on the display device 5 has been illustrated. On the other hand, the trained model may be configured to output Doppler waveform image data in which the position of TR-Vmax for each detected heartbeat is superimposed on the corresponding Doppler waveform. At this time, the trained model inputs the Doppler waveform image data including the Doppler waveform corresponding to at least one heartbeat in the detection function 875, and the Doppler waveform in which the TR-Vmax position for each heartbeat is superimposed on the corresponding Doppler waveform. Output image data.

(Modification example 3)
In the above embodiment, the case where the Doppler waveform image data generated by the image generation circuit 79 is input to the trained model and the TR-Vmax detection process is executed is illustrated. On the other hand, the Doppler waveform data generated by the Doppler processing circuit 77 (that is, the Doppler RAW data stored in the RAW data memory) is input to the trained model as the first Doppler waveform data, and the TR-Vmax detection process is executed. You may try to do so. In such a case, the model for executing the TR-Vmax detection process is generated by using the training data in which the Doppler RAW data is used as the second Doppler waveform data and the coordinates of TR-Vmax on the second Doppler waveform data are used as the teacher data. NS.

(Modification example 4)
In the above embodiment, the case where the device is the ultrasonic diagnostic device 100 has been illustrated. On the other hand, the device according to the present embodiment may be realized by, for example, a medical image processing device, a medical image processing server device, a workstation, or cloud computing. The medical imaging device, the medical image processing server device, the workstation or the cloud computing has, for example, the configuration within the dotted frame 9 shown in FIG. When the device is realized as a medical image processing server device, a workstation, or cloud computing, the input device 3 and the display device 5 may be connected to a network as, for example, a client device. At this time, for example, the internal storage circuit 81, the image memory 83, the communication interface 85, and the processing circuit 87 may be mounted on a server on the network.

When the technical idea in the present embodiment and this application example is realized by a program such as a medical image processing program, the program acquires the first Doppler waveform data regarding the tricuspid valve regurgitation in a computer, and a plurality of programs relating to the tricuspid valve regurgitation. A trained model trained by training data including at least the position of the maximum value (TR-Vmax) of the Doppler waveform in each of the second Doppler waveform data and a plurality of second Doppler waveform data, and the first Doppler waveform. Using the data, it is possible to detect the position of the maximum value of the Doppler waveform in the first Doppler waveform data. For example, TR-Vmax detection processing can also be realized by installing the program on a computer in a PACS server, an integrated server, an ultrasonic diagnostic apparatus 100, or the like in a hospital information system and expanding these programs on a memory. At this time, a program that allows a computer to execute the method can be stored and distributed in a storage medium such as a magnetic disk (hard disk, etc.), an optical disk (CD-ROM, DVD, etc.), or a semiconductor memory. .. Since the processing procedure and effect in the program are the same as those in the embodiment, the description thereof will be omitted.

According to at least one embodiment and modifications described above, the position of TR-Vmax can be accurately specified even when the Doppler waveform image has a lot of noise or folds.

Although some embodiments have been described, these embodiments are presented as examples and are not intended to limit the scope of the invention. These embodiments can be implemented in various other embodiments, and various omissions, replacements, changes, and combinations of embodiments can be made without departing from the gist of the invention. These embodiments and modifications thereof are included in the scope and gist of the invention, as well as in the scope of the invention described in the claims and the equivalent scope thereof.

1 Ultrasonic probe 3 Input device 5 Display device 7 Device body 9 Medical image processing device, medical image processing server device, or cloud computing 71 Ultrasonic transmission circuit 73 Ultrasonic reception circuit 75 B mode processing circuit 77 Doppler processing circuit 79 Image Generation circuit 81 Internal storage circuit 83 Image memory 85 Communication interface 87 Processing circuit 100 Ultrasonic diagnostic device 871 System control function 873 Acquisition function 875 Detection function 877 Display control function

Claims (10)

An acquisition unit that acquires the first Doppler waveform data related to tricuspid valve regurgitation,
A trained model trained by training data including at least the position of the maximum value of the Doppler waveform in each of the plurality of second Doppler waveform data relating to the tricuspid valve regurgitation and the plurality of second Doppler waveform data, and the first Using the Doppler waveform data, a detection unit that detects the position of the maximum value of the Doppler waveform in the first Doppler waveform data, and
A device equipped with. The acquisition unit acquires the first Doppler waveform data including the Doppler waveform of at least one heartbeat, and obtains the first Doppler waveform data.
The plurality of second Doppler waveform data includes at least one heartbeat Doppler waveform.
The device according to claim 1. The acquisition unit acquires an ECG waveform synchronized with the Doppler waveform included in the first Doppler waveform data, and obtains the ECG waveform.
The detection unit uses the trained model learned by the training data including the ECG waveform and the first Doppler waveform data to obtain the maximum value of the Doppler waveform in the first Doppler waveform data. Detect the position,
The device according to claim 1 or 2. The acquisition unit acquires an exhalation waveform synchronized with the Doppler waveform included in the first Doppler waveform data, and obtains the expiratory waveform.
The detection unit uses the trained model learned by the training data including the exhalation waveform and the first Doppler waveform data to obtain the maximum value of the Doppler waveform in the first Doppler waveform data. Detect the position,
The device according to any one of claims 1 to 3. The plurality of second Doppler waveform data includes at least Doppler waveform data whose noise is equal to or higher than the reference value.
The device according to any one of claims 1 to 4. The plurality of second Doppler waveform data includes at least Doppler waveform data including folding back of the Doppler waveform.
The device according to any one of claims 1 to 4. The plurality of second Doppler waveform data includes Doppler waveforms corresponding to a plurality of types of heart rates.
The device according to any one of claims 1 to 6. When the first Doppler waveform data includes Doppler waveforms of a plurality of heartbeats, the detection unit detects the position of the maximum value of the Doppler waveform for each heartbeat.
The device according to any one of claims 1 to 7. Using the detected coordinates of the maximum value of the Doppler waveform and the first Doppler waveform data, an image in which the position of the maximum value of the Doppler waveform is superimposed on the first Doppler waveform data is displayed on the display unit. It also has a display control unit,
The device according to any one of claims 1 to 8. On the computer
Obtained the first Doppler waveform data related to tricuspid valve regurgitation,
A trained model trained by training data including at least the position of the maximum value of the Doppler waveform in each of the plurality of second Doppler waveform data relating to the tricuspid valve regurgitation and the plurality of second Doppler waveform data, and the first Using the Doppler waveform data to detect the position of the maximum value of the Doppler waveform in the first Doppler waveform data,
A program that realizes.

Images