JP2021115211A - Ultrasonic diagnostic device, image processing method, learning device, and program - Google Patents

Abstract

To provide an ultrasonic diagnostic device that allows an image with good image quality to be acquired while reducing an influence of deterioration in a frame rate.SOLUTION: An ultrasonic diagnostic device includes an ultrasonic probe for transmitting/receiving an ultrasonic wave to/from a subject; and an estimation arithmetic unit for estimating data based on a reception signal acquired by transmission/reception of ultrasonic waves with a plurality of different transmission waveforms from third data based on a reception signal acquired by the transmission/reception of an ultrasonic wave of one transmission waveform by the ultrasonic probe by using a model machine-learned by using learning data including first data based on a reception signal acquired by the transmission/reception of an ultrasonic wave of one transmission waveform, and second data based on a reception signal acquired by the transmission/reception of ultrasonic waves with a plurality of different transmission waveforms.SELECTED DRAWING: Figure 2

Description

The present invention relates to an ultrasonic diagnostic apparatus, an image processing method, a learning apparatus and a program, and more particularly to a technique for improving the image quality of the ultrasonic diagnostic apparatus.

Ultrasound diagnostic devices are widely used in clinical practice as diagnostic imaging devices due to their simplicity, high resolution, and real-time performance. A general method of ultrasonic image generation includes the formation of a transmission beam and the phasing addition processing of a received signal. The transmission beam is formed by inputting voltage waveforms with a time delay given to a plurality of conversion elements and converging ultrasonic waves in the living body. The phasing addition of the received signal receives the ultrasonic waves reflected by the structure in the living body by a plurality of conversion elements, gives a time delay to the obtained received signal in consideration of the path length with respect to the point of interest, and further. It is carried out by adding. By the formation of the transmission beam and the phasing addition processing, the reflected signal from the point of interest is selectively extracted and imaged. By controlling the transmission beam to scan in the imaging region, an image of the region to be observed can be obtained.

In such an ultrasonic diagnostic apparatus, imaging (harmonic imaging) of a harmonic component with respect to a fundamental wave component of a transmission signal is widely performed. The method of imaging the harmonic components generated in the process of ultrasonic waves propagating in the subject is called tissue harmonic imaging (THI: Tissue Harmonic Imaging). Tissue Harmonic Imaging extracts the harmonic components generated due to the inherent non-linearity of the subject (living tissue) from the received signal and images the inside of the subject. Since the harmonic component generated by the non-linear effect is used, the effect of improving the resolution and reducing the artifact can be obtained. The Pulse Inversion method is one of the methods for extracting harmonic components in Tissue Harmonic Imaging. In the Pulse Inversion method, the fundamental wave component is canceled and the harmonic component is emphasized by adding the received signal obtained by transmitting and receiving the first transmission waveform and the second transmission waveform whose phase is inverted.

Patent Document 1 discloses a Pulse Inversion method in which penetration is improved while maintaining resolution by using a difference tone component in addition to a harmonic component. Patent Document 2 discloses a medical imaging device using a restorer configured by a neural network.

JP-A-2002-301668 Japanese Unexamined Patent Publication No. 2019-25044

In order to perform Tissue Harmonic Imaging by the Pulse Inversion method, it is necessary to send and receive multiple ultrasonic pulses to the same part of the subject. Therefore, there is a problem that the frame rate is lowered (simply speaking, the frame rate becomes 1 / N when N transmissions and receptions are performed, as compared with the case where an image is generated by one transmission and reception). Further, when the subject or the ultrasonic probe moves during N transmissions / receptions, there is also a problem that the image quality such as resolution and contrast deteriorates.

The present invention has been made in view of the above-mentioned problems, and an object of the present invention is to provide an ultrasonic diagnostic apparatus capable of obtaining an image with good image quality while reducing the influence of a decrease in frame rate.

A first aspect of the present invention includes an ultrasonic probe that transmits and receives ultrasonic waves to a subject, first data based on a received signal obtained by transmitting and receiving ultrasonic waves of one transmission waveform, and a plurality of pieces. Using a model machine-learned using training data including second data based on received signals obtained by transmitting and receiving ultrasonic waves of different transmission waveforms, ultrasonic waves of one transmission waveform by the ultrasonic probe. An ultrasonic wave having an estimation calculation unit that estimates data based on a received signal obtained by transmitting and receiving ultrasonic waves having a plurality of different transmission waveforms from a third data based on the received signal obtained by the transmission and reception of the ultrasonic waves. It is a diagnostic device.

The second aspect of the present invention is a learning device that performs machine learning of a learning model used in the estimation calculation unit of the ultrasonic diagnostic apparatus of the first aspect, and is obtained by transmitting and receiving ultrasonic waves of one transmission waveform. It has a learning unit that performs machine learning of the learning model using learning data including the first data as input data and the second data obtained by transmitting and receiving ultrasonic waves of a plurality of different transmission waveforms as correct answer data. It is a learning device characterized by this.

A third aspect of the present invention is a transmission / reception step of transmitting / receiving ultrasonic waves to a subject, first data obtained by transmitting / receiving ultrasonic waves of one transmission waveform, and ultrasonic waves having a plurality of different transmission waveforms. Using a model machine-learned using training data including a second data obtained by transmission and reception, a plurality of different data from the third data obtained by transmission and reception of ultrasonic waves of one transmission waveform in the transmission and reception step. It is an image processing method including an estimation calculation step for estimating data obtained by transmitting and receiving ultrasonic waves of a transmission waveform.

According to the present invention, it is possible to provide an ultrasonic diagnostic apparatus capable of obtaining an image with good image quality while reducing the influence of a decrease in frame rate.

Block diagram showing an example of the configuration of an ultrasonic diagnostic device A block diagram showing an example of the functions of the received signal processing block of the first embodiment. Diagram illustrating extraction of high frequency components in Tissue Harmonic Imaging Diagram showing an example of a learning device that learns a learning model Diagram explaining training data Diagram showing an example of GUI for creating training data The figure which shows an example of the display of the ultrasonic image on the display device. A block diagram showing an example of the functions of the received signal processing block of the second embodiment.

<First Embodiment>
The first embodiment of the present invention will be described. FIG. 1 is a block diagram showing an example of the hardware configuration of the ultrasonic diagnostic apparatus.

The ultrasonic diagnostic apparatus 1 generally includes an ultrasonic probe (ultrasonic probe) 102, a probe connection portion 103, a transmission electric circuit 104, a reception electric circuit 105, a reception signal processing block 106, an image processing block 107, and a display device 108. , Has a system control block 109. The ultrasonic diagnostic apparatus 1 transmits an ultrasonic pulse from the ultrasonic probe 102 to the subject 100, receives the reflected ultrasonic wave reflected inside the subject 100, and receives image information (ultrasonic) inside the subject 100. It is a system for generating an ultrasonic image). The ultrasonic image obtained by the ultrasonic diagnostic apparatus 1 is used in various clinical examinations.

The ultrasonic probe 102 is an electron scanning probe, and has a plurality of vibrators 101 arranged one-dimensionally or two-dimensionally at its tip. The vibrator 101 is an electromechanical conversion element that performs mutual conversion between an electric signal (voltage pulse signal) and an ultrasonic wave (acoustic wave). The ultrasonic probe 102 transmits ultrasonic waves to the subject 100 from the plurality of vibrators 101, and receives the reflected ultrasonic waves from the subject 100 by the plurality of vibrators 101. The reflected acoustic wave reflects the difference in acoustic impedance in the subject 100.

The transmission electric circuit 104 is a transmission unit that outputs pulse signals (drive signals) to a plurality of vibrators 101. By applying pulse signals to the plurality of vibrators 101 with a time lag, ultrasonic waves having different delay times are transmitted from the plurality of vibrators 101 to form a transmitted ultrasonic beam. The direction and focus of the transmitted ultrasonic beam can be controlled by selectively changing the vibrator 101 to which the pulse signal is applied (that is, the driving vibrator 101) or changing the delay time (application timing) of the pulse signal. .. By sequentially changing the direction and focus of the transmitted ultrasonic beam, the observation area inside the subject 100 is scanned. The transmission electric circuit 104 transmits a pulse signal having a predetermined drive waveform to the vibrator 101 to generate a transmission ultrasonic wave having a predetermined transmission waveform in the vibrator 101. The receiving electric circuit 105 is a receiving unit that inputs an electric signal output from the vibrator 101 that has received the reflected ultrasonic waves as a receiving signal. The received signal is input to the received signal processing block 106. The operation of the transmitting electric circuit 104 and the receiving electric circuit 105, that is, the transmission / reception of ultrasonic waves is controlled by the system control block 109. In the present specification, the analog signal output from the vibrator 101 and the digital data obtained by sampling (digitally converting) the analog signal are referred to as received signals without particular distinction. However, the received signal may be referred to as received data for the purpose of clearly indicating that it is digital data depending on the context.

The reception signal processing block 106 is an image generation unit that generates image data based on the reception signal obtained from the ultrasonic probe 102. The image processing block 107 performs image processing such as brightness adjustment, interpolation, and filter processing on the image data generated by the received signal processing block 106. The display device 108 is a display unit for displaying image data and various information, and is composed of, for example, a liquid crystal display or an organic EL display. The system control block 109 is a control unit that collectively controls the transmission electric circuit 104, the reception electric circuit 105, the reception signal processing block 106, the image processing block 107, the display device 108, and the like.

(Configuration of received signal processing block)
FIG. 2 is a block diagram showing an example of the function of the received signal processing block 106. The reception signal processing block 106 includes a phasing addition processing block 201, a signal storage block 202, an arithmetic processing block 203, a B mode processing block 204, and an estimation arithmetic block 205.

The phasing addition processing block 201 performs phasing addition on the received signal obtained from the reception electric circuit 105, and stores the added reception signal in the signal storage block 202. The phase-adjusting addition process is a process of forming a received ultrasonic beam by adding the received signals of a plurality of oscillators 101 by changing the delay time for each oscillator 101, and is also referred to as Delay and Sum (DAS) beamforming. be called. The phasing addition processing is performed based on the element arrangement given by the system control block 109 and various conditions for image generation (aperture control, signal filter).

The arithmetic processing block 203 generates an ultrasonic signal based on a harmonic component by a conventional Tissue Harmonic Imaging (THI) method. The received signal stored in the signal storage block 202 and the ultrasonic signal for THI are transmitted to the B mode processing block 204. The arithmetic processing block 203 corresponds to the arithmetic processing unit of the present invention.

The B-mode processing block 204 performs envelope detection processing and logarithmic compression processing on the received signal from the signal storage block 202 and the arithmetic processing block 203 and the ultrasonic signal for THI, and determines the signal strength at each point in the observation region. Generate image data expressed in brightness intensity.

The estimation calculation block 205 (estimation calculation unit) uses a model to transmit and receive ultrasonic waves of a plurality of different transmission waveforms from the third data obtained by transmitting and receiving ultrasonic waves of one transmission waveform by an ultrasonic probe. Estimate the data obtained by. In the present embodiment, the estimation calculation block 205 uses a learned model obtained by machine learning obtained from the received signal, and uses a pseudo-THI equivalent image from a B-mode image based on the received signal stored in the signal storage block 202. Is estimated (generated). The estimation calculation block 205 corresponds to the estimation calculation unit of the present invention.

As used herein, "THI image" means an image generated from an ultrasonic signal for THI. It can be said that the THI image is an image generated by a conventional THI imaging method. Further, the "pseudo-THI image" means an image corresponding to the THI image obtained by performing image processing (estimation calculation processing) on the B-mode image based on the received signal. The generation of the pseudo THI image by the estimation calculation block 205 can be called the THI image estimation process. Further, in order to clarify the distinction from the pseudo THI image, the THI image obtained by the conventional method may be referred to as a normal THI image.

The reception signal processing block 106 outputs a normal B mode image, a normal THI image, and a pseudo THI image to the image processing block 107. The reception signal processing block 106 does not always output these images, and may output only one of these images. The image output from the received signal processing block 106 is displayed on the display device 108 after being subjected to predetermined processing by the image processing block 107.

The received signal processing block 106 may be composed of one or more processors and a memory. In that case, the functions of the blocks 2001 to 205 shown in FIG. 2 are realized by a computer program. For example, the functions of the blocks 201 to 205 can be provided by the CPU reading and executing the program stored in the memory. In addition to the CPU, the reception signal processing block 106 may include a processor (GPU, FPGA, etc.) that is in charge of the calculation of the calculation processing block 203 and the calculation of the estimation calculation block 205. In particular, it is effective to use an FPGA for the arithmetic processing block 203 to which a large amount of data is input at the same time, and to use a GPU to efficiently execute an arithmetic such as the estimation arithmetic block 205. The memory may include a memory for non-temporarily storing a program, a memory for temporarily storing data such as a received signal, a working memory used by the CPU, and the like.

(THI imaging: Pulse Inversion method)
Here, the Pulse Inversion (PI) method, which is an image generation method by THI imaging, will be described. THI imaging is a method of generating an image based on a harmonic component by reducing the fundamental frequency component of a transmitted ultrasonic wave from a plurality of received signals obtained by transmitting and receiving ultrasonic waves having different transmission waveforms.

When an ultrasonic wave having a fundamental wave component f1 is transmitted to a subject, as shown in FIG. 3A, a first received signal 31 representing a reflected ultrasonic wave including the fundamental wave component f1 and the harmonic component 2f1 is received. By adjusting the fundamental wave component f1 and the harmonic component 2f1 so as to be included in the effective frequency band of the ultrasonic probe 102, both the fundamental wave component f1 and the harmonic component 2f1 can be received. The frequency of the fundamental wave is generally lower than the center frequency of the ultrasonic probe 102. The first received signal 31 is phasing-added in the received signal processing block 106 and stored in the signal storage block 202.

Next, as shown in FIG. 3B, when an ultrasonic wave having a frequency spectrum whose phase is inverted from that of the first transmitted waveform is transmitted to the subject, a second reflected ultrasonic wave containing the fundamental wave component f1 and the harmonic component 2f1 is represented. The reception signal 32 of is received. The second received signal 32 is phasing-added in the received signal processing block 106 and stored in the signal storage block 202.

The phase change of the second harmonic is twice the phase change of the fundamental wave. Therefore, when the phase of the fundamental wave is changed (inverted) by 180 degrees, the phase of the second harmonic is changed by 360 degrees. That is, the harmonic components 2f1 included in the first received signal 31 and the second received signal 32 are in phase.

The phasing-added received signals 31 and 32 are added by the arithmetic processing block 203. As a result, as shown in FIG. 3C, the fundamental wave component f1 is canceled and the ultrasonic signal 33 having a frequency spectrum in which the harmonic component 2f1 is emphasized is generated. The ultrasonic signal 33 corresponds to the above-mentioned ultrasonic signal for THI.

The B-mode processing block 204 performs envelope detection processing, logarithmic compression processing, etc. on the ultrasonic signal 33 generated by the arithmetic processing block 203, and expresses the signal strength at each point in the observation area by the luminance strength. Generate the image data. As a result, since only the harmonic component is imaged, an image having excellent resolution and contrast can be obtained as compared with the normal B mode in which the fundamental wave component is imaged.

Here, an example in which the ultrasonic signal is transmitted and received twice is described, but the number of times of transmission and reception is not limited to two, and the phase of the transmission waveform is adjusted so as to add and cancel the fundamental wave component. If so, the number of transmissions and receptions may be any number of times of 2 or more. Ideally, the fundamental wave component is completely canceled, but if the fundamental wave component is reduced by addition, it may not be completely canceled.

A B-mode image can also be generated from the first received signal 31 and the second received signal 32, respectively. Since the B-mode image generated in this way is based on the fundamental wave of the transmitted ultrasonic wave, it is also referred to as a "fundamental wave image" in the present specification. Further, the B mode image based on the first received signal 31 is also referred to as a THI positive pulse mode fundamental wave image or a first fundamental wave image, and the B mode image based on the received signal 32 is a THI negative pulse mode fundamental wave image or. Also called the second fundamental wave image.

Each of the first received signal 31 and the second received signal 32 can be called a received signal obtained by transmitting and receiving ultrasonic waves of one transmission waveform. The first and second fundamental wave images can be called data based on the received signal obtained by transmitting and receiving ultrasonic waves of one transmission waveform, respectively. Further, the first received signal 31 and the second received signal 32 are collectively referred to as a plurality of received signals representing reflected ultrasonic waves generated by transmitting a plurality of ultrasonic waves having different phases of the transmitted waveforms. You can also do it. The ultrasonic signal 33 for THI can also be called a signal in which the fundamental frequency component of the transmitted ultrasonic wave is reduced from the plurality of received signals and the harmonic component is emphasized. Further, the ultrasonic image (THI image) based on the ultrasonic signal 33 can also be called harmonic image data based on the fundamental frequency component of the transmitted ultrasonic wave included in the plurality of received signals.

(Estimation calculation block)
The estimation calculation block 205 will be described. The estimation calculation block 205 performs a process of generating (estimating) a pseudo THI image using the trained model.

The model is a first data (input data) based on a received signal obtained by transmitting and receiving ultrasonic waves of one transmission waveform, and a second data based on a received signal obtained by transmitting and receiving ultrasonic waves of a plurality of different transmission waveforms. Machine learning is performed using training data including (correct answer data). Specific algorithms for machine learning include the nearest neighbor method, the naive Bayes method, and support vector machines. In addition, deep learning (deep learning) in which features and coupling weighting coefficients for learning are generated by themselves using a neural network can also be mentioned. As appropriate, any of the above algorithms that can be used can be applied to this embodiment.

FIG. 4 shows an example of a learning device 40 that performs machine learning of a model. The learning device 40 has a learning unit (learning device) 404 that performs machine learning of a model using a plurality of learning data 401. The learning unit 404 may use any of the machine learning algorithms exemplified above, or may use another machine learning algorithm. The learning data 401 is composed of a set of input data and correct answer data (teacher data). In the present embodiment, the fundamental wave image 402 (an image based on the received signal by transmitting and receiving ultrasonic waves of one transmission waveform) is used as input data, and the THI image 403 (plurality by transmitting and receiving ultrasonic waves of a plurality of different transmission waveforms) is used as correct answer data. (Image based on the received signal of) is used. The learning unit 404 learns the correlation between the fundamental wave image 402 and the THI image 403 based on a plurality of given learning data 401, and creates a trained model 405. As a result, the trained model 405 can acquire the function (ability) of generating the pseudo THI image as the output data when the fundamental wave image is given as the input data. In other words, the trained model 405 acquires the function of estimating the data based on the harmonic component of the transmitted ultrasonic waves included in the received signal from the data based on the received signal obtained by transmitting and receiving the ultrasonic waves of one transmission waveform. do. The trained model 405 is implemented in a program executed by the estimation calculation block 205 of the ultrasonic diagnostic apparatus 1. It is desirable that the learning of the model (the process of generating the trained model 405) is performed before being incorporated into the ultrasonic diagnostic apparatus 1. However, when the ultrasonic diagnostic apparatus 1 has a learning function, learning (new learning or additional learning) may be performed using the image data obtained by the ultrasonic diagnostic apparatus 1.

The training data can be prepared by generating a THI image using the processes described in the arithmetic processing block 203 and the B-mode processing block 204. Specifically, a plurality of received signals are acquired by performing ultrasonic transmission a plurality of times with different phases, and a THI image based on harmonic components extracted from the plurality of received signals is generated. In addition, a B-mode image (fundamental wave image) is generated from each of the plurality of received signals. The waveform (phase) mode of the fundamental wave image and the transmitted ultrasonic wave generated in this way is used as input data, and the THI image is used as correct answer data. Therefore, as a result of one THI imaging, a plurality of sets of training data can be obtained. The learning data may be generated by an apparatus other than the ultrasonic diagnostic apparatus on which the learning model is mounted, or by an ultrasonic diagnostic apparatus on which the learning model is mounted.

The learning data will be described more specifically with reference to FIG. The correct answer data included in the training data is a THI image captured using the THI mode. Among the input data included in the training data, the transmission waveform mode is a mode in which the transmitted ultrasonic wave used when generating the fundamental wave image which is the input data transmits a waveform without inversion in the PI method and a waveform with inversion. Indicates which of the modes to transmit. The transmission mode without inversion is referred to as THI positive pulse mode (or simply positive pulse mode), and the transmission mode with inversion is referred to as THI negative pulse mode (or simply negative pulse mode). Of the input data included in the training data, the fundamental wave image is the data obtained by imaging the received signal corresponding to each transmission waveform in the above THI mode imaging (PI method) (primary wave image in positive pulse mode or negative pulse mode). Fundamental wave image).

FIG. 5 illustrates four learning data of learning data IDs 1 to 4. Training data I
As the input data of D1, the fundamental wave image B1-1 of the positive pulse mode in the first THI imaging and the positive pulse mode which is the transmission waveform mode at that time are used. Further, the THI image obtained as a result of the first THI imaging is used as the correct answer data of the learning data ID1. As the input data of the learning data ID2, the fundamental wave image B1-2 of the negative pulse mode in the first THI imaging and the negative pulse mode which is the transmission waveform mode at that time are used. Further, as the correct answer data of the learning data ID2, the THI image obtained as a result of the first THI imaging is used.

Similarly, the input data of the training data ID3 is the fundamental wave image B2-1 in the positive pulse mode and the positive pulse mode in the second THI imaging, and the correct answer data is the THI image obtained as a result of the second THI imaging. be. The input data of the training data ID4 is the fundamental wave image B2-2 and the negative pulse mode of the negative pulse mode in the second THI imaging, and the correct answer data is the THI image obtained as a result of the second THI imaging.

By learning using the learning data acquired under various conditions, learning for input of various patterns is performed, and it can be expected that an image with good image quality can be stably estimated even when it is actually used. Therefore, it is preferable to perform THI imaging on the same subject under different conditions to obtain learning data. As the subject, a digital phantom that can be imaged by ultrasonic transmission / reception simulation may be used, or an actual phantom or an actual living body may be used.

Further, the training data may be preprocessed. For example, the learning efficiency may be improved by correcting the unevenness of the brightness value due to the attenuation of ultrasonic waves. In addition, the harmonic component generated in the process of ultrasonic waves propagating in the subject increases as the propagation distance increases. On the other hand, since the harmonic component has a higher frequency than the fundamental wave component, the influence of attenuation becomes large during propagation. That is, the depth at which the harmonic components to the extent that the effect of THI imaging can be obtained and the depth at which the SN of the harmonic components can be sufficiently obtained are limited to a certain range. Therefore, from the THI image and the fundamental wave image obtained by THI imaging, it is preferable to extract only the image having the depth in the above range and perform the preprocessing. As a result, the THI image can be estimated from the fundamental wave image even in a shallow part where the effect of THI imaging is small or a deep part where the penetration is not good, and the effect of improving the resolution and contrast can be obtained.

In learning, input data and correct answer data may be preprocessed using a GUI as shown in FIG. The input data 60 and the correct answer candidate data 61 are shown in the display screen, and the indicator 62 for dividing each into a plurality of areas is displayed. In the example of FIG. 6, the image is divided into 16 4 × 4 regions. The acceptance designation box 63 is a user interface for allowing the user to specify acceptance / rejection for each area. While comparing the input data 60 and the correct answer candidate data 61, the user inputs "○" in the area to be adopted as learning data and "x" in the area to be excluded. As a result, it is possible to exclude places where unexpected image deterioration has occurred in the correct answer candidate data 61. For example, it is possible to exclude shallow areas where the effect of THI imaging as described above is judged to be small and deep areas where penetration is not good. In addition, it is possible to exclude a portion where it is judged that the image quality is deteriorated due to insufficient cancellation of the fundamental wave component due to the movement of the subject during the transmission and reception of a plurality of ultrasonic pulses. In FIG. 5, the description is made on the assumption that the entire image is used as one learning data, but when the image is divided into a plurality of regions as shown in FIG. 6, the image (partial image) of each region is 1. It is used as one training data. In this case, the learning model accepts an image having the same size (resolution) as the input data 60 as input, and outputs an image having the same size as the correct answer candidate data 61. In the example of FIG. 6, since there are seven regions to be adopted, seven sets of training data are generated.

In the present embodiment, the fundamental wave image in THI imaging is used as the input data of the training data, but a normal B mode image may be used as the input data. The normal B-mode image here is a fundamental wave image obtained by transmitting ultrasonic waves at the center frequency of the ultrasonic probe 102. As a result, the learning model can estimate the THI image from the normal B-mode image.

Further, in the present embodiment, the fundamental wave image and the transmission waveform mode are illustrated as the input data, but the effect can be obtained even if only the fundamental wave image is used as the input data. Further, by using the transmission frequency when the fundamental wave image is acquired and the band of the bandpass filter as input data, it is possible to estimate accurately according to the situation of the input data. In addition, by inputting data such as which part of the living body the subject is in and in which direction the ultrasonic probe is in contact with the body axis, learning can be performed according to the characteristics of each part. The estimation accuracy is improved. Features of each site include, for example, a fat layer on the surface, a high-luminance region due to the structure of fascia, and a low-luminance region due to thick blood vessels. Furthermore, by adding information such as clinical department, gender, BMI, age, and pathological condition to the input data, a learning model corresponding to the characteristics of each part in more detail can be obtained, and it can be expected that the estimation accuracy will be further improved. ..

Further, the trained model 405 of the estimation calculation block 205 mounted on the ultrasonic diagnostic apparatus 1 may be a model in which image data of all clinical departments are trained, or a model in which image data of each clinical department is trained. .. When a model in which image data for each clinical department is trained is installed, the system control block 109 causes the user of the ultrasonic diagnostic apparatus 1 to input or select clinical department information, and is used according to the clinical department. It is good to change the completed model. It can be expected that the estimation accuracy will be further improved by using different models for each clinical department where the imaging site is limited to some extent.

The trained model 405 obtained by learning using such various imaging conditions and the fundamental wave image as input data and using the THI image as the correct answer data operates on the estimation calculation block 205. As a result, the estimation calculation block 205 can be expected to estimate and output an image corresponding to the THI image having high resolution and contrast with respect to the input fundamental wave image.

(Image generation method)
Next, the details of the process for image generation in the present embodiment will be described with reference to FIG. When an imaging instruction is input from the user using a GUI (not shown), the system control block 109 that receives the instruction from the GUI inputs an ultrasonic transmission instruction to the transmission electric circuit 104. The transmission instruction may include parameters and sound velocity information for calculating the delay time. The transmission electric circuit 104 outputs a plurality of pulse signals (voltage waveforms) to the plurality of vibrators 101 of the ultrasonic probe 102 through the probe connection unit 103 based on the transmission instruction from the system control block 109. Here, the ultrasonic wave having the fundamental wave component f1 as shown in FIG. 3A is used as the transmission waveform in the positive pulse mode.

The transmitted ultrasonic waves transmitted from the plurality of vibrators 101 propagate in the subject and generate reflected ultrasonic waves reflecting the difference in acoustic impedance in the subject. The reflected ultrasonic waves are received by the plurality of vibrators 101 and converted into voltage waveforms (voltage signals). This voltage waveform is input to the receiving electric circuit 105 through the probe connecting portion 103. The receiving electric circuit 105 amplifies the voltage waveform as necessary, digitally samples it, and outputs it to the receiving signal processing block 106 as a receiving signal. In the received signal processing block 106, the received signal obtained by the received electric circuit 105 is phase-aligned based on the element arrangement and various image generation conditions (opening control, signal filter) input from the system control block 109. The addition processing block 201 performs phase adjustment addition. Further, the phasing-added signal is stored in the signal storage block 202. As a result, the received data corresponding to the transmission waveform in the positive pulse mode is stored in the signal storage block 202. By performing the same processing, the received data corresponding to the frequency spectrum (transmission waveform in the negative pulse mode) whose phase is inverted from the transmission waveform in the positive pulse mode as shown in FIG. 3B is stored in the signal storage block 202.

In the arithmetic processing block 203, the fundamental wave component is canceled and the harmonic component is extracted from the plurality of received data corresponding to the transmission waveforms in the positive pulse mode and the negative pulse mode by using the PI method, and the harmonic component is emphasized. Generates a harmonic image. These images are transmitted to the B-mode processing block 204. The B-mode processing block 204 performs envelope detection processing, logarithmic compression processing, and the like to generate THI image data (harmonic image data) in which the signal intensity at each point in the observation region is represented by the luminance intensity.

Further, the B mode processing block 204 also performs envelope detection processing, logarithmic compression processing, and the like on the received data in the positive pulse mode and the received data in the negative pulse mode. The first and second fundamental wave image data are generated from the received data in the positive pulse mode and the negative pulse mode.

The estimation calculation block 205 estimates the first and second fundamental wave image data from the B mode processing block 204 and the imaging conditions (input transmission waveform mode, etc.) related to the fundamental wave from the system control block 209 as inputs. Executes the calculation and outputs the image data. The estimation calculation block 205 estimates the first pseudo THI image data from the first fundamental wave image data using the trained model described above, and obtains the second pseudo THI image data from the second fundamental wave image data. presume.

In this way, the estimation calculation block 205 inputs a plurality of fundamental wave image data obtained from the received data used for generating the normal THI image (harmonic image) into the trained model, and pseudo-corresponds to the normal THI image. A plurality of THI images (estimated images) are estimated. Although the pseudo THI image is estimated from each fundamental wave image data, the pseudo THI image may be estimated from only a part of the fundamental wave image data. The received data of the positive pulse mode and the negative pulse mode in this image generation processing correspond to the "received signal obtained by transmitting and receiving ultrasonic waves of one transmission waveform" in the present invention, respectively. Further, in the present embodiment, the first and second fundamental wave image data correspond to the "third data", respectively. Further, the first and second pseudo THI images estimated from the first and second fundamental wave image data correspond to the data estimated by the estimation calculation unit, respectively.

The THI image data and the first and second pseudo-THI image data are input to the image processing block 107, and after brightness adjustment, interpolation, and other filters are applied, the display device 108 follows the instructions of the system control block 109. Is displayed. By continuously displaying these three image data, it is possible to realize a display with an image quality equivalent to THI at a frame rate faster than that in the normal THI mode.

Further, when it is determined that the image quality of the normal THI image is low, only the first and second pseudo THI image data may be displayed without displaying the normal THI image data. If there is a movement of the subject between obtaining the first received data and the second received data, the cancellation of the fundamental wave component in the PI method becomes insufficient, and the image quality of the THI image data usually deteriorates. In such a case, by displaying only the first and second pseudo THI image data, it is possible to realize a frame rate faster than that of the normal THI mode while preventing deterioration of the image quality. It is desirable to use such a display when imaging a part where the movement of the subject is large.

The ultrasonic diagnostic apparatus 1 usually has a plurality of display modes in which at least one of turning on / off the display of THI image data and turning on / off the pseudo THI image is different. For example, the ultrasonic diagnostic apparatus 1 usually has a first mode for displaying THI image data and pseudo THI image data, and a second mode for displaying only pseudo THI image data. The display mode may include a third mode in which only the normal THI image data is displayed without displaying the pseudo THI image data. The display control of the image based on the display mode is performed by the system control block 109.

The display mode may be specified by the user from the GUI (input means) of the display device 108, or may be determined by the image processing block 107. For example, the display device 108 is displayed with a GUI that allows the user to individually specify on / off of the display of the normal THI image and the pseudo THI image, and the display on / off of the normal THI image and the pseudo THI image is specified via the GUI. It may be accepted from the user.

The image processing block 107 or the system control block 109 may determine the mode selection. For example, the image processing block 107 may determine whether or not the image quality of the normal THI image is higher than the threshold value, and select the display mode based on the determination result. Specifically, when the image quality of the normal THI image is higher than the threshold value, a mode for displaying both the normal THI image and the pseudo THI image is selected, and when the image quality is lower than the threshold value, the normal THI image is displayed. Select the mode to display the pseudo THI image without. Normally, the image quality of a THI image can be determined based on the amount of movement of the subject. For example, the image processing block 107 depends on whether or not the movement of the subject obtained based on the first and second fundamental wave image data or the first and second pseudo THI image data is equal to or more than a predetermined amount. Therefore, it may be determined whether or not the image quality of the normal THI image is higher than the threshold value. Alternatively, the image processing block 107 may determine the image quality of the normal THI image based on whether or not the fundamental wave component included in the signal obtained by summing the first and second received data is a predetermined amount or more. ..

7A to 7D schematically show an example of displaying an image on the display device 108. The display screen 70 includes an image display area 71, a frame rate display area 72, a normal THI image display on / off indicator 73, and a pseudo THI image display on / off indicator 74. The indicators 73 and 74 may also serve as a display on / off switching button (GUI) for the normal THI image and the pseudo THI image.

FIG. 7A shows a display example in a normal THI mode display, that is, a mode in which a normal THI image is displayed and a pseudo THI image (AI-THI image) is not displayed. The frame rate (FR) is 50 fps. Since the normal THI image is displayed, "normal THI: ON" is displayed on the indicator 73, and since the pseudo THI image is not displayed, "AI-THI: OFF" is displayed on the indicator 74.

FIG. 7B shows a display example of a mode for displaying both a normal THI image and a pseudo THI image. In this display mode, the normal THI image and the pseudo THI image are continuously displayed in the image display area 71. Since the pseudo THI image data can be estimated from each of the fundamental wave images based on the first and second received data, the frame rate in this mode is 150 fps, which is three times that of FIG. 7A. In this embodiment, the indicator 74 displays "AI-THI: ON". As a result, it is possible to clearly indicate to the user that the display image includes the estimated image estimated by the estimation calculation block 205. Although the indicator 74 of the present embodiment notifies that the estimated image is displayed by displaying characters, the display of the estimated image may be notified by another method. For example, a method of changing the color of the outer edge of the displayed image or the display area, blinking, changing the background color, saturation, pattern, or the like may be adopted.

FIG. 7C shows a display example in a mode in which a pseudo THI image is displayed and a normal THI image is not displayed. The frame rate in this mode is 100 fps, which is twice that of FIG. 7A. The indicator 73 indicates that the display of the normal THI image is OFF (normal THI: OFF), and the indicator 74 indicates that the display of the pseudo THI image is ON (AI-THI: ON).

FIG. 7D is an example in which the normal THI image 75 and the pseudo THI image 76 are displayed side by side. Normally, only the THI image is displayed on the left side of the screen at a frame rate of 50 fps, and only the pseudo THI image is displayed on the right side of the screen at a frame rate of 100 fps. By using this display screen, not only the estimated image but also the normal THI image which is the correct image can be confirmed at the same time. Such a display screen is useful for evaluating and checking the accuracy and reliability of the estimation calculation block 205.

<Second Embodiment>
Next, another embodiment of the present invention will be described. In the first embodiment, the fundamental wave image obtained in the THI mode is input to the estimation calculation block 205 to estimate the pseudo THI image. In the present embodiment, the image data captured in the normal B mode is input to the estimation calculation block 205 to estimate the pseudo THI image. Also in this embodiment, it is possible to obtain an image having an image quality equivalent to THI without causing a decrease in the frame rate due to the PI method.

The overall configuration of the ultrasonic diagnostic apparatus 1 is the same as that of the first embodiment (FIG. 1). The flow from transmitting ultrasonic waves to the subject 100 to inputting the received signal to the received signal processing block 106 is the same as that of the first embodiment. However, in the first embodiment, both the fundamental wave component f1 and the harmonic component 2f1 are adjusted so as to be included in the effective frequency band of the ultrasonic probe 102, but in the present embodiment, the harmonic component 2f1 is the ultrasonic probe. It may be outside the effective frequency band of 102. That is, in the present embodiment, a transmission waveform having a bandwidth such that the fundamental wave component f1 coincides with the effective frequency band of the ultrasonic probe 102 can be used. Further, the fundamental wave component f1 is included in the effective frequency band of the ultrasonic probe, but the harmonic component 2f1 is not included in the effective frequency band of the ultrasonic probe, that is, the fundamental wave as compared with the case of the first embodiment. It may be a transmission waveform in which the component f1 has a high frequency.

FIG. 8 is a diagram showing details of the received signal processing block 116 in the second embodiment. The reception signal processing block 116 includes a phasing addition processing block 201, a signal storage block 202, a B mode processing block 204, and an estimation calculation block 205. The function and processing of each block are basically the same as those of the block having the same name in the first embodiment. That is, the received signal taken in from the receiving electric circuit 105 is phasing-added by the phasing-adding processing block 201 and stored in the signal storage block 202. Then, the B-mode processing block 204 normally generates a B-mode image and inputs it to the estimation calculation block 205. The estimation calculation block 205 inputs a normal B mode image to the trained model, and obtains a pseudo THI image (estimated image) as an estimation result. In the present embodiment, this estimated image is used for display on the display device 108.

According to this embodiment, a pseudo THI image can be acquired even if the ultrasonic probe 102 cannot receive the harmonic component 2f1. In other words, it is not necessary to lower the frequency of the transmitting ultrasonic waves in order to receive the harmonic components. Since a higher definition image can be obtained as the frequency of the ultrasonic wave is higher, the estimation accuracy of the pseudo THI image in the estimation calculation block 205 is also improved according to the present embodiment.

The trained model included in the estimation calculation block 205 of the present embodiment is generated by the same processing as that of the first embodiment. In the present embodiment, the input data included in the training data may be a normal B-mode image obtained by transmitting and receiving ultrasonic waves at the center frequency of the ultrasonic probe 102. However, a trained model that has been trained based on the same training data as in the first embodiment may be used.

<Other Embodiments>
The above-described embodiment merely shows a specific example of the present invention. The scope of the present invention is not limited to the configuration of the above-described embodiment, and various embodiments can be adopted without changing the gist thereof.

For example, in the first embodiment and the second embodiment, a learning model using a B-mode image as input data and an estimated image as output data is used, but the model input / output does not have to be an image and is a source of image generation. Any data will do. For example, the received data itself obtained by the fundamental wave transmission may be used as the input data, or the received data after the phase adjustment addition processing may be used as the input data. In that case, as the correct answer data, it is preferable to use the received data itself from which only the harmonic components are extracted, or the received data obtained by performing phase adjustment addition processing on the received data from which only the harmonic components are extracted. Even if such a model is used, the same effect as that of the above-described embodiment can be obtained. When the output of the model is not an image, the received signal processing block 106 may further include an image generation unit that generates an image based on the data output from the estimation calculation block 205. This image generation unit is configured to generate an estimated image corresponding to a harmonic image from the data estimated by the estimation calculation block 205 (estimation calculation unit).

Further, the disclosed technology can take an embodiment as a system, an apparatus, a method, a program, a recording medium (storage medium), or the like. Specifically, it may be applied to a system composed of a plurality of devices (for example, a host computer, an interface device, an imaging device, a web application, etc.), or it may be applied to a device composed of one device. good.

Needless to say, the object of the present invention is achieved by doing the following. That is, a recording medium (or storage medium) in which a software program code (computer program) that realizes the functions of the above-described embodiment is recorded is supplied to the system or device. Needless to say, such a storage medium is a computer-readable storage medium. Then, the computer (or CPU or MPU) of the system or device reads and executes the program code stored in the recording medium. In this case, the program code itself read from the recording medium realizes the function of the above-described embodiment, and the recording medium on which the program code is recorded constitutes the present invention.

1: Ultrasonic diagnostic device 102: Ultrasonic probe 106: Received signal processing block 205: Estimated calculation block 405: Trained model

Claims (15)

An ultrasonic probe that sends and receives ultrasonic waves to the subject,
Using training data including first data based on a received signal obtained by transmitting and receiving ultrasonic waves of one transmission waveform and second data based on received signals obtained by transmitting and receiving ultrasonic waves of a plurality of different transmission waveforms. By transmitting and receiving ultrasonic waves of a plurality of different transmission waveforms from the third data based on the received signal obtained by transmitting and receiving ultrasonic waves of one transmission waveform by the ultrasonic probe using the machine-learned model. An estimation calculation unit that estimates data based on the obtained received signal,
An ultrasonic diagnostic apparatus characterized by having. The model is trained using training data in which data based on signals obtained by reducing fundamental frequency components from a plurality of received signals obtained by transmitting and receiving ultrasonic waves of a plurality of different transmission waveforms is used as correct answer data.
The ultrasonic diagnostic apparatus according to claim 1. It further has an image generation unit that generates an estimated image corresponding to a harmonic image obtained by transmitting and receiving ultrasonic waves of a plurality of different transmission waveforms from the data estimated by the estimation calculation unit.
The ultrasonic diagnostic apparatus according to claim 1 or 2. Arithmetic processing that extracts harmonic components from a plurality of received signals based on reflected ultrasonic waves generated by transmitting ultrasonic waves with different phases of the transmitted waveform and generates a harmonic image based on the harmonic components. Has more parts,
Using the model, the estimation calculation unit uses the model to obtain an estimation image corresponding to the harmonic image from the third data based on the received signal obtained by transmitting and receiving ultrasonic waves of one transmission waveform by the ultrasonic probe. To estimate,
The ultrasonic diagnostic apparatus according to any one of claims 1 to 3. The estimation calculation unit inputs a plurality of third data based on the received signal obtained by transmitting and receiving ultrasonic waves of one transmission waveform by the ultrasonic probe into the model, and estimates corresponding to the harmonic image. Estimate multiple images and
Displaying the harmonic image and a plurality of the estimated images on a display device.
The ultrasonic diagnostic apparatus according to claim 3 or 4. It also has a control unit that controls the display image output to the display device.
The control unit has a plurality of display modes in which at least one of the on / off of the display of the estimated image and the on / off of the display of the harmonic image are different.
The ultrasonic diagnostic apparatus according to claim 4 or 5. Further, the user further has an input means capable of individually designating the on / off of the display of the estimated image and the on / off of the display of the harmonic image.
The ultrasonic diagnostic apparatus according to claim 6. The control unit displays both the estimated image and the harmonic image when the image quality of the harmonic image is higher than the threshold value, and when the image quality of the harmonic image is lower than the threshold value, the harmonic image is displayed. Display the estimated image without displaying the wave image,
The ultrasonic diagnostic apparatus according to claim 6 or 7. The control unit displays the estimated image and the harmonic image side by side.
The ultrasonic diagnostic apparatus according to any one of claims 6 to 8. The fundamental frequency of the ultrasonic wave transmitted by the ultrasonic probe is lower than the center frequency of the ultrasonic probe.
The ultrasonic diagnostic apparatus according to any one of claims 3 to 9. The first data is data based on a received signal that receives reflected ultrasonic waves generated by transmission of ultrasonic waves having the central frequency of the ultrasonic probe as a fundamental frequency.
The ultrasonic diagnostic apparatus according to claim 1 or 2. The harmonic of the center frequency of the ultrasonic probe is not included in the effective frequency band of the ultrasonic probe.
The ultrasonic diagnostic apparatus according to claim 11. A learning device that performs machine learning of a model used in the estimation calculation unit of the ultrasonic diagnostic apparatus according to any one of claims 1 to 12.
The learning data including the first data obtained by transmitting and receiving ultrasonic waves of one transmission waveform as input data and the second data obtained by transmitting and receiving ultrasonic waves of a plurality of different transmission waveforms as correct answer data is used. Has a learning unit that performs machine learning of models,
A learning device characterized by that. Sending and receiving steps to send and receive ultrasonic waves to the subject,
Using a machine-learned model using training data including first data obtained by transmitting and receiving ultrasonic waves of one transmission waveform and second data obtained by transmitting and receiving ultrasonic waves of a plurality of different transmission waveforms. The estimation calculation step of estimating the data obtained by transmitting and receiving ultrasonic waves of a plurality of different transmission waveforms from the third data obtained by transmitting and receiving ultrasonic waves of one transmission waveform in the transmission / reception step.
An image processing method comprising. A program for causing a processor to execute each step of the image processing method according to claim 14.

Images