JP2021133123A - Ultrasonic diagnostic device, learning device, image processing method and program - Google Patents

Abstract

To provide an ultrasonic diagnostic device that can acquire an image with excellent image quality by controlling velocity of a sonic speed related to ultrasonic propagation inside an organism.SOLUTION: An ultrasonic diagnostic device includes: an ultrasonic probe for transmitting and receiving an ultrasonic wave from an observation area of a subject; and a sonic speed information acquisition unit that uses a learned model which is mechanically learned using learning data including image data acquired from transmission and reception of an ultrasonic wave and sonic speed information of the ultrasonic wave to acquire sonic speed information on an ultrasonic wave in the observation area from the image data acquired from transmission and reception of the ultrasonic wave.SELECTED DRAWING: Figure 1

Description

The present invention relates to an ultrasonic diagnostic apparatus, a learning apparatus, an image processing method and a program, and particularly to a technique for improving the image quality of an ultrasonic image.

Ultrasound diagnostic devices are widely used in clinical practice as diagnostic imaging devices due to their simplicity, high resolution, and real-time performance. As the image generation method, a method of generating an image by forming a transmission beam and phasing addition processing of a received signal is common. The formation of the transmission beam is achieved by inputting voltage waveforms in which a time delay is given to a plurality of conversion elements and converging ultrasonic waves in the living body. Further, in the phase-aligned addition of the received signal, the ultrasonic waves reflected by the structure in the living body are received by a plurality of conversion elements, and the obtained received voltage signal is delayed in consideration of the path length with respect to the point of interest. It is achieved by giving and adding. The reflected signal from the point of interest is selectively extracted and imaged by the transmission beam formation process and the phasing addition process. Then, by controlling the transmission beam so that the transmission beam scans in the imaging region, an image of the region to be observed can be obtained.

Patent Document 1 discloses a medical imaging device using a restorer configured by a neural network.

Japanese Unexamined Patent Publication No. 2019-25044

However, in the living body to be diagnosed by the ultrasonic diagnostic apparatus, the optimum sound velocity of ultrasonic waves for image generation differs depending on the tissue and usage conditions, that is, the distribution of acoustic parameters related to the propagation of ultrasonic waves in the living body. Exists. Therefore, there is a problem that the image quality is deteriorated as a result of the difference between the above-mentioned time delay (generally calculated from the speed of sound and the assumed distance) and the time delay in the actual living body.

The present invention has been made in view of the above-mentioned problems, and an object of the present invention is to provide an ultrasonic diagnostic apparatus capable of obtaining an image with good image quality by adjusting the sound velocity related to ultrasonic propagation in a living body.

The present disclosure is machine-learned using an ultrasonic probe that transmits and receives ultrasonic waves to the observation region of a subject, and learning data including image data obtained by transmitting and receiving ultrasonic waves and ultrasonic sound velocity information. An ultrasonic diagnostic apparatus characterized by having a sound velocity information acquisition unit that acquires sound velocity information of ultrasonic waves in the observation region from image data obtained by transmitting and receiving ultrasonic waves using the trained model. include.

The present disclosure has a learning unit that performs machine learning of a trained model using learning data including image data of an observation area obtained by transmitting and receiving ultrasonic waves as input data and sound velocity information of ultrasonic waves as correct answer data. Includes a learning device characterized by.

The present disclosure has been machine-learned using a transmission / reception step of transmitting / receiving ultrasonic waves to the observation region of a subject by an ultrasonic probe, image data obtained by transmitting / receiving ultrasonic waves, and sound velocity information of ultrasonic waves. The image processing method includes a sound velocity information acquisition step of acquiring ultrasonic sound velocity information in the observation region from image data obtained by transmitting and receiving ultrasonic waves using the trained model.

The present disclosure includes a program for causing a processor to execute each step of the image processing method.

According to the present invention, it is possible to provide an ultrasonic diagnostic apparatus capable of adjusting the sound velocity according to the sound velocity distribution related to ultrasonic wave propagation in a living body.

A block diagram showing a hardware configuration of an ultrasonic diagnostic apparatus according to an embodiment. Block diagram showing an example of a learning device in one embodiment The figure which shows an example of the learning data using the image data, the observation depth and the adjustment amount of a sound velocity. The figure which shows an example of GUI which creates learning data in one Embodiment The figure which shows the flow of processing in one Embodiment The figure which shows an example of the display of the display device in one Embodiment Block diagram showing the hardware configuration of the ultrasonic diagnostic apparatus in one modified example Block diagram showing details of received signal processing block in one modification

<First Embodiment>
(Configuration of ultrasonic diagnostic equipment)
The first embodiment of the present invention will be described. FIG. 1 is a block diagram showing an example of the hardware configuration of the ultrasonic diagnostic apparatus. The ultrasonic diagnostic apparatus 1 includes an ultrasonic probe 102, a probe connection portion 103, a transmission electric circuit 104, a reception electric circuit 105, a reception signal processing block 106, an image processing block 107, a display device 108, a system control block 109, and a sound velocity suggestion block. Has 110. The ultrasonic diagnostic apparatus 1 transmits an ultrasonic pulse from the ultrasonic probe 102 (ultrasonic probe) to the observation region in the subject 100, and receives the reflected ultrasonic waves reflected inside the subject 100. , A system for generating image information (ultrasonic image) inside the subject 100. The ultrasonic image obtained by the ultrasonic diagnostic apparatus 1 is used in various clinical examinations.

The ultrasonic probe 102 is an electron scanning probe, and has a plurality of vibrators 101 arranged one-dimensionally or two-dimensionally at its tip. The vibrator 101 is an electromechanical conversion element that performs mutual conversion between an electric signal (voltage pulse signal) and an ultrasonic wave (acoustic wave). The ultrasonic probe 102 transmits ultrasonic waves from the plurality of vibrators 101 to the subject 100, and receives reflected ultrasonic waves reflecting the difference in acoustic impedance in the subject 100 by the plurality of vibrators 101.

The transmission electric circuit 104 is a transmission unit that outputs pulse signals (drive signals) to a plurality of vibrators 101. By applying pulse signals to the plurality of vibrators 101 with a time lag, ultrasonic waves having different delay times are transmitted from the plurality of vibrators 101 to form a transmitted ultrasonic beam. The direction and focus of the transmitted ultrasonic beam can be controlled by selectively changing the vibrator 101 to which the pulse signal is applied (that is, the driving vibrator 101) or changing the delay time (application timing) of the pulse signal. .. By sequentially changing the direction and focus of the transmitted ultrasonic beam, the observation area inside the subject 100 is scanned.

The receiving electric circuit 105 is a receiving unit that inputs an electric signal output from the vibrator 101 that has received the reflected ultrasonic wave as a receiving signal. The received signal is input to the received signal processing block 106. The operation of the transmitting electric circuit 104 and the receiving electric circuit 105, that is, the transmission and reception of ultrasonic waves, is controlled by the system control block 109. In the present specification, the analog signal output from the vibrator 101 and the digital data obtained by sampling (digitally converting) the analog signal are referred to as received signals without particular distinction. However, the received signal may be referred to as received data for the purpose of clearly indicating that it is digital data depending on the context.

The reception signal processing block 106 performs phase adjustment addition processing from the reception signal obtained from the reception electric circuit 105 based on various conditions (sound velocity, aperture control, signal filter) of element arrangement and image generation given by the system control block 109. I do. A signal indicating an appropriate sound velocity input from the sound velocity suggestion block 110 to the system control block 109, which will be described later, is also input from the system control block 109 to the received signal processing block 106. Further, the received signal processing block 106 performs envelope detection processing, logarithmic compression processing, and the like on the signal subjected to the phase adjustment addition processing, and the signal intensity at each point in the observation region is represented by the brightness of the image data. To generate.

The image processing block 107 performs image processing such as brightness adjustment, interpolation, and filter processing on the image data generated by the received signal processing block 106. The display device 108 is a display unit for displaying image data and various information, and is composed of, for example, a liquid crystal display or an organic EL display. The system control block 109 is a control unit that comprehensively controls the transmission electric circuit 104, the reception electric circuit 105, the reception signal processing block 106, the image processing block 107, the display device 108, the sound velocity suggestion block 110, and the like.

(Sonic speed suggestion block)
The sound velocity suggestion block 110 will be described. The sound velocity suggestion block 110 may be composed of one or more processors and a memory. In that case, the function of the sound velocity suggestion block 110 is realized by a computer program. For example, the function of the sound velocity suggestion block 110 can be provided by the CPU reading and executing the program stored in the memory. The sound velocity suggestion block 110 may include a processor (GPU, FPGA, etc.) in charge of the calculation of the sound velocity suggestion block 110 in addition to the CPU. Among the various processing blocks constituting the sound velocity suggestion block 110, it is effective to use FPGA for the block in which a large amount of data is input at the same time and GPU for the block for efficiently executing the calculation. The memory may include a memory for storing the program non-temporarily, a memory for temporarily storing data such as a received signal, a working memory used by the CPU, and the like.

The sound velocity suggestion block 110 performs a process of generating (estimating) an appropriate sound wave velocity (appropriate sound velocity value) of ultrasonic waves transmitted from the ultrasonic probe 102 using a trained model that has been trained. The sound velocity suggestion block 110 uses a trained model that has been machine-learned using image data obtained by transmitting and receiving ultrasonic waves and ultrasonic sound velocity adjustment amount data used when generating the image data. Then, the sound velocity suggestion block 110 uses the trained model to acquire the adjustment amount of the sound wave velocity of the ultrasonic wave used when generating the image data from the image data obtained by transmitting and receiving the ultrasonic wave. The sound velocity suggestion block 110 corresponds to the sound velocity information acquisition unit of the present invention. Further, the appropriate sound velocity value of ultrasonic waves, the adjustment amount of sound velocity, and the data showing these values and amounts correspond to the sound velocity information of the present invention.

Machine learning should be used for model learning. Specific algorithms for machine learning include the nearest neighbor method, the naive Bayes method, and support vector machines. In addition, deep learning (deep learning) in which features and coupling weighting coefficients for learning are generated by themselves using a neural network can also be mentioned. As appropriate, any of the above algorithms that can be used can be applied to this embodiment.

FIG. 2 shows an example of a learning device 20 that performs machine learning of a model. The learning device 32 has a learning unit (learning device) 204 that performs machine learning of a model using a plurality of learning data 201. The learning unit 204 may use any of the machine learning algorithms exemplified above, or may use another machine learning algorithm. The training data 201 is composed of a set of input data and correct answer data (teacher data), and in the present embodiment, as the input data, data in which the observation image and the observation depth in the observation image are paired is used. The observation depth is the depth of the observation region in which ultrasonic waves are transmitted, and can be said to be the distance between the object imaged in the observation image and the surface of the ultrasonic probe 102. Further, as the correct answer data, sound velocity information, here, data using the difference up to the appropriate sound velocity in the observed image as a label is used. The appropriate sound velocity is a sound velocity indicating how much the sound velocity should be adjusted to obtain a more desirable image in the observed image, and the difference to the appropriate sound velocity is the adjustment amount of the sound velocity. Further, the image data of the observation image may include RF (Radio Frequency) data (raw data).

The learning unit 204 learns the correlation between the observation image 202 and the difference 203 up to the appropriate sound velocity based on the given learning data 201, and creates the trained model 205. As a result, the trained model 205 can acquire a function (ability) of generating a difference (adjustment amount of sound velocity) up to an appropriate sound velocity as output data when an observation image is given as input data. The trained model 205 is implemented in a program executed by the sound velocity suggestion block 110 of the ultrasonic diagnostic apparatus 1. It is desirable that the learning of the model (the process of generating the trained model 205) is performed before being incorporated into the ultrasonic diagnostic apparatus 1. However, when the ultrasonic diagnostic apparatus 1 has a learning function, learning (new learning or additional learning) may be performed using the image data obtained by the ultrasonic diagnostic apparatus 1.

FIG. 3 is a diagram illustrating learning of the sound velocity suggestion block 110. In the learning, the observation image obtained by imaging the tissue to be observed with the transmission ultrasonic beam and the data including the observation depth when the observation image is imaged are used as input data. Further, the sound velocity suggestion block 110 uses data including a label indicating the difference from the sound velocity to the appropriate sound velocity of the transmitted ultrasonic beam used for capturing the observation image as correct answer data.

Here, the appropriate sound velocity is the sound velocity of the ultrasonic wave used when performing phase adjustment addition in the received signal processing block 106, which can obtain the optimum captured image. The criteria for regarding the captured image as the optimum image can be appropriately determined, for example, the image quality. Further, for example, when the captured image is an image created by simulation, the difference from the sound velocity used for the delay time calculation to the appropriate sound velocity with respect to the sound velocity of the model in the simulation may be included in the above correct answer data as a label. can.

The learning of the sound velocity suggestion block 110 in the present embodiment will be described in more detail. As shown in FIG. 3, the input data of the learning data includes an observation image, an observation depth, and the like. In addition, the correct answer data of the learning data includes a label indicating the difference from the sound velocity of the transmitted ultrasonic beam used for imaging to the appropriate sound velocity. In the example of FIG. 3, the label indicating the difference from the sound velocity of the transmitted ultrasonic beam to the appropriate sound velocity when the observation image of the learning data ID 1 is acquired is +10. As an example, the unit of the label indicating the difference from the sound velocity of the transmitted ultrasonic beam to the appropriate sound velocity is m / s, but an index of any unit may be used for the label. In the example of FIG. 3, "+10" means that when the sound velocity of the transmitted ultrasonic beam when the observation image is acquired is increased by 10 m / s, the sound velocity becomes appropriate.

Similarly, as the input data of the learning data IDs 2 to 4, a set of an observation image, an observation depth, and the like is used. Further, as the correct answer data of the learning data IDs 2 to 4, a label indicating the difference from the sound velocity of the transmitted ultrasonic beam used for imaging to the appropriate sound velocity is used.

Further, the sound velocity suggestion block 110 may perform learning using various learning data such as learning data having different transmission conditions of the transmitted ultrasonic beam and learning data having different tissues to be observed. By learning using more learning data, learning for input data of various patterns is performed, and it can be expected that an image with good image quality can be stably estimated even when it is actually used. As the subject, a digital phantom that can be imaged by ultrasonic transmission / reception simulation may be used, and an actual phantom or an actual living body may be used.

Further, the training data may be preprocessed. For example, the learning efficiency may be improved by correcting the unevenness of the brightness value due to the attenuation of ultrasonic waves. This can be expected to improve the resolution of the estimated image. A process of removing shadows due to floating of the ultrasonic probe at the time of imaging the subject from the input data may be performed. This makes it possible to improve the stability of the estimation accuracy. Alternatively, by using learning data in which both the input data and the correct answer data include shadows due to the float of the probe, an image that can recognize that the probe is floating is estimated even in the estimated image when the actual probe float occurs. You can also expect the effect of doing so.

Further, in the learning, the correct answer data may be further preprocessed by using the GUI as shown in FIGS. 4A and 4B. The GUI is displayed on the display device 108 according to the instruction of the system control block 109. The system control block 109 corresponds to a reception unit that receives an adjustment amount of the sound wave velocity of ultrasonic waves used when generating image data. In the GUI 41 shown in FIG. 4A, the entire image 411 showing the entire observation image and the image 412 of the extraction region obtained by cutting out a part of the entire image 411 are displayed. The extraction area may be an arbitrary area specified by the user, may be a preset area, or is an area determined based on image processing such as calculating the feature value of the image. There may be. Further, the extraction region may be one region or a plurality of regions in the entire image 411. Further, the GUI 41 displays the observation depth in the image 412 of the extraction region. The system control block 109 outputs this observation depth information. The GUI 41 displays a sound velocity adjusting unit 413 that adjusts the sound velocity used when the received signal processing block 106 generates image data.

The sound velocity adjusting unit 413 displays a scale 413a and a slide bar 413b indicating the amount of sound velocity adjustment. The user moves the slide bar 413b to a desired adjustment amount of the scale 413a by operating an input device (not shown) connected to the ultrasonic diagnostic apparatus 1. As a result, the image 412 of the extraction area when the sound velocity is changed according to the adjustment amount specified by the slide bar 413b is changed. Therefore, the user can determine the image 412 of the optimum extraction area while moving the slide bar 413b. The user operates the input device and presses the OK button 414 when the image 412 of the extraction area determined to be optimal is displayed. The amount of sound velocity adjustment specified by the slide bar 413b when the OK button 414 is pressed is determined as the difference to the appropriate sound velocity. The determined appropriate sound velocity is created as correct answer data in association with the image 412 of the extraction region and the observation depth. The user can create correct answer data for the images of a plurality of extraction regions by repeating the above operation for the image of another extraction region of the same whole image or the image of the extraction region of another whole image. When the user determines in the GUI 41 that the image is not suitable as learning data, such as the SN ratio of the image in the displayed extraction area is not good, the user operates the input device and presses the SKIP button 415. As a result, the image 412 of the currently displayed extraction area can be prevented from being included in the training data.

Further, the GUI 42 shown in FIG. 4B may be used instead of the GUI 41. In the GUI 42, instead of changing the image 412 of the extraction area by the slide bar 413b, the images 422, 423, and 424 when the sound velocity is changed by a predetermined adjustment amount are displayed side by side with respect to the image 412 of the same extraction area. Will be done. In the case of FIG. 4B, the predetermined adjustment amounts are “+10”, “0”, and “-10” (all units are m / s), but the value and number of the adjustment amounts may be appropriately determined. In the GUI 42, radio buttons 425, 426, and 427 corresponding to each image 422, 423, and 424 when the sound velocity is changed by a predetermined adjustment amount are displayed. The user operates the input device to enable the radio button corresponding to the image determined to be the most suitable among the images 422, 423, and 424, and presses the OK button 428. When the OK button 428 is pressed, the adjustment amount of the sound velocity corresponding to the radio button that is enabled among the radio buttons 425, 426, and 427 is determined as the difference to the appropriate sound velocity. The determined appropriate sound velocity is created as correct answer data in association with the image in which the radio button is enabled and the observation depth among the images 422, 423, and 424 in the extraction area. When the user determines in the GUI 42 that the image is not suitable as learning data, such as the SN ratio of the image in the displayed extraction area is not good, the user operates the input device and presses the SKIP button 429. As a result, the images 422, 423, and 424 of the currently displayed extraction area can be prevented from being included in the training data.

As described above, the slide bar 413b and the radio buttons 425, 426, 427 can accept the adjustment amount of the sound wave velocity of the ultrasonic wave used when generating the image data. The display device 108 corresponds to a display unit that selectively displays a plurality of adjustment amounts of the sound velocity of ultrasonic waves used when generating image data.

In the present embodiment, related information other than the observation image and the observation depth may be added to the input data. For example, other candidates for input data include NA (numerical aperture), aperture size, frequency, diagnostic area, etc. of a lens that emits a transmitted ultrasonic beam. The NA, aperture size, and frequency are related to changes in the image due to differences in the speed of sound used when generating image data between the actual observation target and the received signal processing block 106. In addition, the diagnostic area is related to the change in the image in that the amount up to the proper sound velocity may change depending on the thickness of the fat layer, which generally has a slow sound velocity. Furthermore, by adding information such as clinical department, gender, BMI, age, and pathological condition to the input data, it is possible to obtain a learned model that corresponds to the characteristics of each part to be observed in more detail. , It can be expected that the estimation accuracy will be higher.

Further, the trained model of the sound velocity suggestion block 110 mounted on the ultrasonic diagnostic apparatus 1 may be a model in which image data of all clinical departments is trained, or a model in which image data of each clinical department is trained. When a model in which image data for each clinical department is trained is installed, the system control block 109 causes the user of the ultrasonic diagnostic apparatus 1 to input or select clinical department information, and is used according to the clinical department. It is good to change the finished model. It can be expected that the estimation accuracy will be further improved by using different models for each clinical department where the imaging site is limited to some extent.

The trained model obtained by learning using such an image and the observation depth as input data and using the label of the difference from the sound velocity at the time of imaging to the appropriate sound velocity as the correct answer data is obtained on the sound velocity suggestion block 110. Operate. As a result, the sound velocity suggestion block 110 estimates the difference (adjustment amount) to the appropriate sound velocity with respect to the image input from the received signal processing block 106, and outputs the estimated sound velocity adjustment amount as the estimation result.

(Image generation method)
Next, the details of the process for image generation in the present embodiment will be described with reference to FIG. An imaging instruction is input from the user using a GUI (not shown). The system control block 109 that receives the instruction from the GUI inputs the ultrasonic transmission instruction to the transmission electric circuit 104. The transmission instruction may include parameters and sound velocity information for calculating the delay time. The transmission electric circuit 104 outputs a plurality of pulse signals (voltage waveforms) to the plurality of vibrators 101 of the ultrasonic probe 102 through the probe connection unit 103 based on the transmission instruction from the system control block 109. At this time, the transmission electric circuit 104 sets the delay time of the pulse signal applied to each transducer 101 according to the transmission direction (deflection angle) and the focus position of the ultrasonic beam. The deflection angle is an angle formed by the normal direction of the surface on which the plurality of vibrators 101 are arranged and the axial direction of the ultrasonic beam.

The ultrasonic waves transmitted from the plurality of vibrators 101 propagate in the subject 100 and are reflected at the boundary of the acoustic impedance in the subject 100. The plurality of vibrators 101 receive the reflected ultrasonic waves reflecting the difference in acoustic impedance and convert them into voltage waveforms. This voltage waveform is input to the receiving electric circuit 105 through the probe connecting portion 103. The receiving electric circuit 105 amplifies the voltage waveform as necessary, digitally samples it, and outputs it to the receiving signal processing block 106 as a receiving signal.

FIG. 5 shows a flow of image generation / display processing executed by each block of the ultrasonic diagnostic apparatus 1 in the present embodiment. In step S50, the received signal processing block 106 generates an observation image. Specifically, the received signal processing block 106 stores the input received signal in the internal memory. Further, the received signal processing block 106 performs delay processing and addition processing according to parameters and sound velocity information for calculating the delay time input from the system control block 109. Further, the received signal processing block 106 obtains the envelope by the envelope detection process, and stores the obtained envelope data in the internal memory. The reception signal processing block 106 generates an observation image by repeating the above processing with respect to the input reception signal.

Next, in step S51, the sound velocity suggestion block 110 calculates the difference to the appropriate sound velocity with respect to the observed image. Specifically, the image generated in step 50 is output to the image processing block 107 and the sound velocity suggestion block 110. The sound velocity suggestion block 110 outputs the difference up to the appropriate sound velocity to the system control block 109 by using the input image and the information of the observation depth input from the system control block 109. The difference up to the appropriate sound velocity may be calculated using the average value of the difference in the entire observation image or the distribution of the difference for each region set in the observation image.

Next, in step S52, the display device 108 displays the corrected image corrected based on the difference up to the proper sound velocity and the difference up to the proper sound velocity. Specifically, the system control block 109 inputs the information of the appropriate sound velocity to the reception signal processing block 106 by using the information of the difference from the sound velocity suggestion block 110 to the appropriate sound velocity input. The reception signal processing block 106 acquires the reception signal saved in the internal memory by the above processing from the internal memory. Then, the received signal processing block 106 re-executes the delay processing and the addition processing on the acquired received signal using the sound velocity information of the appropriate sound velocity input from the system control block 109. Further, the reception signal processing block 106 obtains the envelope by the envelope detection process, and outputs the image data generated based on the obtained envelope to the image processing block 107 as the corrected image data. The reception signal processing block 106 corresponds to an image generation unit that generates modified image data obtained by modifying the image data obtained by transmitting and receiving ultrasonic waves by using the adjustment amount of the sound velocity. As a result, the image processing block 107 acquires two types of observation images, the uncorrected observation image generated in step S50 and the corrected observation image generated in step S52. The image processing block 107 outputs to the display device an image that has undergone processing such as brightness adjustment and interpolation, and application of other filters to the image input from the reception signal processing block 106. Then, the ultrasonic diagnostic apparatus 1 ends the processing of this flow.

6A to 6C schematically show an example of an image display screen in the display device 108. Images 611, 621, and 631 generated by the above processing are displayed on the display screens 610, 620, and 630, respectively.

In the example of FIG. 6A, the image 611 displayed on the display screen 610 is a modified image generated by the above processing. Further, on the display screen 610, the difference between the sound velocity used for capturing the observation image which is the basis of the image 610 and the appropriate sound velocity (“1500 + 15” in the figure) is displayed. In the case of the example of FIG. 6A, of "1500 + 15" in the figure, "1500" is the speed of sound input from the system control block 109 to the transmission electric circuit 104, that is, the delay time of each vibrator for forming the transmission beam. Represents the speed of sound when calculated. Further, among "1500 + 15" in the figure, "+15" represents the difference from the sound velocity suggestion block 110 to the appropriate sound velocity output to the system control block 109. Further, the display screen 610 has an index (“sound velocity suggestion ON” in the example of FIG. 6A) indicating that the image display mode is a mode for displaying the corrected image after being adjusted by the difference to the appropriate sound velocity. Is displayed.

In the example of FIG. 6B, the image 621 displayed on the display screen 620 is an observation image that is the basis of the corrected image generated by the above processing, that is, an image before correction based on an appropriate sound velocity. Further, it is indicated by "+0" of "1500 + 0" that there is a correction candidate (correction image) on the display screen 620. Of "1500 + 0" in the figure, "1500" represents the sound velocity input from the system control block 109 to the transmission electric circuit 104, that is, the sound velocity when the delay time of each vibrator for forming the transmission beam is calculated. .. Further, on the display screen 620, an index (“sound velocity suggestion OFF” in the example of FIG. 6B) indicating that the image display mode is not the mode for displaying the image after being adjusted by the difference to the appropriate sound velocity is displayed. Will be done.

The user operates the input device to switch the image display mode to the mode for displaying the modified image. As a result, the display of the index "sound velocity suggestion OFF" on the display screen 620 is changed to "sound velocity suggestion ON", and the display content of the display screen 620 is switched to the display of the corrected image as described in the example of the display screen 610. If there is no correction candidate, such as when there is no difference between the sound velocity when calculating the delay time of each vibrator for forming the transmission beam and the appropriate sound velocity, "+0" of "1500 + 0" is displayed on the display screen 620. It is shown that there are no correction candidates by hiding.

In the example of FIG. 6C, the image 631 displayed on the display screen 630 is a modified image generated by the above processing. Further, on the display screen 630, a distribution diagram 632 showing the distribution of the difference up to the appropriate sound velocity according to the observation depth in the image 631 is displayed together with the observation depth (“f1” and “f2” in the figure). As for the distribution of the difference up to the appropriate sound velocity shown by the distribution diagram 632, the representative value such as the overall average value or the variation may be indicated by a number, the distribution of the variation itself may be displayed, and the sound velocity average in the selected region. It may indicate a value and variation. Further, on the display screen 630, a scale 633, which is an legend showing the difference (adjustment amount) to the appropriate sound velocity in the distribution diagram 632, is displayed. By referring to the distribution diagram 632 based on the scale 633, the user can confirm the difference (adjustment amount) up to the appropriate sound velocity, which is the sound velocity information corresponding to the observation depth. Further, since the user can confirm the appropriate sound velocity in the region of interest from the distribution map 632, the time required for adjusting the sound velocity can be shortened.

The display form of the observed image and the modified image on the display device 108 is not limited to the above, and for example, the observed image before modification and the modified image may be displayed in parallel on one display screen. Further, a simple display form may be used in which the difference up to the appropriate sound velocity calculated by the display device 108 is only displayed. Even in this case, the user can shorten the time required for adjusting the sound velocity while grasping how much it is preferable to change the sound velocity. In addition, the display of whether the displayed image is before or after correction is a character (in the above example, "sound velocity suggestion ON".
It does not have to be displayed by "sound velocity suggestion OFF"). For example, a method of changing the color of the outer edge of the display image or the display area, blinking, changing the background color, saturation, pattern, or the like may be adopted.

<Second Embodiment>
Next, the ultrasonic diagnostic apparatus according to the second embodiment will be described. Since the overall configuration of the ultrasonic diagnostic apparatus 1 is the same as that of the first embodiment (FIG. 1), the same reference numerals are given to the same configurations as those of the first embodiment in the following description, and detailed description thereof will be omitted. do.

In the second embodiment, the process of outputting the image data from the received signal processing block 106 to the image processing block 107 and the sound velocity suggestion block 110 is the same as that of the first embodiment. Further, the process until the difference from the sound velocity suggestion block 110 to the system control block 109 to the appropriate sound velocity is output is the same as that of the first embodiment. In the second embodiment, the transmission delay is recalculated using the difference to the appropriate sound velocity, the ultrasonic wave is transmitted to the subject again, and the ultrasonic wave reflected from the observation target is received by the receiving electric circuit 105. .. Then, even in the phase adjustment addition in the received signal processing block 106, the delay time is calculated using the difference up to the appropriate sound velocity.

Further, in the second embodiment, when the second ultrasonic wave is transmitted and received, the difference up to the appropriate sound velocity is not used as it is for the sound velocity set at the time of the first transmission and reception, but a fixed ratio of the difference is corrected. Is set as the adjustment amount. For example, when the sound velocity set at the time of the first transmission / reception is 1500 m / s, even if the difference to the appropriate sound velocity is calculated as + 10 m / s, 50% of the difference is set as the adjustment amount at the time of the second transmission / reception. 1505 m / s (= 1500 + 0.5 × 10) is set as the speed of sound. By setting the adjustment amount in this way, the change in the image before and after the correction becomes gentle, and it is possible to reduce the discomfort that the user receives when changing the displayed image in image observation.

Further, in the second embodiment, since the ultrasonic wave is transmitted so as to converge to a specific position in the transmission of the ultrasonic wave, even if the distribution of the difference up to the appropriate sound velocity is known, the transmission using the corrected delay time is used. It is difficult to do this everywhere because the number of transmissions increases in reality. Therefore, a technique called a multi-stage focus technique is adopted in which ultrasonic transmission is performed by gradually changing the convergence depth in the same transmission direction. In the second embodiment, the convergence of ultrasonic transmission in the multi-stage focus technique is performed on the observation depths (observation depths f1 and f2 in the example of FIG. 6C) that greatly change in the depth direction of the subject in the distribution of the difference up to the appropriate sound velocity. The depth is set. As a result, transmission that reflects the difference up to the appropriate sound velocity can be performed efficiently.

<Other Embodiments>
The above embodiments are merely specific examples of the present invention. The scope of the present invention is not limited to the configuration of the above-described embodiment, and various embodiments can be adopted without changing the gist thereof.

In the above embodiment, the input to the sound velocity suggestion block 110 is image data, but the received data after the delay processing may be input, whereby the difference up to the appropriate sound velocity may be calculated. Further, in the process of creating correct answer data using the GUIs 41 and 42 illustrated in FIG. 4 with the difference up to the appropriate sound velocity as a label, the appropriate sound velocity is determined by changing the image data of the extraction area while changing the sound velocity adjustment amount. By doing so, the difference up to the appropriate sound velocity is used as the correct answer data. However, a trained model is created that outputs the difference up to the proper sound velocity by inputting the received data after the delay processing and the difference between the received data after the delay processing and the appropriate sound velocity, which is the basis of the image data in the extraction area, as the correct answer data. May be done. When using a digital phantom, by changing the sound velocity value set by the model and the sound velocity value used to calculate the delay time, correct answer data of the difference between the received data after delay processing and the appropriate sound velocity is created. be able to.

FIG. 7 is a block diagram showing an example of the hardware configuration of the ultrasonic diagnostic apparatus according to the modified example. In the ultrasonic diagnostic apparatus 7 according to this modification, the same reference numerals are given to the same configurations as those in the above embodiment, and detailed description thereof will be omitted. Further, FIG. 8 is a block diagram showing details of a received signal processing block of the ultrasonic diagnostic apparatus 7. As shown in FIG. 8, the received signal processing block 706 includes a delay processing block 7061, an addition processing block 7062, an envelope detection processing block 7063, and a sound velocity suggestion block 7064.

The delay processing block 7061 performs delay processing on the received signal output from the receiving electric circuit 105. When this modification is applied to the first embodiment, the sound velocity used for this delay processing is the sound velocity input from the system control block 109 to the transmission electric circuit 104, that is, the delay time of each vibrator for forming the transmission beam. Is the speed of sound when calculated. Further, when the present modification is applied to the second embodiment, the sound velocity used for this delay processing is the sound velocity set at the time of the first transmission / reception. The received data after the delay processing is input to the sound velocity suggestion block 7064, and the sound velocity suggestion block 7064 outputs the difference up to the appropriate sound velocity.

Then, when this modification is applied to the first embodiment, the difference up to the appropriate sound velocity is output to the delay processing block 7061, and the delay processing is performed again. Then, the addition processing by the addition processing block 7062 and the envelope detection processing by the envelope detection processing block 7063 are performed to generate image data. The generated image data is output to the image processing block 107.

Further, when this modification is applied to the second embodiment, the difference up to the proper sound velocity is output to the system control block 109 and then input to the transmission electric circuit 104. Then, the transmission delay is recalculated by the transmission circuit 104, the ultrasonic wave is transmitted to the observation target again from the ultrasonic probe 102, and the reception signal is acquired by the reception electric circuit 105. Then, the delay time calculation process using the difference to the appropriate sound velocity by the delay process block 7061, the addition process by the addition process block 7062, and the envelope detection process by the envelope detection process block 7063 are performed, and the image data is obtained. Generated. The generated image data is output to the image processing block 107.

Further, in the above embodiment and the modified example, the input data used for creating the trained model includes the sound velocity of the ultrasonic wave used when generating the image data (1500 m / s in the above example). Is also good. Further, as an output using the trained model, the adjusted sound velocity (1510 m / s in the above example) may be output in addition to or instead of the difference up to the appropriate sound velocity.

Further, the disclosed technology can take an embodiment as a system, an apparatus, a method, a program, a recording medium (storage medium), or the like. Specifically, it may be applied to a system composed of a plurality of devices (for example, a host computer, an interface device, an imaging device, a web application, etc.), or it may be applied to a device composed of one device. good.

Needless to say, the object of the present invention is achieved by doing the following. That is, a recording medium (or storage medium) on which the program code (computer program) of the software that realizes the functions of the above-described embodiment is recorded is supplied to the system or device. Needless to say, such a storage medium is a computer-readable storage medium. Then, the computer (or CPU or MPU) of the system or device reads and executes the program code stored in the recording medium. In this case, the program code itself read from the recording medium realizes the function of the above-described embodiment, and the recording medium on which the program code is recorded constitutes the present invention.

1: Ultrasonic diagnostic device 100: Subject 102: Ultrasonic probe 110: Sound velocity suggestion block 201: Learning data 202: Observation image 203: Difference to appropriate sound velocity 205: Trained model

Claims (19)

An ultrasonic probe that sends and receives ultrasonic waves to the observation area of the subject,
Using a trained model machine-learned using training data including image data obtained by transmitting and receiving ultrasonic waves and sound velocity information of ultrasonic waves, image data obtained by transmitting and receiving ultrasonic waves is used in the observation region. A sound velocity information acquisition unit that acquires ultrasonic sound velocity information,
An ultrasonic diagnostic apparatus characterized by having. The ultrasonic diagnostic apparatus according to claim 1, wherein the learning data includes the sound velocity information as correct answer data. The ultrasonic diagnostic apparatus according to claim 1 or 2, wherein the image data obtained by transmitting and receiving the ultrasonic wave includes RF (Radio Frequency) data. The ultrasonic diagnostic apparatus according to any one of claims 1 to 3, wherein the trained model is further machine-learned using the depth of the observation region in the image data. It also has a reception unit that receives the adjustment amount of the sound wave of the ultrasonic wave used when generating the image data.
The ultrasonic diagnostic apparatus according to any one of claims 1 to 4, wherein the trained model is machine-learned using the received adjustment amount. A display unit that can selectably display a plurality of adjustment amounts of ultrasonic sound waves in the observation region, and
A reception unit that accepts the adjustment amount selected from the plurality of adjustment amounts, and
The ultrasonic diagnostic apparatus according to claim 1, further comprising. The image data used for machine learning of the trained model is the data of the received signal of the ultrasonic wave by the ultrasonic probe or the data obtained by phasing-adding the received signal. The ultrasonic diagnostic apparatus according to any one of 6. The ultrasonic diagnostic apparatus according to any one of claims 1 to 7, wherein the trained model is a neural network. An image generation unit that uses the sound velocity information acquired by the sound velocity information acquisition unit to generate modified image data obtained by modifying the image data obtained by transmitting and receiving ultrasonic waves.
A display unit that displays the generated modified image data, and
The ultrasonic diagnostic apparatus according to any one of claims 1 to 8, further comprising. The ultrasonic wave according to claim 9, wherein the display unit displays the modified image data and an index indicating that the modified image data is image data modified by using the sound velocity information. Diagnostic device. The ultrasonic diagnostic apparatus according to claim 9 or 10, wherein the display unit displays the modified image data, the depth of the observation region in the modified image data, and the distribution of the sound velocity information. Claims 1 to 8 further include an image generation unit that generates an ultrasonic image by using the image data obtained by transmitting and receiving ultrasonic waves and the sound velocity information acquired by the sound velocity information acquisition unit. The ultrasonic diagnostic apparatus according to any one of the above. Any one of claims 1 to 11, wherein the sound velocity information acquired by the sound velocity information acquisition unit is an appropriate sound velocity value of the sound velocity of the ultrasonic wave used when generating the image data. The ultrasonic diagnostic apparatus according to the section. The method according to any one of claims 1 to 11, wherein the sound velocity information acquired by the sound velocity information acquisition unit is an adjustment amount of the sound velocity of the ultrasonic wave used when generating the image data. The described ultrasonic diagnostic apparatus. It is characterized by having a learning unit that performs machine learning of a trained model using learning data including image data of an observation area obtained by transmitting and receiving ultrasonic waves as input data and sound velocity information of ultrasonic waves as correct answer data. Learning device. A learning device that performs machine learning of a trained model used in a sound velocity information acquisition unit that acquires ultrasonic sound velocity information of the ultrasonic diagnostic apparatus according to any one of claims 1 to 14.
A learning device characterized by performing machine learning of a trained model using learning data including image data of an observation area obtained by transmitting and receiving ultrasonic waves as input data and sound velocity information of ultrasonic waves as correct answer data. A reception unit that receives ultrasonic sound velocity information used when generating image data,
The learning device according to claim 16, wherein the trained model is machine-learned using the received sound velocity information. A transmission / reception step of transmitting and receiving ultrasonic waves to the observation area of the subject using an ultrasonic probe,
Using a trained model machine-learned using the image data obtained by transmitting and receiving ultrasonic waves and the sound velocity information of ultrasonic waves, the sound velocity of ultrasonic waves in the observation region from the image data obtained by transmitting and receiving ultrasonic waves. Sound speed information acquisition step to acquire information,
An image processing method characterized by having. A program for causing a processor to execute each step of the image processing method according to claim 18.

Images