CN117115452A

CN117115452A - Controllable medical ultrasonic image denoising method, system and computer storage medium

Info

Publication number: CN117115452A
Application number: CN202311169180.1A
Authority: CN
Inventors: 檀韬; 姜明富; 林灿堂; 吴大维; 尤辰智
Original assignee: Macau Polytechnic University
Current assignee: Macau Polytechnic University
Priority date: 2023-09-12
Filing date: 2023-09-12
Publication date: 2023-11-24
Anticipated expiration: 2043-09-12
Also published as: CN117115452B

Abstract

The invention discloses a controllable medical ultrasonic image denoising method, a system and a computer storage medium, wherein the method comprises the steps of generating a simulated noise ultrasonic image data set, preprocessing a simulated noise ultrasonic image, establishing a denoising model, training the denoising model, establishing a joint loss function and the like, establishing a controllable medical ultrasonic image denoising model, then firstly downsampling a true medical ultrasonic image to be denoised, inputting a subgraph of the downsampled true medical ultrasonic image and 2 characteristic parameters with set values into the trained denoising model, adaptively denoising the true medical ultrasonic image by the denoising model, flexibly adjusting denoising intensity, and finally outputting medical ultrasonic images denoised by different denoising intensities. The invention can adapt to a wider denoising range, can adaptively denoise, can flexibly adjust the denoising intensity, can effectively remove the noise of the medical ultrasonic image, and simultaneously retains the detailed information of the medical ultrasonic image.

Description

Controllable medical ultrasonic image denoising method, system and computer storage medium

Technical Field

The invention belongs to the technical field of medical image and image processing, and particularly relates to a controllable medical ultrasonic image denoising method, a controllable medical ultrasonic image denoising system and a computer storage medium.

Background

Ultrasonic imaging is widely adopted in clinical diagnosis with the advantages of safety, noninvasive property, convenience, simple operation and the like, and is one of important tools for clinical diagnosis. However, the practical ultrasonic image generally has the problems of more speckle noise, low signal-to-noise ratio, low contrast ratio and the like. Especially, excessive speckle noise seriously reduces the quality of images, directly increases the difficulty of accurately distinguishing between focus parts and benign tissue areas of medical workers, and makes automatic identification, segmentation, analysis and feature extraction of image focuses more difficult in a computer-aided diagnosis system.

During the ultrasound image acquisition, due to the reflection, scattering and refraction properties of the ultrasound echoes, and the non-uniformity and uncertainty properties with respect to the spatial distribution of parts of the human tissue, a large number of randomly distributed scattering particles are formed when the ultrasound waves are incident on the human body, and the interaction between the scattering particles generates a relevant scattering beam. During echo reflection, due to interference effect of reflected echoes and mutual interference between scattered beams, when echoes of different beams overlap, addition and subtraction of amplitudes occur due to different phases of the echoes, so that random fluctuation of an electric signal is caused in output after envelope detection of a transducer, and speckle particles with different brightness are generated in an ultrasonic image, and the noise is often called multiplicative speckle noise. In addition, in the operation process of the ultrasonic equipment, certain additive Gaussian noise can be generated by internal devices, circuits, electromagnetic interference and the like of the transducer, and the noise forms complex noise of the medical ultrasonic image together. These noises can affect the clinical diagnosis of the doctor and can also affect the further analysis and processing of the medical ultrasound images, so that researchers have studied the denoising of medical ultrasound images.

In terms of conventional denoising methods, for example:

jain proposes a denoising theory based on a model consisting of multiplicative noise and additive noise, the multiplicative noise is converted into the additive noise, the image denoising utilizes different logarithm conversion, logarithm compression and the like to calculate and obtain an approximate value of an original pixel point without noise, so that a denoised image is obtained.

Gupta, S.et al propose a method for posterior estimation in wavelet domain, which uses multiplicative Rayleigh distribution and additive Gaussian distribution of speckle noise to build a logarithmic transformation denoising model, and can well inhibit speckle noise.

Tsui P-H.et al, on the basis of Nakagam distributed Noise, uses multi-focus images to reconstruct ultrasonic images to inhibit Noise, and can well remove artifacts.

Merem Hacini et al propose a method for denoising a uniform region by creating an adaptive window as a filter as a function of the image structure in a multiplicative regularization manner using a weighted total variation function as a multiplication factor, which can remove speckle noise while edges of the image can be preserved.

Saniie J. the homomorphic filtering algorithm denoising is a transform domain filtering algorithm, which uses homomorphic processing to perform spectrum smoothing denoising, performs Fourier transform on the image of the transform window, determines different frequency ranges of noise and uncontaminated image in an interactive mode, selects a proper frequency domain band-pass filter to perform filtering processing, filters out frequency domain noise, and performs inverse Fourier transform to obtain a denoising image.

In general, the above-mentioned traditional denoising methods are mainly a spatial domain-based filtering algorithm, a transform domain-based filtering algorithm and a diffusion theory-based filtering algorithm, and most of the traditional denoising methods rely on fixed reasoning formulas, have complex parameter adjustment, cannot consider the whole complicated noise model, are difficult to keep balance in terms of removing noise and preserving image structure and detail, and cannot process an ultrasonic image from time to time.

With the development of machine learning and artificial intelligence, in recent years, a method based on machine learning and deep learning has achieved good performance in the field of image denoising, for example:

yancheng LI proposes an ultrasonic image denoising model of a residual coder-decoder network based on a multi-attention fusion model (RED-MAM), which can obtain more and enough features from an ultrasonic image and maintain the diversity of image feature information, and the PSNR of which is improved by 0.5dB compared with that of a BM3D method, but the training set of which lacks a real ultrasonic image.

Zhang L proposes that denoising of an ultrasound image by a model of the generated countermeasure network with residual dense connectivity and weighted joint loss can remove some speckle noise, but training data of the method is formed by using a simulated noise ultrasound image and gaussian noise, a data set is constructed so that the ultrasound image cannot be close to a real ultrasound image, and therefore, removing noise of the ultrasound image can lose part of image details and cause image distortion.

On Karao ğ lu proposed that the U-shaped network (DUNet) and the generation countermeasure network (dganet) remove noise of the ultrasonic image, and although a certain denoising effect is obtained, the speckle noise of the rayleigh distribution can be removed, other speckle noise of the ultrasonic image is difficult to remove, and some image details cannot be kept.

K Zhang proposes a Gaussian noise denoising model (FFDNet) based on controllable denoising level, and denoising can be performed by manually adjusting denoising intensity for natural image denoising, but the method cannot automatically calculate denoising intensity based on manual adjustment of denoising intensity, so that the denoising intensity is difficult to select, only additive Gaussian noise of an ultrasonic image can be removed, and speckle noise of the ultrasonic image cannot be effectively removed.

Shen z. A parallel network (USNet) is proposed for removing speckle noise, three paths of parallel convolutional neural networks are used to realize speckle suppression of an ultrasonic image, three different sub-networks are connected together, the width of the whole network can be increased, more image features are obtained, parameters do not need to be adjusted in the method, and speckle noise can be removed rapidly for different types of ultrasonic images and better textures can be reserved.

The emmaraasana, S. the adjustable weighted kernel norm specification minimizes the removal of ultrasonic image noise, and can flexibly construct modules and select denoising algorithms, but the denoising image structure detail reserved by the method has poor effect.

Tian, c. double convolutional nerves (DudeNet), this model consists of Feature Extraction Block (FEB), enhancement Block (EB), compression Block (CB) and Reconstruction Block (RB), and global features of an image can be fused with local features to obtain finer image features.

Goudarzi, S. proposes a beam forming method using deep learning, reconstructing an ultrasound image from an original plane wave image using an ideal Point Spread Function (PSF) close to Plane Wave Imaging (PWI), and the model can be widely applied to high-quality reconstruction of the ultrasound image without adjusting parameters and denoising.

In summary, some of the above denoising methods based on machine learning or deep learning cannot fully utilize deep information of spots and textures of an original ultrasonic image, and most models can only process noise at a single level, so that the model has a small noise processing range, and automatic adjustment of denoising intensity is difficult.

Disclosure of Invention

In order to overcome the defects of the prior denoising method, the invention provides a controllable medical ultrasonic image denoising method, a system and a computer storage medium, so as to realize self-adaptive adjustment of denoising intensity, and the detail information of an image is reserved while the noise of an ultrasonic image is effectively removed.

In order to solve the technical problems, the invention is realized by the following technical scheme:

a controllable medical ultrasonic image denoising model establishment method comprises the following steps:

step 1) generation of a simulated noise ultrasound image dataset:

generating a simulation noise ultrasonic image with different noise levels by taking a large number of clean natural images as references and using corresponding simulation software and different imaging parameter groups, and dividing the simulation noise ultrasonic image into a simulation noise ultrasonic image data set for model training, verification and test;

step 2) preprocessing of the simulated noise ultrasonic image:

downsampling the analog noise ultrasonic image in the analog noise ultrasonic image data set, randomly selecting values of 2 characteristic parameters between a true value and a set value, and connecting the downsampled subgraph with the 2 characteristic parameters to form a new input image data set;

and 3) building a denoising model:

constructing an initial denoising model by adopting a multi-stage residual attu space pyramid pooling (MRASPP) module, a Nonlinear Mapping Convolutional Neural Network (NMCNNB) module, an adaptive noise level and a variable denoising intensity module;

the multistage residual attu space pyramid pooling module is established based on residual error structures, attu space pyramid pooling and matrix addition connection and is responsible for extracting features of a new input image in the new input image dataset;

The nonlinear mapping convolutional neural network module is established based on a convolutional layer of CNN and is responsible for extracting deep features from the output of the multistage residual Atu space pyramid pooling module to finally generate a feature image;

the self-adaptive noise level and variable denoising intensity module is established based on bifurcated up-sampling, a full-connection layer and a joint loss function, is responsible for carrying out noise prediction on the characteristic images with known 2 real values of characteristic parameters, obtains noise prediction images predicted by different denoising intensities by setting different denoising intensities, and finally obtains images denoised by different denoising intensities by respectively subtracting the noise prediction images by using an analog ultrasonic image; on the other hand, the method is responsible for carrying out characteristic parameter prediction on the characteristic images with unknown 2 real values of characteristic parameters, and obtaining predicted values of 2 characteristic parameters;

training of the denoising model:

training a denoising model by using the simulated noise ultrasonic image data set, and randomly selecting 2 simulated noise ultrasonic images with the characteristic parameters as the true values when the denoising model is input, wherein the denoising model directly uses the true values of the 2 characteristic parameters to obtain denoised images with different denoising intensities; for a simulated noise ultrasonic image with 2 characteristic parameters as set values when a denoising model is input, firstly predicting predicted values of the 2 characteristic parameters by the denoising model, then inputting a sub-image obtained by downsampling the simulated noise ultrasonic image together with the predicted values of the 2 characteristic parameters into the denoising model again, and obtaining an image denoised by different denoising intensities by using the predicted values of the 2 characteristic parameters by the denoising model;

In the training process, the parameters of the denoising model when predicting the noise level and 2 characteristic parameters are adjusted by utilizing the joint loss function of the denoising model, and finally, the trained controllable medical ultrasonic image denoising model is obtained.

Further, in step 1), the specific method for generating the simulated noise ultrasound image dataset includes:

acquiring a certain number of clean natural images, carrying out gray scale treatment on each clean natural image, and then respectively inputting the clean natural images into simulation noise ultrasonic image simulation software;

simulating each gray-scale clean natural image by using 3 groups of different imaging parameters, and correspondingly generating simulated noise ultrasonic images with 3 different noise levels; the simulated noise ultrasonic image is aligned with the original clean natural image, and the size of the simulated noise ultrasonic image and the original clean natural image are the same;

all the simulated noise ultrasonic images are divided into 3 groups of simulated noise ultrasonic image data sets according to a certain quantity proportion, and the 3 groups of simulated noise ultrasonic image data sets are respectively used as a training set, a verification set and a test set of the denoising model.

Further, in the step 2), the specific method for preprocessing the simulated noise ultrasonic image is as follows:

downsampling the size of an analog noise ultrasound image to 4A subgraph; the original ultrasonic image has the size ofThe size of the ultrasound image after downsampling is +.>Wherein C represents an image size Channel, W represents an image size Width, and H represents an image size height;

randomly selecting values of 2 characteristic parameters between a true value and a set value, wherein the 2 characteristic parameters are 2 parameters fc (ultrasonic image frequency) and pitch (ultrasonic image array element center-to-center distance) related to noise level in imaging parameters of the analog noise ultrasonic image respectively; the real value is the actual value of the parameters fc and pitch adopted in the generation process of the simulated noise ultrasonic image; the set value is assumed to be a numerical value-1 of manually set parameters fc and pitch;

the parameters fc and pitch with good values are respectively used as the other 2 channels of the analog noise ultrasonic image, are connected with the 4 sub-images after downsampling to form a new input image of 6 channels, and are input into a denoising model, wherein the size of the new input image is that 。

Further, in step 3), the specific method for generating the feature image is as follows:

firstly, inputting the new input image into the multistage residual attu space pyramid pooling module to extract features for the first time, wherein the specific steps comprise:

carrying out first-layer Arteru space pyramid pooling treatment on the new input image, connecting the extracted feature images together, and then carrying out further treatment on a first-layer convolution layer with a convolution kernel of 3 multiplied by 3 to extract a feature image 1;

carrying out second-layer Arteru space pyramid pooling treatment on the extracted feature map 1, connecting the extracted feature maps together, and then carrying out further treatment on a second-layer convolution layer with a convolution kernel of 3 multiplied by 3 to extract a feature map 2;

carrying out third-layer Abbe space pyramid pooling treatment on the extracted feature map 2, connecting the extracted feature maps together, and then carrying out further treatment on a third-layer convolution layer with a convolution kernel of 3 multiplied by 3 to extract a feature map 3;

three layers of the Atu space pyramid pooling are composed of 3 cavity convolutions, namely expansion convolutions, so as to solve the contradiction between the size of the receptive field and the resolution ratio in the process of extracting the features; the 3 hole convolutions are as follows: the first cavity convolution, the expansion rate of which is 1, and the receptive field of the convolution kernel of which is 3 multiplied by 3, is common convolution; a second cavity convolution with an expansion ratio of 2 and a convolution kernel with a receptive field of 7 x 7; a third cavity convolution with an expansion ratio of 4 and a convolution kernel with a receptive field of 15×15;

Connecting the feature map 1, the feature map 2 and the feature map 3 obtained by three times of extraction with the new input image by using a residual error structure and matrix addition connection to form a multi-stage residual attu space pyramid pooling output image, wherein the multi-stage residual attu space pyramid pooling output image extracts abundant structural detail features of the simulated noise ultrasonic image and retains original features of the simulated noise ultrasonic image;

and then inputting the multi-stage residual atlas space pyramid pooling output image into the nonlinear mapping convolutional neural network module to extract features for the second time, wherein the specific steps comprise:

processing the multi-level residual atlas-shaped spatial pyramid pooled output image through a 1 st layer CNN convolution layer, wherein the 1 st layer CNN convolution layer consists of convolution (Conv) and a correction linear unit (Rectified Linear Units, reLU);

sequentially processing the output of the CNN convolution layers 1 through the CNN convolution layers 2 to 14, wherein the CNN convolution layers 2 to 14 consist of convolution (Conv), batch normalization (Batch Normalization, BN) and correction linear units (Rectified Linear Units, reLU);

processing the output of the 14 th CNN convolution layer by a 15 th CNN convolution layer, wherein the 15 th CNN convolution layer consists of convolution (Conv);

The convolution kernels of the CNN convolution layers of the 1 st layer to the 15 th layer are 3 multiplied by 3, 0 is used for filling, a nonlinear mapping convolution neural network output image is obtained after convolution of the CNN convolution layers of the 15 th layer, and deep features, namely the feature images, are obtained by the nonlinear mapping convolution neural network output image.

Further, in step 4), the specific method for generating the image denoised by different denoising intensities is as follows:

upsampling the feature image, the upsampling being opposite to the downsampling, reducing the feature image to a 3-channel complete noise prediction image; the other 2 channels in the complete noise prediction image are characterized by parameters fc and pitch;

and taking out the 3 channel characteristics of the complete noise image, and setting a plurality of different denoising intensities, thereby obtaining a plurality of noise prediction graphs with different noise intensities.

Further, in step 4), the specific method for predicting the predicted values of the parameters fc and pitch is as follows:

firstly, carrying out self-adaptive pooling processing on the characteristic image to obtain a self-adaptive pooled output image which is matched with the characteristic image and provided with 12 channels;

then carrying out leveling treatment on the adaptive pooled output image, leveling the values of the adaptive pooled output image with 12 channels into one dimension so as to carry out full-connection operation;

And finally, carrying out full-connection operation on all image features of the flattened self-adaptive pooled output image by using 3 full-connection layers to respectively obtain predicted values of the parameters fc and pitch.

Further, in step 4), the joint loss function uses a weighted sum of loss functions, including denoising loss, parameter fc and pitch loss, where the formula is:

the method comprises the steps of carrying out a first treatment on the surface of the Wherein,in order to remove the noise loss,for the parameters fc and pitch loss, +.>As a result of the mean square error of the noise level,parameters fc and pitch, respectively, affecting ultrasound image generation +.>For a regularization term related to noise, +.>Weight factors lost for parameters fc and pitch,/->，/>Is a weighting factor for joint loss.

A controllable medical ultrasound image denoising method, comprising: the method comprises the steps of obtaining a true medical ultrasonic image to be denoised, firstly performing downsampling on the true medical ultrasonic image, then inputting a subgraph obtained by downsampling of the true medical ultrasonic image and 2 characteristic parameters taking the value as a set value into a controllable medical ultrasonic image denoising model obtained by the controllable medical ultrasonic image denoising model building method, firstly evaluating predicted values of parameters fc and pitch by the denoising model, then performing self-adaptive denoising on the true medical ultrasonic image according to the predicted values of the parameters fc and pitch, flexibly adjusting denoising intensity, finally outputting the medical ultrasonic image denoised by different denoising intensities, and retaining detail information of the image while effectively removing noise.

A controllable medical ultrasound image denoising system, comprising: a data receiving unit, a data processing unit and a data output unit;

the data receiving unit is in charge of acquiring a clean natural image and a real ultrasonic image;

the data processing unit is responsible for executing the operation corresponding to the controllable medical ultrasonic image denoising model establishment method and/or the operation corresponding to the controllable medical ultrasonic image denoising method;

the data processing unit comprises a data set generation module, a preprocessing module, a multi-stage residual attu space pyramid pooling (MRASPP) module, a Nonlinear Mapping Convolutional Neural Network (NMCNNB) module, an adaptive noise level and a variable denoising intensity module; wherein,

the data set generation module is responsible for generating simulated noise ultrasonic image data sets with different noise levels according to a large number of clean natural images and different imaging parameter sets by using the simulation noise ultrasonic image simulation software contained in the data set generation module, and dividing the simulated noise ultrasonic image data sets into simulated noise ultrasonic image data sets for training, verifying and testing of a denoising model according to a certain number proportion;

the preprocessing module is responsible for downsampling an analog noise ultrasonic image, randomly selecting values of 2 characteristic parameters between a true value and a set value, and connecting the downsampled subgraph with the 2 characteristic parameters to form a new input image data set; the method comprises the steps of taking charge of downsampling a real medical ultrasonic image, and connecting the downsampled subgraph with 2 characteristic parameters with values as set values to form a new input image;

The multi-stage residual attu space pyramid pooling (MRASPP) module is responsible for extracting features from the preprocessed new input image and generating a multi-stage residual attu space pyramid pooling output image; the multi-level residual attu space pyramid pooling (MRASPP) module is established based on residual error structures, attu space pyramid pooling and matrix addition connection and comprises three layers of attu space pyramid pooling layers and three layers of convolution layers which are sequentially connected at intervals; each layer of the Arteru space pyramid pooling layer comprises a first cavity convolution with the expansion rate of 1 and the receptive field of a convolution kernel of 3 multiplied by 3, a second cavity convolution with the expansion rate of 2 and the receptive field of the convolution kernel of 7 multiplied by 7, and a third cavity convolution with the expansion rate of 4 and the receptive field of the convolution kernel of 15 multiplied by 15;

the Nonlinear Mapping Convolutional Neural Network (NMCNNB) module is responsible for extracting deep features from the multi-level residual atlas space pyramid pooling output image and generating a feature image; the Nonlinear Mapping Convolutional Neural Network (NMCNNB) module is built based on 15 layers of CNN convolutional layers, and comprises a 1 st layer of CNN convolutional layers consisting of convolutional and correction linear units, 2 nd to 14 th layers of CNN convolutional layers consisting of convolutional, batch normalization and correction linear units and a 15 th layer of CNN convolutional layers consisting of convolutions, wherein the convolution kernel of each layer of CNN convolutional layer is 3 multiplied by 3, and 0 is used for filling;

The self-adaptive noise level and variable denoising intensity module is responsible for carrying out noise prediction on the second characteristic image with known real values of 2 characteristic parameters, obtaining noise prediction images predicted by adopting different denoising intensities by setting different denoising intensities, and finally obtaining images denoised by different denoising intensities by respectively subtracting the noise prediction images by using an analog ultrasonic image; on the other hand, the method is responsible for carrying out characteristic parameter prediction on the second characteristic image with unknown 2 real values of characteristic parameters and obtaining predicted values of 2 characteristic parameters; the self-adaptive noise level and variable denoising strength module is established based on a bifurcated up-sampling module, a full-connection module and a joint loss function, wherein the full-connection layer comprises a self-adaptive pooling layer, a leveling layer and a 3-layer full-connection layer which are sequentially connected;

and the data output unit is responsible for displaying the denoising ultrasonic image generated after denoising with different denoising intensities.

A computer apparatus, comprising: the medical ultrasonic image denoising system comprises a processor, a memory, a communication interface and a communication bus, wherein the processor, the memory and the communication interface are communicated with each other through the communication bus, the memory is used for storing at least one executable instruction, and the executable instruction enables the processor to execute the operation corresponding to the controllable medical ultrasonic image denoising model building method and/or the operation corresponding to the controllable medical ultrasonic image denoising method.

A computer storage medium, where at least one executable instruction is stored, where the executable instruction causes a processor to perform operations corresponding to the above-mentioned controllable medical ultrasound image denoising model creation method and/or operations corresponding to the above-mentioned controllable medical ultrasound image denoising method.

The invention uses Field II simulation software, sets three groups of ultrasonic transducers with different parameter (including frequency, array element width, array element interval, etc.) intervals according to the working principle of the ultrasonic transducers, generates three groups of ultrasonic images with different noise characteristics and intensities as training data sets, and then trains a model by using a training method of the three groups of mixed training data sets.

The invention uses the joint loss function to realize self-adaptive adjustment of the noise level of the ultrasonic image, and adapts to different denoising intensities.

The invention adopts the main parameters of the array element center frequency (fc) and the array element distance (pitch) which influence noise, can simulate different noise levels to train the denoising model, enables the denoising model to adaptively denoise, can flexibly adjust the denoising intensity, and accords with the parameter setting of a real ultrasonic image generation scene.

The invention adopts a model bifurcation mode, can adaptively configure the denoising of parameters, and randomly takes the values of the parameters fc and pitch as the true value or the set value-1 generated by the simulation image during training. When the predicted parameters fc and pitch are required, the values of the input parameters fc and pitch are-1. And the predicted parameters fc and pitch are brought into the model, so that the denoising can be performed accurately, and a denoised image can be obtained. The model is thus able to adapt to all cases of simulated images of known parameters fc and pitch values and real images of unknown parameters fc and pitch values.

The invention uses an MRASPP module, an NMCNNB module and a three-layer full-connection module to construct a model. The MRASPP module comprises a plurality of expansion convolutions and variable receptive fields, and is more effective for extracting features with different levels of image outlines and details. The NMCNNB module contains 15 layers of convolutions that enable extraction of deep features of the image. The three-layer full connection module is used for extracting characteristic values of the parameters fc and pitch so as to predict the values of the parameters fc and pitch.

The invention applies the images before and after denoising to the comparison of downstream image classification tasks. The real breast ultrasonic image is used for denoising, then the results of ultrasonic images before and after denoising for benign and malignant tumor classification are compared, and the Googlene and VGG16 classifier is found to be sensitive to the denoised image, the ACC of the real ultrasonic image classification after denoising is improved by 11.68%,6.11% and 1.09% respectively, and the AUC is improved by 2.83%. The invention is therefore very suitable for the denoising task of medical ultrasound images.

The foregoing description is only an overview of the present invention, and is presented in terms of preferred embodiments of the present invention and detailed description of the invention with reference to the accompanying drawings. Specific embodiments of the present invention are given in detail by the following examples and the accompanying drawings.

Drawings

The accompanying drawings, which are included to provide a further understanding of the application and are incorporated in and constitute a part of this specification, illustrate embodiments of the application and together with the description serve to explain the application and do not constitute a limitation on the application. In the drawings:

FIG. 1 is a flow chart of a controllable medical ultrasound image denoising method according to the present application;

FIG. 2 is a schematic illustration of the formation of a simulated noise ultrasound image dataset in accordance with the present application;

FIG. 3 is a schematic diagram of a multi-level residual Arragluty space pyramid pooling module in the present application;

FIG. 4 is a schematic diagram of a nonlinear mapping convolutional neural network module in accordance with the present application;

FIG. 5 is a schematic diagram of the denoising inference process of the present application when the parameters fc and pitch are known;

FIG. 6 is a schematic diagram of the denoising inference process of the present application when unknown parameters fc and pitch;

FIG. 7 is a schematic diagram of the denoising reasoning process of the present application for a real medical ultrasound image;

fig. 8 is a block diagram of a controllable medical ultrasound image denoising system according to the present application.

Detailed Description

The preferred embodiments of the present application will be described in detail below with reference to the attached drawings, so that the objects, features and advantages of the present application will be more clearly understood. It should be understood that the embodiments shown in the drawings are not intended to limit the scope of the application, but rather are merely illustrative of the true spirit of the application.

In the following description, for the purposes of explanation of various disclosed embodiments, certain specific details are set forth in order to provide a thorough understanding of the various disclosed embodiments. One skilled in the relevant art will recognize, however, that an embodiment may be practiced without one or more of the specific details. In other instances, well-known devices, structures, and techniques associated with the present application may not be shown or described in detail to avoid unnecessarily obscuring the description of the embodiments.

Throughout the specification and claims, unless the context requires otherwise, the word "comprise" and variations such as "comprises" and "comprising" will be understood to be open-ended, meaning of inclusion, i.e. to be interpreted to mean "including, but not limited to.

Reference throughout this specification to "one embodiment" or "an embodiment" means that a particular feature, structure, or characteristic described in connection with the embodiment is included in at least one embodiment. Thus, appearances of the phrases "in one embodiment" or "in an embodiment" in various places throughout this specification are not necessarily all referring to the same embodiment. Furthermore, the particular features, structures, or characteristics may be combined in any suitable manner in one or more embodiments.

As used in this specification and the appended claims, the singular forms "a," "an," and "the" include plural referents unless the context clearly dictates otherwise. It should be noted that the term "or" is generally employed in its sense including "and/or" unless the context clearly dictates otherwise.

In addition, the technical features of the different embodiments of the present invention described below may be combined with each other as long as they do not collide with each other.

Referring to fig. 1, the invention firstly provides a controllable medical ultrasonic image denoising model, and the method for establishing and training the denoising model specifically comprises the following steps:

1. generating a simulated noise ultrasound image dataset;

in a clinical setting, it is difficult to acquire a large number of sharp ultrasound images along with noisy ultrasound images. Furthermore, due to the unique properties of medical images, the fine structure of the denoised image must be maintained unchanged. To address this dilemma, the present invention utilizes plane wave imaging techniques to simulate a large number of noisy ultrasound images, which are aligned with clean natural images, thereby constructing a training dataset.

Plane wave imaging techniques significantly increase the imaging frame rate by activating the entire aperture of the transducer array simultaneously, thereby generating an image of the entire area in a single transmission. However, since there is no focus at the time of transmission, the plane wave alone cannot meet the image quality criteria of diagnostic medical ultrasound imaging. In order to improve the image quality, a Coherent Plane Wave Compounding (CPWC) method is proposed. The CPWC transmits plane waves at various angles and coherently combines the echo signals of each wave to produce an image with high resolution.

CPWC employs a uniform linear transducer array comprising N elements for both transmission and reception. Delayed echo signals received from array apertures are based on their arrival at an imaging pointIs delay compensated. The delay-compensated signals are coherently summed to produce an output of a single Plane Wave Imaging (PWI), specifically formulated as follows,in the formula (1), ω is a weighting factor of the length N,is defined as a vector containing the RF data recorded by the ith element, corresponding to each pixel in the imaging field。

Assume that the transmit beam is at M different anglesManipulated, the final output of CPWC imaging can be obtained by coherently adjusting the ratio of the sub-imaging points +.>Is obtained by summing the values obtained for each plane wave of (c) and can be defined as the formula +.>

In equation (1), it can be expressed as a convolution of the Tissue Reflectance Function (TRF) and the Point Spread Function (PSF) of the imaging system. In the model of the present invention, TRF represents pixel information of an original natural image, which takes the position and size of a pixel into consideration and replaces the position and scattering intensity of a scatterer, respectively. Furthermore, the point spread function in ultrasound imaging studies can be expressed as an impulse response function, the value of which is determined by the frequency spectrum of the excitation signal and the frequency response of the transducer. More specifically, the PSF depends on the geometry of the transducer and the excitation signal.

However, due to various assumptions of the ultrasound imaging system, the acquired signalsOnly an approximation of the real signal, more specifically the collected ultrasound signal, including the original signal and various noise and interference terms, can be represented, in which case the signal pattern (1) can be modified to the following formula,

in the formula (3), the amino acid sequence of the compound,component representing multiplicative noise, ++>Component representing additive noise, ++>For the original signal +.>To observe the signal.

The estimated ultrasound noise level may thus be expressed as the difference between the acquired signal and the real signal, in other words, it may also be expressed as the distance between the ultrasound image and the clean natural image in pixel values, and may be expressed by the following formula,

in the formula (4), N (x, Y) represents a noise image, Y (x, Y) represents a transmitted ultrasound image, and O (x, Y) represents a natural image.

In the invention, the data set generation module is responsible for generating simulated noise ultrasonic images with different noise levels by taking a large number of clean natural images as references and using corresponding simulation software and different imaging parameter sets, and dividing the simulated noise ultrasonic images into simulated noise ultrasonic image data sets for model training, verification and testing.

In one embodiment of the present invention, three public data sets are used, the Waterloo Exploration Dataset (WED), berkeley Segmentation Dataset (BSD 400), berkeley Segmentation Dataset (BSD 68), respectively. Wherein 4744 natural images are arranged in the open data set WED, and the sizes of the images are random; the public data set BSD400 contains 400 natural images, and the sizes of the images are 180 multiplied by 180; the public dataset BSD68 contains 68 natural images with the image size of 256×256.

To simulate an ultrasound image, the natural images of all three public data sets are first converted into gray-scale images. The gray scale map is then fed into Field II simulation software which simulates CPWC imaging principles and generates a simulated noise ultrasound image from the input natural image. In the process of generating analog noise ultrasound images by Field II simulation software, it is necessary to set plane wave imaging parameters. Referring to table 1, 3 different sets of plane wave imaging parameters are used herein. The values of the parameters fc, width, element kerf and pitch in each group of plane wave imaging parameters are different.

TABLE 1 plane wave imaging parameters used in FieldII simulation software

Referring to fig. 2, 3 sets of different plane wave imaging parameters in table 1 are used first, and the gray-scale public data sets WED, BSD400, and BSD68 are generated into 9 analog noise ultrasound image data sets by using Field II simulation software, then the 3 analog noise ultrasound image data sets corresponding to the same public data set are mixed into one set, and finally the 3 sets of training data sets including Dataset a, dataset B, and Dataset C are obtained. The size of the simulated noise ultrasound image in each set of training data is the same as the size of the original natural image. According to the image quantity of the public data sets WED, BSD400 and BSD68, the training data set Dataset A corresponding to the public data set WED is used as a training set of a denoising model, the training data set Dataset B corresponding to the public data set BSD400 is used as a verification set of the denoising model, and the training data set Dataset C corresponding to the public data set BSD68 is used as a verification set of the denoising model.

2. Preprocessing the simulated noise ultrasonic image;

in order to improve the efficiency of processing the simulated noise ultrasonic image x by the denoising model, the invention downsamples the simulated noise ultrasonic image x in the simulated noise ultrasonic image data set into 4 subgraphs, and the size of the original simulated noise ultrasonic image x is The size of the analog noise ultrasound image x after downsampling is +.>Where C denotes an image size Channel, W denotes an image size Width, and H denotes an image size height.

Then 2 characteristic parameters of the analog noise ultrasonic image x are selected, wherein the 2 characteristic parameters are respectively a parameter fc (ultrasonic image using frequency) and pitch (ultrasonic image using array element center-to-center distance) in plane wave imaging parameters used in Field II simulation software imaging. The parameters fc and pitch are both related to the noise level of the simulated noise ultrasound image, which varies with the parameters fc and pitch. The abstract formula of the denoising method of the present invention can thus be expressed as:

y = F(x,M,θ ₁ ,θ ₂ ) （5）

M = N(θ ₁ ,θ ₂ ) （6）

in the formulas (5) and (6), y represents the denoised ultrasonic image, x represents the simulated noise ultrasonic image, M represents the noise level, and θ ₁ Representing the parameters fc, θ ₂ Representing the parameter pitch.

Because the parameters fc and pitch of each simulated noise ultrasound image x are random and different in the process of generating the simulated noise ultrasound image x, in order to provide a more realistic match between the real medical ultrasound image and the simulated noise ultrasound image, the values of the parameters fc and pitch are randomly selected during the training of the model, and the actual values of the parameters fc and pitch are selected with 50% probability, namely the actual values of the parameters fc and pitch adopted by the Field II simulation software during the process of generating the simulated noise ultrasound image, and the set values of the parameters fc and pitch are selected with 50% probability, namely the assumed values-1 of the parameters fc and pitch set manually.

In order to make the parameters fc and pitch correspond to the noise level M one by one when inputting the denoising model, the invention uses the parameters fc and pitch with randomly selected values as the other 2 channels of the analog noise ultrasonic image respectively, and connects the parameters fc and pitch with the analog noise ultrasonic image x which is downsampled to form 4 subgraphs to form a new input image x 'of 6 channels, finally forming a new input image data set and inputting the new input image data set into the denoising model, wherein the size of the new input image x' is as follows。

3. Establishing a denoising model;

the invention adopts a multi-stage residual attu space pyramid pooling (MRASPP) module, a Nonlinear Mapping Convolutional Neural Network (NMCNNB) module, an adaptive noise level and a variable denoising intensity module to construct an initial denoising model.

The MRASPP module is based on residual error structure, abte robust space pyramid pooling (Atrous Spatial Pyramid Pooling, ASPP) and matrix addition connection establishment, and is responsible for extracting features of a new input image x' in the new input image dataset.

Referring to fig. 3, in one embodiment of the present invention, the MRASPP module includes three ASPP layers and a convolution layer with a convolution kernel of 3×3.

Each ASPP layer consists of 3 hole convolutions, i.e. dilation convolutions. If the convolution is allowed to expand the receptive field of extracting the image features and the resolution of the feature map is not allowed to drop too much, the two requirements are contradictory, as too much loss of resolution will lose much detailed information about the image boundaries. The need to acquire a larger receptive field requires the use of a larger convolution kernel or a larger step size for pooling, which is computationally intensive and which can lose resolution. The cavity convolution is used for solving the contradiction, so that the convolution can obtain a larger receptive field, and the resolution is not lost too much.

The 3 hole convolutions in the ASPP layer of each layer are respectively as follows:

the first cavity convolution has an expansion rate of 1, the receptive field of the convolution kernel is 3 multiplied by 3, and the first cavity convolution is the common convolution;

a second cavity convolution with an expansion ratio of 2 and a convolution kernel with a receptive field of 7 x 7;

and the third cavity convolution has an expansion rate of 4 and a receptive field of 15×15 of the convolution kernel.

Inputting the new input image x after pretreatment into a first ASPP layer Feature map to be extracted after 3 hole convolution operations in ASPP layer of first layerConnected together and then further extracted by a first convolution layer into the feature map 1, i.e. feature map +.>As shown in the following formula>。

The extracted feature map is then processedInputting into a second ASPP layer, connecting the feature images extracted after the convolution operation of 3 holes in the second ASPP layer, and further extracting the feature image 2, namely the feature image ∈2, through a second convolution layer with a convolution kernel of 3×3>。

And then extracting the characteristic diagramAnd inputting a third ASPP layer, connecting the feature graphs extracted after the 3 hole convolution operations in the third ASPP layer, and then further extracting the feature graph 3, namely the feature graph through the third ASPP layer with the convolution kernel of 3 multiplied by 3.

Feature mapAnd feature map->Extraction method and feature map of (2)>Likewise, a total of three ASPP layers are used herein. With the superposition of ASPP multi-layer convolution layers, the neural network may have gradient vanishing or gradient explosion during counter-propagation for the purpose ofThe present invention solves this problem by using a residual structure and a matrix addition connection to extract the three extracted feature map +.>Feature map->And feature map- >Is connected with the new output images x 'of 6 channels formed after downsampling to form an MRASPP output image y', as shown in the following formula,

in the formula (8), the amino acid sequence of the compound,representing MRASPP output image,/->Representing element-by-element additions.

The invention refers to the whole residual structure, the Atu space pyramid pooling and the matrix addition connection joint as multi-stage residual Atu space pyramid pooling, namely MRASPP. The invention extracts rich structural detail characteristics of the image through MRASPP, and retains original characteristics of the image through a residual structure.

The NMCNNB module is established based on a convolutional layer of CNN and is responsible for extracting deep features from an MRASPP output image y', and finally generates a feature image.

Referring to fig. 4, in one embodiment of the present invention, the NMCNNB module includes a CNN convolutional layer setup of 15 layers, including:

the layer 1 convolution layer consists of convolution (Conv) and correction linear units (Rectified Linear Units, reLU); the 2 nd to 14 th convolution layers consist of convolutions (Conv), batch normalization (Batch Normalization, BN) and modified linear units (Rectified Linear Units, reLU); layer 15 convolution layer is convolution (Conv); the convolution kernels of each CNN convolution layer are 3×3, filled with 0.

And sequentially processing the MRASPP output image y' through the 15 CNN convolution layers. The ReLU activation function increases the nonlinear relation among layers of the neural network, and the model after the ReLU is sparse can better mine relevant features and fit training data. The BN normalizes training data, can effectively accelerate the convergence speed of a model, prevent gradient explosion and gradient disappearance, and can effectively prevent the problem of overfitting of a denoising model. After convolution by the last CNN convolution layer, an NMCNNB output image, namely a characteristic image z', is obtained. The feature image z' is a deeper level feature.

The training data of each batch is the batch size, and then the size of the characteristic image z' isWhere 12 denotes a size Channel of an image, W denotes an image size Width (W), and H denotes an image size height (H).

The self-adaptive noise level and variable denoising strength module is established based on the bifurcated up-sampling module, the full-connection layer module and the joint loss function, so that the noise prediction and the self-adaptive parameter pc and pitch prediction can be simultaneously carried out.

Because the parameters fc and pitch are known actual values when the Field II simulation software generates a part of the simulated noise ultrasonic image, the invention inputs the actual values of the parameters fc and pitch and the simulated noise ultrasonic image into the denoising model and uses the actual values of the parameters fc and pitch to infer the denoised ultrasonic image when the denoising model training is carried out by using the step of the simulated noise ultrasonic image.

Referring to fig. 5, when preprocessing an analog noise ultrasonic image, parameters fc and pitch are selected to be true values, the up-sampling module directly up-samples the characteristic image z 'with known parameters fc and pitch true values, and the up-sampling is opposite to the down-sampling operation, so that the characteristic image z' is restored to a complete noise prediction image with 3 channels; the other 2 channels in the complete noise prediction image are characterized by parameters fc and pitcTrue value of h. The training data size of each batch is the batch size, and the size of the complete noise prediction image isWhere 3 denotes a size Channel of an image, W denotes an image size Width (W), and H denotes an image size height (H). By setting 3 different denoising intensities, the denoising intensities set by the invention are 100%, 50% and 20%, and a noise prediction image 1, a noise prediction image 2 and a noise prediction image 3 predicted by different denoising intensities are obtained. And finally, respectively subtracting the noise prediction image 1, the noise prediction image 2 and the noise prediction image 3 from the original analog ultrasonic image to obtain 3 ultrasonic images subjected to denoising with different denoising intensities.

Because of the existence of various ultrasonic instruments and various brands of ultrasonic instruments in the market, the parameter setting is not uniform, and the parameters fc and pitch values of a real medical ultrasonic image data set are difficult to obtain, when the noise removal model training is carried out by using the step-by-step simulation noise ultrasonic image, the parameters fc and pitch of the part of the simulation noise ultrasonic image are both-1, and simultaneously, the parameter fc and pitch reasoning and the estimated noise reasoning are carried out.

Referring to fig. 6, when preprocessing an analog noise ultrasound image, the parameters fc and pitch are set to be-1, and the parameters fc and pitch are predicted by the full connection layer module. The full-connection layer module of the invention uses a PyTorch self-adaptive-pooling layer (adaptive-pooling), a leveling layer (flat) and a three-layer full-connection layer (linear) to predict parameters fc and pitch with unknown true values, so that the denoising level is more consistent with the parameters for generating an actual ultrasonic image. Firstly, the characteristic image z' needs to be pooled, the adaptive-pooling adopted by the invention is the pooling kernel size of the known pooling layer, the filling value padding, the step size stride and the input tensor size input_size are the output tensor size output_size: output_size= (input_size+2 ∗ packing-kernel_size)/stride+1. The output image size output size may automatically generate an adaptively pooled output image of 12 channels matching the feature image z'. And then the value of the self-adaptive pooled output image of the 12 channels is leveled into one dimension by the flatten so as to carry out subsequent full-connection operation. And finally, performing full-connection operation on all image features of the flattened self-adaptive pooled output image by using three layers of linear to respectively obtain predicted values of parameters fc and pitch.

After obtaining the predicted values of the parameters fc and pitch, continuing to infer a denoising model, then inputting 4 sub-images obtained by downsampling the simulated noise ultrasonic image together with the predicted values of the parameters fc and pitch into the denoising model again, and upsampling the characteristic image z 'with the predicted values of the parameters fc and pitch by the upsampling module, wherein the upsampling is opposite to the downsampling operation, and reducing the characteristic image z' into a complete noise predicted image with 3 channels; the other 2 channels in the complete noise prediction image are characterized by the predicted values of parameters fc and pitch. The training data size of each batch is the batch size, and the size of the complete noise prediction image isWhere 3 denotes a size Channel of an image, W denotes an image size Width (W), and H denotes an image size height (H). By setting 3 different denoising intensities, the denoising intensities set by the invention are 100%, 50% and 20%, and a noise prediction image 1, a noise prediction image 2 and a noise prediction image 3 predicted by different denoising intensities are obtained. And finally, respectively subtracting the noise prediction image 1, the noise prediction image 2 and the noise prediction image 3 from the original analog ultrasonic image to obtain 3 ultrasonic images subjected to denoising with different denoising intensities.

Through the 2 times of reasoning, the invention can obtain the denoised ultrasonic image by the self-adaptive reasoning of the denoised model no matter whether the simulation noise ultrasonic image with the real values of the parameters fc and pitch are known or not.

4. Training, verifying and testing a denoising model;

and training the denoising model by using the training set of the simulated noise ultrasonic image dataset. During training, for a simulated noise ultrasonic image with parameters fc and pitch as real values randomly selected when a denoising model is input, the denoising model directly utilizes the real values of the parameters fc and pitch to obtain denoised images with different denoising intensities; for the analog noise ultrasonic image with parameters fc and pitch as set values-1 when the denoising model is input, the denoising model predicts predicted values of the parameters fc and pitch, then the sub-image after the analog noise ultrasonic image is downsampled is input into the denoising model again together with the predicted values of the parameters fc and pitch, and the denoising model obtains the image after denoising with different denoising intensities by using the predicted values of the parameters fc and pitch.

And verifying the denoising model by using a verification set of the simulated noise ultrasonic image data set. And testing the denoising model by using a test set of the simulated noise ultrasonic image data set.

In the training process, parameters of the denoising model are adjusted by utilizing a joint loss function of the denoising model, the denoising model is optimized, and finally, a trained controllable medical ultrasonic image denoising model is obtained.

5. Constructing a joint loss function;

conventionally, based on the learned image denoising with the loss per pixel between the denoised image and the real image as an optimization target, excellent quantitative scores can be obtained. Recent studies have shown, however, that relying solely on noisy pixels to minimize pixel errors can lead to loss of detail and smooth out the results. In the present invention, a weighted sum of loss functions is used, which includes denoising loss, parameter fc (ultrasound image frequency) and pitch (ultrasound image array element center-to-center spacing) loss.

The object of the denoising method of the present invention is to solve the problem of the following formula,

in the formula (9), the amino acid sequence of the compound,for different denoising intensity coefficients, +.>For the result of the mean square error of the noise level, the training set images of the invention are each +. >Random variation, by->Acquiring +.>，Representing a noise image and a clean image pair, the difference between the two being +.>。/>Controlling the balance between denoising intensity and image detail, +.>，/>Parameters fc and pitch respectively that affect ultrasound image generation,is a regularization term related to noise.

The main parameters fc and pitch affecting noise are selected, different noise levels can be simulated to train the denoising model, so that the denoising model can adaptively denoise, the denoising intensity can be flexibly adjusted, and the parameter setting of a real ultrasonic image generation scene is met. So that the expression (9) can be rewritten as the following expression,

equation (10) can be expressed as the following equation,

in the formula (11), the amino acid sequence of the compound,for denoising losses.

Since the present invention is to adaptively predict the parameters fc and pitch, the predicted values of fc and pitch are obtained by the following formula,

in the same way as in the formula (10), in the formula (12)Parameters fc and pitch respectively that affect ultrasound image generation,the MSE weights are adjusted for the parameters fc and pitch lost weight factors. The model bifurcation mode is adopted, the denoising of the parameters can be configured in a self-adaptive mode, and the parameters fc and pitch are randomly valued as the true value or the set value-1 generated by the simulated noise ultrasonic image during training. When it is desired to predict +. >At this time, the input parameters fc and pitch are set to-1. Reusing predicted parametersAnd the image can be accurately denoised by bringing a denoising model into the image after denoising.

The model is thus able to adapt to all situations of a simulated noise ultrasound image knowing the values of the parameters fc and pitch and a real image not knowing the values of the parameters fc and pitch.

Equation (12) can be expressed as follows,

in the formula (13), the amino acid sequence of the compound,for the parameters fc and pitch.

The present invention then uses joint loss to implement the loss function of the entire model,

in the formula (14), the amino acid sequence of the compound,in order to remove the noise loss,for the parameters fc and pitch loss, +.>As a result of the mean square error of the noise level,parameters fc and pitch, respectively, affecting ultrasound image generation +.>For a regularization term related to noise, +.>Weight factors lost for parameters fc and pitch,/->，/>Is a weighting factor for joint loss. According to the result of training, the present invention can be fine-tuned +.>，/>。

Referring to fig. 7, the invention provides a controllable medical ultrasonic image denoising method, which comprises the following steps: the method comprises the steps of obtaining a true medical ultrasonic image to be denoised, firstly, performing downsampling on the true medical ultrasonic image, wherein as a plurality of ultrasonic instruments and a plurality of brands of ultrasonic instruments exist in the market, parameter setting is not uniform, so that the parameter fc and pitch values of a data set of the true ultrasonic image are difficult to obtain, directly setting the parameter fc and pitch of the true ultrasonic image to be-1, then inputting a sub-image obtained by downsampling of the true medical ultrasonic image and the parameter fc and pitch with the value of the parameter fc and pitch of the parameter-1 into a controllable medical ultrasonic image denoising model obtained by the controllable medical ultrasonic image denoising model establishment method, firstly, evaluating predicted values of the parameter fc and pitch by the denoising model, then inputting the predicted values of the evaluated parameter fc and pitch into the denoising model together with the sub-image obtained by downsampling of the true medical ultrasonic image, and performing self-adaptive denoising on the true medical ultrasonic image by the denoising model according to the predicted values of the parameter fc and pitch, finally, outputting medical images with different denoising intensities, and effectively removing detail information at the same time.

Referring to fig. 8, the present invention provides a controllable medical ultrasound image denoising system, comprising: a data receiving unit, a data processing unit and a data output unit.

The data receiving unit is responsible for acquiring clean natural images and real ultrasonic images.

The data processing unit is responsible for executing the operation corresponding to the controllable medical ultrasonic image denoising model establishment method and/or the operation corresponding to the controllable medical ultrasonic image denoising method.

the preprocessing module is responsible for downsampling an analog noise ultrasonic image, randomly selecting values of parameters fc and pitch between a true value and a set value, and connecting the downsampled subgraph with the parameters fc and pitch to form a new input image data set; the method comprises the steps of performing downsampling on a real medical ultrasonic image, and connecting a downsampled subgraph with parameters fc and pitch with values of set value-1 to form a new input image;

The self-adaptive noise level and variable denoising intensity module is responsible for carrying out noise prediction on the second characteristic image with known parameters fc and pitch true values, obtaining noise prediction images predicted by adopting different denoising intensities by setting different denoising intensities, and finally obtaining images denoised by different denoising intensities by respectively subtracting the noise prediction images by using an analog ultrasonic image; on the other hand, the second characteristic image with unknown parameters fc and pitch true values is subjected to characteristic parameter prediction, and predicted values of the parameters fc and pitch are obtained; the self-adaptive noise level and variable denoising strength module is established based on a bifurcated up-sampling module and a full-connection module and comprises denoising loss, parameters fc and pitch (joint loss function after super-loss weighted sum), and the full-connection layer comprises a self-adaptive pooling layer, a leveling layer and a 3-layer full-connection layer which are sequentially connected.

The present invention also provides a computer apparatus comprising: the medical ultrasonic image denoising system comprises a processor, a memory, a communication interface and a communication bus, wherein the processor, the memory and the communication interface are communicated with each other through the communication bus, the memory is used for storing at least one executable instruction, and the executable instruction enables the processor to execute the operation corresponding to the controllable medical ultrasonic image denoising model building method and/or the operation corresponding to the controllable medical ultrasonic image denoising method.

The invention also provides a computer storage medium, at least one executable instruction is stored in the computer storage medium, and the executable instruction enables a processor to execute the operation corresponding to the controllable medical ultrasonic image denoising model establishment method and/or the operation corresponding to the controllable medical ultrasonic image denoising method.

The above description is only of the preferred embodiments of the present invention and is not intended to limit the present invention, but various modifications and variations can be made to the present invention by those skilled in the art. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present invention should be included in the protection scope of the present invention.

Claims

1. A controllable medical ultrasonic image denoising model establishment method is characterized by comprising the following steps:

step 1) generation of a simulated noise ultrasound image dataset:

step 2) preprocessing of the simulated noise ultrasonic image:

and 3) building a denoising model:

an initial denoising model is constructed by adopting a multistage residual Arteru space pyramid pooling module, a nonlinear mapping convolutional neural network module, a self-adaptive noise level and a variable denoising intensity module;

training of the denoising model:

2. The method for constructing a controllable medical ultrasound image denoising model according to claim 1, wherein in step 1), the specific method for generating the simulated noise ultrasound image dataset is as follows:

3. The method for constructing a controllable medical ultrasound image denoising model according to claim 1, wherein in step 2), the specific method for preprocessing the simulated noise ultrasound image is as follows:

downsampling the size of the simulated noise ultrasound image into 4 subgraphs; the original ultrasonic image has the size ofThe size of the ultrasound image after downsampling is +.>Wherein C represents an image size Channel, W represents an image size Width, and H represents an image size height;

randomly selecting values of 2 characteristic parameters between the true value and the set value, wherein the 2 characteristic parameters are 2 parameters fc and pitch related to noise level in imaging parameters of the simulated noise ultrasonic image respectively; the real value is the actual value of the parameters fc and pitch adopted in the generation process of the simulated noise ultrasonic image; the set value is assumed to be a numerical value-1 of manually set parameters fc and pitch;

4. The method for constructing a denoising model of a controllable medical ultrasound image according to claim 1, wherein in step 3), the specific method for generating the feature image is as follows:

the multistage residual Arteru space pyramid pooled output image is processed by a 1 st layer CNN convolution layer, wherein the 1 st layer CNN convolution layer consists of convolution and correction linear units;

the output of the CNN convolution layer 1 is sequentially processed by the CNN convolution layers 2 to 14, and the CNN convolution layers 2 to 14 consist of convolution, batch normalization and correction linear units;

the output of the 14 th CNN convolution layer is processed by the 15 th CNN convolution layer, and the 15 th CNN convolution layer is formed by convolution;

5. The method for constructing a controllable medical ultrasound image denoising model according to claim 1, wherein in step 4), the specific method for generating the image denoised by different denoising intensities is as follows:

6. The method for constructing a controllable medical ultrasound image denoising model according to claim 1, wherein in step 4), the specific method for predicting the predicted values of the parameters fc and pitch is as follows:

7. The method for constructing a controllable medical ultrasound image denoising model according to claim 1, wherein in step 4), the joint loss function adopts a weighted sum of loss functions, including denoising loss, parameters fc and pitch loss, and the formula is:

；

wherein,in order to remove the noise loss,for the parameters fc and pitch loss, +.>As a result of the mean square error of the noise level,parameters fc and pitch, respectively, affecting ultrasound image generation +.>For a regularization term related to noise, +.>Weight factors lost for parameters fc and pitch,/->，/>Is a weighting factor for joint loss.

8. A method for denoising a controllable medical ultrasound image, comprising:

acquiring a real medical ultrasonic image to be denoised, firstly downsampling the real medical ultrasonic image, then inputting a subgraph of the downsampling of the real medical ultrasonic image and 2 characteristic parameters taking the value as a set value into a controllable medical ultrasonic image denoising model obtained by the controllable medical ultrasonic image denoising model building method according to any one of claims 1-7, adaptively denoising the real medical ultrasonic image, flexibly adjusting denoising intensity, finally outputting medical ultrasonic images denoised by different denoising intensities, and retaining detail information of the images while effectively removing noise.

9. A controllable medical ultrasound image denoising system, comprising: a data receiving unit, a data processing unit and a data output unit;

the data processing unit is responsible for executing operations corresponding to the controllable medical ultrasonic image denoising model establishment method according to any one of claims 1 to 7 and/or operations corresponding to the controllable medical ultrasonic image denoising method according to claim 8;

the data processing unit comprises a data set generation module, a preprocessing module, a multi-stage residual Arteru space pyramid pooling module, a nonlinear mapping convolutional neural network module, a self-adaptive noise level and variable denoising intensity module; wherein,

The multistage residual attu space pyramid pooling module is responsible for extracting features of the preprocessed new input image and generating a multistage residual attu space pyramid pooling output image; the multistage residual attu space pyramid pooling module is established based on residual error structures, attu space pyramid pooling and matrix addition connection and comprises three layers of attu space pyramid pooling layers and three layers of convolution layers which are sequentially connected at intervals; each layer of the Arteru space pyramid pooling layer comprises a first cavity convolution with the expansion rate of 1 and the receptive field of a convolution kernel of 3 multiplied by 3, a second cavity convolution with the expansion rate of 2 and the receptive field of the convolution kernel of 7 multiplied by 7, and a third cavity convolution with the expansion rate of 4 and the receptive field of the convolution kernel of 15 multiplied by 15;

the nonlinear mapping convolutional neural network module is responsible for extracting deep features from the multi-level residual atlas space pyramid pooling output image and generating a feature image; the nonlinear mapping convolutional neural network module is built based on 15 layers of CNN convolutional layers, and comprises a 1 st layer of CNN convolutional layers consisting of convolutional and correction linear units, 2 nd to 14 th layers of CNN convolutional layers consisting of convolutional, batch normalization and correction linear units, and a 15 th layer of CNN convolutional layers consisting of convolutions, wherein the convolution kernel of each layer of CNN convolutional layer is 3 multiplied by 3, and 0 is used for filling;

10. A computer storage medium, wherein at least one executable instruction is stored in the computer storage medium, and the executable instruction causes a processor to perform operations corresponding to the controllable medical ultrasound image denoising model creation method according to any one of claims 1 to 7, and/or operations corresponding to the controllable medical ultrasound image denoising method according to claim 8.