CN115908204B

CN115908204B - Noise reduction processing method and device for medical image acquired by radiological imaging equipment

Info

Publication number: CN115908204B
Application number: CN202310138789.6A
Authority: CN
Inventors: 刘春燕; 解菁
Original assignee: Beijing Wemed Medical Equipment Co Ltd
Current assignee: Beijing Wemed Medical Equipment Co Ltd
Priority date: 2023-02-21
Filing date: 2023-02-21
Publication date: 2023-06-06
Anticipated expiration: 2043-02-21
Also published as: CN115908204A

Abstract

The application relates to a noise reduction processing method, device and medium for medical images acquired by a radiation imaging device. The noise reduction processing method comprises the steps of acquiring a first medical image containing noise, wherein the first medical image is acquired by using a radiation imaging device and is sensitive to the change of an image value; and obtaining a third medical image with noise suppression and fidelity of image values based on the first medical image and the trained noise reduction network. The noise reduction network is constructed based on a plurality of feature extraction sub-networks and an image restoration sub-network in stages, the feature extraction sub-network comprises an encoder-decoder sub-network and an image value adjusting part which are connected in series, and a first transverse connection edge structure with an inter-stage offset correction function is arranged between an encoder and a decoder in the encoder-decoder sub-network; the image restoration sub-network includes an original resolution sub-network and an image value adjusting section connected in series. The noise reduction processing method has better noise reduction effect, and can realize the fidelity of the image value.

Description

Noise reduction processing method and device for medical image acquired by radiological imaging equipment

Technical Field

The present invention relates to the field of medical image processing, and more particularly, to a noise reduction processing method, apparatus, and medium for medical images acquired with a radiological imaging device.

Background

Digital Subtraction Angiography (DSA) is a widely used radioscopic technique called "gold standard" for vascular imaging, and is widely used in the fields of percutaneous coronary intervention, nerve intervention, peripheral intervention, intravascular intervention, etc. of tumors. From a radiophysics perspective, the imaging quality of an image is positively correlated with the intensity of radiation, however, too high a radiation dose can be detrimental to the health of the physician and patient, and therefore the physician needs to gauge the relationship between radiation dose level and imaging quality. When the radiation dose level is lower, the DSA gray level image possibly contains more noise points, and the effect of the low radiation dose image can be improved to a certain extent by utilizing the existing image noise reduction algorithm, so that the method has important significance for reducing the radiation dose and protecting the health and safety of doctors and patients.

The image noise reduction algorithm of the prior art can be classified into a method based on manual design features and a method based on deep learning. In the method of manually designing features, BM3D (Block-Matching and 3D filtering) and NLM (Non-Local Means) achieve good noise reduction performance. However, the comparison of the effect of the method of manually designing the features depends on the quality of the design features, the calculation process is time-consuming, and the noise reduction effect for noise with complex distribution is poor. The noise reduction algorithm based on deep learning achieves the effect superior to the noise reduction algorithm of the manual design features in the aspects of feature extraction capability, noise reduction indexes, visual effect of images, noise reduction speed and the like. However, in both the method based on the manual design feature and the method based on the deep learning, when the image sharpness is improved by performing the noise reduction processing on the DSA image, for example, the noise reduced image may deviate from the original image in the overall brightness of the image, or the relative brightness between different positions of the image may generate undesired distortion, and such deviation and distortion in brightness may possibly cause trouble to the doctor in interpreting the medical image, and in severe cases, misdiagnosis and missed diagnosis may be caused by misdiagnosis.

Disclosure of Invention

The present application is provided to address the above-mentioned deficiencies in the prior art. There is a need for a noise reduction processing method, apparatus and medium for a medical image acquired by a radiological imaging device, which can improve the noise reduction effect of the medical image, and simultaneously avoid the offset or distortion of the image value caused by noise processing as much as possible, so as to realize the fidelity of the image value of the medical image after noise reduction, thereby providing a more reliable basis for the interpretation of the medical image after noise reduction for a doctor user, or providing a more accurate and reliable basis for the subsequent image processing such as blood vessel segmentation, and the like, thereby improving the accuracy of diagnosis based on the medical image, and reducing the occurrence of misdiagnosis, missed diagnosis, and the like.

According to a first aspect of the present application, there is provided a noise reduction processing method for a medical image acquired with a radiological imaging device, comprising acquiring a first medical image containing noise, each pixel in the first medical image having a corresponding image value, the first medical image being acquired with the radiological imaging device and being sensitive to a change in the image value; inputting the first medical image and a second medical image generated after the first medical image is subjected to first preprocessing into a trained noise reduction network, wherein the noise reduction network is constructed based on a plurality of feature extraction sub-networks and an image restoration sub-network in stages, the feature extraction sub-networks in each stage comprise encoder-decoder sub-networks and image value adjusting parts which are connected in series, a first transverse connection edge structure with an inter-stage offset correction function is arranged between an encoder and a decoder in the encoder-decoder sub-networks, and the second medical image corresponding to the stage is used as an input image; the image restoration sub-network is positioned above the plurality of feature extraction sub-networks, comprises an original resolution sub-network and an image value adjusting part which are connected in series, and takes the first medical image as an input image; and outputting a third medical image with noise suppression and image value fidelity by the image restoration sub-network.

According to a second aspect of the present application, there is provided a noise reduction processing apparatus for medical images acquired with a radiological imaging device, the noise reduction processing apparatus comprising an interface configured to contain noisy first medical images, each pixel in the first medical images having a corresponding image value, the first medical images being acquired with the radiological imaging device and being sensitive to variations in the image values, and a processor. The processor is configured to perform the steps of the method for noise reduction processing of medical images acquired with the radiological imaging device according to various embodiments of the present application.

According to a third aspect of the present application, there is provided a computer readable storage medium having instructions stored thereon, wherein the instructions, when executed by a processor, perform the steps of a method for noise reduction processing of medical images acquired with a radiological imaging device according to various embodiments of the present application.

According to the noise reduction processing method, the noise reduction processing device and the noise reduction processing medium for the medical image acquired by the radiation imaging equipment, when the image feature extraction is performed by utilizing the multi-stage feature extraction sub-network, offset correction among stages is performed at each stage, so that the image features among different layers can be subjected to cross-stage feature fusion in a manner of smaller offset, and therefore higher-quality image feature representation of each stage is obtained. On the basis, the image value adjusting part at each stage is used for further adjusting the high-quality feature images from the angle of image value correction, so that the feature images generated at each stage are improved in noise reduction effect, and meanwhile, the fidelity of the image values relative to the original medical images is realized, therefore, misjudgment and missed diagnosis caused by offset or distortion of the image values when a doctor uses the medical images after noise reduction to perform diagnosis can be avoided, and a more accurate and reliable basis can be provided for further processing of the medical images after noise reduction.

Drawings

In the drawings, which are not necessarily drawn to scale, like numerals may describe similar components in different views. The accompanying drawings illustrate various embodiments by way of example in general and not by way of limitation, and together with the description and claims serve to explain the disclosed embodiments. Wherever possible, the same reference numbers will be used throughout the drawings to refer to the same or like parts. Such embodiments are illustrative and not intended to be exhaustive or exclusive of the present apparatus or method.

Fig. 1 shows a flowchart of a noise reduction processing method for a medical image acquired with a radiological imaging device according to an embodiment of the present application.

Fig. 2 shows a schematic diagram of a structure of a noise reduction network for performing noise reduction processing on a first medical image according to an embodiment of the present application.

Fig. 3 shows a schematic view of a first transverse connecting edge structure according to an embodiment of the present application.

Fig. 4 is a schematic diagram showing the composition and principle of an image value adjusting section according to an embodiment of the present application.

Fig. 5 shows a schematic diagram of the structure of another noise reduction network for noise reduction processing of a first medical image according to an embodiment of the present application.

Fig. 6 shows a schematic diagram of the partial composition of a noise reduction processing device for medical images acquired with a radiological imaging device and its related components according to an embodiment of the present application.

Detailed Description

In order to better understand the technical solutions of the present application, the following detailed description of the present application is provided with reference to the accompanying drawings and the specific embodiments. Embodiments of the present application will now be described in further detail with reference to the accompanying drawings and specific examples, but are not intended to be limiting of the present application.

The terms "first," "second," and the like, as used herein, do not denote any order, quantity, or importance, but rather are used to distinguish one element from another. The word "comprising" or "comprises" and the like means that elements preceding the word encompass the elements recited after the word, and not exclude the possibility of also encompassing other elements.

The application proposes a noise reduction processing method for medical images acquired with a radiological imaging device. The noise reduction processing method will be described with reference to fig. 1 and 2. Fig. 1 shows a flowchart of a noise reduction processing method for a medical image acquired with a radiological imaging device according to an embodiment of the present application, and fig. 2 shows a schematic diagram of a structure of a noise reduction network for noise reduction processing of a first medical image according to an embodiment of the present application.

As shown in fig. 1, in step 101, first a first medical image I0 (as shown in fig. 2) containing noise may be acquired, each pixel in the first medical image I0 having a corresponding image value, the first medical image I0 being acquired with a radiological imaging device (not shown in fig. 2) and the first medical image I0 being sensitive to changes in image values.

In some embodiments, the radiological imaging device may be, for example, a Digital Subtraction Angiography (DSA) device, and accordingly, its acquired first medical image I0 may be a DSA image. Taking DSA gray images as an example, the image value of each pixel is the gray value of the pixel, but the DSA gray images are different from the common gray images, the pixel depth is very large, the pixel depth refers to the binary bit number occupied by storing the gray value of each pixel, and the pixel depth of the DSA gray images is generally more than 8, for example, 16 or more, which means that the resolution of the images on the image values is higher, the image values carry more fine image features, and a doctor can determine whether a specific part has a lesion, the development degree of the lesion and the like by utilizing the detailed information. It will thus be appreciated that such medical images are sensitive to changes in image values which tend to lead to the destruction of detailed information in the medical image, for example, many finer blood vessels in the DSA image, and deviations and distortions in image values are likely to lead to a change in their boundaries, and thus to a greater deviation in the determination of their thickness, which may endanger the health of the patient and even lead to life hazards when performing interventional procedures. Therefore, when the image contains noise, such as image blurring and ghosting, or has more noise points, due to the reasons of lower radiation dose of the DSA device, excessive patient thickness or movement of the device, the image needs to be subjected to noise reduction treatment, so that the image is clearer, and meanwhile, the image after the noise reduction treatment also restores the due image value of each pixel in the image as much as possible, thereby avoiding medical image distortion caused by integral deviation or relative distortion of the image value and even losing application value.

In further embodiments, the radiological imaging device may also be, for example, a Computed Radiography (CR), a Digital Radiography (DR), an electronic Computed Tomography (CT) device, etc., and the corresponding acquired first medical image I0 may also be a CR image, a DR image, a CT image, etc., where the image values of the images also carry a lot of image detail information, i.e. are sensitive to changes in the image values. The type of the first medical image is not specifically enumerated herein, and the noise reduction processing method according to the embodiment of the present application is applicable as long as the medical image to be subjected to the noise reduction processing has a characteristic sensitive to a change in an image value, and it is desirable that the image after the noise reduction processing retains an image value as close as possible to the original image and generates as little distortion of the image value as possible.

After the first medical image is acquired, in step 102, the first medical image I0, and a second medical image generated by first preprocessing the first medical image, such as I1 and I2, may be input into the trained noise reduction network 20, wherein the noise reduction network 20 is constructed based on the staged feature extraction sub-networks and the image restoration sub-network 23. Referring to fig. 2, the noise reduction network 20 may include, for example, a feature extraction sub-network 21, a feature extraction sub-network 22, and the like, and the actual number may be equal to or more than 2.

In some embodiments, the feature extraction sub-network of each stage includes an encoder-decoder sub-network and an image value adjustment section connected in series, a first lateral connection edge structure having an inter-stage offset correction function is included between the encoder and the decoder in the encoder-decoder sub-network, and a second medical image corresponding to the stage is taken as an input image. As shown in fig. 2, the feature extraction sub-network 21 includes an encoder-decoder sub-network 211 and an image value adjusting section 212 connected in series, and a first lateral connection edge structure 211c having an inter-stage offset correction function is included between an encoder 211a and a decoder 211b in the encoder-decoder sub-network 211. Similarly, the feature extraction sub-network 22 may include an encoder-decoder sub-network 221 and an image value adjustment section 222 connected in series, and a first lateral connection edge structure 221c having an inter-stage offset correction function is included between the encoder 221a and the decoder 221b in the encoder-decoder sub-network 221.

As shown in fig. 2, the image restoration subnetwork 23 directly takes the first medical image I0 as an input image, and the feature extraction subnetwork of each stage takes the first medical image after corresponding first preprocessing as an input image, where the first preprocessing may include, for example, performing segmentation on the first medical image I0 by a number corresponding to the stage where the first medical image I0 is located, and performing channel transformation on the segmented image. For example, the input image I1 of the feature extraction sub-network 21 of the first stage is a multi-channel image obtained by equally dividing the first medical image I0 by 4 and channel-transforming the channels corresponding to the 4 image blocks in the order from left to right and from top to bottom. Similarly, the input image I2 of the second stage feature extraction sub-network 22 is a multi-channel image obtained by equally dividing the first medical image I0 by 2 and performing channel conversion on channels corresponding to 2 image blocks in the order from top to bottom, and the specific manner of the first preprocessing is not limited in this application, as long as the receptive field of the image in the lower stage (the first stage is the lowest stage) is smaller than the receptive field of the image in the higher stage, so that the multi-stage feature extraction sub-network can extract multi-scale feature information of the image through feature extraction and image processing in different stages.

As shown in fig. 2, the image restoration subnetwork 23 is located above the plurality of feature extraction subnetworks, and includes a raw resolution subnetwork 231 and an image value adjusting section 232 connected in series, which takes the first medical image I0 as an input image, and finally outputs a third medical image I0' with noise suppressed and image value fidelity.

According to the noise reduction processing method, aiming at the characteristic that the medical image acquired by the radiation imaging equipment is sensitive to the change of the image value, when the image characteristic extraction is carried out by utilizing the multi-stage characteristic extraction sub-network, offset correction among stages is carried out in each stage, so that the image characteristics among different layers can be subjected to cross-stage characteristic fusion in a mode of smaller offset, and further higher-quality image characteristic representation of each stage is obtained. Based on inter-stage feature fusion offset correction, the image value adjusting part of each stage can perform further image value correction on the basis of a feature image with more accurate image value representation, the first transverse connecting edge structure with the inter-stage offset correction function and the image value adjusting part arranged at each stage work cooperatively in a serial connection mode, so that the feature image generated by each stage is improved in noise reduction effect, and fidelity of the image value relative to an original medical image is realized, thereby avoiding false diagnosis and missed diagnosis caused by false diagnosis and false treatment opportunity and other adverse consequences caused by offset or distortion of the image value when a doctor user uses the medical image after noise reduction to perform diagnosis, and providing more accurate and reliable basis for further processing of the medical image after noise reduction.

Taking the first lateral connecting edge structure 221c in fig. 2 as an example, it has a feature selecting portion 221c1 and a feature aligning portion 221c2, where the feature selecting portion 221c1 is used to select the useful feature in the encoder feature map 301 of the present stage, and the feature aligning portion 221c2 is used to first correct the offset of the upsampled feature map before the upsampled feature map corresponding to the feature map output by the decoder of the previous stage is cross-stage feature fused with the encoder feature map of the present stage, so as to align the upsampled feature map with the encoder feature map of the present stage as much as possible. The method comprises the following specific steps:

the feature selection unit 221c1 obtains a feature map 301w after the feature selection of the present stage based on the present-stage encoder feature map 301, wherein the present-stage encoder feature map 301 is output from the encoder 221a in the present-stage encoder-decoder sub-network 221. For example only, as shown in fig. 3, the global adaptive pooling may be performed on the encoder feature map 301 at the present stage to obtain a first feature selection weight 301a, then a channel compression convolution and a channel expansion convolution are sequentially performed to obtain a second feature selection weight 301b, and the encoder feature map 301 at the present stage and the second feature selection weight 301b are multiplied by channels to obtain a feature map 301w after feature selection at the present stage.

In some embodiments, the offset of the upsampled feature map 302u corresponding to the previous stage decoder feature map 302 may be predicted by the feature alignment section 221c2 based on the feature map 301w after the present stage feature selection. Specifically, as shown in fig. 3, a smaller-sized previous-stage decoder feature map 302 (which is output by the decoder 211b in the previous-stage encoder-decoder sub-network 211) may be first up-sampled, so as to obtain an up-sampled feature map 302u having the same size as the present-stage feature map, and the up-sampled feature map 302u and the feature map 301w after the present-stage feature selection are combined according to channels, a convolution kernel offset 304 is determined based on the combined feature map 303, and then a deformable convolution operation is performed on the up-sampled feature map 302u based on the convolution kernel offset 304, so as to obtain a feature map 305 after feature alignment, thereby implementing correction of inter-stage offset of the up-sampled feature map 302 u.

In some embodiments, further, the output feature map 306 of the first lateral connecting edge structure 221c of the present stage may be obtained based on the feature map 301 of the encoder of the present stage, the feature map 301w after feature selection of the present stage, and the feature map 305 after feature alignment. Specifically, the corresponding elements of the three feature maps may be simply added according to the channel, or different weights may be given to different feature maps and then added, and the specific algorithm is not limited in this application.

Further, the output feature map 306 of the first lateral connecting edge structure 221c of the present stage is decoded by the decoder 221b in the encoder-decoder sub-network 221 of the present stage to output a decoder feature map (not shown) of the present stage as an output of the encoder-decoder sub-network 221 of the present stage.

Taking the DSA image as an example, the blood vessels, bones, soft tissues and other information in the image are interwoven together, wherein the blood vessels are the information of the DSA image which is relatively "useful", such as the first transverse connecting edge structure 221c shown in fig. 3, the "useful information" in the feature map can be selected by the feature selection part 221c1 first, and the subsequent feature alignment processing is focused on the useful information. On the basis of feature selection, the feature alignment part 221c2 is utilized to correct the feature map offset generated by the reasons of feature extraction on different image sizes and receptive fields, up-sampling processing and the like, so that when cross-stage feature fusion is performed, the parts corresponding to useful information in the feature maps of different stages are aligned as much as possible, especially for the first medical image sensitive to image value change in the application, the above-mentioned feature selection and feature alignment processing performed at each stage can effectively improve the quality of image noise reduction and provide better fused image feature representation for the subsequent processing steps.

Fig. 4 shows a schematic diagram of an image value adjustment section according to an embodiment of the present application. As shown in fig. 4, in the image value adjustment unit 400, an image value gain adjustment matrix 403 and an image value offset adjustment matrix 404 in the present stage can be calculated based on the present-stage decoder feature map 307 and the first medical image I0. The method comprises the following steps:

first, the present stage decoder profile 307 may be sequentially subjected to global adaptive pooling, channel compression convolution (which may further include activation with, for example, a Swish activation function), channel expansion convolution (which may further include activation with, for example, a Sigmoid activation function) to obtain weights 401 of the respective channels of the present stage decoder profile 307, and multiplying the weights 401 with the present stage decoder profile 307 by corresponding channels to obtain a first profile 402 having weights at the present stage.

The first medical image I0, for example, represented in terms of image values of pixels, is then stitched to the first feature map 402 in accordance with the corresponding channels, and a residual convolution operation is performed using the original residual convolution block to obtain a second feature map (not shown) with image value transformation information at this stage. Then, a corresponding convolution operation (not shown) is performed on the second feature map to obtain an image value gain adjustment matrix 403 and an image value offset adjustment matrix 404 at this stage. It should be noted that the image value gain adjustment matrix 403 at this stage calculated according to the above steps characterizes a proportional adjustment coefficient that needs to be applied to the decoder feature map 307 at this stage in the case of adjusting the image value to a desired value, and the image value offset adjustment matrix 404 is an adjustment amount that needs to be additively offset to the image value based on the proportional adjustment.

Next, the image value gain adjustment matrix 403 and the image value offset adjustment matrix 404 are used to perform image value adjustment on the decoder feature map 307 in this stage, so as to generate an image value adjusted feature map in this stage. Specifically, the present-stage decoder feature map 307 may be multiplied by the image-value gain adjustment matrix 403 at the corresponding position (corresponding to the dot product operation between the matrix of the present-stage decoder feature map 307 and the image-value gain adjustment matrix 403), and then added to the image-value offset adjustment matrix 404 at the corresponding position to obtain a feature map having an image-value representation, which is the output of the image-value adjustment unit 400, which is the feature map after the image-value adjustment at the present stage.

In the noise reduction network 500 shown in fig. 5, it can be seen that in each of the feature extraction sub-network and the image restoration sub-network, the convolution-plus-channel attention structure block may also be used to perform convolution-plus-channel attention processing on the input image of the corresponding stage, so as to obtain the shallow features of the input image of the corresponding stage.

In some embodiments, for each feature extraction sub-network, a supervised attention section is further connected after the image value adjustment section, the supervised attention section generating an image noise representation of the present stage and a restored image of the present stage based on the image value adjusted feature map of the present stage (i.e. the output of the image value adjustment section of the present stage) and the input image of the present stage, and taking the image noise representation of the present stage as input to the feature extraction sub-network of the next stage, or as input to the image restoration sub-network. Corresponding to fig. 5, for example, in the feature extraction sub-network of the first stage, the image value adjustment section is followed by the supervised attention section 511, and the supervised attention section 511 generates the image noise representation 512 of the present stage and the restored image I1' of the present stage based on the output of the image value adjustment section of the present stage and the input image I1 of the present stage, wherein the image noise representation 512 of the present stage is spliced with the shallow features of the next stage (i.e., the second stage) extracted via the convolution plus channel attention structure block and serves as the input of the sub-network of the following encoder-decoder. Similarly, in the feature extraction sub-network of the first stage, the image value adjustment section is followed by the supervised attention section 521, and the supervised attention section 521 generates the image noise representation 522 of the present stage and the restored image I2' of the present stage based on the output of the image value adjustment section of the present stage and the input image I2 of the present stage, unlike the first stage, since the present stage is the image restoration stage above, the image noise representation 522 is input into the original resolution sub-network after being spliced with the shallow features extracted via the convolution plus channel attention structure block of the image restoration stage.

As shown in fig. 5, in the last stage, that is, the image restoration stage, the image value adjusting part in the image restoration sub-network may be further connected with a feature extraction and channel adjustment sub-network 531, and the image restoration process is specifically as follows:

the first medical image I0 and the previous stage image noise representation 522 are utilized by the original resolution subnetwork in the image restoration subnetwork to generate a second feature map having original resolution and fusing the features of the respective stage images through a plurality of original residual convolution structure blocks ORB1, …, ORBn, etc. in the original resolution subnetwork, wherein "…" refers to each ORB block, such as ORB2, ORB3, etc., in the middle of ORB1 to ORBn, n is the total number of original residual convolution structure blocks in series, which is a natural number that can be customized. It is noted that unlike previous feature extraction sub-networks, the last stage of image restoration sub-network does not employ an encoder-decoder sub-network that would downsample-upsample the image, but instead employs an original resolution sub-network that maintains the original resolution of the image, so that the fine texture and useful detail information that the medical image of the present application expects to preserve can be preserved in the final restored image.

Then, image value adjustment is performed again on the second feature map based on the second feature map having the original resolution and the first medical image by an image value adjustment section in the image restoration sub-network to generate a second feature map having the original resolution and the image value adjusted.

Finally, the feature extraction and channel adjustment sub-network 531 obtains a third medical image with noise suppression of the same size as the first medical image and fidelity of the image value based on the second feature map with original resolution and adjusted image value and the first medical image.

In the noise reduction network structure and the processing steps shown in fig. 5, it can be seen that the first medical image with the original resolution is used not only for the image restoration sub-network of the image restoration stage but also for the image value adjusting part of each stage, thereby, the accumulation of the integral deviation or distortion of the image value can be avoided, the deviation and distortion of the image value caused by the feature extraction of the present stage or the cross-stage feature fusion can be corrected in time and with a common reference, and the authenticity of the image value of the image after noise reduction is effectively ensured.

Furthermore, it will be appreciated that in some embodiments, the first medical image is an image set including a plurality of images, for example, an original medical image set including a plurality of batches and a plurality of channels may be acquired, and in order to improve the noise reduction effect, data enhancement processing is typically performed on the acquired medical image set in advance, and a medical image set after the data enhancement processing is performed on the original medical image set is taken as the first medical image. In some embodiments, the data enhancement processing may include at least one of random rotation, random size expansion, random clipping, random flipping, or a combination thereof, and training the noise reduction network using the medical image set after data enhancement may expand the training data amount, and may further improve model robustness of the noise reduction network.

Before the noise reduction network according to the embodiment of the present application is used to perform noise reduction processing on a medical image acquired by using a radiological imaging device, the noise reduction network needs to be trained first, for example, the noise addition processing may be performed on a low-noise medical image acquired by using the radiological imaging device to generate a high-noise medical image, and a specific noise pattern may be, for example, one or more of gaussian noise, poisson noise, multiplicative noise, and pretzel noise, which is not limited in this application. And then, the low-noise medical image and the high-noise medical image set can be synchronously subjected to data enhancement processing, the high-noise medical image set after data enhancement is used as a training sample image set, the corresponding low-noise medical image set after data enhancement is used as an image truth value of a training sample, and the image truth value of each stage in the noise reduction network can be obtained by carrying out first preprocessing including blocking, which is the same as that of the input image of each stage, on the basis of the truth value of the training sample. And (3) utilizing the paired training sample image sets and corresponding image truth values, carrying out joint training on the noise reduction network based on a combined loss function combining the Charbonnier loss, the multi-scale structure similarity (MS-SSIM) loss and the perception loss, and selecting a network model with the best performance in the training process, thereby obtaining the trained noise reduction network.

Specifically, the charbonier loss (charbonier loss) mainly focuses on image details, such as the similarity between image pixels, and because the noise reduction network according to the application is of a staged structure, multi-scale detail information of images is contained in the noise reduction network, and inter-stage offset is corrected when cross-stage features are fused, and image reconstruction is supervised by using a first medical image with original resolution in an image recovery stage, on the basis, the introduction of the charbonier loss can reduce the difference between the noise-reduced images and target images (such as true images), and improve the peak signal-to-noise ratio (PSNR) of the noise-reduced images and the target images.

MS-SSIM loss (Multi-Scale Structural Similarity Loss, multi-scale structure similarity loss) mainly focuses on the degree of similarity of brightness, contrast and structure of an original image and a target image, and can help the noise-reduced image to have smaller overall deviation in terms of image value.

In addition to the two losses described above, the embodiments of the present application introduce a third loss, namely a perceived loss that is used to measure perceived similarity. As described above, since the image after noise reduction may be provided to a doctor for interpretation, how to reduce the noise of the image from the perspective of human eyes, whether the quality is obviously improved is also a problem that the noise reduction network of the present application needs to pay attention to. The reference image such as the truth image and the image to be noise-reduced can be respectively input into a pre-trained classified neural network such as a VGG network and the like, and the noise reduction effect of the noise reduction network can be optimized from the perspective of perception based on the weighted sum of differences between the output feature images of each layer of the VGG network, wherein the perception is not limited to human eye perception, the classified neural network can be pre-trained according to other mode division modes of the medical image, and the trained classified neural network is used for calculating and optimizing the perception loss.

Preferably, the weight of the Charbonnier loss can be set to be the maximum in the combined loss function, so that the noise reduction network can firstly emphasize improvement of the overall accuracy and generalization performance of the noise-reduced image, and on the basis, the MS-SSIM loss and the perception loss are utilized to further improve the fidelity of the image value of the noise-reduced image, so that the noise-reduced medical image has higher application value no matter used for medical image processing such as interpretation by doctors or further blood vessel segmentation or used for related occasions such as intra-operative navigation.

There is also provided, in accordance with an embodiment of the present application, a noise reduction processing apparatus for medical images acquired with a radiological imaging device. Fig. 6 shows a schematic diagram of the partial composition of a noise reduction processing device for medical images acquired with a radiological imaging device and its related components according to an embodiment of the present application. Fig. 6 shows a noise reduction processing apparatus 600 for medical images acquired with a radiological imaging device, comprising at least an interface 601 and a processor 602. Wherein the interface 601 may be configured to acquire a first medical image comprising noise, each pixel in the first medical image having a corresponding image value, the first medical image being acquired with a radiological imaging device and being sensitive to changes in the image values. In particular, the first medical image may be transmitted to the noise reduction processing apparatus 600 via the interface 601 for noise reduction processing between the acquisition by the radiological imaging device, or the first medical image including noise may be acquired from the memory 800 or the medical image database 900 in which the first medical image is stored via the interface 601.

After the first medical image is acquired by the interface 601, it may be directly transferred to the processor 602, or in case the noise reduction processing device comprises a memory (not shown), it may be stored in the memory first, and when the noise reduction processing is to be performed, the first medical image is read out. The processor 602 may be configured to perform the steps of the method for noise reduction processing of medical images acquired with a radiological imaging device according to various embodiments of the present application. In a plurality of verification experiments for performing noise reduction processing on a DSA image or the like, which are performed by the noise reduction processing apparatus 600, one or more criteria such as L1 distance, SSIM, PSNR, etc. are used to evaluate the noise reduced image, and the evaluation results indicate that the fidelity of the image value is improved to different degrees. According to the angle of attention of the user, the further application of the image after noise reduction and the like, different measuring methods are adopted, the image value fidelity degree of the image after noise reduction is measured and evaluated from different dimensions, and the specific method is not limited in the application.

There is also provided, in accordance with an embodiment of the present application, a non-transitory computer readable medium having instructions stored thereon, wherein the instructions, when executed by a processor, perform the steps of a method for noise reduction processing of medical images acquired with a radiological imaging device according to various embodiments of the present application. In some embodiments, a computer-readable medium may include volatile or nonvolatile, magnetic, semiconductor, tape, optical, removable, non-removable, or other types of computer-readable media or computer-readable storage devices. For example, the computer-readable medium may be a storage device or memory module having computer instructions stored thereon. In some embodiments, the computer readable medium may be a disk or flash drive having computer instructions stored thereon.

Furthermore, although exemplary embodiments have been described herein, the scope thereof includes any and all embodiments having equivalent elements, modifications, omissions, combinations (e.g., of the various embodiments across), adaptations or alterations as pertains to the present application. Elements in the claims are to be construed broadly based on the language employed in the claims and are not limited to examples described in the present specification or during the practice of the present application, which examples are to be construed as non-exclusive. It is intended, therefore, that the specification and examples be considered as exemplary only, with a true scope and spirit being indicated by the following claims and their full scope of equivalents.

The above description is intended to be illustrative and not restrictive. For example, the above-described examples (or one or more aspects thereof) may be used in combination with each other. For example, other embodiments may be used by those of ordinary skill in the art upon reading the above description. In addition, in the above detailed description, various features may be grouped together to streamline the application. This is not to be interpreted as an intention that the disclosed features not being claimed are essential to any claim. Rather, the subject matter of the present application is capable of less than all of the features of a particular disclosed embodiment. Thus, the following claims are hereby incorporated into the detailed description as examples or embodiments, with each claim standing on its own as a separate embodiment, and it is contemplated that these embodiments may be combined with one another in various combinations or permutations. The scope of the invention should be determined with reference to the appended claims, along with the full scope of equivalents to which such claims are entitled.

The above embodiments are only exemplary embodiments of the present application and are not intended to limit the present invention, the scope of which is defined by the claims. Various modifications and equivalent arrangements may be made to the present invention by those skilled in the art, which modifications and equivalents are also considered to be within the scope of the present invention.

Claims

1. A noise reduction processing method for a medical image acquired with a radiological imaging device, comprising:

acquiring a first medical image containing noise, each pixel in the first medical image having a corresponding image value, the first medical image acquired with a radiological imaging device and being sensitive to changes in the image values;

inputting the first medical image and a second medical image generated by performing first preprocessing on the first medical image into a trained noise reduction network, wherein the noise reduction network is constructed based on a plurality of feature extraction sub-networks and an image restoration sub-network in stages, the first preprocessing comprises performing segmentation on the first medical image by a quantity corresponding to the stages, and performing channel transformation on the segmented image,

The characteristic extraction sub-network of each stage comprises an encoder-decoder sub-network and an image value adjusting part which are connected in series, a first transverse connection edge structure is arranged between the encoder and the decoder in the encoder-decoder sub-network, the first transverse connection edge structure takes a second medical image corresponding to the stage as an input image, and the characteristic selecting part and the characteristic aligning part are used for correcting the offset between the stages; the image restoration sub-network is positioned above the plurality of feature extraction sub-networks and comprises an original resolution sub-network, an image value adjusting part and a feature extraction and channel adjustment sub-network which are connected in series, wherein the original resolution sub-network, the image value adjusting part and the feature extraction and channel adjustment sub-network take the first medical image as an input image and output a third medical image which has noise suppression of the same size as the first medical image and has a fidelity image value.

2. The noise reduction processing method according to claim 1, characterized in that the processing method further comprises:

obtaining, by the feature selection unit, a feature map after feature selection of the present stage based on the feature map of the present stage encoder, wherein the feature map of the present stage encoder is output by an encoder in the present stage encoder-decoder sub-network;

Predicting, by the feature alignment section, an offset of an up-sampled feature map corresponding to a previous stage decoder feature map based on the feature map after the feature selection of the present stage, and performing inter-stage offset correction on the up-sampled feature map by using deformable convolution to obtain a feature map after feature alignment, wherein the previous stage decoder feature map is output by a decoder in a previous stage encoder-decoder sub-network;

based on the characteristic diagram of the encoder at the stage, the characteristic diagram after characteristic selection at the stage and the characteristic diagram after characteristic alignment, an output characteristic diagram of a first transverse connecting edge structure at the stage is obtained;

the decoder in the stage encoder-decoder sub-network decodes the output characteristic diagram of the first transverse connection edge structure of the stage, and outputs the stage decoder characteristic diagram as the output of the stage encoder-decoder sub-network.

3. The noise reduction processing method according to claim 2, characterized in that the processing method further comprises, by the image value adjusting section of each stage:

calculating to obtain an image value gain adjustment matrix and an image value offset adjustment matrix of the stage based on the decoder feature map of the stage and the first medical image;

And performing image value adjustment on the decoder characteristic diagram at the stage by using the image value gain adjustment matrix and the image value offset adjustment matrix to generate an image value adjusted characteristic diagram at the stage.

4. A noise reduction method according to claim 3, wherein calculating an image value offset adjustment matrix and an image value gain adjustment matrix of the present stage based on the present stage decoder feature map and the first medical image specifically includes:

sequentially performing global self-adaptive pooling, channel compression convolution and channel expansion convolution on the stage decoder feature map to obtain the weight of each channel of the stage decoder feature map, and multiplying the weight with the stage decoder feature map according to the corresponding channel to obtain a first feature map with weight in the stage;

splicing and residual convolution operations are carried out on the first medical image and the first feature image according to the corresponding channels so as to obtain a second feature image with image value transformation information at the stage;

based on the second feature map, calculating to obtain an image value offset adjustment matrix and an image value gain adjustment matrix at the stage by utilizing convolution operation.

5. A noise reduction processing method according to claim 3, wherein performing image value adjustment on the decoder feature map of the present stage by using the image value offset adjustment matrix and the image value gain adjustment matrix to generate the image value adjusted feature map of the present stage specifically comprises:

Multiplying the decoder characteristic diagram of the stage by the image value gain adjustment matrix according to the corresponding position, and adding the decoder characteristic diagram of the stage by the image value offset adjustment matrix according to the corresponding position to generate the characteristic diagram after the image value adjustment of the stage.

6. A noise reduction processing method according to claim 3, wherein for each of the feature extraction sub-networks, after the image value adjustment section, a supervised attention section is further connected, the supervised attention section generating an image noise representation of the present stage and a restored image of the present stage based on the feature map adjusted by the image value of the present stage and the input image of the present stage, and taking the image noise representation of the present stage as an input to a feature extraction sub-network of a next stage or as an input to the image restoration sub-network.

7. The noise reduction processing method according to claim 6, characterized in that the processing method further comprises:

generating, by the original resolution sub-network, a second feature map having the original resolution and incorporating image features of each stage using the first medical image and an image noise representation of a previous stage;

performing, by an image value adjustment portion in the image restoration subnetwork, image value adjustment of the second feature map based on the second feature map and the first medical image to generate an image value adjusted second feature map;

And obtaining a third medical image which has noise suppression of the same size as the first medical image and has fidelity of image values by the characteristic extraction and channel adjustment sub-network based on the second characteristic image and the first medical image after image value adjustment.

8. The noise reduction processing method according to any one of claims 1 to 5, characterized in that the processing method further comprises:

acquiring an original medical image set containing multiple batches and multiple channels;

and taking the medical image set after the data enhancement processing of the original medical image set as the first medical image.

9. The noise reduction processing method according to claim 8, wherein the data enhancement processing includes at least one of random rotation, random size expansion, random clipping, random flipping, or a combination thereof.

10. The noise reduction processing method according to any one of claims 1 to 5, further comprising, in each of the feature extraction sub-network and the image restoration sub-network, performing convolution plus channel attention processing on the input image of the corresponding stage.

11. The noise reduction processing method according to any one of claims 1 to 5, wherein the noise reduction network performs joint training based on a combined loss function combining a Charbonnier loss, an MS-SSIM loss, and a perceptual loss, using a training image set generated by performing data enhancement processing on a medical image set containing noise and image truth values corresponding to respective stages in the noise reduction network.

12. The noise reduction processing method according to any one of claims 1 to 5, wherein the radiological imaging device is a digital subtraction angiography DSA device, and the first medical image is a DSA image.

13. A noise reduction processing apparatus for a medical image acquired with a radiological imaging device, the noise reduction processing apparatus comprising:

an interface configured to acquire a first medical image containing noise, each pixel in the first medical image having a corresponding image value, the first medical image acquired with a radiological imaging device and being sensitive to changes in the image values; and

a processor configured to perform the noise reduction processing method for medical images acquired with a radiological imaging device as claimed in any one of claims 1 to 12.

14. A non-transitory computer readable medium having instructions stored thereon, wherein the instructions, when executed by a processor, perform the steps of the noise reduction processing method for medical images acquired with a radiological imaging device as claimed in any one of claims 1-12.