WO2023202265A1

WO2023202265A1 - Image processing method and apparatus for artifact removal, and device, product and medium

Info

Publication number: WO2023202265A1
Application number: PCT/CN2023/081283
Authority: WO
Inventors: 王红; 李悦翔; 郑冶枫; 孟德宇
Original assignee: 腾讯科技（深圳）有限公司
Priority date: 2022-04-19
Filing date: 2023-03-14
Publication date: 2023-10-26
Also published as: CN115131452A

Abstract

Provided in the embodiments of the present disclosure are an image processing method and apparatus for artifact removal, and a computer program product and a storage medium. The image processing method in the present disclosure comprises: establishing a training data set used for training a neural network; performing artifact removal processing on an artifact-containing image by using an adaptive convolutional dictionary network, so as to obtain a processed image; and on the basis of an artifact-free image, the processed image, and an objective function which has been subjected to image mask processing, performing iterative training on the adaptive convolutional dictionary network, so as to optimize network parameters of the adaptive convolutional dictionary network. By means of the image processing method in the present disclosure, artifacts in an image can be simply and effectively removed to obtain a clearer image which is not interfered with by the artifacts.

Description

Image processing methods, devices, equipment, products and media for artifact removal

This application claims priority to the Chinese patent application filed with the China Patent Office on April 19, 2022, with application number 202210408968.2 and the application title "Image processing method and device for artifact removal", the entire content of which is incorporated by reference. in this application.

Technical field

The present disclosure relates to the field of artificial intelligence, and more particularly, to image processing for artifact removal.

Background technique

As one of the most important sources for humans to obtain information, images occupy a huge proportion in various information libraries. With the rapid development of computer technology, image processing has been widely used in all aspects of human social life, such as industrial testing, medicine, intelligent robots, etc. Images are often used in various fields to describe and express the characteristics and logical relationships of things due to their vividness and intuitiveness. They have a wide range of applications. Therefore, the development of image processing technology and information processing in various fields are extremely important.

Image processing technology is a technology that uses computers to process image information. It mainly includes image digitization, image enhancement and restoration, image data encoding, image segmentation and image recognition, etc. Among them, the purpose of image restoration technology is to restore the degraded image to its original true appearance as much as possible. For example, it is used in the field of removing artifacts in images, such as removing noise in images, removing raindrops in rainy images taken on rainy days, removing metal artifacts in CT images, etc. However, current image processing methods face technical bottlenecks such as complex mathematical models, limited application scope, and difficulty in obtaining some data when removing artifacts.

Therefore, there is a need for an image processing method that can be widely used and effectively identify artifacts in images and remove them from artifact-bearing images, so as to help people obtain clearer images without interference from artifacts.

Contents of the invention

In order to solve the above problems, the present disclosure provides an image processing method, device, computer equipment, computer program product and storage medium for artifact removal. The image processing method for artifact removal provided by the present disclosure includes: establishing a training data set for training a neural network, wherein the training data set includes multiple groups of image samples, and each group of image samples includes images with artifacts. (Y) and its corresponding artifact-free image (X) and image mask (I); for any group of image samples among the multiple groups of image samples, use an adaptive convolution dictionary network to The artifact-bearing image (Y) is subjected to artifact removal processing to obtain a processed image, based on the artifact-free image (X), the processed image, and the objective function processed by the image mask (I) The adaptive convolutional dictionary network is iteratively trained to optimize network parameters of the adaptive convolutional dictionary network, wherein the adaptive convolutional dictionary network includes a basic artifact dictionary, and the basic artifact dictionary is The convolution dictionary does not change with the sample and includes an artifact convolution kernel, and the adaptation for the image sample is determined by a plurality of artifact convolution kernels in the basic artifact dictionary and a weighting coefficient that changes with the sample. A convolution kernel, wherein the artifact image in the artifact-bearing image is determined by convolution of the adaptive convolution kernel with image features of the artifact-bearing image, and from the artifact-bearing image The artifact image is removed to obtain the processed image.

The image processing method of the present disclosure can simply and effectively remove artifacts in images to obtain clearer images free from artifact interference.

Embodiments of the present disclosure also provide an image processing method for artifact removal, including: obtaining an input image to be processed; using an adaptive convolution dictionary network to process the input image to obtain artifact removal results. After processing Image, wherein the adaptive convolutional dictionary network is trained based on the artifact database (D) and includes a T-level network, wherein the level 1 network outputs level 1 image features and level 1 based on the input image Artifact removal image; the t-th level network outputs the t-level image features and the t-th level artifact based at least in part on the t-1th level image features and the t-1th level artifact removal image output by the t-1th level network. Remove the image, where t is greater than 1 and less than or equal to T; the T-th level network removes the image based on the T-1-th level image features and T-1-th level artifacts output by the T-1-th level network, and outputs the T-th level artifact removal The image is the processed image after artifact removal.

An embodiment of the present disclosure provides an image processing device for artifact removal, including: a training data set establishment module configured to: establish a training data set for training a neural network, wherein the training data set includes Multiple sets of image samples, each set of image samples includes an image with artifacts (Y) and its corresponding image without artifacts (X) and image mask (I); the adaptive convolutional dictionary network is configured as: Any group of image samples among the plurality of groups of image samples performs artifact removal processing on the artifact-containing image (Y) to obtain a processed image; the training module is configured to: based on the artifact-free image The adaptive convolutional dictionary network is iteratively trained using the shadow image (X), the processed image, and the objective function processed by the image mask (I) to optimize the performance of the adaptive convolutional dictionary network. Network parameters; wherein, the adaptive convolution dictionary network includes a basic artifact dictionary, the basic artifact dictionary is a convolution dictionary that does not change with samples and includes an artifact convolution kernel, and through the basic artifact dictionary Multiple artifact convolution kernels and weighting coefficients that change with the sample are used to determine the adaptive convolution kernel for the image sample, wherein the adaptive convolution kernel and the image features of the image with artifacts are convolution to determine the artifact image in the image with artifacts, and remove the artifact image from the image with artifacts to obtain the processed image.

An embodiment of the present disclosure provides an image processing device for artifact removal, including: an image acquisition module configured to: acquire an input image to be processed; an image processing module configured to: utilize an adaptive convolution dictionary A network that processes the input image to obtain a processed image with artifacts removed; wherein the adaptive convolutional dictionary network is trained based on the artifact database (D) and includes a T-level network, wherein the The level 1 network outputs the level 1 image features and the level 1 artifact removal image based on the input image; the level t network is based at least in part on the level t-1 image features and the t-1 level output by the level t-1 network. The level t artifact removal image outputs the t level image features and the t level artifact removal image, where t is greater than 1 and less than or equal to T; the T level network is based on the T-1 level image features output by the T-1 level network. and the T-1-th level artifact removal image, and the T-th level artifact removal image is output as the processed image after artifact removal.

Embodiments of the present disclosure provide a computer program product. The computer program product includes a computer program. When the computer program is run by a processor, the computer program executes the above method.

Embodiments of the present disclosure provide a computer-readable storage medium on which a computer program is stored. When the computer program is executed by a processor, the above method is performed.

An embodiment of the present disclosure provides a computer device, including:

Processors, communication interfaces, memory and communication buses;

Wherein, the processor, the communication interface and the memory complete communication with each other through the communication bus; the communication interface is an interface of a communication module;

The memory is used to store a computer program and transmit the computer program to the processor; the processor is used to call the computer program in the memory to execute the above method.

The image processing method of the present disclosure can perform artifact removal processing only based on the image to be processed without additional Get the chord diagram of the image. The adaptive convolutional dictionary network adopted in this disclosure makes full use of the prior structure of the artifact image, resulting in better artifact removal and stronger model generalization. In addition, the mathematical model used in the image processing method of the present disclosure has clear physical meaning and strong interpretability, and the physical meaning of each network module is clearer, which facilitates understanding and application by those skilled in the art. The image processing method of the present disclosure can simply and effectively remove artifacts in images to obtain clearer images free from artifact interference.

Description of the drawings

Here, in the attached picture:

1A-1B are schematic diagrams respectively showing images with artifacts and images without artifacts according to embodiments of the present disclosure;

Figures 2A-2B are schematic diagrams showing an image processing method based on chord diagrams;

3A-3B are schematic flowcharts illustrating an image processing method for artifact removal according to an embodiment of the present disclosure;

FIG. 4 is an exemplary schematic diagram illustrating an image processing model based on a weighted adaptive convolution dictionary according to an embodiment of the present disclosure;

5A-5B are schematic diagrams illustrating the iterative update process of each level of the network in the adaptive convolutional dictionary network according to an embodiment of the present disclosure;

6 is a schematic flowchart illustrating an image processing process for artifact removal according to an embodiment of the present disclosure;

7A-7B are schematic diagrams illustrating the composition of an image processing device for artifact removal according to an embodiment of the present disclosure;

8 is an architecture diagram illustrating a computing device according to an embodiment of the present disclosure; and

9 is a schematic diagram illustrating a storage medium according to an embodiment of the present disclosure.

Detailed ways

In order to make the objects, technical solutions and advantages of the present disclosure more apparent, example embodiments according to the present disclosure will be described in detail below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present disclosure, rather than all embodiments of the present disclosure, and it should be understood that the present disclosure is not limited to the example embodiments described here.

Furthermore, in this specification and the drawings, steps and elements that are substantially the same or similar are denoted by the same or similar reference numerals, and repeated descriptions of these steps and elements will be omitted.

Furthermore, in this specification and the drawings, elements are described in singular or plural form depending on the embodiment. However, the singular and plural forms are chosen as appropriate for the circumstances presented merely for convenience of explanation and are not intended to limit the disclosure thereto. Thus, the singular may include the plural and the plural may also include the singular unless the context clearly dictates otherwise.

In this specification and the drawings, steps and elements that are substantially the same or similar are denoted by the same or similar reference numerals, and repeated descriptions of these steps and elements will be omitted. Meanwhile, in the description of the present disclosure, the terms “first”, “second”, etc. are only used to differentiate the description and cannot be understood as indicating or implying relative importance or ordering.

To facilitate describing the present disclosure, concepts related to the present disclosure are introduced below.

The method of the present disclosure may be based on artificial intelligence (AI). Artificial intelligence is a theory, method, technology and application system that uses digital computers or machines controlled by digital computers to simulate, extend and expand human intelligence, perceive the environment, acquire knowledge and use knowledge to obtain the best results. In other words, artificial intelligence is a comprehensive technology of computer science that attempts to understand the essence of intelligence and produce a new method that is similar to human intelligence. Intelligent machines that respond. For example, for artificial intelligence-based methods, it can perform machine learning in a manner similar to human perception, such as training neural networks to extract image information, perform image analysis and processing.

Image processing technology is a technology that uses computers to process image information. Mainly include: image digitization, image enhancement and restoration, image data encoding, image segmentation and image recognition, etc. At present, image processing technology is mainly implemented based on computers. Artificial Neural Networks (ANN) is an algorithmic mathematical model that simulates the structure and behavior of biological nervous systems and performs distributed parallel information processing. ANN achieves the purpose of processing information by adjusting the weight relationship between internal neurons and neurons. When processing images, the computer uses the original image or appropriately preprocessed image as the input signal of the neural network, and obtains the processed image signal or classification result at the output end of the neural network.

To sum up, the solutions provided by the embodiments of the present disclosure involve computer technologies such as artificial intelligence, image processing, and neural networks. The embodiments of the present disclosure will be further described below with reference to the accompanying drawings.

Taking a CT image as an example, FIG. 1A is a schematic diagram showing an image with artifacts according to an embodiment of the present disclosure.

X-ray computed tomography (CT) has been widely used in clinical diagnosis. However, during the imaging process, due to the presence of metal implants in the patient's body, such as dental fillings and hip prostheses, projection data is usually lost, causing severe metal artifacts to appear in the reconstructed CT images. . As shown in Figure 1A, the structure of the metal artifact is a group of striped shadows with similar shapes. For different tissue structures and/or metal implants, the thickness and lightness of the strip shadow will be different. As can be seen from Figure 1A, the presence of metal artifacts makes CT images blurred, seriously interfering with the clear presentation of patient body details in CT images, and making it difficult for doctors to make correct judgments based on CT images.

Therefore, an image processing method is needed that can effectively identify artifacts in images and remove them from artifact-bearing images, so that the processed artifact-free image is as shown in Figure 1B. Clear, artifact-free CT images such as those shown in Figure 1B can help doctors make more accurate diagnoses.

FIG. 2A shows a schematic diagram of a chord diagram-based image processing method according to an embodiment of the present disclosure.

As shown in Figure 2A, DuDoNet++ is a joint learning scheme based on CT images and chord diagrams. This scheme uses two network modules, SE-Net and IE-Net, to jointly process CT images with metal artifacts. SE -Net is a network module for chord images, and IE-Net is a network module for CT images. In Figure 2A, S _ma represents the chord image contaminated by metal, S _se represents the chord image after repair and enhancement, X _se represents the CT image obtained by S _se after reverse projection transformation (RIL), and M represents the CT image. The shape characteristics of the metal in the domain (ie, mask), M _p represents the shape characteristics of the metal in the chord diagram domain obtained by M after Radon transformation (FP), X _ma represents the CT image with metal artifacts, X _out represents the output reconstructed image. As can be seen from Figure 2A, CT images with metal artifacts need to be processed jointly by the chord image domain (using SE-Net) and the CT image domain (using IE-Net) to achieve the purpose of image artifact removal. Among them, chord The transformation between the image domain and the CT image domain is performed through a differentiable Radon transform layer.

Similarly, FIG. 2B shows a schematic diagram of another chord diagram-based image processing method according to an embodiment of the present disclosure.

As shown in Figure 2B, DSCMAR is also a joint processing scheme based on CT image chord diagrams. Different from DuDoNet++, this scheme first uses PriorNet to obtain a cleaner repaired image, and then uses SinoNet to further correct and enhance the chord diagrams. Finally, The reconstructed CT image is obtained through filtered back projection (FBP) transformation. In Figure 2B, S _ma represents the chord diagram contaminated by metal, T _r represents the chord diagram with metal characteristics, S _LI represents the chord diagram after linear interpolation, and X _LI represents the chord diagram obtained by S _LI after back-projection transformation. CT image, X _ma represents CT image with metal artifacts Image, X _prior represents the initial repaired CT image processed by PriorNet, S _prior represents the initial repaired chord image obtained by forward projection transformation of X _prior , S _res represents the difference obtained by S _LI and S _prior The chord diagram, S _corr represents the further modified and enhanced chord diagram processed by SinoNet. By filtering the back projection transformation of S _corr , an image without metal artifacts can be obtained.

The image processing method based on chord diagrams shown in FIGS. 2A and 2B can implement image processing for artifact removal as shown in FIGS. 1A and 1B . However, the common limitation of DuDoNet++ and DSCMAR is that in these two solutions, the chord diagram information used is difficult to obtain in practice and usually needs to be provided by the equipment manufacturer. Moreover, the designed network does not embed the prior information unique to the Metal Artifacts Reduction (MAR) task well, and the generalization ability of the network model is limited. In addition, each network module included in these two solutions has weak physical interpretability and is not easy to understand and use by those skilled in the art.

In response to these problems, the present disclosure proposes an image processing method for artifact removal, which can perform reconstruction processing based only on images with artifacts, overcoming the problem of difficulty in obtaining chord diagram data. Moreover, the image processing method of the present disclosure uses a specific weighted convolution dictionary model to encode the prior structure of the artifact, making the network more reliable and with stronger generalization ability. In addition, the adaptive convolutional dictionary network model of the present disclosure has clear physical interpretability and is easy to understand and use by those skilled in the art.

FIG. 4 is an exemplary schematic diagram of an image processing model based on a weighted adaptive convolution dictionary according to an embodiment of the present disclosure.

For CT images with metal artifacts, their non-metal regions can be decomposed into the following model:
I⊙Y＝I⊙X+I⊙A, (1)

in, is the CT image with metal artifacts; H and W are the height and width of the image respectively; ;A is a metallic artifact. According to embodiments of the present disclosure, the non-metallic region mask I may be known, which is used to focus the solved mathematical model on the non-metallic domain and no longer pay attention to the solution of the metallic domain.

It should be understood that for different CT images with metal artifacts, the metal artifacts A contained therein basically show a common or similar pattern, that is, strip structures with similar shapes. At the same time, due to the interaction between normal human tissue and metal artifacts, the artifact patterns contained in different CT images with metal artifacts have some specific attributes, such as pixel intensity.

Considering the above characteristics of CT images with metal artifacts, according to embodiments of the present disclosure, for each artifact image (Y) to be processed, by using a basic artifact dictionary learned in advance from existing samples to build an adaptive convolution kernel for the image And extract the artifact features of the image Then the adaptive convolution kernel based on this image can be with artifact features The metal artifact A of the image is obtained, thereby realizing the extraction of the metal artifact. It should be understood that the basic artifact dictionary It can be either an existing known basic artifact dictionary or a basic artifact dictionary obtained through training and synthesis of existing samples.

According to embodiments of the present disclosure, a weighted convolution dictionary model can be used to encode the metal artifact A:

in, is a sample-invariant dictionary, including d convolution kernels, representing the common patterns of different metal artifacts. In short, A common database representing different metal artifact types in all CT images with metal artifacts; is the weighting coefficient that changes with the sample; Represents a specific convolution kernel, which represents a recurring pattern of a certain metal artifact, p×p is the size of the convolution kernel; is the feature layer, which represents the recurrence of local patterns. Position; N is the number of real specific convolution kernels used to encode A; It is a two-dimensional plane convolution operation, and p and d are both positive integers. also, and

That is, the basic artifact dictionary is a convolutional dictionary that does not change with samples and includes a first number (d) of artifact convolution kernels. The second number of adaptive convolution kernels for the image sample can be determined by a plurality of artifact convolution kernels in the basic artifact dictionary and a weighting coefficient (K) that changes with the sample.

Putting equation (2) into equation (1), the model of the non-metallic area corresponding to the final CT image with metal artifacts can be obtained as:

FIG. 5A shows a schematic structure of an adaptive convolutional dictionary network according to an embodiment of the present disclosure, which includes a T-level network. In each level of network, the weighting coefficient K, feature layer and clean CT image X are updated. Figure 5B shows a schematic structure of a network at each level according to an embodiment of the present disclosure.

The image processing method according to the embodiment of the present disclosure will first be described below in conjunction with Figures 4, 5A and 5B, and then the mathematical model of the adaptive convolutional dictionary network of Figures 5A and 5B will be described.

3B is a schematic flowchart 320 illustrating a model usage process of an image processing method for artifact removal according to an embodiment of the present disclosure.

In S321, the input image to be processed is obtained.

According to an embodiment of the present disclosure, the input image to be processed may be an image obtained after applying its corresponding image mask (I) to the original CT image, that is, I⊙Y. Optionally, the input image to be processed may be an image with artifacts, or may be an image with artifacts that has undergone preprocessing (eg, denoising processing, normalization processing). It should be noted that the image mask (I) in this disclosure is a mask of the area to be studied or the area of interest.

In S322, the input image is processed using an adaptive convolutional dictionary network to obtain a processed image with artifacts removed.

Wherein, the adaptive convolutional dictionary network is trained based on the artifact database (D) and includes a T-level network (as shown in Figure 5A), wherein the level 1 network outputs a level 1 image based on the input image Features and level 1 artifact removal images; the t-th level network is based at least in part on the t-1th level image features and the t-1th level artifact removal image output by the t-1th level network, and outputs the t-th level image features and The t-th level artifact removal image, where t is greater than 1 and less than or equal to T; the T-th level network outputs the T-1th level image feature and the T-1th level artifact removal image based on the T-1th level network output The T-level artifact removal image is used as the processed image after artifact removal. The processed image with artifacts removed can be output to the user through a display or through film, drawings, etc.

According to an embodiment of the present disclosure, the adaptive convolution dictionary network includes a basic artifact dictionary, which is a convolution dictionary that does not change with the input image and includes an artifact convolution kernel. In the t-th level network, First, determine the t-th level weighting coefficient of the t-th level network, and then determine the adaptive convolution for the t-th level network through the artifact convolution kernel in the basic artifact dictionary and the t-th level weighting coefficient. convolution kernel; then based on the adaptive convolution kernel and the image characteristics of the t-th level network Characteristics determine the t-th level artifact removal image.

The number of artifact convolution kernels can be recorded as the first number, which is an integer value greater than 1. This application does not limit the number of adaptive convolution kernels, and their number can be recorded as the second number.

According to an embodiment of the present disclosure, each level of the network may include a weighted coefficient update network, an image feature update network, and an artifact removal image update network, wherein the weighted coefficient update network, the image feature update network, and the artifact removal image update network include residual Differential network structure and normalization processing layer. It should be understood that the network structures of the weighted coefficient update network, image feature update network, and artifact removal image update network can be diverse, and the network structure is usually related to the dimensional characteristics of the solution variables corresponding to the network. For example, the weighted coefficient update network solves for coefficients, so it usually includes a linear layer, while the image feature update network and the artifact removal image update network solve for two-dimensional images, so it usually includes a convolutional layer.

According to an embodiment of the present disclosure, the artifact image may be a metal artifact, the input image to be processed may be a CT image with metal artifacts, and the image mask (I) may be the same as a CT image with metal artifacts. Non-metal area mask corresponding to CT image. In this embodiment, each level of the network may include a weighted coefficient update network, a metal artifact image feature update network, and a metal artifact removal image update network, where the weighted coefficient update network, the metal artifact image feature update network, and the metal artifact removal image update network are The artifact removal image update network includes a residual network structure; and the weighted coefficient update network includes: a linear layer, a rectified linear unit (Rectified Linear Unit, ReLU) layer, a cross-link layer, and a batch normalization (Batch Normalization, BN) layer; metal The artifact image feature update network includes: convolution layer, BN layer, ReLU layer, and cross-link layer; the metal artifact removal image update network includes: convolution layer, BN layer, ReLU layer, and cross-link layer.

3A is a schematic flowchart 310 illustrating a neural network training process of an image processing method for artifact removal according to an embodiment of the present disclosure.

In S311, a training data set for training the neural network is established, wherein the training data set includes multiple groups of image samples, each group of image samples includes an image with artifacts (Y) and its corresponding image without artifacts. (X) and image mask (I).

According to embodiments of the present disclosure, public image libraries and different types of metal masks can be used to synthesize artifacts according to the data simulation process, and the artifact-bearing image and the metal mask corresponding to the artifact-bearing image ( _Me ) as training data. For example, in the application scenario of removing artifacts from CT images with metal artifacts, you can use the public DeepLesion image library and different types of metal masks to synthesize metal artifacts according to the data simulation process, and convert the CT images with metal artifacts into Images and metal masks are used as training data. It should be understood that the different types of masks or different types of metal masks are used to set the size and shape of the implant in the CT image. Based on the size and shape of the implant, the corresponding simulation can be obtained. Artifact shape. There is a corresponding relationship between the metal mask (M _e ) and the non-metal area mask (I), and they can be deduced from each other, that is, Me ₊ I = 1.

According to an embodiment of the present disclosure, in order to process sample data of a larger data range, establishing a training data set for training a neural network may include: normalizing the pixel values of the image with artifacts (Y).

In addition, in order to obtain the diversity of sample data, the establishment of a training data set for training a neural network may include: randomly cropping the artifact-bearing image (Y) to obtain image blocks, and classifying all the artifacts according to a predetermined probability. The above image blocks are randomly flipped.

For example, you can crop the numerical range of images with artifacts in the training data (for example, remove values that are too large or negative values, so that unnecessary numerical ranges are no longer retained, but the required organizational information will not be lost) , and then normalize The pixel value of the image is within the threshold range [0,1]. Optionally, the normalized data can also be converted to the [0,255] range to facilitate computer processing.

According to embodiments of the present disclosure, each training image and the corresponding mask can also be randomly cropped to form a smaller image block (for example, it can be an image block of 64x64 pixel size), and then with a predetermined probability (for example, it can be 0.5) Perform random horizontal mirror flipping and random vertical mirror flipping respectively to obtain more diverse training sample data.

For any group of image samples among the plurality of groups of image samples, optimization of the network parameters of the adaptive convolutional dictionary network can be achieved by executing the following S312-S313.

In S312, an adaptive convolutional dictionary network is used to perform artifact removal processing on the image with artifacts (Y) to obtain a processed image.

According to an embodiment of the present disclosure, as shown in Figure 4, the adaptive convolution dictionary network may include a basic artifact dictionary, which is a convolution dictionary that does not change with samples and includes an artifact convolution kernel, and may The adaptive convolution kernel for the image sample is determined through a plurality of artifact convolution kernels in the basic artifact dictionary and a weighting coefficient that changes with the sample. Furthermore, the artifact image in the artifact-bearing image can be determined by convolving the adaptive convolution kernel with the image features of the artifact-bearing image, and the artifact-bearing image can be removed from the artifact-bearing image. Artifact image to obtain the processed image.

In S313, the adaptive convolutional dictionary network is iteratively trained based on the artifact-free image (X), the processed image, and the objective function processed by the image mask (I) to optimize Network parameters of the adaptive convolutional dictionary network.

According to an embodiment of the present disclosure, the objective function may be a loss objective function constructed based on the artifact-free image (X) and the processed image, wherein the objective function is constructed based on the artifact-free image (X) and the processed image. The processed image and the objective function are used to iteratively train the adaptive convolutional dictionary network to optimize the network parameters of the adaptive convolutional dictionary network, including: calculating the loss objective function and backpropagating the result to the adaptive convolutional dictionary network. The adaptive convolutional dictionary network is described, and the network parameters of the adaptive convolutional dictionary network are optimized based on the adaptive moment estimation (Adaptive moment estimation, Adam) algorithm.

According to an embodiment of the present disclosure, the objective function may also be a loss objective function constructed based on the artifact-free image (X) and the processed image and processed by the image mask (I).

According to an embodiment of the present disclosure, the artifact convolution kernel indicates an artifact pattern, and the image feature indicates the location of the artifact pattern, wherein the adaptive convolution dictionary network includes a T-level network, wherein In the t-level network, the iterative update rule based on proximal gradient descent is used to update the weighted coefficients and image features output by the t-1 level network to obtain the weighted coefficients and image features of the t-level network, where t is greater than An integer that is 1 and less than or equal to T.

According to embodiments of the present disclosure, the image processing method for artifact removal may further include: after the training is completed, The adaptive convolutional dictionary network was tested to evaluate the effect of image processing. Wherein, testing the adaptive convolutional dictionary network includes: preprocessing the artifact-bearing image to be tested and inputting it into the adaptive convolutional dictionary network; using the adaptive convolutional dictionary network to The image with artifacts to be tested is processed to obtain a processed image with artifacts removed.

According to an embodiment of the present disclosure, the artifact image may be a metal artifact, and the image with artifacts may be a CT image with metal artifacts. Therefore, the training data set includes multiple groups of CT image samples, each group of image samples includes CT images with metal artifacts and corresponding CT images without metal artifacts; the adaptive convolution dictionary network includes basic metal Artifact dictionary, the basic metal artifact dictionary is a metal artifact convolution dictionary that does not change with CT image samples and includes a metal artifact convolution kernel, and is passed through a plurality of metal artifact volumes in the basic metal artifact dictionary The adaptive convolution kernel for the CT image sample is determined with a weighting coefficient that changes with the CT image sample, wherein the metal artifact convolution kernel indicates a metal artifact pattern, and the image feature indicates the Where the metal artifact pattern is located.

According to embodiments of the present disclosure, in the application scenario of removing artifacts from CT images, each level of the network may include a weighted coefficient update network, a metal artifact image feature update network, and a metal artifact removal image update network, where the weighted coefficient update network The network, the metal artifact image feature update network, and the metal artifact removal image update network include a residual network structure; and the weighted coefficient update network includes: a linear layer, a ReLU layer, a cross-link layer, and a BN layer; metal artifact image features The update network includes: convolution layer, BN layer, ReLU layer, and cross-link layer; the metal artifact removal image update network includes: convolution layer, BN layer, ReLU layer, and cross-link layer.

The following continues to take the mathematical model for removing metal artifacts from CT images with metal artifacts in Figure 4 as an example. The image used for artifact removal according to the present disclosure will be described in detail with reference to Figures 4, 5A and 5B. Mathematical model of the processing method and structure of the adaptive convolutional dictionary network.

According to embodiments of the present disclosure, a common database representing different metal artifact types in all CT images with metal artifacts can be It is constructed as a convolution layer, based on which an adaptive convolutional dictionary network is constructed, and the optimized parameters of the adaptive convolutional dictionary network are obtained through end-to-end training on the training data set.

For the mathematical model shown in equation (4) above:

The solution goal is to estimate K from Y, and X, the corresponding optimization problem is:

Among them, α, β and γ are compromise parameters, f ₁ (·), f ₂ (·) and f ₃ (·) are regular terms, which respectively represent the weighting coefficient K, feature layer and the prior structure of the clean CT image X, which can be designed as a neural network module to solve.

In order to solve the optimization problem in (5), proximal gradient technology can be used to alternately update the weighting coefficient K and the feature layer and clean CT image X. details as follows:

Update K:

In the (t+1)th iteration, K can be updated as:

The corresponding quadratic approximation form is:

Among them, Ω={K|‖K _n ‖ ₂ =1,n=1,2,…,N}; η ₁ is the update step size, which can be derived:

in, For depth convolution operation; Represents the expansion of the tensor in the third dimension; vec(·) represents the vectorization operation.

Equation (7) can be equivalently written as:

For the general prior term f ₁ (·), equation (9) can be written as:

in,

is a proximal operator, related to the regular term f ₁ (·); Ω can be obtained by Introduce a normalization operation to achieve this.

renew

Similar to the update of N, in the (t+1)th iteration, can be updated to:

Among them, eta ₂ is the update step size, For the general prior term f ₂ (·), equation (11) can be written as:

in, is a proximal operator, related to the regular term f ₂ (·); is the transposed convolution operation.

UpdateX:

Given N ^(t+1) and can be updated to:

in, Further, the update rule of X can be obtained as:

in, is a proximal operator, related to the regular term f ₃ (·).

By expanding the update formulas (10) (12) (14), a complete adaptive convolutional dictionary network (Adaptive Convolutional Dictionary Network, ACDNet) can finally be constructed, in which each network has good physical interpretability.

A schematic structure of an adaptive convolutional dictionary network according to an embodiment of the present disclosure is shown in FIG. 5A , which includes a T-level network. In each level of network, the weighting coefficient K, feature layer and processed CT image X are updated. A schematic structure of each level network according to an embodiment of the present disclosure is shown in Figure 5B.

According to the embodiment of the present disclosure, ACDNet as shown in Figure 5A is composed of T stages (i.e., T-level network). In each stage, the corresponding network structure consists of K-net, and X-net, which are used to implement K, and iterative updates of X.

Next, the correspondence between the network structure in Figures 5A and 5B and the above mathematical model is explained.

In Figure 5A and Figure 5B, specifically,

K-net: in It is a residual structure, specifically: linear layer, ReLU layer, linear layer, cross-link layer and normalization operation layer at dimension d;

in Composed of 3 residual blocks, each residual block includes in turn: convolution layer, BN layer, ReLU layer, convolution layer, BN layer, and cross-link layer;

X-net: in It is composed of 3 residual blocks, and each residual block includes in turn: convolution layer, BN layer, ReLU layer, convolution layer, BN layer, and cross-link layer.

As can be seen from Figure 5A, the first-level network obtains the first-level image feature M ⁽¹⁾ and the first-level artifact removal image X ⁽¹⁾ output by the first-level network based on the input image; the t-level network at least partially Based on the t-1th level image feature M ^(t-1) output by the t-1th level network and the t-1th level artifact removal image X ^(t-1) , obtain and output the t-th level network output The t-th level image feature M ^(t) and the t-th level artifact removal image X ^(t) , where t is greater than 1 and less than or equal to T; the T-th level network is based on the T-1 level image output by the T-1 level network Feature M ^(T-1) and T-1th level artifact removal image X ^(T-1) , obtain and output the Tth level artifact removal image X ^(T) output by the Tth level network as the artifact removal image The processed image of the shadow.

For each level of network in the adaptive convolutional dictionary network, the iterative solution process is shown in Figure 5B. Among them, in the t-th level network, the weighted coefficients and image features output by the t-1 level network are updated using the iterative update rule based on proximal gradient descent to obtain the weighting coefficients and image features of the t-level network, where t is an integer greater than 1 and less than or equal to T. K-net, M-net, and X-net are solved iteratively in a serial manner.

The adaptive convolutional dictionary network can be iteratively trained based on the artifact-free image (X), the processed image, and the objective function processed by the image mask (I) to optimize the network of the adaptive convolutional dictionary network. parameter. Wherein, the objective function may be a loss objective function constructed based on the artifact-free image (X) and the processed image, that is,

Among them, μ _t is the compromise parameter, and ω ₁ and ω ₂ are the weights used to balance various losses. For example, in the simulation experiment, μ _t =0.1 (t = 0, 1,..., T-1), μ _T =1, ω ₁ =ω ₂ =5×10 ^-4 , and T = 10 can be set.

According to embodiments of the present disclosure, the optimization parameters can be updated and solved based on the Adam algorithm, including: Convolution kernel Step sizes eta ₁ , eta ₂ and eta ₃ . During each iteration, the prediction result error is calculated and back-propagated to the convolutional neural network model, the gradient is calculated and the parameters of the convolutional neural network model are updated.

FIG. 6 is a schematic flowchart illustrating an image processing process for artifact removal according to an embodiment of the present disclosure.

As shown in FIG. 6 , the image processing process for artifact removal according to an embodiment of the present disclosure includes a neural network training phase and a testing phase.

In the neural network training stage, images with artifacts can first be preprocessed to establish a training data set for training the neural network, where the training data set includes multiple groups of image samples, and each group of image samples includes Artifact image (Y) and its corresponding artifact-free image (X) and image mask (I). Then based on the preprocessed image samples, the computer iteratively trains ACDNet according to the neural network training settings. During the training process, the parameters of ACDNet are updated based on the predetermined objective function and the Adam optimization algorithm. If the predetermined number of iterations is reached during the training process, the trained model is saved. If the predetermined number of iterations is not reached, ACDNet continues to be trained.

It should be understood that the purpose of using the predetermined number of iterations as the criterion for judging the completion of ACDNet training is to prevent the network from overfitting. Optionally, you can also output the optimized image in a visual way, and then stop training ACDNet after confirming that the optimized image meets the requirements.

In the testing phase, the input image to be processed and its corresponding image mask (I) are input to the computer. The computer loads the trained model and obtains the artifact-removed image through ACDNet forward calculation. The computer can output the artifact-removed image. The post-production images are for user reference.

Similarly, in actual use, the image processing process is similar to the testing stage, so no details will be given.

FIG. 7A is a schematic diagram 710 showing the composition of an image processing device for artifact removal according to an embodiment of the present disclosure. The device 710 is used for a neural network training process during image processing.

According to an embodiment of the present disclosure, the image processing device 710 for artifact removal may include: a training data set establishment module 711, an adaptive convolutional dictionary network 712, and a training module 713.

Wherein, the training data set establishment module 711 can be configured to: establish a training data set for training the neural network, wherein the training data set includes multiple groups of image samples, and each group of image samples includes images with artifacts (Y ) and its corresponding artifact-free image (X) and image mask (I).

Optionally, the training data set establishment module 711 may be configured to: perform pixel values of the image with artifacts (Y). Perform normalization processing; and\or, randomly crop the image with artifacts (Y) to obtain image blocks, and perform random flip processing on the image blocks according to a predetermined probability.

The adaptive convolutional dictionary network 712 may be configured to: perform artifact removal processing on the artifact-bearing image (Y) for at least one group of image samples among the plurality of groups of image samples to obtain a processed image. .

The adaptive convolution dictionary network 712 may include a basic artifact dictionary, which is a convolution dictionary that does not change with samples and includes an artifact convolution kernel. The artifact convolution kernel and the weighting coefficient that change with the sample are used to determine the adaptive convolution kernel for the image sample. Furthermore, the artifact image in the artifact-bearing image can be determined by convolving the adaptive convolution kernel with the image features of the artifact-bearing image, and the artifact-bearing image can be removed from the artifact-bearing image. Artifact image to obtain the processed image.

The training module 713 may be configured to iterate the adaptive convolutional dictionary network based on the artifact-free image (X) and the processed image, and an objective function processed by the image mask (I) Train to optimize the network parameters of the adaptive convolutional dictionary network.

According to an embodiment of the present disclosure, the objective function may be a loss objective function constructed based on the artifact-free image (X) and the processed image, wherein the training module 713 may be configured to: calculate the loss objective function and The results are backpropagated to the adaptive convolutional dictionary network, and the network parameters of the adaptive convolutional dictionary network are optimized based on the Adam algorithm.

According to an embodiment of the present disclosure, the artifact convolution kernel indicates an artifact pattern, and the image feature indicates the location of the artifact pattern, wherein the adaptive convolution dictionary network includes a T-level network, wherein In the t-level network, the iterative update rule based on proximal gradient descent is used to update the weighted coefficients and image features output by the t-1 level network to obtain the weighted coefficients and image features of the t-level network, where t is greater than An integer that is 1 and less than or equal to T

According to an embodiment of the present disclosure, each level of the network includes a weighted coefficient update network, an image feature update network, and an artifact removal image update network, wherein the weighted coefficient update network, the image feature update network, and the artifact removal image update network include residual Network structure and normalization processing layer.

According to an embodiment of the present disclosure, the image processing device 710 for artifact removal may further include: a testing module 714 configured to: test the adaptive convolutional dictionary network after the training is completed, wherein the Testing the adaptive convolutional dictionary network includes: preprocessing the artifact-containing image to be tested and inputting it into the adaptive convolutional dictionary network; using the adaptive convolutional dictionary network to preprocess the artifact-containing image to be tested. Artifact images are processed to obtain a processed image with artifacts removed.

FIG. 7B is a schematic diagram 720 showing the composition of an image processing device for artifact removal according to an embodiment of the present disclosure. The device 720 is used in a model usage process during image processing.

According to an embodiment of the present disclosure, the image processing device 720 for artifact removal may include: an image acquisition module 721 and an image processing module 722.

Wherein, the image acquisition module 721 may be configured to: acquire the input image to be processed.

According to embodiments of the present disclosure, the input image to be processed may be an image obtained after applying its corresponding image mask to the original CT image. For example, the image acquisition module 721 may receive the original CT image and its corresponding image mask, and apply its corresponding image mask to the original CT image, thereby acquiring the input image to be processed.

According to embodiments of the present disclosure, the input image to be processed may be an image with artifacts, or may be a pre-processed image (for example Such as denoising processing, normalization processing) images with artifacts.

The image processing module 722 may be configured to use an adaptive convolutional dictionary network to process the input image to obtain a processed image with artifacts removed.

Wherein, the adaptive convolutional dictionary network is trained based on the artifact database (D) and includes a T-level network, wherein the level 1 network outputs level 1 image features and level 1 artifacts based on the input image. Remove the image; the t-th level network outputs the t-level image features and the t-th level artifact removal image based at least in part on the t-1 level image features and the t-1 level artifact removal image output by the t-1 level network. , where t is greater than 1 and less than or equal to T; the T-level network is based on the T-1-th level image features and the T-1-th level artifact removal image output by the T-1-th level network, and outputs the T-th level artifact removal image as The processed image after artifact removal.

According to an embodiment of the present disclosure, the adaptive convolution dictionary network includes a basic artifact dictionary, which is a convolution dictionary that does not change with the input image and includes an artifact convolution kernel, which determines the t-th level network The t-level weighting coefficient determines the adaptive convolution kernel for the t-th level network through multiple artifact convolution kernels in the basic artifact dictionary and the t-th level weighting coefficient; and based on the The adaptive convolution kernel and the image features of the t-th level network determine the t-th level artifact removal image.

Generally speaking, the various example embodiments of the present disclosure may be implemented in hardware or special purpose circuits, software, firmware, logic, or any combination thereof. Some aspects may be implemented in hardware, while other aspects may be implemented in firmware or software that may be executed by a controller, microprocessor, or other computing device. While aspects of embodiments of the present disclosure are illustrated or described as block diagrams, flowcharts, or using some other graphical representation, it will be understood that the blocks, devices, systems, techniques, or methods described herein may be used as non-limiting Examples are implemented in hardware, software, firmware, special purpose circuitry or logic, general purpose hardware or controllers, or other computing devices, or some combination thereof.

For example, methods or apparatuses according to embodiments of the present disclosure may also be implemented with the aid of the architecture of the computing device 3000 shown in FIG. 8 . As shown in Figure 8, computing device 3000 may include a bus 3010, one or more CPUs 3020, read only memory (ROM) 3030, random access memory (RAM) 3040, communication port 3050 connected to a network, input/output components 3060, hard disk 3070, etc. The storage device in the computing device 3000, such as the ROM 3030 or the hard disk 3070, can store various data or files used for processing and/or communication of the methods provided by the present disclosure, as well as program instructions executed by the CPU. Computing device 3000 may also include user interface 3080. Of course, the architecture shown in FIG. 8 is only exemplary, and when implementing different devices, one or more components in the computing device shown in FIG. 8 may be omitted according to actual needs.

According to yet another aspect of the present disclosure, a computer-readable storage medium is also provided. Figure 9 shows a schematic diagram 4000 of a storage medium in accordance with the present disclosure.

As shown in Figure 9, computer readable instructions 4010 are stored on computer storage medium 4020. When the computer readable instructions 4010 are executed by the processor, the methods according to the embodiments of the present disclosure described with reference to the above figures may be performed. Computer-readable storage media in embodiments of the present disclosure may be volatile memory or non-volatile memory, or may include both volatile and non-volatile memory. Non-volatile memory may be read-only memory (ROM), programmable read-only memory (PROM), erasable programmable read-only memory (EPROM), electrically erasable programmable read-only memory (EEPROM), or flash memory. Volatile memory may be random access memory (RAM), which acts as an external cache. By way of illustration, but not limitation, many forms of RAM are available, such as static random access memory (SRAM), dynamic random access memory (DRAM), synchronous dynamic random access memory (SDRAM), double data rate synchronous dynamic Follow machine access memory (DDRSDRAM), enhanced synchronous dynamic random access memory (ESDRAM), synchronous link dynamic random access memory (SLDRAM) and direct memory bus random access memory (DR RAM). It should be noted that memory for the methods described herein is intended to include, but is not limited to, these and any other suitable types of memory. It should be noted that memory for the methods described herein is intended to include, but is not limited to, these and any other suitable types of memory.

Embodiments of the present disclosure also provide a computer program product or computer program, which includes a computer program, and the computer program is stored in a computer-readable storage medium. The processor of the computer device reads the computer program from the computer-readable storage medium, and the processor executes the computer program, so that the computer device performs the method according to the embodiment of the present disclosure.

It should be noted that the flowcharts and block diagrams in the accompanying drawings illustrate the possible implementation architecture, functions and operations of systems, methods and computer program products according to various embodiments of the present disclosure. In this regard, each block in the flowchart or block diagrams may represent a module, segment, or portion of code that contains at least one element for implementing the specified logical function. Executable instructions. It should also be noted that, in some alternative implementations, the functions noted in the block may occur out of the order noted in the figures. For example, two blocks shown one after another may actually execute substantially in parallel, or they may sometimes execute in the reverse order, depending on the functionality involved. It will also be noted that each block of the block diagram and/or flowchart illustration, and combinations of blocks in the block diagram and/or flowchart illustration, can be implemented by special purpose hardware-based systems that perform the specified functions or operations. , or can be implemented using a combination of specialized hardware and computer instructions.

This disclosure uses specific words to describe embodiments of the disclosure. For example, "first/second embodiment", "an embodiment", and/or "some embodiments" means a certain feature, structure or characteristic related to at least one embodiment of the present disclosure. Therefore, it should be emphasized and noted that “one embodiment” or “an embodiment” or “an alternative embodiment” mentioned twice or more at different places in this specification does not necessarily refer to the same embodiment. . In addition, certain features, structures or characteristics of one or more embodiments of the present disclosure may be combined appropriately.

Unless otherwise defined, all terms (including technical and scientific terms) used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this application belongs. It should also be understood that terms such as those defined in ordinary dictionaries should be construed to have meanings consistent with their meanings in the context of the relevant technology and should not be interpreted in an idealized or highly formalized sense unless expressly stated herein Ground is defined this way.

The above is a description of the present application and should not be considered as a limitation thereof. Although several exemplary embodiments of the present application have been described, those skilled in the art will readily appreciate that many modifications are possible in the exemplary embodiments without departing from the novel teachings and advantages of the present application. Accordingly, all such modifications are intended to be included within the scope of the application as defined by the claims. It is to be understood that the above is a description of the present application and should not be construed as limited to the particular embodiments disclosed, and that modifications to the disclosed embodiments as well as other embodiments are intended to be included within the scope of the appended claims. The application is defined by the claims and their equivalents.

Claims

An image processing method for artifact removal, including:

Establish a training data set for training the neural network, wherein the training data set includes multiple groups of image samples, each group of image samples includes an image with artifacts (Y) and its corresponding image without artifacts (X) and ImageMask(I);

For any group of image samples among the plurality of groups of image samples,

Using an adaptive convolutional dictionary network, perform artifact removal processing on the image with artifacts (Y) to obtain the processed image,

The adaptive convolutional dictionary network is iteratively trained based on the artifact-free image (X) and the processed image, and the objective function processed by the image mask (I) to optimize the adaptive Network parameters of convolutional dictionary network,

Wherein, the adaptive convolution dictionary network includes a basic artifact dictionary, which is a convolution dictionary that does not change with samples and includes an artifact convolution kernel, and through multiple artifacts in the basic artifact dictionary The artifact convolution kernel and the weighting coefficient that change with the sample are used to determine the adaptive convolution kernel for the image sample,

Wherein, the artifact image in the artifact-bearing image is determined through convolution of the adaptive convolution kernel and the image features of the artifact-bearing image, and the artifact is removed from the artifact-bearing image. shadow image to obtain the processed image.
The image processing method of claim 1, wherein the first number of artifact convolution kernels indicates an artifact pattern, and the image feature indicates a location where the artifact pattern is located,

Wherein, the adaptive convolutional dictionary network includes a T-level network, wherein in the t-level network, the weighted coefficients and image features output by the t-1th level network are updated using an iterative update rule based on proximal gradient descent. , to obtain the weighting coefficients and image features of the t-th level network, where t is an integer greater than 1 and less than or equal to T.
The image processing method of claim 2, wherein each level of network includes a weighted coefficient update network, an image feature update network, and an artifact removal image update network, wherein,

The weighted coefficient update network, image feature update network, and artifact removal image update network include a residual network structure and a normalization processing layer.
The image processing method according to claim 1, wherein said establishing a training data set for training a neural network includes at least one of the following:

Normalize the pixel values of the artifact-bearing image (Y); and

The image with artifacts (Y) is randomly cropped to obtain image blocks, and the image blocks are randomly flipped according to a predetermined probability.
The image processing method according to claim 1, wherein the objective function is a loss objective function constructed based on the artifact-free image (X) and the processed image,

Wherein, the adaptive convolutional dictionary network is iteratively trained based on the artifact-free image (X), the processed image, and an objective function to optimize network parameters of the adaptive convolutional dictionary network. include:

The loss objective function is calculated and the result is back-propagated to the adaptive convolutional dictionary network, and the network parameters of the adaptive convolutional dictionary network are optimized based on the adaptive moment estimation Adam algorithm.
The image processing method as claimed in claim 1, further comprising: after training is completed, testing the adaptive convolutional dictionary network,

Wherein, the testing of the adaptive convolutional dictionary network includes:

Preprocess the image with artifacts to be tested and input it into the adaptive convolutional dictionary network;

The image with artifacts to be tested is processed using an adaptive convolutional dictionary network to obtain a processed image with artifacts removed.
The image processing method according to claim 1, wherein the artifact image is a metal artifact, and the artifact image is a computed tomography CT image with metal artifact,

The training data set includes multiple groups of CT image samples, each group of image samples includes a CT image with metal artifacts and a corresponding CT image without metal artifacts and a non-metal area mask;

The adaptive convolution dictionary network includes a basic metal artifact dictionary, which is a metal artifact convolution dictionary that does not change with CT image samples and includes a metal artifact convolution kernel, and through the basic metal artifact dictionary Multiple metal artifact convolution kernels in the artifact dictionary and weighting coefficients that change with the CT image sample are used to determine an adaptive convolution kernel for the CT image sample, wherein the metal artifact convolution kernel indicates metal Artifact pattern, the image feature indicates where the metal artifact pattern is located.
The image processing method according to claim 7, wherein each level of the network includes a weighted coefficient update network, a metal artifact image feature update network, and a metal artifact removal image update network, wherein,

The weighted coefficient update network, the metal artifact image feature update network, and the metal artifact removal image update network include a residual network structure; and

The weighted coefficient update network includes: linear layer, modified linear unit ReLU layer, cross-link layer, and batch normalized BN layer;

The metal artifact image feature update network includes: convolution layer, BN layer, ReLU layer, and cross-link layer;

The metal artifact removal image update network includes: convolution layer, BN layer, ReLU layer, and cross-link layer.
An image processing method for artifact removal, including:

Get the input image to be processed;

Using an adaptive convolutional dictionary network, the input image is processed to obtain a processed image with artifacts removed,

Wherein, the adaptive convolutional dictionary network is trained based on the artifact database (D) and includes a T-level network, wherein the level 1 network outputs level 1 image features and level 1 artifacts based on the input image. Remove the image; the t-th level network outputs the t-level image features and the t-th level artifact removal image based at least in part on the t-1 level image features and the t-1 level artifact removal image output by the t-1 level network. , where t is greater than 1 and less than or equal to T; the T-level network is based on the T-1-th level image features and the T-1-th level artifact removal image output by the T-1-th level network, and outputs the T-th level artifact removal image as The processed image after artifact removal.
The image processing method according to claim 9, wherein the adaptive convolution dictionary network includes a basic artifact dictionary, the basic artifact dictionary is a convolution dictionary that does not change with the input image and includes an artifact convolution kernel,

Determine the t-th level weighting coefficient of the t-th level network, and determine the adaptive convolution for the t-th level network through multiple artifact convolution kernels in the basic artifact dictionary and the t-th level weighting coefficient. convolution kernel; and determine the t-th level artifact removal image based on the image features of the adaptive convolution kernel and the t-th level network.
The image processing method according to claim 9, wherein each level of the network includes a weighted coefficient update network, an image feature update network, and an artifact removal image update network, wherein,

The weighted coefficient update network, image feature update network, and artifact removal image update network include a residual network structure and a normalization processing layer.
The image processing method according to claim 9, wherein the input image to be processed is an image after applying the image mask (I).
The image processing method according to claim 12, wherein the artifact is a metal artifact, the input image to be processed is a computed tomography CT image with metal artifact, and the image mask (I) is Non-metal area mask corresponding to CT image with metal artifacts,

Among them, each level of network includes a weighted coefficient update network, a metal artifact image feature update network, and a metal artifact removal image update network, where,

The weighted coefficient update network, the metal artifact image feature update network, and the metal artifact removal image update network include a residual network structure; and

The weighted coefficient update network includes: linear layer, modified linear unit ReLU layer, cross-link layer, and batch normalized BN layer;

The metal artifact image feature update network includes: convolution layer, BN layer, ReLU layer, and cross-link layer;

The metal artifact removal image update network includes: convolution layer, BN layer, ReLU layer, and cross-link layer.
An image processing device for artifact removal, including:

A training data set creation module configured to: create a training data set for training a neural network, wherein the training data set includes multiple groups of image samples, and each group of image samples includes an artifact image (Y) and its Corresponding artifact-free image (X) and image mask (I);

An adaptive convolutional dictionary network configured to: perform artifact removal processing on the artifact-bearing image (Y) for any group of image samples among the plurality of groups of image samples to obtain a processed image;

a training module configured to iteratively train the adaptive convolutional dictionary network based on the artifact-free image (X), the processed image, and an objective function processed by the image mask (I) , to optimize the network parameters of the adaptive convolutional dictionary network;

Wherein, the adaptive convolution dictionary network includes a basic artifact dictionary, which is a convolution dictionary that does not change with samples and includes an artifact convolution kernel, and through multiple artifacts in the basic artifact dictionary The artifact convolution kernel and the weighting coefficient that change with the sample are used to determine the adaptive convolution kernel for the image sample,

Wherein, the artifact image in the artifact-bearing image is determined through convolution of the adaptive convolution kernel and the image features of the artifact-bearing image, and the artifact is removed from the artifact-bearing image. shadow image to obtain the processed image.
An image processing device for artifact removal, including:

The image acquisition module is configured to: acquire the input image to be processed;

An image processing module configured to: use an adaptive convolutional dictionary network to process the input image to obtain a processed image with artifacts removed;

Wherein, the adaptive convolutional dictionary network is trained based on the artifact database (D) and includes a T-level network, wherein the level 1 network outputs level 1 image features and level 1 artifacts based on the input image. Remove the image; the t-th level network outputs the t-level image features and the t-th level artifact removal image based at least in part on the t-1 level image features and the t-1 level artifact removal image output by the t-1 level network. , where t is greater than 1 and less than or equal to T; the T-th level network is based on the T-1-th level network output level image features and the T-1th level artifact removal image, and output the T-th level artifact removal image as the processed image after artifact removal.
A computer program product, which includes a computer program that, when run by a processor, is used to implement the method according to any one of claims 1-13.
A computer-readable storage medium having a computer program stored thereon, the computer program being used to implement the method according to any one of claims 1-13 when executed by a processor.
A kind of computer equipment, described computer equipment includes:

Processors, communication interfaces, memory and communication buses;

Wherein, the processor, the communication interface and the memory complete communication with each other through the communication bus; the communication interface is an interface of a communication module;

The memory is used to store a computer program and transmit the computer program to the processor;

The processor is configured to call a computer program in a memory to execute the method described in any one of claims 1-13.