CN116993852A - Training method of image reconstruction model, main control equipment and image reconstruction method - Google Patents
Training method of image reconstruction model, main control equipment and image reconstruction method Download PDFInfo
- Publication number
- CN116993852A CN116993852A CN202311250563.1A CN202311250563A CN116993852A CN 116993852 A CN116993852 A CN 116993852A CN 202311250563 A CN202311250563 A CN 202311250563A CN 116993852 A CN116993852 A CN 116993852A
- Authority
- CN
- China
- Prior art keywords
- image
- undersampled
- spectrum
- target
- training
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Granted
Links
- 238000000034 method Methods 0.000 title claims abstract description 79
- 238000012549 training Methods 0.000 title claims abstract description 76
- 238000001228 spectrum Methods 0.000 claims abstract description 94
- 238000003384 imaging method Methods 0.000 claims abstract description 37
- 238000005457 optimization Methods 0.000 claims abstract description 11
- 208000037170 Delayed Emergence from Anesthesia Diseases 0.000 claims description 37
- 238000012545 processing Methods 0.000 claims description 19
- 238000000605 extraction Methods 0.000 claims description 13
- 238000007781 pre-processing Methods 0.000 claims description 5
- 238000005070 sampling Methods 0.000 abstract description 4
- 230000006870 function Effects 0.000 description 17
- 230000008569 process Effects 0.000 description 15
- 239000004973 liquid crystal related substance Substances 0.000 description 12
- 230000009466 transformation Effects 0.000 description 10
- 230000004913 activation Effects 0.000 description 8
- 238000010586 diagram Methods 0.000 description 6
- 238000013507 mapping Methods 0.000 description 5
- 230000003595 spectral effect Effects 0.000 description 5
- 230000015556 catabolic process Effects 0.000 description 3
- 238000013527 convolutional neural network Methods 0.000 description 3
- 238000006731 degradation reaction Methods 0.000 description 3
- 238000005516 engineering process Methods 0.000 description 3
- 238000004364 calculation method Methods 0.000 description 2
- 238000010276 construction Methods 0.000 description 2
- 238000013135 deep learning Methods 0.000 description 2
- 238000012217 deletion Methods 0.000 description 2
- 230000037430 deletion Effects 0.000 description 2
- 230000003287 optical effect Effects 0.000 description 2
- 238000011282 treatment Methods 0.000 description 2
- 108010076504 Protein Sorting Signals Proteins 0.000 description 1
- 230000001133 acceleration Effects 0.000 description 1
- 230000006978 adaptation Effects 0.000 description 1
- 230000009286 beneficial effect Effects 0.000 description 1
- 238000006243 chemical reaction Methods 0.000 description 1
- 230000008094 contradictory effect Effects 0.000 description 1
- 230000000694 effects Effects 0.000 description 1
- 239000000284 extract Substances 0.000 description 1
- 239000012634 fragment Substances 0.000 description 1
- 238000010438 heat treatment Methods 0.000 description 1
- 238000012986 modification Methods 0.000 description 1
- 230000004048 modification Effects 0.000 description 1
- 238000010606 normalization Methods 0.000 description 1
- 230000000717 retained effect Effects 0.000 description 1
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T11/00—2D [Two Dimensional] image generation
- G06T11/001—Texturing; Colouring; Generation of texture or colour
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/04—Architecture, e.g. interconnection topology
- G06N3/0464—Convolutional networks [CNN, ConvNet]
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/08—Learning methods
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T5/00—Image enhancement or restoration
- G06T5/10—Image enhancement or restoration by non-spatial domain filtering
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/20—Special algorithmic details
- G06T2207/20048—Transform domain processing
- G06T2207/20052—Discrete cosine transform [DCT]
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/20—Special algorithmic details
- G06T2207/20081—Training; Learning
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/20—Special algorithmic details
- G06T2207/20084—Artificial neural networks [ANN]
Abstract
The invention discloses a training method of an image reconstruction model, which comprises the following steps: constructing a training set, wherein the training set comprises an undersampled image and an image spectrum corresponding to the undersampled image; inputting the undersampled image into an initial network model to obtain a target image; calculating a loss value according to the target image and an image spectrum corresponding to the target image; optimizing an initial network model according to the loss value; and when the optimization times of the initial network model reach the target value, outputting the optimized initial network model as an image reconstruction model. The invention discloses a training method of an image reconstruction model, which aims to solve the problem of poor image quality of a discrete cosine single-pixel imaging method under the condition of extremely low sampling rate. In addition, the invention also discloses a main control device and an image reconstruction method.
Description
Technical Field
The present invention relates to the field of computer imaging technologies, and in particular, to a training method for an image reconstruction model, a main control device, and an image reconstruction method.
Background
Discrete cosine single-pixel imaging is an orthogonal transformation single-pixel imaging method, and based on two-dimensional discrete cosine transformation, the discrete cosine transformation spectrum coefficient of an imaged object image is acquired, and image reconstruction is realized through two-dimensional inverse discrete cosine transformation. In discrete cosine single-pixel imaging, a group of cosine fringe structure patterns with different frequencies are loaded into a spatial light modulator, optical space information of an imaged object is modulated, signal values corresponding to the fringe patterns are detected by a single-pixel detector, discrete cosine transform coefficients of the object image are constructed according to the signal values, a discrete cosine transform spectrum is formed, and inverse transform is performed on the acquired discrete cosine transform spectrum to complete image reconstruction. Generally, the energy of the discrete cosine transform spectrum of an image, i.e., coefficient values with larger absolute values of the values, is more concentrated in the low frequency part. Therefore, the discrete cosine single-pixel imaging generally only collects a part of coefficient values with larger absolute values, and reduces the collection times, so that the time required for collecting the image is shortened, and an undersampling scheme of the image is realized.
However, when discrete cosine single pixel imaging is achieved by the undersampling scheme, the imaged image spectrum is incomplete. And due to the lack of a large number of frequency spectrum coefficients, the problems of image blurring, ringing effect and the like of the image can be caused, so that the image quality is poor.
Disclosure of Invention
The invention mainly aims to provide a training method, main control equipment and an image reconstruction method of an image reconstruction model, and aims to solve the problem of poor image quality of a discrete cosine single-pixel imaging method under the condition of extremely low sampling rate.
In order to achieve the above object, the present invention provides a training method of an image reconstruction model, the training method of the image reconstruction model comprising:
constructing a training set, wherein the training set comprises an undersampled image and an image spectrum corresponding to the undersampled image;
inputting the undersampled image into an initial network model to obtain a target image;
calculating a loss value according to the target image and an image spectrum corresponding to the target image;
optimizing the initial network model according to the loss value; and
and when the optimization times of the initial network model reach a target value, outputting the optimized initial network model as the image reconstruction model.
Preferably, calculating the loss value from the target image and the image spectrum corresponding to the target image includes:
the loss value is calculated from the discrete cosine transform spectrum of the target image and the corresponding image spectrum.
Preferably, before inputting the undersampled image into the initial network model to obtain the target image, the training method of the image reconstruction model further includes:
the method comprises the steps of constructing an initial network model, wherein the initial network model comprises a feature extraction module, a residual error module and a reconstruction module, and the residual error module comprises a plurality of residual error blocks which are sequentially connected.
Preferably, inputting the undersampled image into the initial network model to obtain the target image comprises:
inputting the undersampled image into the feature extraction module to obtain low-level features;
inputting the low-level features into the residual error module to obtain residual error features;
and inputting the residual characteristics into the reconstruction module to obtain the target image.
Preferably, inputting the low-level features into the residual module to obtain residual features includes:
inputting the low-level features into a first one of the residual blocks to obtain output features;
the low-level features and the output features are used as input features to be input into a second residual block together, and the output features and the input features of the previous residual block are input into the next residual block together from the second residual block until the last residual block outputs the high-level features according to the input features;
and calculating the residual error characteristic according to the low-layer characteristic and the high-layer characteristic.
Preferably, constructing the training set includes:
preprocessing the color image to obtain a target gray level image;
performing discrete cosine transform processing on the target gray level image to obtain the image spectrum;
setting partial spectrum coefficients in the image spectrum as preset values to obtain undersampled spectrum;
and performing inverse discrete cosine transform processing on the undersampled spectrum to obtain the undersampled image.
Preferably, preprocessing the color image to obtain the target gray-scale image includes:
converting the color image into an initial gray scale image;
normalizing the initial gray level image to obtain a normalized gray level image; and
and cutting the normalized gray level image to obtain the target gray level image.
Preferably, after performing inverse discrete cosine transform processing on the undersampled spectrum to obtain the undersampled image, the training method of the image reconstruction model further includes:
and normalizing the gray value of the undersampled image.
The invention further proposes a master control device comprising:
a memory for storing program instructions; and
and a processor for executing the program instructions to implement the training method of the image reconstruction model as described above.
The invention further provides an image reconstruction method, which comprises the following steps:
receiving a discrete cosine transform spectrum sent by a single-pixel imaging device;
performing inverse discrete cosine transform processing on the discrete cosine transform spectrum to obtain an image to be reconstructed;
inputting the image to be reconstructed into an image reconstruction model to obtain a target reconstruction image, wherein the image reconstruction model is trained by the training method of the image reconstruction model.
The technical scheme of the invention has the beneficial effects that: the method comprises the steps of constructing an initial network model into a convolutional neural network, constructing a training set with undersampled images, carrying out iterative training on the initial network model through the training set, taking the complete transformation spectrum of the undersampled images as a target, taking a trained image reconstruction model as a post-processor based on a deep learning method, enabling the image reconstruction model to have the capability of recovering the image spectrum, recovering the spectrum of the undersampled images imaged by discrete cosine single pixels as much as possible, obviously removing the quality problem of the undersampled images, and obviously improving the imaging quality of the discrete cosine single pixel imaging.
Drawings
FIG. 1 is a first flowchart of a training method of an image reconstruction model according to an embodiment of the present invention;
FIG. 2 is a second flowchart of a training method of an image reconstruction model according to an embodiment of the present invention;
FIG. 3 is a first sub-flowchart of a training method of an image reconstruction model according to an embodiment of the present invention;
FIG. 4 is a second sub-flowchart of a training method of an image reconstruction model according to an embodiment of the present invention;
FIG. 5 is a third sub-flowchart of a training method of an image reconstruction model according to an embodiment of the present invention;
FIG. 6 is a fourth sub-flowchart of a training method of an image reconstruction model according to an embodiment of the present invention;
FIG. 7 is a schematic diagram of the initial network model shown in FIG. 2;
FIG. 8 is a schematic diagram of the residual block of the initial network model shown in FIG. 7;
FIG. 9 is a schematic flow chart of constructing the training set shown in FIG. 3;
fig. 10 is a schematic diagram of an internal structure of a master control device according to an embodiment of the present invention;
FIG. 11 is a flowchart of an image reconstruction method according to an embodiment of the present invention;
fig. 12 is a flowchart of an image reconstruction method according to an embodiment of the present invention;
FIG. 13 is a first schematic view of the single pixel imaging device shown in FIG. 11;
fig. 14 is a second schematic view of the single pixel imaging device shown in fig. 11.
The achievement of the objects, functional features and advantages of the present invention will be further described with reference to the accompanying drawings, in conjunction with the embodiments.
Detailed Description
The following description of the embodiments of the present invention will be made more clearly and fully with reference to the accompanying drawings, in which it is evident that the embodiments described are only some, but not all embodiments of the invention. All other embodiments, which can be made by those skilled in the art based on the embodiments of the invention without making any inventive effort, are intended to be within the scope of the invention.
It should be noted that all directional indicators (such as up, down, left, right, front, and rear … …) in the embodiments of the present invention are merely used to explain the relative positional relationship, movement, etc. between the components in a particular posture (as shown in the drawings), and if the particular posture is changed, the directional indicator is changed accordingly.
It will also be understood that when an element is referred to as being "mounted" or "disposed" on another element, it can be directly on the other element or intervening elements may also be present. When an element is referred to as being "connected" to another element, it can be directly connected to the other element or intervening elements may also be present.
Furthermore, the description of "first," "second," etc. in this disclosure is for descriptive purposes only and is not to be construed as indicating or implying a relative importance or implicitly indicating the number of technical features indicated. Thus, a feature defining "a first" or "a second" may explicitly or implicitly include at least one such feature. In addition, the technical solutions of the embodiments may be combined with each other, but it is necessary to base that the technical solutions can be realized by those skilled in the art, and when the technical solutions are contradictory or cannot be realized, the combination of the technical solutions should be considered to be absent and not within the scope of protection claimed in the present invention.
Please refer to fig. 1, which is a first flowchart of a training method of an image reconstruction model according to an embodiment of the present invention. The training method of the image reconstruction model is used for carrying out iterative training on the initial network model to obtain the image reconstruction model, and the image reconstruction model is used for reconstructing an undersampled image obtained by discrete cosine single-pixel imaging so as to improve the definition of the image. It will be appreciated that the spectrum of the undersampled image is incomplete.
The training method of the image reconstruction model specifically comprises the following steps.
Step S102, constructing a training set.
In this embodiment, the training set includes an undersampled image and an image spectrum corresponding to the undersampled image. The undersampled images are gray-scale images with fixed sizes, and each undersampled image corresponds to an image frequency spectrum.
The specific process of how the training set is constructed will be described in detail below.
Step S104, the undersampled image is input into the initial network model to obtain a target image.
And inputting the undersampled image into an initial network model for iterative training to obtain a target image corresponding to the undersampled image. In this embodiment, the initial network model is a convolutional neural network.
The specific process of how the undersampled image is input into the initial network model to obtain the target image will be described in detail below.
Step S106, calculating a loss value according to the target image and the image spectrum corresponding to the target image.
A loss value is calculated from the target image and the corresponding image spectrum. Wherein the target images correspond to the undersampled images, and therefore each target image also corresponds to an image spectrum.
In this embodiment, the loss value is calculated from the discrete cosine transform spectrum of the target image and the corresponding image spectrum. Specifically, the loss value is calculated according to a preset loss function, and the loss function is as follows:
。
wherein, the liquid crystal display device comprises a liquid crystal display device,representing a frequency domain based loss function; />Representing an image spectrum; />A discrete cosine transform spectrum representing the target image; />Representing the spectral component coordinates. Wherein, the liquid crystal display device comprises a liquid crystal display device,；/>representing a two-dimensional discrete cosine transform; />Representing a target image; />Representing a mapping process of the initial network model; />Representing the network weights of the initial network model.
Step S108, optimizing the initial network model according to the loss value.
And optimizing the initial network model according to a preset optimizer. In this embodiment, the initial network model is optimized by minimizing the error, i.e., the loss value, between the discrete cosine transform spectrum of the target image and the corresponding image spectrum. Specifically, the optimizer optimizes the initial network model according to an objective function of:
。
wherein, the liquid crystal display device comprises a liquid crystal display device,representing the number of undersampled images simultaneously input to the initial network model.
And step S110, when the optimization times of the initial network model reach the target value, outputting the optimized initial network model as an image reconstruction model.
In this embodiment, the number of cycles of iterative training, that is, the target value, is preset. And inputting all undersampled images in the training set into an initial network model, and optimizing the initial network model into one iteration training. And (3) recording corresponding optimization times after each iteration training, and judging whether the optimization times reach a target value. When the optimization times do not reach the target value, training is continued; and when the optimization times reach the target value, outputting the optimized initial network model as an image reconstruction model.
In this embodiment, each training requires inputting all undersampled images in the training set into the initial network model and performing a feedforward calculation. Calculating the discrete cosine transform spectrum of the target image and the loss value of the corresponding image spectrum according to the loss function, and deriving and feeding back the network weight according to the gradient optimizer to optimize the parameters of the initial network model. And when the optimization times of the initial network model reach the target value, namely the iterative training times of the initial network model reach the target value, taking the initial network model optimized for the last time as an image reconstruction model.
In some embodiments, several undersampled images in the training set may be simultaneously input into the initial network model for training.
In the above embodiment, the initial network model is constructed as the convolutional neural network, and a training set with undersampled images is constructed, the initial network model is iteratively trained through the training set, the complete transformation spectrum of the undersampled images is recovered as a target, and the image reconstruction model after training is used as a post-processor based on a deep learning method, so that the image reconstruction model has the capability of recovering the image spectrum, the spectrum of the undersampled images imaged by the discrete cosine single pixels can be recovered as much as possible, the quality problem of the undersampled images is obviously removed, and the imaging quality of the discrete cosine single pixel images is obviously improved.
Referring to fig. 2, fig. 7 and fig. 8 in combination, fig. 2 is a second flowchart of a training method of an image reconstruction model according to an embodiment of the present invention, fig. 7 is a schematic structural diagram of an initial network model according to an embodiment of the present invention, and fig. 8 is a schematic structural diagram of a residual block according to an embodiment of the present invention. The training method of the image reconstruction model further includes the following steps before executing step S104.
Step S103, constructing an initial network model.
In this embodiment, the initial network model includes a feature extraction module a, a residual module B, and a reconstruction module C. The characteristic extraction module A, the residual error module B and the reconstruction module C are sequentially connected.
Specifically, the feature extraction module a includes a convolution layer and an activation function. The feature extraction module A is used for extracting low-level features of the undersampled image. Wherein the activation function may be a linear rectification function (ReLU).
Specifically, the residual error module B comprises a plurality of residual error blocks and a convolution layer, wherein the plurality of residual error blocks are sequentially connected, and the convolution layer is positioned at the output end of the residual error module B. The residual module B is used for carrying out high-level mapping on the low-level features. Wherein the input and output of the residual block B have a long jump connection. In this embodiment, each residual block includes two convolution layers and an activation function, where one convolution layer, the activation function, and the other convolution layer are sequentially connected. Wherein, the input end and the output end of the residual block are provided with a jump connection; the activation function may be a linear rectification function (ReLU).
Specifically, the reconstruction module C includes two convolution layers and an activation function, where one convolution layer, the activation function, and the other convolution layer are sequentially connected. Wherein the activation function may be a linear rectification function (ReLU).
It will be appreciated that the structure of the initial network model is not limited thereto, and in some possible embodiments, the initial network model may be other network structures capable of achieving the same function, which is not limited herein.
In the above embodiment, the feature extraction module extracts the low-level features of the undersampled image, and performs high-level mapping on the low-level features according to the residual module, learns the residual features, solves the problem of network degradation possibly caused by too deep network level, and finally completes image reconstruction through the reconstruction module.
Referring to fig. 3 and fig. 9 in combination, fig. 3 is a first sub-flowchart of a training method of an image reconstruction model according to an embodiment of the present invention, and fig. 9 is a schematic flowchart of a training set construction process according to an embodiment of the present invention. Step S102 specifically includes the following steps.
Step S202, preprocessing the color image to obtain a target gray-scale image.
In this embodiment, since the image data learned by the initial network model is a gray image, a color image is used as a basis for constructing a training set, a plurality of color images are collected, and each color image is preprocessed to obtain a corresponding target gray image. Wherein the color image may be a high definition color self-heating image. Correspondingly, the target gray image is also a clear image.
The specific process of how the color image is preprocessed to obtain the target gray image will be described in detail below.
In step S204, discrete cosine transform processing is performed on the target gray-scale image to obtain an image spectrum.
And performing discrete cosine transform processing on all the target gray images to obtain an image frequency spectrum corresponding to the target gray images. In this embodiment, the image spectrum is a discrete cosine transform spectrum obtained by performing discrete cosine transform on the target gray-scale image, and is a complete spectrum without any subtraction. The image spectrum can be used as a sample label of the corresponding target gray image.
In this embodiment, the discrete cosine transform (Discrete Cosine Transform, DCT) process can be expressed by a first formula. Specifically, the first formula is:
。
wherein, the liquid crystal display device comprises a liquid crystal display device,representing an image spectrum; />Representing discrete frequency coordinates; />Representing a two-dimensional discrete cosine transform; />Representing a target gray-scale image with a pixel size +.>；/>Representing discrete spatial coordinates. Wherein, the liquid crystal display device comprises a liquid crystal display device,
。
in step S206, a portion of the spectrum coefficients in the image spectrum are set to a preset value to obtain an undersampled spectrum.
In this embodiment, a portion of spectrum coefficients in the image spectrum are set to a preset value according to a preset number, so as to obtain an undersampled spectrum corresponding to the image spectrum. Wherein. The preset value is 0. It will be appreciated that another portion of the spectral coefficients in the image spectrum are acquired and retained. Specifically, the reserved spectral coefficients are the larger-valued portions. That is, the undersampled spectrum is a residual spectrum obtained by setting a part of spectral coefficients in the complete spectrum to 0. Accordingly, the undersampled spectrum is a discrete cosine transform spectrum. The undersampling process of the image spectrum to obtain undersampled spectrum occurs in the frequency domain, and the undersampling process is to delete the spectrum coefficient of the image spectrum, namely, the value corresponding to the spectrum coefficient is replaced by 0.
Step S208, discrete cosine inverse transformation processing is performed on the undersampled spectrum to obtain an undersampled image.
And performing inverse discrete cosine transform processing on the undersampled spectrum, and simulating an undersampling process of discrete cosine single-pixel imaging to obtain a corresponding undersampled image. In this embodiment, since a part of the spectrum coefficients in the undersampled spectrum are missing, that is, the spectrum coefficients are 0, the undersampled image obtained by the undersampled spectrum inverse transformation is blurred. Wherein the undersampled image is of a size consistent with the size of the target gray scale image.
In this embodiment, the inverse discrete cosine transform process can be expressed by a second formula. Specifically, the second formula is:
。
wherein, the liquid crystal display device comprises a liquid crystal display device,representing an undersampled image; />Representing an undersampled spectrum; />Representing a two-dimensional inverse discrete cosine transform.
In this embodiment, after performing inverse discrete cosine transform processing on the undersampled spectrum to obtain an undersampled image, the training method of the image reconstruction model further includes: and carrying out normalization processing on the gray value of the undersampled image.
Specifically, the gray values of the undersampled image are normalized from the [0, 255] range to the [0,1] range.
Please refer to fig. 4 in combination, which is a second sub-flowchart of the training method of the image reconstruction model according to an embodiment of the present invention. Step S202 specifically includes the following steps.
Step S302, converting the color image into an initial gray scale image.
And carrying out graying treatment on the color image to obtain a corresponding initial gray image.
Step S304, the initial gray level image is normalized to obtain a normalized gray level image.
In this embodiment, the gray value of the initial gray image is normalized from the [0, 255] range to the [0,1] range, so as to obtain a corresponding normalized gray image.
Step S306, the normalized gray-scale image is cut to obtain a target gray-scale image.
And slicing the normalized gray level image to obtain a target gray level image. Taking 256×256 pixel imaging resolution as an example, the normalized gray-scale image is sliced into 256×256 pixel fragments, i.e., the target gray-scale image.
In this embodiment, a clear, undegraded target gray image is obtained by graying and slicing the color image.
In the above embodiment, since the image data of the initial network model is learned to be a gray image, the imaging resolution of single-pixel imaging is usually a gray image with a fixed resolution, and a large number of gray images are difficult to obtain, so that a color image is used as a basis for constructing a training set, and a large number of color images are converted to obtain a gray image, that is, the color image is converted, normalized and cut to obtain a target gray image. That is, since the image imaged by the discrete cosine single pixel is a gray-scale image with a fixed size, both the gray-scale conversion and the slicing operation are performed on the color image in the process of constructing the data set in order to satisfy the characteristics of the image imaged by the discrete cosine single pixel. The normalized gray level image is cut into a target gray level image with a fixed size, and then a series of treatments such as discrete cosine transform, spectral coefficient deletion and discrete cosine inverse transform are carried out on the target gray level image, so that the image data participating in the training process comprises an undersampled image and an intact spectrum without deletion, namely an image spectrum.
In this embodiment, except for the color image, the initial gray image, and the normalized gray image, the sizes of the remaining images are all fixed and the same throughout the training set construction and the initial network model training.
Please refer to fig. 5 in combination, which is a third sub-flowchart of a training method of an image reconstruction model according to an embodiment of the present invention. Step S104 specifically includes the following steps.
In step S402, the undersampled image is input to a feature extraction module to obtain low-level features.
And inputting the undersampled image into a feature extraction module A, and carrying out feature extraction on the undersampled image by the feature extraction module A to obtain the low-level features of the undersampled image.
In step S404, the low-level features are input to the residual module to obtain residual features.
And inputting the low-level features into a residual error module B, and performing high-level mapping on the low-level features by the residual error module B to obtain residual error features.
The specific process of how the low-level features are input into the residual module to obtain the residual features will be described in detail below.
In step S406, the residual feature is input to the reconstruction module to obtain the target image.
And inputting the residual characteristics into a reconstruction module C, completing image reconstruction by the reconstruction module C according to the residual characteristics, and finally outputting a high-quality clear image, namely a target image.
Please refer to fig. 6 in combination, which is a fourth sub-flowchart of a training method of an image reconstruction model according to an embodiment of the present invention. Step S404 specifically includes the following steps.
In step S502, the low-level features are input into the first residual block to obtain output features.
In this embodiment, after the low-level features are input into the residual error module B, the low-level features are input into the first residual error block, and the first residual error block maps the low-level features to obtain the output features.
In step S504, the low-level features and the output features are input together as input features into the second residual block, and from the second residual block, the output features and the input features of the previous residual block are input together into the subsequent residual block until the last residual block outputs the high-level features according to the input features.
In this embodiment, the low-level feature input to the first residual block and the output feature output from the first residual block are used together as input features and input to the second residual block. The second residual block maps the input features to obtain output features. Correspondingly, the input characteristic of the second residual block and the output characteristic of the second residual block are taken as new input characteristics together, and are input into a third residual block, and so on. That is, starting from the second residual block, the input feature input to the previous residual block and the output feature output from the previous residual block are input to the next residual block together as new input features. And mapping the input features by the last residual block to finally obtain high-level features.
Step S506, calculating residual characteristics according to the low-layer characteristics and the high-layer characteristics.
In this embodiment, the sum of the low-level features and the high-level features is calculated as the residual feature. Specifically, the long jump connection of the residual module B may add the low layer feature and the high layer feature as residual features, so that the residual module B learns the residual features, and solves the problem of network degradation that may be caused by too deep a network hierarchy.
In the above embodiment, the skip connection of the residual block may add the input feature and the output feature as the input feature of the next residual block, and may solve the problem of network degradation that may be caused by the network hierarchy being too deep.
Please refer to fig. 10 in combination, which is a schematic diagram illustrating an internal structure of a master control device according to an embodiment of the present invention. The master device 10 includes a memory 11 and a processor 12. The memory 11 is used for storing program instructions and the processor 12 is used for executing the program instructions to implement the training method of the image reconstruction model described above.
The processor 12 may be, in some embodiments, a central processing unit (Central Processing Unit, CPU), controller, microcontroller, microprocessor or other data processing chip for executing program instructions stored in the memory 11.
The memory 11 includes at least one type of readable storage medium including flash memory, a hard disk, a multimedia card, a card memory (e.g., SD or DX memory, etc.), magnetic memory, magnetic disk, optical disk, etc. The memory 11 may in some embodiments be an internal storage unit of a computer device, such as a hard disk of a computer device. The memory 11 may in other embodiments also be an external storage device of the computer device, such as a plug-in hard disk, a Smart Media Card (SMC), a Secure Digital (SD) Card, a Flash memory Card (Flash Card) or the like, which are provided on the computer device. Further, the memory 11 may also include both an internal storage unit and an external storage device of the computer device. The memory 11 may be used not only for storing application software installed in a computer device and various types of data, such as code for implementing a training method of an image reconstruction model, but also for temporarily storing data that has been output or is to be output.
Referring to fig. 11 and 12 in combination, fig. 11 is a flowchart of an image reconstruction method according to an embodiment of the present invention, and fig. 12 is a flowchart of an image reconstruction method according to an embodiment of the present invention. The image reconstruction method is used for reconstructing an undersampled image obtained by discrete cosine single-pixel imaging so as to obtain a clear target reconstruction image. The Shan Xiangsu imaging is a novel calculation imaging method, a group of time-varying structure pattern sequences with coding information are loaded by using a spatial light modulation technology to modulate and code the spatial information of reflected light of an imaging object, then a single-pixel detector without spatial resolution capability, such as a photodiode and the like, is used for detecting for a plurality of times to obtain a group of signal value sequences corresponding to the structure pattern sequences one by one, and the structure patterns and the sampled signal sequences are combined through a certain algorithm to complete imaging. Common single-pixel imaging methods include compressed sensing methods based on random pattern sampling, orthogonal transformation single-pixel imaging methods based on orthogonal base pattern sampling, and the like. The discrete cosine single-pixel imaging adopted in the present embodiment is an orthogonal transformation single-pixel imaging method.
The image reconstruction method specifically comprises the following steps.
In step S602, a discrete cosine transform spectrum transmitted by the single-pixel imaging device is received.
In this embodiment, the single-pixel imaging device samples the imaging object to obtain the discrete cosine transform spectrum. Specifically, as shown in fig. 13 and 14, fig. 13 is a structured light imaging scheme in which a structured pattern is projected onto an imaging object for active modulation; fig. 14 is a single pixel camera scheme using a spatial light modulator for passive modulation of spatial information of light reflected by an imaged object.
Step S604, performing inverse discrete cosine transform processing on the discrete cosine transform spectrum to obtain an image to be reconstructed.
And performing inverse discrete cosine transform processing on the acquired discrete cosine transform spectrum to obtain an image to be reconstructed. Wherein the image to be reconstructed is an undersampled image.
Step S606, inputting the image to be reconstructed into an image reconstruction model to obtain a target reconstructed image.
Inputting the image to be reconstructed into an image reconstruction model, and performing image reconstruction on the image to be reconstructed by the image reconstruction model to obtain a target reconstruction image. The image reconstruction model is obtained by training the training method of the image reconstruction model.
In the above embodiment, according to the image reconstruction model with the capability of recovering the image spectrum, the image to be reconstructed is reconstructed, and the target reconstructed image reconstructed by the image reconstruction model can obviously remove the quality problem existing in the image to be reconstructed, thereby obviously improving the imaging quality of discrete cosine single-pixel imaging. The trained image reconstruction model is mostly linear operation in the reasoning process of the reconstructed image, and the whole operation process can be controlled to be completed in millisecond level by utilizing the parallel acceleration operation of the GPU, so that the image reconstruction model can still hardly obviously increase the operation time under the condition of greatly improving the imaging quality.
In some possible embodiments, the image reconstruction method may further perform iterative optimization solution on the image to be reconstructed by using an optimization solver in compressed sensing, so as to implement image reconstruction.
The above description of the preferred embodiments of the present invention should not be taken as limiting the scope of the invention, but rather should be understood to cover all modifications, variations and adaptations of the present invention using its general principles and the following detailed description and the accompanying drawings, or the direct/indirect application of the present invention to other relevant arts and technologies.
Claims (9)
1. A method for training an image reconstruction model, the method comprising:
constructing a training set, wherein the training set comprises an undersampled image and an image spectrum corresponding to the undersampled image;
inputting the undersampled image into an initial network model to obtain a target image;
calculating a loss value according to the target image and an image spectrum corresponding to the target image;
optimizing the initial network model according to the loss value; and
when the optimization times of the initial network model reach a target value, outputting the optimized initial network model as the image reconstruction model;
wherein calculating a loss value from the target image and an image spectrum corresponding to the target image includes:
the loss value is calculated from the discrete cosine transform spectrum of the target image and the corresponding image spectrum.
2. The method of training an image reconstruction model according to claim 1, wherein before inputting the undersampled image into an initial network model to obtain a target image, the method of training an image reconstruction model further comprises:
the method comprises the steps of constructing an initial network model, wherein the initial network model comprises a feature extraction module, a residual error module and a reconstruction module, and the residual error module comprises a plurality of residual error blocks which are sequentially connected.
3. The method of training an image reconstruction model according to claim 2, wherein inputting the undersampled image into an initial network model to obtain a target image comprises:
inputting the undersampled image into the feature extraction module to obtain low-level features;
inputting the low-level features into the residual error module to obtain residual error features;
and inputting the residual characteristics into the reconstruction module to obtain the target image.
4. A method of training an image reconstruction model according to claim 3, wherein inputting the low-level features into the residual module to obtain residual features comprises:
inputting the low-level features into a first one of the residual blocks to obtain output features;
the low-level features and the output features are used as input features to be input into a second residual block together, and the output features and the input features of the previous residual block are input into the next residual block together from the second residual block until the last residual block outputs the high-level features according to the input features;
and calculating the residual error characteristic according to the low-layer characteristic and the high-layer characteristic.
5. The method of training an image reconstruction model according to claim 1, wherein constructing a training set comprises:
preprocessing the color image to obtain a target gray level image;
performing discrete cosine transform processing on the target gray level image to obtain the image spectrum;
setting partial spectrum coefficients in the image spectrum as preset values to obtain undersampled spectrum;
and performing inverse discrete cosine transform processing on the undersampled spectrum to obtain the undersampled image.
6. The method of claim 5, wherein preprocessing the color image to obtain the target gray scale image comprises:
converting the color image into an initial gray scale image;
normalizing the initial gray level image to obtain a normalized gray level image; and
and cutting the normalized gray level image to obtain the target gray level image.
7. The method according to claim 5, wherein after performing inverse discrete cosine transform processing on the undersampled spectrum to obtain the undersampled image, the method further comprises:
and normalizing the gray value of the undersampled image.
8. A master device, the master device comprising:
a memory for storing program instructions; and
a processor for executing the program instructions to implement the training method of the image reconstruction model as claimed in any one of claims 1 to 7.
9. An image reconstruction method, characterized in that the image reconstruction method comprises:
receiving a discrete cosine transform spectrum sent by a single-pixel imaging device;
performing inverse discrete cosine transform processing on the discrete cosine transform spectrum to obtain an image to be reconstructed;
inputting the image to be reconstructed into an image reconstruction model to obtain a target reconstructed image, the image reconstruction model being trained by the training method of the image reconstruction model according to any one of claims 1 to 7.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202311250563.1A CN116993852B (en) | 2023-09-26 | 2023-09-26 | Training method of image reconstruction model, main control equipment and image reconstruction method |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202311250563.1A CN116993852B (en) | 2023-09-26 | 2023-09-26 | Training method of image reconstruction model, main control equipment and image reconstruction method |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN116993852A true CN116993852A (en) | 2023-11-03 |
| CN116993852B CN116993852B (en) | 2024-01-30 |
Family
ID=88523528
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202311250563.1A Active CN116993852B (en) | 2023-09-26 | 2023-09-26 | Training method of image reconstruction model, main control equipment and image reconstruction method |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN116993852B (en) |
Citations (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| WO2020134826A1 (en) * | 2018-12-24 | 2020-07-02 | 深圳先进技术研究院 | Parallel magnetic resonance imaging method and related equipment |
| CN111696168A (en) * | 2020-06-13 | 2020-09-22 | 中北大学 | High-speed MRI reconstruction method based on residual self-attention image enhancement |
| CN114998458A (en) * | 2021-11-29 | 2022-09-02 | 厦门理工学院 | Undersampled magnetic resonance image reconstruction method based on reference image and data correction |
-
2023
- 2023-09-26 CN CN202311250563.1A patent/CN116993852B/en active Active
Patent Citations (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| WO2020134826A1 (en) * | 2018-12-24 | 2020-07-02 | 深圳先进技术研究院 | Parallel magnetic resonance imaging method and related equipment |
| CN111696168A (en) * | 2020-06-13 | 2020-09-22 | 中北大学 | High-speed MRI reconstruction method based on residual self-attention image enhancement |
| CN114998458A (en) * | 2021-11-29 | 2022-09-02 | 厦门理工学院 | Undersampled magnetic resonance image reconstruction method based on reference image and data correction |
Also Published As
| Publication number | Publication date |
|---|---|
| CN116993852B (en) | 2024-01-30 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN102693528B (en) | Noise suppressed in low light images | |
| CN112329702B (en) | Method and device for rapid face density prediction and face detection, electronic equipment and storage medium | |
| US9535928B2 (en) | Combining information of different levels for content-based retrieval of digital pathology images | |
| CN113284037B (en) | Ceramic watermark carrier recovery method based on deep neural network | |
| CN105338219B (en) | Video image denoising method and apparatus | |
| CN111429371B (en) | Image processing method and device and terminal equipment | |
| Wu et al. | An image authentication and recovery system based on discrete wavelet transform and convolutional neural networks | |
| CN114529793A (en) | Depth image restoration system and method based on gating cycle feature fusion | |
| CN116993852B (en) | Training method of image reconstruction model, main control equipment and image reconstruction method | |
| CN110766708B (en) | Image comparison method based on contour similarity | |
| CN117152182A (en) | Ultralow-illumination network camera image processing method and device and electronic equipment | |
| CN117191816A (en) | Method and device for detecting surface defects of electronic component based on multispectral fusion | |
| Xiao et al. | Effective PRNU extraction via densely connected hierarchical network | |
| Guo et al. | Image blind deblurring using an adaptive patch prior | |
| CN109493280B (en) | Image processing method, device, terminal and storage medium | |
| Zhang et al. | Feature compensation network based on non-uniform quantization of channels for digital image global manipulation forensics | |
| CN114630130B (en) | Face-changing video tracing method and system based on deep learning | |
| CN116385281A (en) | Remote sensing image denoising method based on real noise model and generated countermeasure network | |
| CN116228899A (en) | Single-pixel imaging method and device combining supervision and unsupervised learning | |
| CN113744158A (en) | Image generation method and device, electronic equipment and storage medium | |
| CN114677584A (en) | Water immersion identification method and system for power distribution station of double-attention power system | |
| KR20220079125A (en) | System and method for semi-supervised single image depth estimation and computer program for the same | |
| CN113240589A (en) | Image defogging method and system based on multi-scale feature fusion | |
| Xu et al. | A deep neural network for coarse-to-fine image dehazing with interleaved residual connections and semi-supervised training | |
| CN116091758B (en) | Image processing method, device, equipment and storage medium based on detail extraction |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |