WO2021134874A1

WO2021134874A1 - Training method for deep residual network for removing a moire pattern of two-dimensional code

Info

Publication number: WO2021134874A1
Application number: PCT/CN2020/076819
Authority: WO
Inventors: 陈昌盛; 陆涵; 黄继武
Original assignee: 深圳大学
Priority date: 2019-12-31
Filing date: 2020-02-26
Publication date: 2021-07-08
Also published as: CN111340729A; CN111340729B

Abstract

A training method for a deep residual network for removing a moire pattern of a two-dimensional code, comprising: preparing an original two-dimensional code image having a moire pattern (S100); inputting the original two-dimensional code image into a preprocessing module, and then performing down-sampling processing to form a zoomed-out preprocessed image (S300); inputting the preprocessed image into a first residual module for up-sampling processing to form a first output image of which the image size is zoomed in to the size of the original two-dimensional code image (S400); inputting the first output image into a second residual module to form a second output image for recovering lost image information of the first output image (S500); and performing feature fusion on the second output image and the original two-dimensional code image to form a feature fusion image, and inputting the feature fusion image into a third residual module to perform purification processing to form a moire pattern-removed image in which a moire pattern is removed (S600). Therefore, the moire pattern in an original two-dimensional code image can be removed more effectively.

Description

Training method of deep residual network for removing moiré in QR code

Technical field

The present disclosure generally relates to a training method of a deep residual network for removing moiré in a two-dimensional code.

Background technique

At present, there are three commonly used methods to remove moiré in images. Multiphase component layer decomposition technology (LDPC) is used to remove the moiré in the image captured by the camera; the wavelet domain filtering method is used to remove the moiré in the scanned image; The image restoration method realizes the moiré removal.

The above three methods can remove the moiré in the image to varying degrees, but they also have the following three limitations:

(1) Reduce the versatility of the moiré removal technology. The use of multi-phase component layer decomposition technology can achieve the removal of moiré on the image taken by the camera. But in fact, this technology can only remove a small amount of moiré interference, and cannot play a significant role in images with large-scale moiré interference, especially for moiré color bands, and will eventually smooth image details. The wavelet domain filtering can remove the moiré in the scanned image, but the moiré must have a mesh structure. Using traditional image restoration methods or improving DnCNN (Denoising Convolutional Neural Network) to achieve moiré removal, because these models are not specifically designed for moiré removal problems, they can only achieve sub-optimal performance. In fact, the moiré pattern is anisotropic and varies randomly. The above-mentioned technology can only solve the problem of removing the moiré pattern for a specific moiré pattern, which has certain limitations.

(2) The impact on the original image cannot be completely avoided. Multi-phase component layer decomposition technology is used to solve the problem of moiré removal, the removal effect is not obvious and the details of the original image will be smoothed, that is, the restored image has a certain degree of distortion. Using image restoration methods such as reference denoising adaptive encoder to remove moiré, while removing the moiré in the high-frequency range, the high-frequency components of the original image will also be removed. Because the image loses high-frequency information, there will also be a certain amount of blur. None of the above technologies can completely avoid the moiré removal without affecting the original image content.

(3) Increased use complexity. Because the moiré is anisotropic and non-uniform. For an image with various moiré patterns, we have to combine a variety of existing technologies to make them work for different moiré patterns. This approach increases the complexity of the moiré removal problem, requires multiple steps to achieve the moiré removal, and cannot achieve a better removal effect.

Summary of the invention

In view of the above-mentioned existing conditions, the present invention aims to provide a training method for a deep residual network that can effectively remove moiré in a two-dimensional code.

To this end, the present disclosure provides a method for training a deep residual network for removing moiré in a two-dimensional code, which includes: preparing an original two-dimensional code image with moiré; preparing to process the original two-dimensional code The network device includes a preprocessing module, a first residual module, a second residual module, and a third residual module; the original two-dimensional code image is input into the preprocessing module, and the original The two-dimensional code image is blurred to increase the pixels in the two-dimensional code area in the simulated two-dimensional code image, and then down-sampling is performed to form a pre-processed image with reduced image; then the pre-processed image is input to the first residual image The difference module performs up-sampling processing to form a first output image whose image size is enlarged to the size of the original two-dimensional code image; then the first output image is input to the second residual module to form a restoration of the first output image. Output the second output image of the missing image information of the image; and then perform feature fusion of the second output image and the original two-dimensional code image to form a feature fusion image, and then input the feature fusion image into the third The residual module performs a purification process to form a moiré-removed image.

In the present disclosure, the pre-processed image is processed by the first residual module, the second residual module and the third residual module in the deep residual network, which can conveniently and effectively remove the moire in the original two-dimensional code image. Pattern.

In addition, in the training method of the deep residual network for removing the moiré of the two-dimensional code involved in the present disclosure, optionally, the first residual module includes a convolution kernel connected in sequence with a size of 3×3 and 64 The first convolutional layer with feature maps, the first ReLU activation layer, the 16 residual blocks in series, the second convolutional layer with 64 feature maps and the batch normalization layer with a convolution kernel size of 3×3 , The third convolutional layer with a convolution kernel size of 3×3 and 256 feature maps, a fourth convolution layer with a convolution kernel size of 1×1 and 3 feature maps, and a tanh activation layer. As a result, the moiré in the preprocessed image can be removed more effectively.

In addition, in the training method of the deep residual network for removing the moiré of the two-dimensional code involved in the present disclosure, optionally, the second residual module includes 10 successively connected layers consisting of a convolution kernel with a size of 5×5. And the convolutional layer with 64 feature maps and the second ReLU activation layer are concatenated into the first combined layer and 10 layers are activated by the convolutional layer with a convolution kernel size of 3×3 and 64 feature maps and the third ReLU The second combination layer formed by series connection. In this way, it is possible to restore the image information lost in the first output image.

In addition, in the training method of the deep residual network for removing the moiré of the two-dimensional code involved in the present disclosure, optionally, the third residual module includes a convolution kernel connected in sequence with a size of 3×3 and 128 The fifth convolutional layer with three feature maps, the fourth ReLU activation layer, and the sixth convolutional layer with a convolution kernel size of 3×3 and three feature maps. Thus, the moiré in the second output image can be effectively removed.

In addition, in the training method of the deep residual network for removing the moiré of the two-dimensional code involved in the present disclosure, optionally, the original two-dimensional code image is a synthesized simulated two-dimensional code image. As a result, the original two-dimensional code image can be generated more conveniently.

In addition, in the training method of the deep residual network for removing the moiré of the two-dimensional code involved in the present disclosure, optionally, the formation of the simulated two-dimensional code image includes the following steps: re-sampling the input image to generate a A mosaic composed of RGB pixels and displayed on the display; random projection transformation is used to simulate the different relative positions and directions between the display and the camera; the radiation distortion function is used to simulate the distortion of the camera lens; a flat top is used Gaussian filter to simulate anti-aliasing filtering; re-sampling the input image to simulate the input of the camera sensor; adding Gaussian noise to the input image to simulate sensor noise; demosaicing processing; using a denoising filter for processing Denoising processing; compressing the input image; and outputting a decompressed image to form the simulated two-dimensional code image. As a result, the simulated two-dimensional code image can be generated more conveniently.

In addition, in the training method of the deep residual network for removing the moiré of the two-dimensional code involved in the present disclosure, optionally, the two-dimensional code image on the display corresponding to the input image uses the same The projection transformation and the lens distortion function are processed. As a result, it is possible to easily obtain the two-dimensional code image on the display corresponding to the input image one-to-one.

In addition, in the training method of the deep residual network for removing the moiré pattern of the two-dimensional code involved in the present disclosure, optionally, the mean square error function is used

Train the deep residual network as a loss function, where M and N are the height and width of the simulated two-dimensional code image, H(G(I)) is the two-dimensional code image output by the deep residual network, and J is and The two-dimensional code image on the display corresponding to the simulated two-dimensional code image is stored, and the model and parameters of the deep residual network when the training loss is the least are stored. Therefore, the deep residual network can be effectively trained.

In addition, in the training method of the deep residual network for removing the moiré of the two-dimensional code involved in the present disclosure, optionally, it also includes a real two-dimensional code image with moiré, based on the minimum training loss. The model and parameters of the deep residual network, and use the real two-dimensional code image to perform a migration learning operation on the deep residual network. As a result, the deep residual network can be trained more effectively.

In addition, in the training method of the deep residual network for removing the moiré of the two-dimensional code involved in the present disclosure, optionally, the angle transformation is performed on the real two-dimensional code image to obtain the same image as the real two-dimensional code image. A corresponding two-dimensional code image on the display. In this way, it is possible to obtain the two-dimensional code image on the display corresponding to the real two-dimensional code image one-to-one.

Description of the drawings

The embodiments of the present disclosure will now be explained in further detail only by referring to the examples of the accompanying drawings, in which:

FIG. 1 is a schematic diagram showing the modules of the deep residual network involved in this embodiment.

FIG. 2 is a schematic diagram showing the specific structure of the deep residual network involved in this embodiment.

FIG. 3 is a schematic flowchart showing a method for training a deep residual network for removing moiré in a two-dimensional code according to this embodiment.

FIG. 4(a) is a schematic diagram showing the synthesized simulated two-dimensional code image related to this embodiment, and FIG. 4(b) is a diagram showing the de-moiré image related to FIG. 4(a) related to this embodiment. .

FIG. 5(a) is a schematic diagram showing a captured image according to this embodiment, FIG. 5(b) is a schematic diagram showing a reconstructed image according to this embodiment, and FIG. 5(c) is a schematic diagram showing the image The embodiment relates to the de-moiré image of FIG. 5(a).

Symbol Description:

Network device...1, preprocessing module...10, first residual module...20, second residual module...30, first combination layer...31, second combination layer...32, third residual module...40.

Detailed ways

Hereinafter, with reference to the drawings, preferred embodiments of the present disclosure will be described in detail. In the following description, the same symbols are assigned to the same components, and repeated descriptions are omitted. In addition, the drawings are only schematic diagrams, and the ratio of dimensions between components or the shapes of components may be different from actual ones.

FIG. 1 is a schematic diagram showing a module of a network device 1 related to this embodiment. FIG. 2 is a schematic diagram showing a specific structure of the network device 1 according to this embodiment. FIG. 3 is a schematic flowchart showing a training method for removing moiré from a two-dimensional code based on the network device 1 according to this embodiment.

Using digital cameras or smartphones to take photos of photoelectric displays (or display screens) can facilitate us to record and transmit data. However, because the pixel grid between the camera sensor and the display device cannot be completely matched, the captured images often have moiré patterns. Pattern interference. The existing de-moiring technologies mostly process natural images, and the effect is not particularly ideal. It is a hot issue to construct a display-shooting communication channel based on the image two-dimensional code, but the moiré distortion seriously hinders the effectiveness of this type of communication system. After investigation, it is found that there is no moiré removal technology proposed for two-dimensional codes.

In view of the above-mentioned problems, the present disclosure designs a training method of a deep residual network based on the two-dimensional code moiré removal of the network device 1. 1 to 3, the network device 1 of the present disclosure may include a preprocessing module 10, a first residual module 20, a second residual module 30, and a third residual module 40, the preprocessing module 10, the first residual module The module 20, the second residual module 30, and the third residual module 40 are connected in series in sequence.

The training method of the deep residual network for removing the moiré of the two-dimensional code of the present disclosure includes the following steps: preparing an original two-dimensional code image I with moiré (step S100); preparing a network device for processing the original two-dimensional code , The network device includes a preprocessing module, a first residual module, a second residual module, and a third residual module (step S200); the original two-dimensional code image I is input into the preprocessing module 10, and the original two-dimensional code image I perform blur processing to increase the pixels in the QR code area in the simulated two-dimensional code image, and then perform down-sampling processing to form a reduced image preprocessed image I'(step S300); then input the preprocessed image I'to the first The residual module 20 performs up-sampling processing to form a first output image whose image size is enlarged to the size of the original two-dimensional code image I (step S400); then the first output image is input to the second residual module 30 to form a restored first output image. Output the second output image of the missing image information of the image (step S500); and then perform feature fusion of the second output image and the original two-dimensional code image I to form a feature fusion image, and then input the feature fusion image to the third residual module 40 performs a purification process to form a moiré-removed image (step S600).

In the present disclosure, the first residual module 20, the second residual module 30, and the third residual module 40 in the network device 1 process the preprocessed image I', which can easily and effectively remove the original two-dimensional code. Moiré in image I.

In some examples, the lost image information of the first output image may be, for example, the lack of pixel points, the loss of pixel coordinates, and the like.

In some examples, the first residual module 20 may include a first convolutional layer with a convolution kernel size of 3×3 and 64 feature maps, a first ReLU (modified linear unit) activation layer, and a concatenated 16 residual blocks, a second convolutional layer with a convolution kernel size of 3×3 and 64 feature maps, a batch normalization layer (Batch Norm), a convolution kernel size of 3×3 and 256 features The mapped third convolutional layer, the fourth convolutional layer with the size of the convolution kernel of 1×1 and 3 feature maps, and the tanh (hyperbolic tangent) activation layer. In this case, the first residual module 20 can be used to remove moiré in the pre-processed image I′ that is significantly different from the pre-processed image I′, such as some colored band-shaped moiré. Of course, the size of the convolution kernel and the number of feature maps in the first residual module 20 are not fixed, and can be adjusted according to different network devices 1 to be trained, and there is no limitation here.

In some examples, the second residual module 30 may include 10 successively connected layers, a first combination layer composed of a convolutional layer with a convolution kernel size of 5×5 and 64 feature maps and a second ReLU activation layer in series. The 31 and 10 layers are a second combination layer 32 composed of a convolutional layer with a convolution kernel size of 3×3 and 64 feature maps and a third ReLU activation layer in series. That is, each layer of the 10-layer first combination layer 31 can be composed of a convolutional layer with a size of 5×5 and 64 feature maps and a second ReLU activation layer in series, and then 10 layers In the same way, each layer of the 10-layer second combination layer 32 can be composed of a convolutional layer with a size of 3×3 and 64 feature maps and a layer The third ReLU activation layer is connected in series, and then each of the 10 layers is connected in series to form the second combined layer 32. In this case, the second residual module 30 can be used to restore the lost detail information of the first output image. Of course, the size of the convolution kernels and the number of feature maps in the first combination layer 31 and the second combination layer 32 are not fixed, and can be adjusted according to the network device 1 to be trained, and there is no limitation here.

In some examples, the third residual module 40 may include a fifth convolution layer with a convolution kernel size of 3×3 and 128 feature maps, a fourth ReLU activation layer, and a convolution kernel size of 3×3. 3 and a sixth convolutional layer with 3 feature maps. Thus, the moiré in the second output image can be effectively removed. The input to the third residual module 40 is the second output image and the original two-dimensional code image I, that is, the second output image and the original two-dimensional code image I are feature-fused and input to the third residual module 40. In this case, two convolutional layers can be used to purify the second output image and the original two-dimensional code image I to further remove some comparative rules and moiré that is not easily separated from the original two-dimensional code image I. The final moiré-removed de-moiré image is obtained.

FIG. 4(a) is a schematic diagram showing a synthesized pseudo two-dimensional code image according to this embodiment, and FIG. 4(b) is a diagram showing a de-moiré image related to figure (a) according to this embodiment.

4(a) and 4(b), in some examples, the original two-dimensional code image I is a synthesized simulated two-dimensional code image. As a result, the original two-dimensional code image I can be generated more conveniently.

It is understandable that, in order to make the network device 1 of the present disclosure more accurately recognize the moiré in the real environment, ideally we can only use the image pair composed of the real shot image and the image on the corresponding display for training. The network device1. However, it is more difficult to obtain a pair of images that exactly match in space. It is easy for the network to misidentify unmatched edges as moiré, and there are many common problems in real images: lens distortion, camera shake, etc. Seriously affect the alignment of the image pair. Considering the difficulty of using real images to construct high-quality large image sets, we use simulated two-dimensional code images with real moiré to train the network. In order to simulate the moiré generation process more accurately, we strictly implement the entire process from the image display on the LCD monitor to the use of the camera to capture the image and then to the digital processing inside the camera.

In some instances, the formation of a simulated two-dimensional code image includes the following steps: re-sampling the input image to generate a mosaic composed of RGB pixels, and display it on the display; random projection transformation to simulate the difference between the display and the camera The relative position and direction of the camera; use the radiation distortion function to simulate the distortion of the camera lens; use a flat-top Gaussian filter to simulate anti-aliasing filtering; resample the input image to simulate the input of the camera sensor; add Gaussian noise to the input image To simulate sensor noise; demosaicing; using denoising filters for denoising; compressing the input image; and outputting the decompressed image to form a simulated two-dimensional code image. As a result, the simulated two-dimensional code image can be generated more conveniently.

In some examples, bayer CFA may be used to resample the input image to simulate the input of the camera sensor.

In addition, in some examples, the input image may be compressed in JPEG, TIFF, RAW, and other formats.

In some examples, the two-dimensional code image on the display corresponding to the input image can be processed using the same projection transformation and lens distortion function. Since it is necessary to simulate a one-to-one correspondence between the two-dimensional code image and the two-dimensional code image on the display to train the network device 1, the same projection transformation and lens distortion function can be used to process the two-dimensional code image on the display .

In some examples, the mean square error function can be used

Train the network device 1 as a loss function, where M and N are the height and width of the simulated two-dimensional code image, H(G(I')) is the two-dimensional code image output by the network device 1, and J is the same as the simulated two-dimensional code image Corresponding to the two-dimensional code image on the display, and save the model and parameters of the network device 1 when the training loss is the smallest. As a result, the network device 1 can be effectively trained.

In some examples, the loss function can also be implemented by a loss function, or a cross-entropy loss function, an exponential loss function, or a Hinge loss function (SVM), which can be selected according to actual conditions in specific applications, and there is no limitation here.

In this embodiment, as described above, the simulated two-dimensional code image can be input to the network device 1 as the original image I for training, so as to obtain the corresponding moiré image (as shown in FIG. 4(b)). As a result, it is possible to reduce the difficulty caused by a real two-dimensional code image using a large amount of data.

In some examples, in order to achieve the moiré removal of the two-dimensional code, the network device 1 model proposed in the present disclosure can be divided into two stages: (a) A large number of simulated two-dimensional code images with synthetic moiré are used to pair the network. Carry out pre-training (refer to the training of the simulated two-dimensional code image as above), so that the network can play the performance of removing moiré for the two-dimensional code. (b) Considering the characteristics of the moiré in the real environment, a relatively small amount of real-photographed real two-dimensional code images with moiré can be used in training for migration learning (Fine-tune) based on the network device 1, so that the network can target the real image The moiré pattern has a more ideal removal effect. The network device 1 trained in two stages can achieve good de-moiring performance on real-photographed two-dimensional code images with moiré.

Therefore, in some examples, the present disclosure may also include a real two-dimensional code image with moiré, based on the model and parameters of the network device 1 when the training loss is minimal, and use the real two-dimensional code image to perform the network device 1 Transfer learning operation. As a result, the network device 1 can be trained more effectively.

The migration learning operation is detailed below.

After pre-training the network with the simulated two-dimensional code image, the network has the function of removing moiré, but the simulated two-dimensional code image cannot fully simulate the situation in the real environment, so it is necessary to use the real two-dimensional code image captured The network is fine-tuned to enable the network to exert practical effects while enhancing the robustness of the network. The model and parameters at the time of the minimum loss during the pre-training process are saved, and on this basis, the real two-dimensional code image with moiré is used to perform the migration learning operation on the network.

Since the network device 1 is a supervised learning method, the input and output images of the network should have no other difference except whether there is moiré or not. However, it is difficult to find the matching label for the real two-dimensional code image that is actually shot, so it can be used The fixed modules in the two-dimensional code: the positioning pattern, the calibration pattern and the detection pattern are used for angular transformation of the captured two-dimensional code image so that it can correspond one-to-one with the two-dimensional code image on the display. The reconstructed image (as shown in Figure 5(b)) after the angle transformation is used as the network input of the migration learning stage, and the corresponding two-dimensional code image on the screen is used as the network output. The transfer learning process still uses the mean square error (MSE) function as the loss function. After the transfer learning operation of the real data (real two-dimensional code image), the network can have a more ideal removal effect on the moiré in the real shooting environment.

In some examples, when the specific training parameters are set to batchsize, the batchsize can be 4, the number of iterations can be 12×10 ⁵ times, the learning rate can be 10 ^-5 , and the optimizer can be Adam.

In the pre-training stage, you can use MATLAB to perform moiré simulation processing on 600 different two-dimensional codes displayed on the display to generate 60,000 512×512 simulated two-dimensional code images with moiré, of which 50,000 can be used For training, 10,000 sheets are used for testing.

In some examples, the deep learning framework tensorflow can be used to implement the network device 1. The training and testing environment of the present disclosure may be a server equipped with NVIDIA Tesla P100 GPU and Intel Xeon E5-2695 v4 CPU.

In some examples, the Dell U2414H display can be used to display different QR code images, and three smart phones (Apple iPhone 8plus, Xiaomi MI 8, Meizu m1 metal, etc.) can be used to capture the QR code images.

Referring to Fig. 5(a), in some examples, in the migration learning stage, 10,000 real moiré real two-dimensional code images can be collected to perform migration learning operations on the network device 1, so that the network can perform better demolition The effect of the pattern. You can collect 2000 images from each version of the mobile phone from version 2 to version 6, and use the mobile phone to take the same QR code image on the screen 20 times from different angles and different distances. Then you can reconstruct the 10,000 real two-dimensional code images taken, and perform angle transformations based on the fixed modules in the two-dimensional code: positioning pattern, calibration pattern and three detection patterns, and get one-to-one with the two-dimensional code image displayed on the screen. The corresponding reconstructed image is shown in Figure 5(b).

In addition, referring again to FIG. 1, a real two-dimensional code image taken by a mobile phone and other devices can be used as input, and the reconstructed image can be obtained by performing perspective transformation, angle transformation, and other operations to obtain a reconstructed image, which can be used as the input of the network device 1 for migration learning . In this case, the trained network device 1 can have a more ideal removal effect on the two-dimensional code image taken in the real environment.

Although the present disclosure has been specifically described above with reference to the drawings and embodiments, it can be understood that the foregoing description does not limit the present disclosure in any form. Those skilled in the art can make modifications and changes to the present disclosure as needed without departing from the essential spirit and scope of the present disclosure, and these modifications and changes fall within the scope of the present disclosure.

Claims

A training method of a deep residual network for removing moiré patterns from two-dimensional codes, which is characterized in that:

include:

Prepare the original QR code image with moiré patterns;

Preparing a network device for processing the original two-dimensional code, the network device including a preprocessing module, a first residual module, a second residual module, and a third residual module;

The original two-dimensional code image is input to the preprocessing module, the original two-dimensional code image is blurred to increase the pixels in the two-dimensional code area in the simulated two-dimensional code image, and then down-sampling is performed to form an image reduction Preprocessed image;

Inputting the preprocessed image to the first residual module for up-sampling processing to form a first output image whose image size is enlarged to the size of the original two-dimensional code image;

Inputting the first output image to the second residual module to form a second output image that restores the image information lost in the first output image; and

Perform feature fusion of the second output image and the original two-dimensional code image to form a feature fusion image, and then input the feature fusion image to the third residual module for purification processing to form moiré removal Pattern image.
The training method according to claim 1, wherein:

The first residual module includes a first convolution layer with a convolution kernel size of 3×3 and 64 feature maps, a first ReLU activation layer, 16 residual blocks connected in series, and a convolution kernel size. The second convolutional layer that is 3×3 and has 64 feature maps, the batch normalization layer, the third convolutional layer with a convolution kernel size of 3×3 and 256 feature maps, and the convolution kernel size is 1. ×1 The fourth convolutional layer with 3 feature maps and the tanh activation layer.
The training method according to claim 1, wherein:

The second residual module includes 10 successively connected first combination layers composed of a convolutional layer with a convolution kernel size of 5×5 and 64 feature maps and a second ReLU activation layer, and 10 layers consisting of a convolutional layer and a second ReLU activation layer. The second combination layer is a convolutional layer with a size of 3×3 and 64 feature maps and a third ReLU activation layer in series.
The training method according to claim 1, wherein:

The third residual module includes a fifth convolutional layer with a convolution kernel size of 3×3 and 128 feature maps, a fourth ReLU activation layer, a convolution kernel size of 3×3 and 3 The sixth convolutional layer of feature maps.
The training method according to claim 1, wherein:

The original two-dimensional code image is a synthesized simulated two-dimensional code image.
The training method according to claim 5, wherein:

The formation of the simulated two-dimensional code image includes the following steps:

Resample the input image to generate a mosaic composed of RGB pixels and display it on the monitor;

Randomly perform projection transformation to simulate different relative positions and directions between the display and the camera;

Using a radiation distortion function to simulate the distortion of the lens of the camera;

Use flat-top Gaussian filter to simulate anti-aliasing filtering;

Re-sampling the input image to simulate the input of the camera sensor;

Adding Gaussian noise to the input image to simulate sensor noise;

Demosaicing

Use denoising filter for denoising processing;

Compress the input image; and

The decompressed image is output to form the simulated two-dimensional code image.
The training method according to claim 6, wherein:

The two-dimensional code image on the display corresponding to the input image is processed using the same projection transformation and lens distortion function.
The training method according to claim 1, wherein:

Using the mean square error function
Train the deep residual network as a loss function, where M and N are the height and width of the simulated two-dimensional code image, H(G(I)) is the two-dimensional code image output by the deep residual network, and J is and The two-dimensional code image on the display corresponding to the simulated two-dimensional code image is stored, and the model and parameters of the deep residual network when the training loss is the least are stored.
The training method according to claim 8, wherein:

It also includes a real two-dimensional code image with moiré, based on the model and parameters of the deep residual network when the training loss is minimal, and uses the real two-dimensional code image to perform migration learning on the deep residual network operating.
The training method according to claim 8, wherein:

Performing an angle transformation on the real two-dimensional code image to obtain a two-dimensional code image on the display corresponding to the real two-dimensional code image one-to-one.