CN113222814B - Resolution processing method, device, equipment and storage medium for image - Google Patents

Resolution processing method, device, equipment and storage medium for image Download PDF

Info

Publication number
CN113222814B
CN113222814B CN202110433837.5A CN202110433837A CN113222814B CN 113222814 B CN113222814 B CN 113222814B CN 202110433837 A CN202110433837 A CN 202110433837A CN 113222814 B CN113222814 B CN 113222814B
Authority
CN
China
Prior art keywords
image
resolution
processed
deep learning
learning model
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Active
Application number
CN202110433837.5A
Other languages
Chinese (zh)
Other versions
CN113222814A (en
Inventor
韩浩瀚
曹锋铭
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Shenzhen Saiante Technology Service Co Ltd
Original Assignee
Shenzhen Saiante Technology Service Co Ltd
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Shenzhen Saiante Technology Service Co Ltd filed Critical Shenzhen Saiante Technology Service Co Ltd
Priority to CN202110433837.5A priority Critical patent/CN113222814B/en
Publication of CN113222814A publication Critical patent/CN113222814A/en
Application granted granted Critical
Publication of CN113222814B publication Critical patent/CN113222814B/en
Active legal-status Critical Current
Anticipated expiration legal-status Critical

Links

Classifications

    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T3/00—Geometric image transformation in the plane of the image
    • G06T3/40—Scaling the whole image or part thereof
    • G06T3/4053—Super resolution, i.e. output image resolution higher than sensor resolution
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/04—Architecture, e.g. interconnection topology
    • G06N3/045—Combinations of networks
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/04—Architecture, e.g. interconnection topology
    • G06N3/048—Activation functions
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/08—Learning methods
    • G06T5/70—
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/20—Special algorithmic details
    • G06T2207/20048—Transform domain processing
    • G06T2207/20064—Wavelet transform [DWT]
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/20—Special algorithmic details
    • G06T2207/20081—Training; Learning
    • Y—GENERAL TAGGING OF NEW TECHNOLOGICAL DEVELOPMENTS; GENERAL TAGGING OF CROSS-SECTIONAL TECHNOLOGIES SPANNING OVER SEVERAL SECTIONS OF THE IPC; TECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS
    • Y02—TECHNOLOGIES OR APPLICATIONS FOR MITIGATION OR ADAPTATION AGAINST CLIMATE CHANGE
    • Y02T—CLIMATE CHANGE MITIGATION TECHNOLOGIES RELATED TO TRANSPORTATION
    • Y02T10/00—Road transport of goods or passengers
    • Y02T10/10—Internal combustion engine [ICE] based vehicles
    • Y02T10/40—Engine management systems

Abstract

The invention relates to the field of artificial intelligence, and discloses a resolution processing method, device, equipment and storage medium for an image, which are used for constructing a deep learning model based on a wavelet super-resolution frame and performing super-resolution processing on the image, so that the loss of original image information is relieved, the chessboard effect is slowed down, and the salt and pepper noise of an output image is restrained. The resolution processing method of the image comprises the following steps: acquiring an image to be processed and corresponding parameters of the image to be processed; leading the corresponding parameters of the image to be processed into an optimized deep learning model, and processing the corresponding parameters of the image to be processed through a preset activation function to obtain a first image; folding and subtracting the first image and introducing a scale factor matrix to generate a second image; and judging whether the resolution of the second image reaches a preset standard, obtaining a judging result, and outputting a target image based on the judging result. In addition, the invention also relates to a blockchain technology, and the target image can be stored in a blockchain node.

Description

Resolution processing method, device, equipment and storage medium for image
Technical Field
The present invention relates to the field of wavelet transform, and in particular, to a method, an apparatus, a device, and a storage medium for processing resolution of an image.
Background
The single image super-Resolution algorithm is mainly an algorithm for inputting a Low Resolution (LR) image to reconstruct to obtain a High Resolution (HR) image, and the current single image super-Resolution algorithm is mainly divided into three types: an image super-resolution algorithm based on interpolation, an image super-resolution algorithm based on a reconstruction model and an image super-resolution algorithm based on learning.
The single image super-resolution algorithm based on interpolation is simple and high in processing speed, but at image abrupt change positions, such as edges and textures, the processing effect is poor, saw teeth and blocking effect are easy to occur, the single image super-resolution processing effect based on a reconstruction model is relatively good, the image is required to have better priori knowledge and is not suitable for image reconstruction with larger amplification factors, the single image super-resolution based on learning can recover finer textures, the reconstruction effect is good, but the algorithm is complex, the calculated amount is large, a sufficient number of learning sample image data sets are needed, universality is avoided, the single image super-resolution based on deep learning is a type of single image super-resolution based on learning, the single image super-resolution based on deep learning is a mainstream method applied to super-resolution by virtue of the strong fitting capability of deep learning, most of frames are subjected to up-sampling operation after extracting features, the obtained up-sampled images are compared with original image images, the loss function learning up-sampled parameters are built, and the original image information is lost after the single image super-resolution algorithm based on deep learning.
Disclosure of Invention
The invention provides an image resolution processing method, device, equipment and storage medium, which are used for constructing a deep learning model based on a wavelet super-resolution frame, performing super-resolution processing on an image, introducing a scale coefficient matrix to modulate the output image, relieving the loss condition of original image information, relieving chessboard effect and inhibiting salt and pepper noise of the output image.
The first aspect of the present invention provides a resolution processing method for an image, including: acquiring an image to be processed and corresponding parameters of the image to be processed, wherein the corresponding parameters of the image to be processed comprise height, width and color channel number; the parameters corresponding to the image to be processed are imported into an optimized deep learning model, and the parameters corresponding to the image to be processed are processed through a preset activation function to obtain a first image; performing doubling subtraction operation on the first image and introducing a scale factor matrix to generate a second image; judging whether the resolution of the second image reaches a preset standard or not, obtaining a judging result, and outputting a target image based on the judging result, wherein the resolution of the target image is higher than that of the image to be processed.
Optionally, in a first implementation manner of the first aspect of the present invention, the step of adding the first component to the second componentThe parameters corresponding to the image to be processed are imported into the optimized deep learning model, the parameters corresponding to the image to be processed are processed through a preset activation function, and the obtaining of the first image comprises the following steps: corresponding parameters I of the image to be processed H×W×C Leading in the neural network NReLU of the optimized deep learning model to obtain a first intermediate result; invoking an activation function sigma to map the first intermediate result to a preset range, and generating a first image, wherein the activation function sigma is according to a first formulaGenerating, wherein the value range of sigma is 0 to 1.
Optionally, in a second implementation manner of the first aspect of the present invention, the performing a fold-in-half subtraction operation on the first image and introducing a scale factor matrix, and generating a second image includes: substituting the first image into a preset doubling-back subtraction function lambda to obtain a second intermediate result, wherein the doubling-back subtraction function lambda is calculated according to a second formula lambda (X) =x 0-0.5 -X 0.5-1 Generating; performing element-by-element multiplication operation on the second intermediate result and a scale factor matrix A to generate a second image, wherein the scale factor matrix A is according to a third formulaGeneration, wherein, is a multiplication operation of elements among matrices.
Optionally, in a third implementation manner of the first aspect of the present invention, the determining whether the resolution of the second image reaches a preset standard, to obtain a determination result, and outputting a target image based on the determination result, where the resolution of the target image is higher than the image to be processed includes: reading the resolution of the second image, comparing the resolution of the second image with a preset standard resolution, and outputting a target image if the resolution of the second image reaches the preset standard resolution; and if the resolution of the second image does not reach the preset standard resolution, the second image is guided into the optimized deep learning model again, the operation is repeated until the generated image resolution reaches the preset standard resolution, and the target image is output.
Optionally, in a fourth implementation manner of the first aspect of the present invention, before the acquiring the image to be processed and the parameters corresponding to the image to be processed, the method further includes: and constructing an optimized deep learning model.
Optionally, in a fifth implementation manner of the first aspect of the present invention, the constructing an optimized deep learning model includes: obtaining a model training image and corresponding parameters of the model training image, wherein the corresponding parameters of the model training image comprise height, width and color channel number; leading corresponding parameters of the model training image into a preset neural network to obtain an initial training result, and calling an activation function sigma to map the initial training result to a preset range to obtain an intermediate training result; performing doubling subtraction processing on the intermediate training result to obtain a target training result, performing element-by-element multiplication operation on the target training result and the scale matrix, and outputting an initial deep learning model; and optimizing the initial deep learning model based on a preset loss function, and generating an optimized deep learning model.
Optionally, in a sixth implementation manner of the first aspect of the present invention, the optimizing the initial deep learning model based on a preset loss function, and generating the optimized deep learning model includes: calculating a loss function of the initial deep learning model according to a preset loss function calculation formula to obtain a target loss function L; and acquiring a standard image, importing the standard image into the initial deep learning model, calling a gradient descent algorithm and the target loss function L to optimize the initial deep learning model, and generating an optimized deep learning model.
A second aspect of the present invention provides an image resolution processing apparatus comprising: the device comprises an acquisition module, a processing module and a processing module, wherein the acquisition module is used for acquiring an image to be processed and corresponding parameters of the image to be processed, and the corresponding parameters of the image to be processed comprise height, width and color channel number; the processing module is used for importing the corresponding parameters of the image to be processed into the optimized deep learning model, and processing the corresponding parameters of the image to be processed through a preset activation function to obtain a first image; the generation module is used for carrying out doubling-in subtraction operation on the first image and introducing a scale coefficient matrix to generate a second image; and the output module is used for judging whether the resolution of the second image reaches a preset standard or not, obtaining a judging result, and outputting a target image based on the judging result, wherein the resolution of the target image is higher than that of the image to be processed.
Optionally, in a first implementation manner of the second aspect of the present invention, the processing module includes: an importing unit for importing the parameters I corresponding to the image to be processed H×W×C Leading in the neural network NReLU of the optimized deep learning model to obtain a first intermediate result; a mapping unit for calling an activation function sigma to map the first intermediate result to a preset range to generate a first image, wherein the activation function sigma is according to a first formulaGenerating, wherein the value range of sigma is 0 to 1.
Optionally, in a second implementation manner of the second aspect of the present invention, the generating module includes: a foldback subtraction unit, configured to substitute the first image into a preset foldback subtraction function Λ, to obtain a second intermediate result, where the foldback subtraction function Λ is according to a second formula Λ (X) =x 0-0.5 -X 0.5-1 Generating; a calculation unit for performing a multiplication operation on the second intermediate result and a scale factor matrix a element by element to generate a second image, wherein the scale factor matrix a is according to a third formulaGeneration, wherein, is a multiplication operation of elements among matrices.
Optionally, in a third implementation manner of the second aspect of the present invention, the output module includes: the contrast unit is used for reading the resolution of the second image, comparing the resolution of the second image with a preset standard resolution, and outputting a target image if the resolution of the second image reaches the preset standard resolution; and the output unit is used for importing the second image into the optimized deep learning model again if the resolution of the second image does not reach the preset standard resolution, repeating the operation until the generated image resolution reaches the preset standard resolution, and outputting a target image.
Optionally, in a fourth implementation manner of the second aspect of the present invention, before the acquiring module, the apparatus further includes: and the construction module is used for constructing the optimized deep learning model.
Optionally, in a fifth implementation manner of the second aspect of the present invention, the building module includes: the system comprises an acquisition unit, a display unit and a display unit, wherein the acquisition unit is used for acquiring a model training image and model training image corresponding parameters, and the model training image corresponding parameters comprise height, width and color channel number; the first training unit is used for importing corresponding parameters of the model training image into a preset neural network to obtain an initial training result, and calling an activation function sigma to map the initial training result to a preset range to obtain an intermediate training result; the second training unit is used for carrying out doubling and subtraction on the intermediate training result to obtain a target training result, carrying out element-by-element multiplication operation on the target training result and the scale matrix, and outputting an initial deep learning model; and the generating unit is used for optimizing the initial deep learning model based on a preset loss function and generating an optimized deep learning model.
Optionally, in a sixth implementation manner of the second aspect of the present invention, the generating unit is specifically configured to: calculating a loss function of the initial deep learning model according to a preset loss function calculation formula to obtain a target loss function L; and acquiring a standard image, importing the standard image into an initial deep learning model, calling a gradient descent algorithm and the target loss function L to optimize the initial deep learning model, and generating an optimized deep learning model.
A third aspect of the present invention provides an image resolution processing apparatus comprising: a memory and at least one processor, the memory having instructions stored therein; the at least one processor invokes the instructions in the memory to cause the resolution processing device of the image to perform the resolution processing method of the image described above.
A fourth aspect of the present invention provides a computer-readable storage medium having instructions stored therein, which when run on a computer, cause the computer to perform the above-described resolution processing method of an image.
In the technical scheme provided by the invention, an image to be processed and corresponding parameters of the image to be processed are obtained, wherein the corresponding parameters of the image to be processed comprise height, width and color channel number; the parameters corresponding to the image to be processed are imported into an optimized deep learning model, and the parameters corresponding to the image to be processed are processed through a preset activation function to obtain a first image; performing doubling subtraction operation on the first image and introducing a scale factor matrix to generate a second image; judging whether the resolution of the second image reaches a preset standard or not, obtaining a judging result, and outputting a target image based on the judging result, wherein the resolution of the target image is higher than that of the image to be processed. In the embodiment of the invention, a deep learning model is constructed based on a wavelet super-resolution frame, super-resolution processing is carried out on the image, and a scale coefficient matrix is introduced to modulate the output image, so that the loss condition of original image information is relieved, the chessboard effect is slowed down, and the salt and pepper noise of the output image is restrained.
Drawings
FIG. 1 is a schematic diagram of an embodiment of a resolution processing method of an image according to an embodiment of the present invention;
FIG. 2 is a schematic diagram of another embodiment of a resolution processing method of an image according to an embodiment of the present invention;
FIG. 3 is a schematic diagram of an embodiment of an apparatus for processing resolution of an image according to an embodiment of the present invention;
FIG. 4 is a schematic diagram of another embodiment of an apparatus for processing resolution of an image according to an embodiment of the present invention;
fig. 5 is a schematic diagram of an embodiment of an apparatus for processing resolution of an image according to an embodiment of the present invention.
Detailed Description
The embodiment of the invention provides an image resolution processing method, device, equipment and storage medium, which are used for constructing a deep learning model based on a wavelet super-resolution frame, performing super-resolution processing on an image, introducing a scale coefficient matrix to modulate the output image, relieving the loss condition of original image information, slowing down the chessboard effect and inhibiting the salt and pepper noise of the output image.
The terms "first," "second," "third," "fourth" and the like in the description and in the claims and in the above drawings, if any, are used for distinguishing between similar objects and not necessarily for describing a particular sequential or chronological order. It is to be understood that the data so used may be interchanged where appropriate such that the embodiments described herein may be implemented in other sequences than those illustrated or otherwise described herein. Furthermore, the terms "comprises," "comprising," or any other variation thereof, are intended to cover a non-exclusive inclusion, such that a process, method, system, article, or apparatus that comprises a list of steps or elements is not necessarily limited to those steps or elements expressly listed or inherent to such process, method, article, or apparatus.
For easy understanding, the following describes a specific flow of an embodiment of the present invention, referring to fig. 1, and one embodiment of a method for processing resolution of an image in an embodiment of the present invention includes:
101. and acquiring the image to be processed and corresponding parameters of the image to be processed, wherein the corresponding parameters of the image to be processed comprise the height, the width and the number of color channels.
The server acquires an image to be processed and parameters corresponding to the image to be processed, wherein the parameters corresponding to the image to be processed comprise height, width and color channel number. The height of the image is indicated by letter H, the width of the image is indicated by letter W, the number of color (RGB) channels is indicated by letter C, the number of channels of the image, that is, the bit depth of the image, refers to the number of binary bits describing the proportion of each pixel value in the image, the larger the bit depth is, the more the number of colors the image can represent, and the color image generally comprises three channels of RED (RED), GREEN (GREEN) and BLUE (BLUE), and the pixel value of each channel is between 0 and 255.
It is to be understood that the execution subject of the present invention may be an image resolution processing apparatus, and may also be a terminal or a server, which is not limited herein. The embodiment of the invention is described by taking a server as an execution main body as an example.
102. And importing the corresponding parameters of the image to be processed into the optimized deep learning model, and processing the corresponding parameters of the image to be processed through a preset activation function to obtain a first image.
The server imports the parameters corresponding to the image to be processed into the optimized deep learning model, and processes the parameters corresponding to the image to be processed through a preset activation function to obtain a first image. Specifically, the server corresponds the parameter I to the image to be processed H×W×C Leading in the neural network NReLU of the optimized deep learning model to obtain a first intermediate result; the server calls an activation function sigma to map the first intermediate result to a preset range to generate a first image, wherein the activation function sigma is according to a first formulaGenerating, wherein the value range of sigma is 0 to 1. The method comprises the steps that a server imports corresponding parameters of a model training image into a preset neural network NReLU, wherein the NReLU is selected as an activation function in a neural network of ReLU or Leaky ReLU, wherein a single-side inhibition and sparse model can be realized by a correction linear unit (Rectified Linear Unit, reLU) activation function, the correction linear unit (Leaky ReLU) with leakage is a variant of the ReLU and has high nonlinearity, the function output has a small gradient to negative value input, the neural network in the embodiment uses one of the ReLU activation function or the Leaky ReLU activation function, the server calls the activation function sigmoid to limit the output range to be in a range of 0-1, a first image is generated, the activation function sigmoid is expressed by a letter sigma, the signal gain of the nonlinear sigmoid function to a central area is large, the signal gain to two-side areas is small, and the characteristic of the signal is generatedThe space mapping has good effect, and the formula of the sigma function is +.>
103. And performing folio subtraction operation on the first image and introducing a scale factor matrix to generate a second image.
And the server performs doubling subtraction operation on the first image and introduces a scale factor matrix to generate a second image. Specifically, the server substitutes the first image into a preset doubling subtraction function Λ to obtain a second intermediate result, wherein the doubling subtraction function Λ is calculated according to a second formula Λ (X) =x 0-0.5 -X 0.5-1 Generating; the server performs element-by-element multiplication operation on the second intermediate result and a scale factor matrix A to generate a second image, wherein the scale factor matrix A is according to a third formulaGeneration, wherein, is a multiplication operation of elements among matrices. The fold-back subtraction operation is to achieve positive and negative values, and the positive and negative values are extracted respectively because the ReLU activation function cannot express the negative values well, and the step comprises the following formula Λ (X) =X 0-0.5 -X 0.5-1 Generating a true sparse result by matching with a ReLU activation function, mapping the result to a range from-1 to 1, wherein a scale matrix is represented by a letter A and used for suppressing salt and pepper noise in the result, <' > and>wherein I is H×W×C And training the corresponding parameters of the image for the model.
104. Judging whether the resolution of the second image reaches a preset standard or not, obtaining a judging result, and outputting a target image based on the judging result, wherein the resolution of the target image is higher than that of the image to be processed.
The server judges whether the resolution of the second image reaches a preset standard or not, a judging result is obtained, a target image is output based on the judging result, and the resolution of the target image is higher than that of the image to be processed. Specifically, the server reads the resolution of the second image, compares the resolution of the second image with a preset standard resolution, and outputs a target image if the resolution of the second image reaches the preset standard resolution; if the resolution of the second image does not reach the preset standard resolution, the second image is guided into the optimized deep learning model again, the operation is repeated until the generated image resolution reaches the preset standard resolution, and the target image is output. The server synthesizes the image with resolution higher than that of the input image by one time through the wavelet inverse transformation algorithm, the resolution of the input image is doubled by one time when the model is input each time, the output image reaches the target resolution, and the embodiment can be applied to the traditional shooting television image which is transmitted to the model frame by frame, and the high-definition television effect is output.
In the embodiment of the invention, a deep learning model is constructed based on a wavelet super-resolution frame, super-resolution processing is carried out on the image, and a scale coefficient matrix is introduced to modulate the output image, so that the loss condition of original image information is relieved, the chessboard effect is slowed down, and the salt and pepper noise of the output image is restrained.
Referring to fig. 2, another embodiment of a method for processing resolution of an image according to an embodiment of the present invention includes:
201. and constructing an optimized deep learning model.
And constructing an optimized deep learning model by the server. Specifically, a server acquires a model training image and corresponding parameters of the model training image, wherein the corresponding parameters of the model training image comprise height, width and color channel number; the server imports corresponding parameters of the model training image into a preset neural network to obtain an initial training result, and invokes an activation function sigma to map the initial training result to a preset range to obtain an intermediate training result; the server performs doubling subtraction processing on the intermediate training result to obtain a target training result, performs element-by-element multiplication operation on the target training result and the scale matrix, and outputs an initial deep learning model; and the server optimizes the initial deep learning model based on a preset loss function, and generates an optimized deep learning model. Calculating a loss function of an initial deep learning model according to a preset loss function calculation formula to obtain a target loss function L, wherein the preset loss function calculation formula is L= Σabs (W (S (I), M (S (I))) -I), S (I) is one half of a model training image and is sampled, abs represents taking absolute values element by element, W (S (I), M (S (I)) represents a wavelet image reconstruction operator, finally obtaining the target loss function L, obtaining a standard image by a server after calculating the target loss function, introducing the standard image into the initial deep learning model, calling a gradient descent algorithm and the target loss function L to optimize the initial deep learning model to generate an optimized deep learning model, the standard image is an image with image definition reaching a preset standard, and the gradient descent algorithm (Gradient Descent Optimization) is adopted to optimize the model, wherein the gradient descent method is an algorithm for searching the minimum of the target function by solving the derivative of the target function, so that the minimum of the target loss function L is realized.
202. And acquiring the image to be processed and corresponding parameters of the image to be processed, wherein the corresponding parameters of the image to be processed comprise the height, the width and the number of color channels.
The server acquires an image to be processed and parameters corresponding to the image to be processed, wherein the parameters corresponding to the image to be processed comprise height, width and color channel number. The height of the image is indicated by letter H, the width of the image is indicated by letter W, the number of color (RGB) channels is indicated by letter C, the number of channels of the image, that is, the bit depth of the image, refers to the number of binary bits describing the proportion of each pixel value in the image, the larger the bit depth is, the more the number of colors the image can represent, and the color image generally comprises three channels of RED (RED), GREEN (GREEN) and BLUE (BLUE), and the pixel value of each channel is between 0 and 255.
203. And importing the corresponding parameters of the image to be processed into the optimized deep learning model, and processing the corresponding parameters of the image to be processed through a preset activation function to obtain a first image.
The server imports the parameters corresponding to the image to be processed into the optimized deep learning model, and processes the parameters corresponding to the image to be processed through a preset activation function to obtain a first image. Specifically, the server corresponds the parameter I to the image to be processed H×W×C Leading in the neural network NReLU of the optimized deep learning model to obtain a first intermediate result; the server calls an activation function sigma to map the first intermediate result to a preset range to generate a first image, wherein the activation function sigma is according to a first formulaGenerating, wherein the value range of sigma is 0 to 1. The server imports model training image corresponding parameters into a preset neural network NReLU, wherein the NReLU is an activation function and is selected in a neural network of ReLU or Leaky ReLU, wherein the modification linear unit (Rectified Linear Unit, reLU) activation function can realize single-side inhibition and sparse model, has high nonlinearity, the modification linear unit activation function with leakage (Leaky ReLU) is a variant of ReLU, the function output has small gradient to negative value input, the neural network in the embodiment uses one of the ReLU activation function or the Leaky ReLU activation function, the server calls the activation function sigmoid to limit the output range to be in the range of 0 to 1 to generate a first image, the activation function sigmoid is expressed by letter sigma, the signal gain of the nonlinear sigmoid function on a central area is large, the signal gain on two side areas is small, the function has good effect on the characteristic space mapping of the signal, and the formula of the sigma function is that>
204. And performing folio subtraction operation on the first image and introducing a scale factor matrix to generate a second image.
And the server performs doubling subtraction operation on the first image and introduces a scale factor matrix to generate a second image. Specifically, the server substitutes the first image into a preset doubling subtraction function Λ to obtain a second intermediate result, wherein the doubling subtraction function Λ is calculated according to a second formula Λ (X) =x 0-0.5 -X 0.5-1 Generating; the server performs element-by-element multiplication operation on the second intermediate result and a scale factor matrix A to generate a second image, wherein the scale factor matrix A is according to a third formulaGeneration, wherein, is a multiplication operation of elements among matrices. The fold-back subtraction operation is to achieve positive and negative values, and the positive and negative values are extracted respectively because the ReLU activation function cannot express the negative values well, and the step comprises the following formula Λ (X) =X 0-0.5 -X 0.5-1 Generating a true sparse result by matching with a ReLU activation function, mapping the result to a range from-1 to 1, wherein a scale matrix is represented by a letter A and used for suppressing salt and pepper noise in the result, <' > and>wherein I is H×W×C And training the corresponding parameters of the image for the model.
205. Judging whether the resolution of the second image reaches a preset standard or not, obtaining a judging result, and outputting a target image based on the judging result, wherein the resolution of the target image is higher than that of the image to be processed.
The server judges whether the resolution of the second image reaches a preset standard or not, a judging result is obtained, a target image is output based on the judging result, and the resolution of the target image is higher than that of the image to be processed. Specifically, the server reads the resolution of the second image, compares the resolution of the second image with a preset standard resolution, and outputs a target image if the resolution of the second image reaches the preset standard resolution; if the resolution of the second image does not reach the preset standard resolution, the second image is guided into the optimized deep learning model again, the operation is repeated until the generated image resolution reaches the preset standard resolution, and the target image is output. The server synthesizes the image with resolution higher than that of the input image by one time through the wavelet inverse transformation algorithm, the resolution of the input image is doubled by one time when the model is input each time, the output image reaches the target resolution, and the embodiment can be applied to the traditional shooting television image which is transmitted to the model frame by frame, and the high-definition television effect is output.
In the embodiment of the invention, a deep learning model is constructed based on a wavelet super-resolution frame, super-resolution processing is carried out on the image, and a scale coefficient matrix is introduced to modulate the output image, so that the loss condition of original image information is relieved, the chessboard effect is slowed down, and the salt and pepper noise of the output image is restrained.
The method for processing the resolution of the image in the embodiment of the present invention is described above, and the apparatus for processing the resolution of the image in the embodiment of the present invention is described below, referring to fig. 3, an embodiment of the apparatus for processing the resolution of the image in the embodiment of the present invention includes:
the acquiring module 301 is configured to acquire an image to be processed and parameters corresponding to the image to be processed, where the parameters corresponding to the image to be processed include a height, a width, and a number of color channels;
the processing module 302 is configured to introduce the parameters corresponding to the image to be processed into the optimized deep learning model, and process the parameters corresponding to the image to be processed through a preset activation function to obtain a first image;
a generating module 303, configured to perform a folio subtraction operation on the first image and introduce a scale factor matrix to generate a second image;
and the output module 304 is configured to determine whether the resolution of the second image meets a preset standard, obtain a determination result, and output a target image based on the determination result, where the resolution of the target image is higher than the image to be processed.
In the embodiment of the invention, a deep learning model is constructed based on a wavelet super-resolution frame, super-resolution processing is carried out on the image, and a scale coefficient matrix is introduced to modulate the output image, so that the loss condition of original image information is relieved, the chessboard effect is slowed down, and the salt and pepper noise of the output image is restrained.
Referring to fig. 4, another embodiment of an apparatus for processing resolution of an image according to an embodiment of the present invention includes:
the acquiring module 301 is configured to acquire an image to be processed and parameters corresponding to the image to be processed, where the parameters corresponding to the image to be processed include a height, a width, and a number of color channels;
the processing module 302 is configured to introduce the parameters corresponding to the image to be processed into the optimized deep learning model, and process the parameters corresponding to the image to be processed through a preset activation function to obtain a first image;
a generating module 303, configured to perform a folio subtraction operation on the first image and introduce a scale factor matrix to generate a second image;
and the output module 304 is configured to determine whether the resolution of the second image meets a preset standard, obtain a determination result, and output a target image based on the determination result, where the resolution of the target image is higher than the image to be processed.
Optionally, the processing module 302 includes:
an importing unit 3021 for importing the parameters I corresponding to the image to be processed H×W×C Leading in the neural network NReLU of the optimized deep learning model to obtain a first intermediate result;
a mapping unit 3022 for calling an activation function σ to map the first intermediate result to a preset range, and generating a first image, wherein the activation function σ is according to a first formulaGenerating, wherein the value range of sigma is 0 to 1.
Optionally, the generating module 303 includes:
a folio subtraction unit 3031, configured to substitute the first image into a preset folio subtraction function Λ to obtain a second intermediate result, where the folio subtraction function Λ is according to a second formula Λ (X) =x 0-0.5 -X 0.5-1 Generating;
a computing unit 3032, configured to perform a multiplication operation on the second intermediate result and a scale factor matrix a, to generate a second image, where the scale factor matrix a is according to a third formulaGeneration, wherein, is a multiplication operation of elements among matrices.
Optionally, the output module 304 includes:
the comparing unit 3041 is configured to read the resolution of the second image, compare the resolution of the second image with a preset standard resolution, and output a target image if the resolution of the second image reaches the preset standard resolution;
and an output unit 3042, configured to, if the resolution of the second image does not reach the preset standard resolution, re-import the second image into the optimized deep learning model, repeat the above operation until the resolution of the generated image reaches the preset standard resolution, and output the target image.
Optionally, before the acquiring module 301, the resolution processing apparatus of the image further includes:
a construction module 305 is configured to construct an optimized deep learning model.
Optionally, the building module 305 includes:
the acquiring unit 3051 is configured to acquire a model training image and parameters corresponding to the model training image, where the parameters corresponding to the model training image include a height, a width, and a number of color channels;
the first training unit 3052 is configured to import parameters corresponding to the model training image into a preset neural network to obtain an initial training result, and invoke the activation function sigma to map the initial training result to a preset range to obtain an intermediate training result;
the second training unit 3053 is configured to perform a doubling subtraction process on the intermediate training result to obtain a target training result, perform a unit-by-unit multiplication operation on the target training result and the scale matrix, and output an initial deep learning model;
and the generating unit 3054 is used for optimizing the initial deep learning model based on a preset loss function and generating an optimized deep learning model.
Optionally, the generating unit 3054 is specifically configured to:
calculating a loss function of the initial deep learning model according to a preset loss function calculation formula to obtain a target loss function L; and acquiring a standard image, importing the standard image into an initial deep learning model, calling a gradient descent algorithm and a target loss function L to optimize the initial deep learning model, and generating an optimized deep learning model.
In the embodiment of the invention, a deep learning model is constructed based on a wavelet super-resolution frame, super-resolution processing is carried out on the image, and a scale coefficient matrix is introduced to modulate the output image, so that the loss condition of original image information is relieved, the chessboard effect is slowed down, and the salt and pepper noise of the output image is restrained.
The resolution processing apparatus for an image in the embodiment of the present invention is described in detail above in fig. 3 and 4 from the point of view of a modularized functional entity, and the resolution processing device for an image in the embodiment of the present invention is described in detail below from the point of view of hardware processing.
Fig. 5 is a schematic structural diagram of an image resolution processing device according to an embodiment of the present invention, where the image resolution processing device 500 may have a relatively large difference due to different configurations or performances, and may include one or more processors (central processing units, CPU) 510 (e.g., one or more processors) and a memory 520, and one or more storage media 530 (e.g., one or more mass storage devices) storing application programs 533 or data 532. Wherein memory 520 and storage medium 530 may be transitory or persistent storage. The program stored in the storage medium 530 may include one or more modules (not shown), each of which may include a series of instruction operations in the resolution processing apparatus 500 for an image. Still further, the processor 510 may be arranged to communicate with a storage medium 530 to execute a series of instruction operations in the storage medium 530 on the resolution processing device 500 of an image.
The resolution processing device 500 for images may also include one or more power supplies 540, one or more wired or wireless network interfaces 550, one or more input/output interfaces 560, and/or one or more operating systems 531, such as Windows Serve, mac OS X, unix, linux, freeBSD, and the like. It will be appreciated by those skilled in the art that the resolution processing apparatus structure of the image shown in fig. 5 does not constitute a limitation of the resolution processing apparatus of the image, and may include more or less components than illustrated, or may combine some components, or may be a different arrangement of components.
The present invention also provides an image resolution processing apparatus, the computer apparatus including a memory and a processor, the memory storing computer readable instructions which, when executed by the processor, cause the processor to perform the steps of the image resolution processing method in the above embodiments.
The present invention also provides a computer readable storage medium, which may be a non-volatile computer readable storage medium, or a volatile computer readable storage medium, having stored therein instructions that, when executed on a computer, cause the computer to perform the steps of the resolution processing method of an image.
The blockchain is a novel application mode of computer technologies such as distributed data storage, point-to-point transmission, consensus mechanism, encryption algorithm and the like. The Blockchain (Blockchain), which is essentially a decentralised database, is a string of data blocks that are generated by cryptographic means in association, each data block containing a batch of information of network transactions for verifying the validity of the information (anti-counterfeiting) and generating the next block. The blockchain may include a blockchain underlying platform, a platform product services layer, an application services layer, and the like.
It will be clear to those skilled in the art that, for convenience and brevity of description, specific working procedures of the above-described systems, apparatuses and units may refer to corresponding procedures in the foregoing method embodiments, which are not repeated herein.
The integrated units, if implemented in the form of software functional units and sold or used as stand-alone products, may be stored in a computer readable storage medium. Based on such understanding, the technical solution of the present invention may be embodied essentially or in part or all of the technical solution or in part in the form of a software product stored in a storage medium, including instructions for causing a computer device (which may be a personal computer, a server, or a network device, etc.) to perform all or part of the steps of the method according to the embodiments of the present invention. And the aforementioned storage medium includes: a U-disk, a removable hard disk, a read-only memory (ROM), a random access memory (random access memory, RAM), a magnetic disk, or an optical disk, or other various media capable of storing program codes.
The above embodiments are only for illustrating the technical solution of the present invention, and not for limiting the same; although the invention has been described in detail with reference to the foregoing embodiments, it will be understood by those of ordinary skill in the art that: the technical scheme described in the foregoing embodiments can be modified or some technical features thereof can be replaced by equivalents; such modifications and substitutions do not depart from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims (10)

1. A resolution processing method of an image, characterized in that the resolution processing method of an image comprises:
acquiring an image to be processed and corresponding parameters of the image to be processed, wherein the corresponding parameters of the image to be processed comprise height, width and color channel number;
the parameters corresponding to the image to be processed are imported into an optimized deep learning model, and the parameters corresponding to the image to be processed are processed through a preset activation function to obtain a first image;
performing doubling subtraction operation on the first image and introducing a scale factor matrix to generate a second image;
judging whether the resolution of the second image reaches a preset standard or not, obtaining a judging result, and outputting a target image based on the judging result, wherein the resolution of the target image is higher than that of the image to be processed.
2. The method for processing the resolution of the image according to claim 1, wherein the step of introducing the parameters corresponding to the image to be processed into the optimized deep learning model, and processing the parameters corresponding to the image to be processed by a preset activation function, to obtain the first image includes:
corresponding parameters I of the image to be processed H×W×C Importing optimized depthLearning a neural network NReLU of the model to obtain a first intermediate result;
invoking an activation function sigma to map the first intermediate result to a preset range, and generating a first image, wherein the activation function sigma is according to a first formulaGenerating, wherein the value range of sigma is 0 to 1.
3. The method of claim 1, wherein performing a folio subtraction operation on the first image and introducing a scale factor matrix to generate a second image comprises:
substituting the first image into a preset doubling-back subtraction function lambda to obtain a second intermediate result, wherein the doubling-back subtraction function lambda is calculated according to a second formula lambda (X) =x 0-0.5 -X 0.5-1 Generating;
performing element-by-element multiplication operation on the second intermediate result and a scale factor matrix A to generate a second image, wherein the scale factor matrix A is according to a third formulaGeneration, wherein, is a multiplication operation of elements among matrices.
4. The method according to claim 1, wherein determining whether the resolution of the second image reaches a preset criterion, obtaining a determination result, and outputting a target image based on the determination result, the target image having a resolution higher than the image to be processed, comprises:
reading the resolution of the second image, comparing the resolution of the second image with a preset standard resolution, and outputting a target image if the resolution of the second image reaches the preset standard resolution;
and if the resolution of the second image does not reach the preset standard resolution, the second image is guided into the optimized deep learning model again, the operation is repeated until the generated image resolution reaches the preset standard resolution, and the target image is output.
5. The method of resolution processing of an image according to any one of claims 1 to 4, wherein prior to said acquiring the image to be processed and the corresponding parameters of the image to be processed, the method further comprises:
and constructing an optimized deep learning model.
6. The method of image resolution processing according to claim 5, wherein the constructing an optimized deep learning model includes:
obtaining a model training image and corresponding parameters of the model training image, wherein the corresponding parameters of the model training image comprise height, width and color channel number;
leading corresponding parameters of the model training image into a preset neural network to obtain an initial training result, and calling an activation function sigma to map the initial training result to a preset range to obtain an intermediate training result;
performing doubling subtraction processing on the intermediate training result to obtain a target training result, performing element-by-element multiplication operation on the target training result and the scale matrix, and outputting an initial deep learning model;
and optimizing the initial deep learning model based on a preset loss function, and generating an optimized deep learning model.
7. The method of claim 6, wherein optimizing the initial deep learning model based on a preset loss function, and generating an optimized deep learning model comprises:
calculating a loss function of the initial deep learning model according to a preset loss function calculation formula to obtain a target loss function L;
and acquiring a standard image, importing the standard image into the initial deep learning model, calling a gradient descent algorithm and the target loss function L to optimize the initial deep learning model, and generating an optimized deep learning model.
8. A resolution processing apparatus of an image, characterized in that the resolution processing apparatus of an image comprises:
the device comprises an acquisition module, a processing module and a processing module, wherein the acquisition module is used for acquiring an image to be processed and corresponding parameters of the image to be processed, and the corresponding parameters of the image to be processed comprise height, width and color channel number;
the processing module is used for importing the corresponding parameters of the image to be processed into the optimized deep learning model, and processing the corresponding parameters of the image to be processed through a preset activation function to obtain a first image;
the generation module is used for carrying out doubling-in subtraction operation on the first image and introducing a scale coefficient matrix to generate a second image;
and the output module is used for judging whether the resolution of the second image reaches a preset standard or not, obtaining a judging result, and outputting a target image based on the judging result, wherein the resolution of the target image is higher than that of the image to be processed.
9. A resolution processing apparatus of an image, characterized in that the resolution processing apparatus of an image comprises:
a memory and at least one processor, the memory having instructions stored therein;
the at least one processor invokes the instructions in the memory to cause the resolution processing device of the image to perform the resolution processing method of the image of any one of claims 1-7.
10. A computer readable storage medium having instructions stored thereon, which when executed by a processor, implement the resolution processing method of an image according to any of claims 1-7.
CN202110433837.5A 2021-04-22 2021-04-22 Resolution processing method, device, equipment and storage medium for image Active CN113222814B (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN202110433837.5A CN113222814B (en) 2021-04-22 2021-04-22 Resolution processing method, device, equipment and storage medium for image

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN202110433837.5A CN113222814B (en) 2021-04-22 2021-04-22 Resolution processing method, device, equipment and storage medium for image

Publications (2)

Publication Number Publication Date
CN113222814A CN113222814A (en) 2021-08-06
CN113222814B true CN113222814B (en) 2023-08-04

Family

ID=77088403

Family Applications (1)

Application Number Title Priority Date Filing Date
CN202110433837.5A Active CN113222814B (en) 2021-04-22 2021-04-22 Resolution processing method, device, equipment and storage medium for image

Country Status (1)

Country Link
CN (1) CN113222814B (en)

Citations (6)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN104008539A (en) * 2014-05-29 2014-08-27 西安理工大学 Image super-resolution rebuilding method based on multiscale geometric analysis
CN108596833A (en) * 2018-04-26 2018-09-28 广东工业大学 Super-resolution image reconstruction method, device, equipment and readable storage medium storing program for executing
CN111415316A (en) * 2020-03-18 2020-07-14 山西安数智能科技有限公司 Defect data synthesis algorithm based on generation of countermeasure network
CN111724299A (en) * 2020-05-21 2020-09-29 同济大学 Super-realistic painting image style migration method based on deep learning
CN111932464A (en) * 2020-09-18 2020-11-13 北京百度网讯科技有限公司 Super-resolution model using and training method, device, equipment and medium
CN112561890A (en) * 2020-12-18 2021-03-26 深圳赛安特技术服务有限公司 Image definition calculation method and device and computer equipment

Family Cites Families (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN107633218B (en) * 2017-09-08 2021-06-08 百度在线网络技术（北京）有限公司 Method and apparatus for generating image

Patent Citations (6)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN104008539A (en) * 2014-05-29 2014-08-27 西安理工大学 Image super-resolution rebuilding method based on multiscale geometric analysis
CN108596833A (en) * 2018-04-26 2018-09-28 广东工业大学 Super-resolution image reconstruction method, device, equipment and readable storage medium storing program for executing
CN111415316A (en) * 2020-03-18 2020-07-14 山西安数智能科技有限公司 Defect data synthesis algorithm based on generation of countermeasure network
CN111724299A (en) * 2020-05-21 2020-09-29 同济大学 Super-realistic painting image style migration method based on deep learning
CN111932464A (en) * 2020-09-18 2020-11-13 北京百度网讯科技有限公司 Super-resolution model using and training method, device, equipment and medium
CN112561890A (en) * 2020-12-18 2021-03-26 深圳赛安特技术服务有限公司 Image definition calculation method and device and computer equipment

Non-Patent Citations (1)

* Cited by examiner, † Cited by third party
Title
线性模型的遥感图像时空融合;方帅;姚振稷;曹风云;;中国图象图形学报(第03期);165-178 *

Also Published As

Publication number Publication date
CN113222814A (en) 2021-08-06

Similar Documents

Publication Publication Date Title
US20050025378A1 (en) Method for bilateral filtering of digital images
CN113222834B (en) Visual data tensor completion method based on smoothness constraint and matrix decomposition
CN101527829B (en) Method and device for processing video data
US11455535B2 (en) Systems and methods for sensor-independent illuminant determination
EP2467823A1 (en) Image reconstruction method and system
WO2023005818A1 (en) Noise image generation method and apparatus, electronic device, and storage medium
CN104021529B (en) Blurred image non-blind restoration method
US7512264B2 (en) Image processing
Karam et al. Monte-Carlo acceleration of bilateral filter and non-local means
Huang et al. Two-step approach for the restoration of images corrupted by multiplicative noise
Jin et al. A variational approach for image decolorization by variance maximization
CN113222814B (en) Resolution processing method, device, equipment and storage medium for image
CN113935917A (en) Optical remote sensing image thin cloud removing method based on cloud picture operation and multi-scale generation countermeasure network
Chen et al. Inexact alternating direction method based on proximity projection operator for image inpainting in wavelet domain
Tanwar et al. Privacy preserving image scaling using 2D bicubic interpolation over the cloud
CN116595525A (en) Threshold mechanism malicious software detection method and system based on software map
CN114862711B (en) Low-illumination image enhancement and denoising method based on dual complementary prior constraints
Arya et al. Robust image compression using two dimensional discrete cosine transform
CN115941853A (en) Image hiding method, system, device and medium based on compressed sensing and adaptive embedding
JPH06266747A (en) Dct/inverse dct arithmetic unit
CN112836688B (en) Feature extraction method and device of tile image, electronic equipment and storage medium
US20230154140A1 (en) Neural network-based high-resolution image restoration method and system
CN111402173B (en) Mixed noise removing method and device, electronic equipment and storage medium
Koikara et al. A novel distributed image steganography method based on block-dct
CN113822809A (en) Dim light enhancement method and system

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination
TA01 Transfer of patent application right

Effective date of registration: 20211022

Address after: Room 201, building a, No.1 Qianwan 1st Road, Shenzhen Hong Kong cooperation zone, Shenzhen, Guangdong Province (settled in Shenzhen Qianhai business secretary Co., Ltd.)

Applicant after: Shenzhen saiante Technology Service Co.,Ltd.

Address before: 1-34 / F, Qianhai free trade building, 3048 Xinghai Avenue, Mawan, Qianhai Shenzhen Hong Kong cooperation zone, Shenzhen, Guangdong 518000

Applicant before: Ping An International Smart City Technology Co.,Ltd.

TA01 Transfer of patent application right
GR01 Patent grant
GR01 Patent grant