CN109360153A - Image processing method, super-resolution model generating method, device and electronic equipment - Google Patents
Image processing method, super-resolution model generating method, device and electronic equipment Download PDFInfo
- Publication number
- CN109360153A CN109360153A CN201811260269.8A CN201811260269A CN109360153A CN 109360153 A CN109360153 A CN 109360153A CN 201811260269 A CN201811260269 A CN 201811260269A CN 109360153 A CN109360153 A CN 109360153A
- Authority
- CN
- China
- Prior art keywords
- cnn model
- super
- image
- model
- resolution
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Granted
Links
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T3/00—Geometric image transformation in the plane of the image
- G06T3/40—Scaling the whole image or part thereof
- G06T3/4053—Super resolution, i.e. output image resolution higher than sensor resolution
Abstract
The embodiment of the invention provides a kind of image processing method, super-resolution model generating method, device and electronic equipments, wherein a kind of image processing method is applied to browser, which comprises obtain image to be processed；The image to be processed is handled by the first convolutional neural networks CNN model pre-loaded in super-resolution model, obtains the first information；Wherein, the first CNN model is the CNN model of the browser identified documentation format formatted to default CNN model；The first information is handled by the sub-pix convolutional layer in the super-resolution model, obtains super-resolution image.The present invention realizes the image quality for improving the super-resolution image obtained on a web browser.
Description
Technical field
The present invention relates to technical field of computer information processing, more particularly to a kind of image processing method, super-resolution
Model generating method, device and electronic equipment.
Background technique
Image super-resolution technology, which refers to, is reconstructed into high-definition picture for low-resolution image, related in picture or video
It is very widely used in web application, cell phone system, monitoring device, satellite image and field of medical imaging.
In recent years, the image super-resolution technical research and application based on deep learning become the heat of academia and industry
Point.These methods obtain profound super-resolution model based on large nuber of images training, which includes CNN
(Convolutional Neural Network, convolutional neural networks) layer and sub-pix convolutional layer.By the super-resolution mould
Type is disposed on the server, and then converts and load the super-resolution model on a web browser, passes through the super-resolution model
Low-resolution image is handled, high-definition picture is finally exported.Specifically: by the identifiable checkponit of server
The super-resolution model comprising CNN layers and sub-pix convolutional layer of format, is converted to SavedModel format, Jin Ertong first
It is the identifiable object module of browser end that tfjs-converter tool, which is crossed, by the model conversion of SavedModel format.Pass through
The object module handles the image to be processed of acquisition, obtains the target image of super-resolution.
Inventors have found that using the existing super-resolution model comprising CNN layers and sub-pix convolutional layer to image into
It, will be comprising CNN layers and sub-pix convolutional layer due to the function limitation of tfjs-converter tool in the method for row processing
Super-resolution model conversion be the identifiable object module of browser end after, image is handled with the object module
Super-resolution image, compared to the super-resolution image directly obtained on the server with super-resolution model, image quality under
Drop.Therefore, the lower problem of the image quality of the super-resolution image obtained on a web browser for the relevant technologies, not yet proposes at present
Effective solution scheme.
Summary of the invention
The embodiment of the present invention is designed to provide a kind of image processing method, super-resolution model generating method, device
And electronic equipment, to improve the image quality of the super-resolution image obtained on a web browser.Specific technical solution is as follows:
In a first aspect, being applied to browser, the method packet the embodiment of the invention discloses a kind of image processing method
It includes:
Obtain image to be processed；
By the image to be processed by pre-loaded the first convolutional neural networks CNN model in super-resolution model into
Row processing, obtains the first information；Wherein, the first CNN model be default CNN model is formatted described in
The CNN model of browser identified documentation format；
The first information is handled by the sub-pix convolutional layer in the super-resolution model, obtains super-resolution figure
Picture.
Optionally, the image to be processed is passed through into the first convolutional Neural pre-loaded in super-resolution model described
Network C NN model is handled, before obtaining the first information, the method also includes:
By the default CNN model of checkponit format, be converted to by Tensorflow data transformation interface
The CNN model of SavedModel format；
By the CNN model of the SavedModel format, the first CNN is converted to by tfjs-converter tool
Model；
Load the first CNN model.
Optionally, the acquisition image to be processed includes:
By the fromPixels interface of Tensorflow, the image to be processed is obtained.
Optionally, the first information is handled by the sub-pix convolutional layer in the super-resolution model described,
After obtaining super-resolution image, the method also includes:
In graphics processor GPU, based on Tensor data structure and preset tinter to the super-resolution figure
As carrying out rendering processing, target image is obtained.
Second aspect, the embodiment of the invention discloses a kind of super-resolution model generating methods, are applied to browser, described
Method includes:
By the default CNN model of server identified documentation format, the browser identified documentation format is converted to
First CNN model；
Load the first CNN model；
Sub-pix convolutional layer is generated by compiler, by the first CNN model and the sub-pix convolutional layer shape
At model be determined as super-resolution model.
Optionally, the server identified documentation format is checkponit format；It is described that server be can recognize into text
The default CNN model of part format, is converted to the first CNN model of the browser identified documentation format, comprising:
By the default CNN model of checkponit format, be converted to by Tensorflow data transformation interface
The CNN model of SavedModel format；
By the CNN model of the SavedModel format, the browser is converted to by tfjs-converter tool
First CNN model of identified documentation format.
Optionally, load the first CNN model, comprising:
By the loadFrozenModel interface of tfjs-converter tool, the first CNN model is loaded.
To achieve the above object of the invention, the embodiment of the invention also discloses a kind of image processing apparatus, are applied to browser,
Described device includes:
Image collection module, for obtaining image to be processed；
First information determining module, for the image to be processed to be passed through in super-resolution model pre-loaded first
Convolutional neural networks CNN model is handled, and the first information is obtained；Wherein, the first CNN model is to default CNN model
The CNN model of the browser identified documentation format formatted；
Super-resolution image determining module, for the first information to be passed through the sub-pix in the super-resolution model
Convolutional layer processing, obtains super-resolution image.
Optionally, the first information determining module, described device further include:
First transform subblock, for being turned by Tensorflow data by the default CNN model of checkponit format
Alias is converted to the CNN model of SavedModel format；
First CNN model determines submodule, for passing through tfjs- for the CNN model of the SavedModel format
Converter tool is converted to the first CNN model；
First CNN model loads submodule, for loading the first CNN model.
Optionally, described image obtains module, specifically for passing through the fromPixels interface of Tensorflow, obtains institute
State image to be processed.
Optionally, described device further include:
Target image determining module, in graphics processor GPU, based on Tensor data structure and preset
Color device carries out rendering processing to the super-resolution image, obtains target image.
The third aspect, the embodiment of the invention also discloses a kind of super-resolution model generating means, are applied to browser, institute
Stating device includes:
First CNN model determining module, for being converted to institute for the default CNN model of server identified documentation format
State the first CNN model of browser identified documentation format；
First CNN model loading module, for loading the first CNN model；
Super-resolution model determining module, for generating sub-pix convolutional layer by compiler, by the first CNN mould
The model that type and the sub-pix convolutional layer are formed is determined as super-resolution model.
Optionally, the server identified documentation format is checkponit format；The first CNN model determines mould
Block, comprising:
Transform subblock, for being connect by Tensorflow data conversion by the default CNN model of checkponit format
Mouth is converted to the CNN model of SavedModel format；
First CNN model determines submodule, for passing through tfjs- for the CNN model of the SavedModel format
Converter tool is converted to the first CNN model of the browser identified documentation format.
Optionally, the first CNN model loading module, specifically for passing through tfjs-converter tool
LoadFrozenModel interface loads the first CNN model.
Fourth aspect, the embodiment of the invention also discloses a kind of electronic equipment, including processor, communication interface, memory
And communication bus, wherein processor, communication interface, memory complete mutual communication by communication bus；
Memory, for storing computer program；
Processor when for executing the program stored on memory, is realized any described in above-mentioned image processing method
Method and step.
5th aspect, the embodiment of the invention also discloses a kind of electronic equipment, including processor, communication interface, memory
And communication bus, wherein processor, communication interface, memory complete mutual communication by communication bus；
Memory, for storing computer program；
Processor when for executing the program stored on memory, is realized in above-mentioned super-resolution model generating method
Any method and step.
Another aspect, it is described computer-readable to deposit the embodiment of the invention also discloses a kind of computer readable storage medium
It is stored with computer program in storage media, when the computer program is executed by processor, realizes in above-mentioned image processing method
Any method and step.
Another aspect, it is described computer-readable to deposit the embodiment of the invention also discloses a kind of computer readable storage medium
It is stored with computer program in storage media, when the computer program is executed by processor, realizes that above-mentioned super-resolution model is raw
At the method and step any in method.
The embodiment of the invention provides a kind of image processing methods, model treatment method, apparatus, electronic equipment, realize
It is the super-resolution image that obtains on a web browser, close with the super-resolution image image quality that server obtains.The embodiment of the present invention
In, by separately handling CNN model and sub-pix convolutional layer, as first the identifiable default CNN model of server is turned
It changes the identifiable first CNN model of browser into, and then disposes sub-pix convolutional layer again, and in the prior art due to tfjs-
The function limitation of converter tool will be browser comprising the super-resolution model conversion of CNN layers and sub-pix convolutional layer
After holding identifiable object module, is compared, improved with the super-resolution image that the object module handles image
The image quality of super-resolution image, so that after carrying out super-resolution processing to image by the super-resolution model separately handled
Obtained super-resolution image can have similar image quality with the super-resolution image obtained after server-side processes.
Detailed description of the invention
In order to more clearly explain the embodiment of the invention or the technical proposal in the existing technology, to embodiment or will show below
There is attached drawing needed in technical description to be briefly described, it should be apparent that, the accompanying drawings in the following description is only this
Some embodiments of invention for those of ordinary skill in the art without creative efforts, can be with
It obtains other drawings based on these drawings.
Fig. 1 is a kind of image processing method flow chart of the embodiment of the present invention；
Fig. 2 is picture comparison diagram in a kind of image processing method of the embodiment of the present invention；
Fig. 3 is pre-loaded first convolutional neural networks CNN model in a kind of image processing method of the embodiment of the present invention
Method flow diagram；
Fig. 4 is image rendering method flow chart in a kind of image processing method of the embodiment of the present invention；
Fig. 5 is a kind of super-resolution model generating method flow chart of the embodiment of the present invention；
Fig. 6 (a) is a kind of image processing method flow chart of the prior art；
Fig. 6 (b) is a kind of image processing method flow chart of the embodiment of the present invention；
Fig. 7 is a kind of image processing apparatus structural schematic diagram of the embodiment of the present invention；
Fig. 8 is a kind of super-resolution model generating means structural schematic diagram of the embodiment of the present invention；
Fig. 9 is a kind of electronic equipment structural schematic diagram of the embodiment of the present invention；
Figure 10 is a kind of electronic equipment structural schematic diagram disclosed by the embodiments of the present invention.
Specific embodiment
Following will be combined with the drawings in the embodiments of the present invention, and technical solution in the embodiment of the present invention carries out clear, complete
Site preparation description, it is clear that described embodiments are only a part of the embodiments of the present invention, instead of all the embodiments.It is based on
Embodiment in the present invention, it is obtained by those of ordinary skill in the art without making creative efforts every other
Embodiment shall fall within the protection scope of the present invention.
The scaling of image resolution ratio and image in application layer effect be it is the same, in system or image procossing related software
The width height compared with original size greatly directly is arranged to image to see details clearly, or in the webpage of browser in enlarged drawing,
These are the super-resolution processings in the image done by specific algorithm.Traditional image scaling processing is mainly based upon slotting
It is worth thought, including arest neighbors interpolation method, bilinear interpolation, quadratic interpolattion, bi-cubic interpolation method, cubic spline interpolation
Method, sinc function interpolation etc..Such method have lower computation complexity, the fast feature of processing speed, but amplify after
Poor in image definition strong man meaning.
In recent years, the image super-resolution research based on deep learning becomes a heat of academia and industry with application
Point, these methods model profound based on the training of mass picture data, exports high-resolution image by model prediction, raw
At high-definition picture very big promotion is had based on the algorithm of difference compared to tradition in effect.But it is based on deep learning
Image super-resolution method computation complexity it is higher, can only generally be deployed in server end, utilize powerful GPU
(Graphics Processing Unit, graphics processor) or special API (Application Programming
Interface, application programming interface) chip provides processing capability in real time.
Image is carried out on the browser of terminal (such as PC (personal computer, personal computer), mobile phone)
High-resolution image is downloaded to browser again compared to Image Super Resolution Processing is done in server end by super-resolution processing
Mode, there is the advantage saved bandwidth and flow, save network latency.It is instructed in the related technology using deep learning method
The super-resolution model got include CNN (Convolutional Neural Network, convolutional neural networks) layer and
Sub-pix convolutional layer.On the server by super-resolution model deployment, then convert and load on a web browser the oversubscription
Resolution model handles low-resolution image by the super-resolution model, finally exports high-definition picture.
However, being handled using the existing super-resolution model comprising CNN layers and sub-pix convolutional layer image
Method in, due to the function limitation of tfjs-converter tool, by the super-resolution comprising CNN layers and sub-pix convolutional layer
Rate model conversion is the super-resolution that is handled with the object module image after the identifiable object module of browser end
Rate image, compared to the super-resolution image directly obtained on the server with super-resolution model, image quality is declined.Cause
This, the super-resolution image image quality more phase how realizing the super-resolution image obtained on a web browser, being obtained with server
Closely, it is still a problem to be solved.
To solve the above problems, super-resolution model is cut into two parts by the embodiment of the present invention, first part is CNN mould
Type, second part is sub-pix convolutional layer, and then the identifiable CNN model file format of server end is converted to browser end
First CNN model is deployed in browser end by identifiable first CNN model, and on the basis of the first CNN model again
Sub-pix convolutional layer is disposed, the super-resolution model of the embodiment of the present invention is obtained, by the super-resolution model to figure to be processed
As carrying out super-resolution processing, obtain high-resolution target image, realize the super-resolution image obtained on a web browser, with
The super-resolution image image quality that server obtains is more close.The following contents illustrates:
In a first aspect, the embodiment of the invention discloses a kind of image processing methods, as shown in Figure 1.Fig. 1 is that the present invention is implemented
A kind of image processing method flow chart of example, is applied to browser, and method includes:
S101 obtains image to be processed；
In the embodiment of the present invention, carried out at super-resolution by be processed image of the super-resolution model to low resolution
Reason.The super-resolution model includes pre-loaded the first CNN model on a web browser, and on the basis of the first CNN model
The sub-pix convolutional layer of deployment.
CNN is a kind of feedforward neural network, is made of one or more convolutional layers and full-mesh layer, while also including association
Weight and pond layer.Compared with other deep learning structures, convolutional neural networks can provide in terms of image and speech recognition
Better result.
In this step, the image to be processed of low resolution can be obtained by browser searches to super-resolution model, or
Person obtains low-resolution image that user uploads to super-resolution model.
Optionally, image to be processed is obtained in S101 includes:
By the fromPixels interface of Tensorflow, image to be processed is obtained.
TensorFlow is the second generation artificial intelligence learning system based on DistBelief research and development, is by complicated data
The system that structural transmission carries out analysis and treatment process into artificial intelligence nerve net.TensorFlow can be used for speech recognition
Or the multinomial machine deep learning such as image recognition field.
In the present embodiment, it can be trained in a browser based on the machine learning frame Tensorflow.js of Tensorflow
And the JavaScript line of bottom can be used as in the JavaScript of TensorFlow in deployment neural network model
Property algebra library or advanced API form model on a web browser, obtain the super-resolution model of the embodiment of the present invention, it is super by this
The image processing method of the resolution model realization embodiment of the present invention.
In this step, after browser obtains the image to be processed of low resolution, pass through TensorFlow's
FromPixels interface obtains image to be processed to super-resolution model.
Image to be processed is passed through the first convolutional neural networks CNN model pre-loaded in super-resolution model by S102
It is handled, obtains the first information；Wherein, the first CNN model is the browser formatted to default CNN model
The CNN model of identified documentation format.
The deployment of CNN model is preset in the embodiment of the present invention on the server, which is general image procossing
CNN has been directed to that details are not described herein again specifically by implementing personnel's setting in the prior art.First CNN model is that server end can be known
The default CNN model of other file format, the CNN model for the browser identified documentation format that conversion process obtains.
In this step, image to be processed is handled by the first CNN model in super-resolution model, by this
One or more convolutional layers of one CNN model, extract the image to be processed and feature identification, and obtaining corresponding to should be to
The first information of image is handled, which is to the enhanced information of low resolution image quality to be processed.
The first information is handled by the sub-pix convolutional layer in super-resolution model, obtains super-resolution image by S103.
Sub-pix convolutional layer is a kind of picture up-sampling method, if r is amplification factor, this layer of meeting is by the low of r*r dimension
Resolution figure image is assembled into high-definition picture.
In this step, the first information will be obtained in S102, amplified by the sub-pix convolutional layer of super-resolution model
Processing, and handled by rendering, obtain high-resolution target image.
The embodiment of the invention provides in a kind of image processing method, by by CNN model and sub-pix convolutional layer
Separately processing, as first by the identifiable default CNN model conversion of server at the identifiable first CNN model of browser, into
And sub-pix convolutional layer is disposed again, it and in the prior art due to the function limitation of tfjs-converter tool, will include CNN layers
And after the super-resolution model of sub-pix convolutional layer is all converted to the identifiable object module of browser end, with the object module
The super-resolution image handled image is compared, and the image quality of super-resolution image is improved, so that by separately locating
It manages obtained super-resolution model and carries out the super-resolution image obtained after super-resolution processing to image, it can be with server end
The super-resolution image obtained after processing has similar image quality.
In embodiments of the present invention, it can be realized by the machine learning frame Tensorflow.js of Tensorflow above-mentioned
Image processing method.
Level API (the Application Programming of neural network can be used in TensorFlow.js
Interface, application programming interface) building model, and WebGL (Web Graphics is used in a browser
Library, web graph shape library) the complicated data visualization application of creation.Therefore, it can make full use of browser and computer
Computing resource realize very more machine learning applications.Such as a model is trained to identify picture or voice in page end.
TensorFlow.js can provide high performance, wieldy machine learning building module, allow in browser
Upper training pattern.
In the embodiment of the present invention, by the features described above of TensorFlow.js, super-resolution mould is constructed in browser end
Type is handled image to be processed by the first convolutional neural networks CNN model pre-loaded in the super-resolution model,
Obtain the first information；By the first information by the sub-pix convolutional layer processing in super-resolution model, super-resolution image is obtained.
Alternatively, above-mentioned image processing method can be realized by the machine learning frame Keras.js of Keras.
Keras is a deep learning frame based on Theano, its design reference Torch uses Python
It writes, is the neural network library an of high modularization, support GPU and CPU (Central Processing Unit/
Processor, central processing unit).It can be write in the machine learning frame Keras.js frame of Keras in the embodiment of the present invention
The method for realizing the image procossing of the embodiment of the present invention.As write in super-resolution model using the database Mnist of Keras
The default CNN model in server end is built, and then this is preset into CNN model conversion on a web browser, generating can in browser
First CNN model of identification builds sub-pix convolutional layer on the basis of the first CNN model, forms building in browser end
Super-resolution model, using the image to be processed of the super-resolution model treatment low pixel, generate high pixel obtains super-resolution
Rate image.
The super-resolution image obtained through the embodiment of the present invention can be as shown in Figure 2.
Fig. 2 is picture comparison diagram in a kind of image processing method of the embodiment of the present invention, wherein 201 be image to be processed,
The size of the image can be h*w, wherein h indicates the length of image, and w indicates the width of image；202 be the embodiment of the present invention
Super-resolution model, including the first CNN model and sub-pix convolutional layer；203 is by the super-resolution model treatments
The size of obtained super-resolution image, the super-resolution image can 4h*4w.As by low-resolution image h*w to be processed,
Oversubscription handles to obtain high-resolution super-resolution image 4h*4w.
Optionally, in a kind of embodiment of image processing method of the present invention, image to be processed is being passed through into super-resolution
The first pre-loaded convolutional neural networks CNN model is handled in model, can pre-loaded before obtaining the first information
One convolutional neural networks CNN model, it is specific as shown in Figure 3.Fig. 3 is preparatory in a kind of image processing method of the embodiment of the present invention
Load the method flow diagram of the first convolutional neural networks CNN model, comprising:
S301 is converted to the default CNN model of checkponit format by Tensorflow data transformation interface
The CNN model of SavedModel format.
In the embodiment of the present invention, since the computation complexity of CNN model is higher, default CNN model is deployed in clothes
It is engaged on device, and then this is preset in CNN model conversion to browser, generate the identifiable first CNN model of browser.
This step is, first by the default CNN model of the identifiable checkponit format of server end, to pass through
Tensorflow data transformation interface is converted to the CNN model of SavedModel format.
S302, by the CNN model of SavedModel format, being converted to browser by tfjs-converter tool can know
First CNN model of other file format.
The CNN model of SavedModel format is converted into the first CNN model by tfjs-converter tool.
S303 loads the first CNN model.
After obtaining the first CNN model, the first CNN model is loaded by Tensorflow frame preset interface, such as
By the loadFrozenModel interface of tfjs-converter tool, the first CNN model is loaded.
In the embodiment of the present invention, by using the translation interface of Tensorflow.js frame, the service of being deployed in is realized
The default CNN model of device is converted into the identifiable first CNN model of browser, and then passes through the first CNN mould on a web browser
Type, which is realized, handles the oversubscription of image, compared with the prior art in downloaded to after server end carries out oversubscription processing to image
Browser end, this implementation improve the efficiency of image oversubscription processing.
Optionally, in a kind of embodiment of image processing method of the present invention, the first information is being passed through into super-resolution mould
Sub-pix convolutional layer processing in type, after obtaining super-resolution image, the method also includes:
In graphics processor GPU, based on Tensor data structure and preset tinter to super-resolution image into
Row rendering processing, obtains target image.
In the present embodiment, the first information that will be obtained by the first CNN model treatment passes through the sub- picture of super-resolution model
Plain convolutional layer enhanced processing (such as up-sampling treatment), obtains super-resolution image.Then by the super-resolution image in GPU
Rendering processing is carried out, target image is obtained.
As based on the renderWebGL application programming interface in Tensor data structure, by super-resolution image
Data texturing be loaded into default tinter；By presetting the down-sampling algorithm of tinter, to the texture of super-resolution image
Data are handled, and RGBA data are obtained；To obtaining RGBA data and carry out rendering to show, target image is obtained.
Shader application is one group for the use when executing rendering task of computer picture resource in computer picture field
Instruction, for calculating the color or light and shade of image.It can be used for handling some special-effects or Video post-processing.It is logical
Custom says that tinter tells how computer goes to draw object using a kind of distinctive method.Image pipeline of the tinter in GPU
Middle execution, it defines the position (vertex of fixed point using GLSL (OpenGL Shading Language, OpenGL shading language)
Tinter) and pixel (piece member tinter) color.
It specifically can be as shown in figure 4, Fig. 4 be image rendering method process in a kind of image processing method of the embodiment of the present invention
Figure, comprising:
The data texturing of super-resolution image is loaded into pre- by S401 by renderWebGL application programming interface
If in tinter.
In the embodiment of the present invention, based in TensorFlow frame, write in Tensor data structure in advance
RenderWebGL application programming interface, after super-resolution model is to image procossing to be processed to super-resolution image,
The data texturing of super-resolution image is loaded into default tinter by renderWebGL application programming interface.
For example, passing through the sub-pix convolutional layer enhanced processing of super-resolution model, the texture of obtained super-resolution image
The data format of data are as follows:
G1000B1000R1000G2000B2000R2000
Wherein, G indicates the channel G in rgb color mode, and B indicates the channel B in rgb color mode, and R indicates rgb color
The channel R in mode, 0 indicates the value of insertion, G1000B1000R1000 indicate by sub-pix convolutional layer treated first
Pixel, G2000B2000R2000 indicates by sub-pix convolutional layer treated second pixel point.
S402 handles the data texturing of super-resolution image, obtains by presetting the down-sampling algorithm of tinter
RGBA data.
RGBA is the color space data for representing Red (red) Green (green) Blue (blue) and channel Alpha, is
It attached the color representation data of channel Alpha information in RGB model.
In this step, the data texturing of above-mentioned super-resolution image is passed through into renderWebGL application programming
Interface is loaded into the channel R of default tinter.It presets in tinter according to default bias amount to the super-resolution image at this
Each pixel carries out texture sampling three times, is sequentially completed the down-sampling to the super-resolution image.As first pixel
First time texture sampling data be G1000, second of texture sampling data is B1000, third time texture sampling data are
R1000.Then, first data of each texture sampling of each pixel are obtained as target data, by three groups of sampled datas
Three-dimensional data value of the corresponding each target data as a pixel, and it is upper plus transparency channel after the three-dimensional data value
Data A obtains the RGBA data of each pixel.Wherein, A can be directly disposed as 1.
S403 obtains target image to obtaining RGBA data and carry out rendering to show.
As it can be seen that through the embodiment of the present invention, it can be achieved that rendering processing is carried out to image directly in GPU, compared with existing skill
In art, it is addressable that the super-resolution image after super-resolution model treatment is downloaded into JavaSript program from GPU
In memory, the super-resolution image of one-dimension array format, which is then converted to the CPU such as canvas or image, can recognize format
Image, to carry out image rendering in CPU, the embodiment of the present invention carries out rendering processing directly in GPU, improves at image
The performance of reason.
Second aspect, the embodiment of the invention discloses a kind of super-resolution model generating methods, as shown in Figure 5.Fig. 5 is this
A kind of super-resolution model generating method flow chart of inventive embodiments, is applied to browser, and method includes:
The default CNN model of server identified documentation format is converted to browser identified documentation format by S501
First CNN model.
The embodiment of the present invention will extract characteristics of image and identify that the default CNN model of characteristics of image is deployed in server
End, the default CNN model of the identifiable file format of the server end as generated.This step is that browser end can by server
The default CNN model of the file format of identification, is converted to the first CNN model of browser identified documentation format.
Optionally, server identified documentation format is checkponit format in S501；By server identified documentation
The default CNN model of format, is converted to the first CNN model of browser identified documentation format, comprising:
Step 1 is converted to the default CNN model of checkponit format by Tensorflow data transformation interface
The CNN model of SavedModel format.
Step 2, by the CNN model of SavedModel format, being converted to browser by tfjs-converter tool can
Identify the first CNN model of file format.
S502 loads the first CNN model.
The above-mentioned default CNN model by server identified documentation format, is converted to browser identified documentation format
After first CNN model, the object module is loaded into browser end in this step.
Optionally, the first CNN model is loaded in S502, comprising:
By the loadFrozenModel interface of tfjs-converter tool, the first CNN model is loaded.
S503 generates sub-pix convolutional layer by compiler, and the first CNN model and sub-pix convolutional layer are formed
Model is determined as super-resolution model.
On the basis of the first CNN model that above-mentioned S502 is generated, pass through the sub-pix convolutional layer being arranged in compiler
Compiled code generates sub-pix convolutional layer, and the model that the first CNN model and sub-pix convolutional layer are formed is determined as super-resolution
Rate model.
In a kind of super-resolution model generating method provided in an embodiment of the present invention, by server identified documentation format
Default CNN model conversion at browser identified documentation format the first CNN model, and then by the first CNN model load
Onto browser, sub-pix convolutional layer is constructed on the basis of the first CNN model, and then generate the oversubscription of the embodiment of the present invention
Resolution model can be realized by the super-resolution model and carry out oversubscription processing to the image to be processed of low pixel, generate high pixel
Target image.
A kind of image processing method disclosed in embodiment in order to better illustrate the present invention, can compare the embodiment of the present invention
Image processing method and image processing method in the prior art.Wherein, Fig. 6 (a) is a kind of image processing method of the prior art
Flow chart；Fig. 6 (b) is a kind of image processing method flow chart of the embodiment of the present invention.
The image processing method of the prior art in Fig. 6 (a), comprising:
S611 uploads image to be processed: uploading the rgb format of image to be processed into GPU by fromPixels interface；
Super-resolution model treatment: S612 is handled the image to be processed by super-resolution model；Wherein, should
Super-resolution includes CNN model and sub-pix convolutional layer；
Download information: treated information from GPU is downloaded to the addressable memory of JavaSript program by S613；
S614, format conversion: to treated, one-dimension array information is formatted, and it is identifiable to be converted to CPU
Canvas or image format；
Image rendering: S615 renders information after conversion in CPU, obtains target image.
The image processing method of the embodiment of the present invention in Fig. 6 (b), comprising:
S621 uploads image to be processed: uploading the rgb format of image to be processed into GPU by fromPixels interface；
Super-resolution model treatment: the image to be processed is passed through in super-resolution model pre-loaded first by S622
Convolutional neural networks CNN model is handled, and the first information is obtained；The first information is passed through to the sub-pix of super-resolution model
Convolutional layer processing, obtains super-resolution image, wherein the first CNN model is the default CNN by server identified documentation format
The CNN model for the browser identified documentation format that model conversion obtains；
S623 carries out image rendering in GPU: in graphics processor GPU, based on Tensor data structure and presetting
Tinter rendering processing is carried out to super-resolution image, obtain target image.
Comparison is as it can be seen that in a kind of image processing method of the embodiment of the present invention, by by CNN model and sub-pix volume
Lamination is separately handled, as first by the identifiable default CNN model conversion of server at the identifiable first CNN mould of browser
Type, and then dispose sub-pix convolutional layer again and due to the function limitation of tfjs-converter tool, will include in the prior art
After the super-resolution model conversion of CNN layers and sub-pix convolutional layer is the identifiable object module of browser end, with the target
The super-resolution image that model handles image is compared, and the image quality of super-resolution image is improved, so that by dividing
It opens the super-resolution model that processing obtains and carries out the super-resolution image obtained after super-resolution processing to image, it can be with service
The super-resolution image obtained after the processing of device end has similar image quality.
In addition, the whole process of image oversubscription and rendering processing is carried out directly in GPU in the embodiment of the present invention,
Compared in conventional images processing method, need to download super-resolution image after carrying out image oversubscription from GPU to memory, with
And to after downloading super-resolution image carry out Data Format Transform compare, reduce data downloading and data conversion when
Between, accelerate the speed of image procossing.To sum up, the image processing method of the embodiment of the present invention is optimized based on deep learning figure
The whole efficiency of piece super-resolution processing.
The third aspect, the embodiment of the invention also discloses a kind of image processing apparatus, as shown in Figure 7.Fig. 7 is that the present invention is real
A kind of image processing apparatus structural schematic diagram of example is applied, browser is applied to, device includes:
Image collection module 701, for obtaining image to be processed；
First information determining module 702, for image to be processed to be passed through in super-resolution model pre-loaded first
Convolutional neural networks CNN model is handled, and the first information is obtained；Wherein, the first CNN model is to carry out to default CNN model
The CNN model for the browser identified documentation format that format is converted to；
Super-resolution image determining module 703, for the first information to be passed through the sub-pix convolution in super-resolution model
Layer processing, obtains super-resolution image.
The embodiment of the invention provides in a kind of image processing apparatus, by by CNN model and sub-pix convolutional layer point
Processing is opened, as first by the identifiable default CNN model conversion of server at the identifiable first CNN model of browser, in turn
Dispose sub-pix convolutional layer again, and in the prior art due to the function limitation of tfjs-converter tool, will comprising CNN layers with
And after the super-resolution model of sub-pix convolutional layer is all converted to the identifiable object module of browser end, with the object module pair
The super-resolution image that image is handled is compared, and the image quality of super-resolution image is improved, so that by separately handling
Obtained super-resolution model carries out obtained super-resolution image after super-resolution processing to image, can at server end
The super-resolution image obtained after reason has similar image quality.
Optionally, in a kind of embodiment of image processing apparatus of the present invention, first information determining module 702, comprising:
First transform subblock, for being turned by Tensorflow data by the default CNN model of checkponit format
Alias is converted to the CNN model of SavedModel format；
First CNN model determines submodule, for passing through tfjs- for the CNN model of SavedModel format
Converter tool is converted to the first CNN model of browser identified documentation format；
First CNN model loads submodule, for loading the first CNN model by preset interface.
Optionally, in a kind of embodiment of image processing apparatus of the present invention, image collection module 701 is specifically used for logical
The fromPixels interface for crossing Tensorflow, obtains image to be processed.
Optionally, in a kind of embodiment of image processing apparatus of the present invention, device further include:
Target image determining module, in graphics processor GPU, based on Tensor data structure and preset
Color device carries out rendering processing to super-resolution image, obtains target image.
Fourth aspect, the embodiment of the invention also discloses a kind of super-resolution model generating means, as shown in Figure 8.Fig. 8 is
A kind of super-resolution model generating means structural schematic diagram of the embodiment of the present invention, is applied to browser, and device includes:
First CNN model determining module 801, for being converted to the default CNN model of server identified documentation format
First CNN model of browser identified documentation format；
First CNN model loading module 802, for loading the first CNN model；
Super-resolution model determining module 803, for generating sub-pix convolutional layer by compiler, by the first CNN mould
The model that type and sub-pix convolutional layer are formed is determined as super-resolution model.
In a kind of super-resolution model generating means provided in an embodiment of the present invention, by server identified documentation format
Default CNN model conversion at browser identified documentation format the first CNN model, and then by the first CNN model load
Onto browser, sub-pix convolutional layer is constructed on the basis of the first CNN model, and then generate the oversubscription of the embodiment of the present invention
Resolution model can be realized by the super-resolution model and carry out oversubscription processing to the image to be processed of low pixel, generate high pixel
Target image.
Optionally, in a kind of embodiment of super-resolution model generating means of the present invention, the first CNN model determining module
801, comprising:
Second transform subblock, for being turned by Tensorflow data by the default CNN model of checkponit format
Alias is converted to the CNN model of SavedModel format；
First CNN model determines submodule, for passing through tfjs- for the CNN model of SavedModel format
Converter tool is converted to the first CNN model of browser identified documentation format.
Optionally, in a kind of embodiment of super-resolution model generating means of the present invention, the first CNN model loading module
802, specifically for passing through the loadFrozenModel interface of tfjs-converter tool, load the first CNN model.
5th aspect, the embodiment of the invention also discloses a kind of electronic equipment, as shown in Figure 9.Fig. 9 is the embodiment of the present invention
A kind of electronic equipment structural schematic diagram, including processor 901, communication interface 902, memory 903 and communication bus 904,
In, processor 901, communication interface 902, memory 903 complete mutual communication by communication bus 904；
Memory 903, for storing computer program；
Processor 901 when for executing the program stored on memory 903, realizes following methods step:
Obtain image to be processed；
By image to be processed by the first convolutional neural networks CNN model pre-loaded in super-resolution model at
Reason, obtains the first information；Wherein, the first CNN model is that the browser formatted to default CNN model conversion can
Identify the CNN model of file format；
By the first information by the sub-pix convolutional layer processing in super-resolution model, super-resolution image is obtained.
The communication bus 904 that above-mentioned electronic equipment is mentioned can be Peripheral Component Interconnect standard (Peripheral
Component Interconnect, PCI) bus or expanding the industrial standard structure (Extended Industry Standard
Architecture, EISA) bus etc..The communication bus 904 can be divided into address bus, data/address bus, control bus etc..For
Convenient for indicating, only indicated with a thick line in figure, it is not intended that an only bus or a type of bus.
Communication interface 902 is for the communication between above-mentioned electronic equipment and other equipment.
Memory 903 may include random access memory (Random Access Memory, RAM), also may include
Nonvolatile memory (Non-Volatile Memory, NVM), for example, at least a magnetic disk storage.Optionally, memory
903 can also be that at least one is located remotely from the storage device of aforementioned processor 901.
Above-mentioned processor 901 can be general processor, including central processing unit (Central Processing
Unit, CPU), network processing unit (Network Processor, NP) etc.；It can also be digital signal processor (Digital
Signal Processing, DSP), specific integrated circuit (Application Specific Integrated Circuit,
ASIC), field programmable gate array (Field-Programmable Gate Array, FPGA) or other programmable logic
Device, discrete gate or transistor logic, discrete hardware components.
The embodiment of the invention provides in a kind of electronic equipment, by the way that CNN model and sub-pix convolutional layer are separated
Processing, as first by the identifiable default CNN model conversion of server at the identifiable first CNN model of browser, Jin Erzai
Dispose sub-pix convolutional layer, and in the prior art due to the function limitation of tfjs-converter tool, will comprising CNN layers and
The super-resolution model conversion of sub-pix convolutional layer be the identifiable object module of browser end after, with the object module to image
The super-resolution image handled is compared, and the image quality of super-resolution image is improved, so that being obtained by separately processing
Super-resolution model obtained super-resolution image after super-resolution processing is carried out to image, can with after server-side processes
Obtained super-resolution image has similar image quality.
6th aspect, the embodiment of the invention also discloses a kind of electronic equipment, as shown in Figure 10.Figure 10 is that the present invention is implemented
A kind of electronic equipment structural schematic diagram disclosed in example, including processor 1001, communication interface 1002, memory 1003 and communication are total
Line 1004, wherein processor 1001, communication interface 1002, memory 1003 complete mutual lead to by communication bus 1004
Letter；
Memory 1003, for storing computer program；
Processor 1001 when for executing the program stored on memory 1003, realizes following methods step:
By the default CNN model of server identified documentation format, the first of browser identified documentation format is converted to
CNN model；
Load the first CNN model；
Sub-pix convolutional layer is generated by compiler, the model that the first CNN model and sub-pix convolutional layer are formed
It is determined as super-resolution model.
The communication bus 1004 that above-mentioned electronic equipment is mentioned can be Peripheral Component Interconnect standard (Peripheral
Component Interconnect, PCI) bus or expanding the industrial standard structure (Extended Industry Standard
Architecture, EISA) bus etc..The communication bus 1004 can be divided into address bus, data/address bus, control bus etc..
Only to be indicated with a thick line in figure, it is not intended that an only bus or a type of bus convenient for indicating.
Communication interface 1002 is for the communication between above-mentioned electronic equipment and other equipment.
Memory 1003 may include random access memory (Random Access Memory, RAM), also may include
Nonvolatile memory (Non-Volatile Memory, NVM), for example, at least a magnetic disk storage.Optionally, memory
1003 can also be that at least one is located remotely from the storage device of aforementioned processor 1001.
Above-mentioned processor 1001 can be general processor, including central processing unit (Central Processing
Unit, CPU), network processing unit (Network Processor, NP) etc.；It can also be digital signal processor (Digital
Signal Processing, DSP), specific integrated circuit (Application Specific Integrated Circuit,
ASIC), field programmable gate array (Field-Programmable Gate Array, FPGA) or other programmable logic
Device, discrete gate or transistor logic, discrete hardware components.
In a kind of electronic equipment provided in an embodiment of the present invention, by the default CNN mould of server identified documentation format
Type is converted into the first CNN model of browser identified documentation format, and then the first CNN model is loaded on browser,
Sub-pix convolutional layer is constructed on the basis of the first CNN model, and then generates the super-resolution model of the embodiment of the present invention, is led to
It crosses the super-resolution model and can realize and oversubscription processing is carried out to the image to be processed of low pixel, generate the target image of high pixel.
Another aspect, the embodiment of the invention also discloses a kind of computer readable storage medium, computer-readable storage mediums
It is stored with computer program in matter, when computer program is executed by processor, realizes side any in above-mentioned image processing method
Method step.
The embodiment of the invention provides in a kind of computer readable storage medium, by by CNN model and sub-pix
Convolutional layer is separately handled, as first by the identifiable default CNN model conversion of server at the identifiable first CNN mould of browser
Type, and then dispose sub-pix convolutional layer again and due to the function limitation of tfjs-converter tool, will include in the prior art
After the super-resolution model conversion of CNN layers and sub-pix convolutional layer is the identifiable object module of browser end, with the target
The super-resolution image that model handles image is compared, and the image quality of super-resolution image is improved, so that by dividing
It opens the super-resolution model that processing obtains and carries out the super-resolution image obtained after super-resolution processing to image, it can be with service
The super-resolution image obtained after the processing of device end has similar image quality.
Another aspect, the embodiment of the invention also discloses a kind of computer readable storage medium, computer-readable storage mediums
It is stored with computer program in matter, when computer program is executed by processor, realizes in above-mentioned super-resolution model generating method
Any method and step.
In a kind of computer readable storage medium provided in an embodiment of the present invention, by server identified documentation format
CNN model conversion is preset into the first CNN model of browser identified documentation format, and then the first CNN model is loaded into
On browser, sub-pix convolutional layer is constructed on the basis of the first CNN model, and then generate the super-resolution of the embodiment of the present invention
Rate model can be realized by the super-resolution model and carry out oversubscription processing to the image to be processed of low pixel, generate high pixel
Target image.
Another aspect, the embodiment of the invention also discloses a kind of computer program products comprising instruction, when it is being calculated
When being run on machine, so that computer executes method and step any in image processing method in above-described embodiment.
In a kind of computer program product comprising instruction provided in an embodiment of the present invention, by by CNN model and
Sub-pix convolutional layer is separately handled, as first by the identifiable default CNN model conversion of server at browser identifiable
One CNN model, and then dispose sub-pix convolutional layer again, in the prior art due to the function of tfjs-converter tool by
Limit, by comprising the super-resolution model conversion of CNN layer and sub-pix convolutional layer for after the identifiable object module of browser end,
It is compared with the super-resolution image that the object module handles image, improves the image quality of super-resolution image, make
It obtains the super-resolution model by separately handling and carries out the super-resolution image obtained after super-resolution processing to image, it can
There is similar image quality with the super-resolution image obtained after server-side processes.
Another aspect, the embodiment of the invention also discloses a kind of computer program products comprising instruction, when it is being calculated
When being run on machine, so that computer executes method and step any in super-resolution model generating method in above-described embodiment.
In a kind of computer program product comprising instruction provided in an embodiment of the present invention, by server identified documentation
The default CNN model conversion of format at browser identified documentation format the first CNN model, and then by the first CNN model
It is loaded on browser, sub-pix convolutional layer is constructed on the basis of the first CNN model, and then generate the embodiment of the present invention
Super-resolution model can be realized by the super-resolution model and carry out oversubscription processing to the image to be processed of low pixel, generate high
The target image of pixel.
In the above-described embodiments, can come wholly or partly by software, hardware, firmware or any combination thereof real
It is existing.When implemented in software, it can entirely or partly realize in the form of a computer program product.Computer program product
Including one or more computer instructions.When loading on computers and executing computer program instructions, all or part of real estate
Raw process or function according to the embodiment of the present invention.Computer can be general purpose computer, special purpose computer, computer network,
Or other programmable devices.Computer instruction may be stored in a computer readable storage medium, or from a computer
Readable storage medium storing program for executing to another computer readable storage medium transmit, for example, computer instruction can from a web-site,
Computer, server or data center by wired (such as coaxial cable, optical fiber, Digital Subscriber Line (DSL)) or wireless (such as
Infrared, wireless, microwave etc.) mode transmitted to another web-site, computer, server or data center.Computer
Readable storage medium storing program for executing can be any usable medium or include one or more usable medium collection that computer can access
At the data storage devices such as server, data center.Usable medium can be magnetic medium, (for example, floppy disk, hard disk, magnetic
Band), optical medium (for example, DVD) or semiconductor medium (such as solid state hard disk Solid State Disk (SSD)) etc..
It should be noted that, in this document, relational terms such as first and second and the like are used merely to a reality
Body or operation are distinguished with another entity or operation, are deposited without necessarily requiring or implying between these entities or operation
In any actual relationship or order or sequence.Moreover, the terms "include", "comprise" or its any other variant are intended to
Non-exclusive inclusion, so that the process, method, article or equipment including a series of elements is not only wanted including those
Element, but also including other elements that are not explicitly listed, or further include for this process, method, article or equipment
Intrinsic element.In the absence of more restrictions, the element limited by sentence "including a ...", it is not excluded that
There is also other identical elements in process, method, article or equipment including element.
Each embodiment in this specification is all made of relevant mode and describes, same and similar portion between each embodiment
Dividing may refer to each other, and each embodiment focuses on the differences from other embodiments.Especially for device,
For electronic equipment, storage medium and computer program product embodiments, since it is substantially similar to the method embodiment, so
It is described relatively simple, the relevent part can refer to the partial explaination of embodiments of method.
The above is merely preferred embodiments of the present invention, it is not intended to limit the scope of the present invention.It is all in this hair
Any modification, equivalent replacement, improvement and so within bright spirit and principle, are included within the scope of protection of the present invention.
Claims (16)
1. a kind of image processing method, which is characterized in that be applied to browser, which comprises
Obtain image to be processed；
By the image to be processed by the first convolutional neural networks CNN model pre-loaded in super-resolution model at
Reason, obtains the first information；Wherein, the first CNN model is the browsing formatted to default CNN model
The CNN model of device identified documentation format；
The first information is handled by the sub-pix convolutional layer in the super-resolution model, obtains super-resolution image.
2. the method according to claim 1, wherein the image to be processed is passed through super-resolution mould described
The first pre-loaded convolutional neural networks CNN model is handled in type, and before obtaining the first information, the method is also wrapped
It includes:
By the default CNN model of checkponit format, SavedModel is converted to by Tensorflow data transformation interface
The CNN model of format；
By the CNN model of the SavedModel format, the first CNN mould is converted to by tfjs-converter tool
Type；
Load the first CNN model.
3. the method according to claim 1, wherein the acquisition image to be processed includes:
By the fromPixels interface of Tensorflow, the image to be processed is obtained.
4. the method according to claim 1, wherein the first information is passed through the super-resolution described
Sub-pix convolutional layer processing in model, after obtaining super-resolution image, the method also includes:
In graphics processor GPU, based on Tensor data structure and preset tinter to the super-resolution image into
Row rendering processing, obtains target image.
5. a kind of super-resolution model generating method, which is characterized in that be applied to browser, which comprises
By the default CNN model of server identified documentation format, the first of the browser identified documentation format is converted to
CNN model；
Load the first CNN model；
Sub-pix convolutional layer is generated by compiler, the first CNN model and the sub-pix convolutional layer are formed
Model is determined as super-resolution model.
6. according to the method described in claim 5, it is characterized in that, the server identified documentation format is checkponit
Format；By the default CNN model by server identified documentation format, the browser identified documentation format is converted to
The first CNN model, comprising:
By the default CNN model of checkponit format, SavedModel is converted to by Tensorflow data transformation interface
The CNN model of format；
By the CNN model of the SavedModel format, being converted to the browser by tfjs-converter tool can know
First CNN model of other file format.
7. according to the method described in claim 5, it is characterized in that, load the first CNN model, comprising:
By the loadFrozenModel interface of tfjs-converter tool, the first CNN model is loaded.
8. a kind of image processing apparatus, which is characterized in that be applied to browser, described device includes:
Image collection module, for obtaining image to be processed；
First information determining module, for the image to be processed to be passed through the first convolution pre-loaded in super-resolution model
Neural network CNN model is handled, and the first information is obtained；Wherein, the first CNN model is to carry out to default CNN model
The CNN model for the browser identified documentation format that format is converted to；
Super-resolution image determining module, for the first information to be passed through the sub-pix convolution in the super-resolution model
Layer processing, obtains super-resolution image.
9. device according to claim 8, which is characterized in that described device further include:
First conversion module, for passing through Tensorflow data transformation interface for the default CNN model of checkponit format
Be converted to the CNN model of SavedModel format；
First CNN model determining module, for passing through tfjs-converter for the CNN model of the SavedModel format
Tool is converted to the first CNN model；
First CNN model loading module, for loading the first CNN model.
10. device according to claim 8, which is characterized in that described image obtains module, specifically for passing through
The fromPixels interface of Tensorflow obtains the image to be processed.
11. device according to claim 8, which is characterized in that described device further include:
Target image determining module, for being based on Tensor data structure and preset tinter in graphics processor GPU
Rendering processing is carried out to the super-resolution image, obtains target image.
12. a kind of super-resolution model generating means, which is characterized in that be applied to browser, described device includes:
First CNN model determining module, for being converted to the default CNN model of server identified documentation format described clear
Look at the first CNN model of device identified documentation format；
First CNN model loading module, for loading the first CNN model；
Super-resolution model determining module, for by compiler generation sub-pix convolutional layer, by the first CNN model with
And the model that the sub-pix convolutional layer is formed is determined as super-resolution model.
13. device according to claim 12, which is characterized in that the server identified documentation format is
Checkponit format；The first CNN model determining module, comprising:
Transform subblock, for being turned by Tensorflow data transformation interface by the default CNN model of checkponit format
It is changed to the CNN model of SavedModel format；
First CNN model determines submodule, for passing through tfjs- for the CNN model of the SavedModel format
Converter tool is converted to the first CNN model of the browser identified documentation format.
14. device according to claim 12, which is characterized in that the first CNN model loading module is specifically used for logical
The loadFrozenModel interface of tfjs-converter tool is crossed, the first CNN model is loaded.
15. a kind of electronic equipment, which is characterized in that including processor, communication interface, memory and communication bus, wherein processing
Device, communication interface, memory complete mutual communication by communication bus；
Memory, for storing computer program；
Processor when for executing the program stored on memory, realizes any side in claim 1-4 or 5-7
Method step.
16. a kind of computer readable storage medium, which is characterized in that be stored with computer in the computer readable storage medium
Program when computer program is executed by processor, realizes any method and step in claim 1-4 or 5-7.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201811260269.8A CN109360153B (en) | 2018-10-26 | 2018-10-26 | Image processing method, super-resolution model generation method and device and electronic equipment |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201811260269.8A CN109360153B (en) | 2018-10-26 | 2018-10-26 | Image processing method, super-resolution model generation method and device and electronic equipment |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN109360153A true CN109360153A (en) | 2019-02-19 |
| CN109360153B CN109360153B (en) | 2023-05-02 |
Family
ID=65347088
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201811260269.8A Active CN109360153B (en) | 2018-10-26 | 2018-10-26 | Image processing method, super-resolution model generation method and device and electronic equipment |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN109360153B (en) |
Cited By (10)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN109887018A (en) * | 2019-02-28 | 2019-06-14 | 中国计量大学 | A kind of photon 3D imaging system based on deep learning |
| CN110135464A (en) * | 2019-04-18 | 2019-08-16 | 深兰科技（上海）有限公司 | A kind of image processing method, device, electronic equipment and storage medium |
| CN110430444A (en) * | 2019-08-12 | 2019-11-08 | 北京中科寒武纪科技有限公司 | A kind of video stream processing method and system |
| CN111522771A (en) * | 2020-04-20 | 2020-08-11 | 北京百度网讯科技有限公司 | Fundus image processing method, terminal device and storage medium |
| CN111859210A (en) * | 2019-04-29 | 2020-10-30 | 百度在线网络技术（北京）有限公司 | Image processing method, device, equipment and storage medium |
| CN112906829A (en) * | 2021-04-13 | 2021-06-04 | 成都四方伟业软件股份有限公司 | Digital recognition model construction method and device based on Mnist data set |
| CN113222815A (en) * | 2021-04-26 | 2021-08-06 | 北京奇艺世纪科技有限公司 | Image adjusting method and device, electronic equipment and readable storage medium |
| CN114088399A (en) * | 2021-10-20 | 2022-02-25 | 昆明理工大学 | Bearing fault online diagnosis method and system based on deep separable convolution |
| US11526545B2 (en) | 2020-02-10 | 2022-12-13 | Beijing Baidu Netcom Science Technology Co., Ltd. | Method of processing cover page image of multimedia information, client and electronic device |
| CN111859210B (en) * | 2019-04-29 | 2024-04-26 | 百度在线网络技术（北京）有限公司 | Image processing method, device, equipment and storage medium |
Citations (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN107368857A (en) * | 2017-07-24 | 2017-11-21 | 深圳市图芯智能科技有限公司 | Image object detection method, system and model treatment method, equipment, terminal |
| US20180075581A1 (en) * | 2016-09-15 | 2018-03-15 | Twitter, Inc. | Super resolution using a generative adversarial network |
| CN108259997A (en) * | 2018-04-02 | 2018-07-06 | 腾讯科技（深圳）有限公司 | Image correlation process method and device, intelligent terminal, server, storage medium |
| CN108492265A (en) * | 2018-03-16 | 2018-09-04 | 西安电子科技大学 | CFA image demosaicing based on GAN combines denoising method |
-
2018
- 2018-10-26 CN CN201811260269.8A patent/CN109360153B/en active Active
Patent Citations (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20180075581A1 (en) * | 2016-09-15 | 2018-03-15 | Twitter, Inc. | Super resolution using a generative adversarial network |
| CN107368857A (en) * | 2017-07-24 | 2017-11-21 | 深圳市图芯智能科技有限公司 | Image object detection method, system and model treatment method, equipment, terminal |
| CN108492265A (en) * | 2018-03-16 | 2018-09-04 | 西安电子科技大学 | CFA image demosaicing based on GAN combines denoising method |
| CN108259997A (en) * | 2018-04-02 | 2018-07-06 | 腾讯科技（深圳）有限公司 | Image correlation process method and device, intelligent terminal, server, storage medium |
Non-Patent Citations (1)
| Title |
|---|
| 龙法宁等: "基于深层卷积网络的单幅图像超分辨率重建模型", 《广西科学》 * |
Cited By (11)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN109887018A (en) * | 2019-02-28 | 2019-06-14 | 中国计量大学 | A kind of photon 3D imaging system based on deep learning |
| CN110135464A (en) * | 2019-04-18 | 2019-08-16 | 深兰科技（上海）有限公司 | A kind of image processing method, device, electronic equipment and storage medium |
| CN111859210A (en) * | 2019-04-29 | 2020-10-30 | 百度在线网络技术（北京）有限公司 | Image processing method, device, equipment and storage medium |
| CN111859210B (en) * | 2019-04-29 | 2024-04-26 | 百度在线网络技术（北京）有限公司 | Image processing method, device, equipment and storage medium |
| CN110430444A (en) * | 2019-08-12 | 2019-11-08 | 北京中科寒武纪科技有限公司 | A kind of video stream processing method and system |
| US11526545B2 (en) | 2020-02-10 | 2022-12-13 | Beijing Baidu Netcom Science Technology Co., Ltd. | Method of processing cover page image of multimedia information, client and electronic device |
| CN111522771A (en) * | 2020-04-20 | 2020-08-11 | 北京百度网讯科技有限公司 | Fundus image processing method, terminal device and storage medium |
| CN111522771B (en) * | 2020-04-20 | 2023-08-15 | 北京百度网讯科技有限公司 | Fundus image processing method, terminal device and storage medium |
| CN112906829A (en) * | 2021-04-13 | 2021-06-04 | 成都四方伟业软件股份有限公司 | Digital recognition model construction method and device based on Mnist data set |
| CN113222815A (en) * | 2021-04-26 | 2021-08-06 | 北京奇艺世纪科技有限公司 | Image adjusting method and device, electronic equipment and readable storage medium |
| CN114088399A (en) * | 2021-10-20 | 2022-02-25 | 昆明理工大学 | Bearing fault online diagnosis method and system based on deep separable convolution |
Also Published As
| Publication number | Publication date |
|---|---|
| CN109360153B (en) | 2023-05-02 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN109360153A (en) | Image processing method, super-resolution model generating method, device and electronic equipment | |
| CN109410141A (en) | A kind of image processing method, device, electronic equipment and storage medium | |
| CN109118432B (en) | Image super-resolution reconstruction method based on rapid cyclic convolution network | |
| CN109472270B (en) | Image style conversion method, device and equipment | |
| Dong et al. | Accelerating the super-resolution convolutional neural network | |
| CN110136066B (en) | Video-oriented super-resolution method, device, equipment and storage medium | |
| US20210209459A1 (en) | Processing method and system for convolutional neural network, and storage medium | |
| CN108022212A (en) | High-resolution pictures generation method, generating means and storage medium | |
| CN107729996A (en) | Zero coefficient skips convolutional neural networks engine | |
| WO2011092696A1 (en) | Method and system for generating an output image of increased pixel resolution from an input image | |
| JP2023523047A (en) | Image processing method, apparatus, computer equipment and storage medium | |
| CN111476719A (en) | Image processing method, image processing device, computer equipment and storage medium | |
| WO2021115403A1 (en) | Image processing method and apparatus | |
| WO2023169582A1 (en) | Image enhancement method and apparatus, device, and medium | |
| Kim et al. | Pynet-ca: enhanced pynet with channel attention for end-to-end mobile image signal processing | |
| CN111353938A (en) | Image super-resolution learning method based on network feedback | |
| KR102493492B1 (en) | Method and Device for Fast Adaptation through Meta-learning of Super Resolution Model | |
| CN114494022B (en) | Model training method, super-resolution reconstruction method, device, equipment and medium | |
| Li et al. | Lightweight single image super-resolution with dense connection distillation network | |
| CN113298716A (en) | Image super-resolution reconstruction method based on convolutional neural network | |
| Li et al. | D2c-sr: A divergence to convergence approach for real-world image super-resolution | |
| CN113570612A (en) | Image processing method, device and equipment | |
| Zhao et al. | GPU accelerated high-quality video/image super-resolution | |
| US20150348232A1 (en) | Right sizing enhanced content to generate optimized source content | |
| US20210335018A1 (en) | Data generating device, training device, and data generating method |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |