CN110278486A - Image procossing, the generation method of image resolution ratio processing parameter and device - Google Patents
Image procossing, the generation method of image resolution ratio processing parameter and device Download PDFInfo
- Publication number
- CN110278486A CN110278486A CN201810210845.1A CN201810210845A CN110278486A CN 110278486 A CN110278486 A CN 110278486A CN 201810210845 A CN201810210845 A CN 201810210845A CN 110278486 A CN110278486 A CN 110278486A
- Authority
- CN
- China
- Prior art keywords
- image
- learning model
- deep learning
- block
- image block
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F18/00—Pattern recognition
- G06F18/20—Analysing
- G06F18/24—Classification techniques
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
- H04N21/40—Client devices specifically adapted for the reception of or interaction with content, e.g. set-top-box [STB]; Operations thereof
- H04N21/43—Processing of content or additional data, e.g. demultiplexing additional data from a digital video stream; Elementary client operations, e.g. monitoring of home network or synchronising decoder's clock; Client middleware
- H04N21/44—Processing of video elementary streams, e.g. splicing a video clip retrieved from local storage with an incoming video stream, rendering scenes according to MPEG-4 scene graphs
- H04N21/4402—Processing of video elementary streams, e.g. splicing a video clip retrieved from local storage with an incoming video stream, rendering scenes according to MPEG-4 scene graphs involving reformatting operations of video signals for household redistribution, storage or real-time display
- H04N21/440263—Processing of video elementary streams, e.g. splicing a video clip retrieved from local storage with an incoming video stream, rendering scenes according to MPEG-4 scene graphs involving reformatting operations of video signals for household redistribution, storage or real-time display by altering the spatial resolution, e.g. for displaying on a connected PDA
Abstract
This application discloses a kind of image processing methods, which comprises determines the object type and corresponding first image block of object type that image to be processed includes；Assess the quality of image to be processed；According to the corresponding relationship between object type, picture quality and deep learning model parameter, determine the corresponding deep learning model parameter of quality of the object type, the image to be processed, wherein, the deep learning model parameter is the parameter used when carrying out resolution processes to image block using deep learning model；Resolution processes are carried out to the first image block according to deep learning model parameter, using deep learning model, obtain second image block different from the first image block resolution ratio.Using the above method, the demand that resolution processes are carried out to image is met.
Description
Technical field
This application involves computer image processing technology fields, and in particular to a kind of image processing method and device, this Shen
Please it is related to the generation method and device and a kind of electronic equipment of a kind of image resolution ratio processing parameter simultaneously.
Background technique
Due to the limitation of image capture device hardware or the influence of shooting condition, acquired image size is caused to have
Limit, the visual effect of generation are not ideal enough.If carrying out resolution processes to image, the visual effect of image can be improved.
In addition, in net cast or short field of video applications, it usually needs biggish traffic overhead.If to image into
Row resolution processes can then save traffic overhead.
Therefore, in some scenarios, there is the demand that resolution processes are carried out to image.
Summary of the invention
The application provides a kind of image processing method, to meet the needs of carrying out resolution processes to image.
The application provides a kind of image processing method, which comprises
Determine the object type and corresponding first image block of object type that image to be processed includes；
Assess the quality of image to be processed；
According to the corresponding relationship between object type, picture quality and deep learning model parameter, the object class is determined
Not, the corresponding deep learning model parameter of the quality of the image to be processed, wherein the deep learning model parameter is to utilize
The parameter that deep learning model uses when carrying out resolution processes to image block；
Resolution processes are carried out to the first image block according to deep learning model parameter, using deep learning model,
Obtain second image block different from the first image block resolution ratio.
Optionally, the method also includes: the first image block is divided into multiple first subimage blocks；
It is described that the first image block is carried out at resolution ratio according to deep learning model parameter, using deep learning model
Reason, obtains second image block different from the first image block resolution ratio, comprising:
Resolution ratio is carried out to the multiple first subimage block according to deep learning model parameter, using deep learning model
Processing, obtains multiple second subimage blocks different from the first subimage block resolution ratio；
The multiple second subimage block is merged, second image block is obtained.
Optionally, the image to be processed is divided into multiple first image blocks；
It is described that the first image block is carried out at resolution ratio according to deep learning model parameter, using deep learning model
Reason, obtains second image block different from the first image block resolution ratio, comprising:
Each first image block is carried out at resolution ratio according to deep learning model parameter, using deep learning model respectively
Reason, obtains the second corresponding from each first image block and different resolution ratio image block；
All second image blocks are merged, the image different from the image resolution ratio to be processed is obtained.
Optionally, the image to be processed is divided into multiple first image blocks；
It is described that the first image block is carried out at resolution ratio according to deep learning model parameter, using deep learning model
Reason, obtains second image block different from the first image block resolution ratio, comprising:
According to deep learning model parameter, using deep learning model to a part in the multiple first image block
One image block carries out resolution processes, respectively obtains corresponding from a part of first image block and different resolution ratio second
Image block；
All first image blocks not processed in all second image blocks and the multiple first image block are closed
And obtain the image different from the image resolution ratio to be processed.
Optionally, the object type is object category.
Optionally, corresponding first image block of the object type is the image block for showing a complete object, Huo Zhewei
Show the image block in object parts region.
Optionally, the deep learning model is convolutional network model, and the deep learning model parameter is to utilize convolution
The parameter that network model uses when carrying out resolution processes to image block.
Optionally, the high resolution of second image block is in the resolution ratio of the first image block.
Optionally, the high resolution of second subimage block is in the resolution ratio of first subimage block.
The application also provides a kind of generation method of image resolution ratio processing parameter, which comprises
Determine the object type that the first image includes；
It is the second image of different quality by the first image transcoding；
Second image is subjected to down-sampling processing, obtains the third image different from the first image resolution ratio；
According to the object type, the first image and the third image are split respectively, it is described right to obtain
As the image block of the corresponding different resolution of classification；
It is trained, is used using image block of the deep learning model to the corresponding different resolution of the object type
In being handled image resolution ratio and deep learning model parameter corresponding with the object type, picture quality.
Optionally, described to obtain for being handled image resolution ratio and corresponding with the object type, picture quality
Deep learning model parameter, comprising:
Establish the corresponding relationship between the object type, picture quality and deep learning model parameter.
Optionally, down-sampling coefficient used by the down-sampling is handled carries out image with using the deep learning model
The super-resolution coefficient of super-resolution processing is related.
Optionally, the object type is object category.
Optionally, described image block be include the image block of a complete object, or being includes object parts region
Image block.
Optionally, the deep learning model is convolutional network model, and the deep learning model parameter is convolutional network
Model parameter.
In addition the application provides a kind of image processing apparatus, described device includes:
First image block determination unit, for determining object type and object type corresponding that image to be processed includes
One image block；
Image quality measure unit, for assessing the quality of image to be processed；
Model parameter determination unit, for according to pair between object type, picture quality and deep learning model parameter
It should be related to, determine the corresponding deep learning model parameter of quality of the object type, the image to be processed, wherein described
Deep learning model parameter is the parameter used when carrying out resolution processes to image block using deep learning model；
Second image block acquiring unit, for according to deep learning model parameter, using deep learning model to described the
One image block carries out resolution processes, obtains second image block different from the first image block resolution ratio.
In addition the application provides a kind of generating means of image resolution ratio processing parameter, described device includes:
Object type determination unit, the object type for including for determining the first image；
Image transcoding unit, for being the second image of different quality by the first image transcoding；
Third image generation unit, for will second image progress down-sampling processing, obtain and the first image
The different third image of resolution ratio；
Image block generation unit is used for according to the object type, respectively to the first image and the third image
It is split, obtains the image block of the corresponding different resolution of the object type；
Model parameter generation unit, for utilizing deep learning model to the corresponding different resolution of the object type
Image block is trained, and is obtained for being handled image resolution ratio and depth corresponding with the object type, picture quality
Spend learning model parameter.
The application also provides a kind of electronic equipment characterized by comprising
Processor；And
Memory, for the program of image processing method, which is powered and passes through the processor and run at the image
After the program of reason method, following step is executed:
Determine the object type and corresponding first image block of object type that image to be processed includes；
Assess the quality of image to be processed；
According to the corresponding relationship between object type, picture quality and deep learning model parameter, the object class is determined
Not, the corresponding deep learning model parameter of the quality of the image to be processed；
The first image block is handled according to deep learning model parameter, using deep learning model, obtain with
The second different image block of the first image block resolution ratio.
In addition the application provides a kind of electronic equipment characterized by comprising
Processor；And
Memory, the program of the generation method for image resolution ratio processing parameter, the equipment are powered and pass through the place
After the program for the generation method that reason device runs the image resolution ratio processing parameter, following step is executed:
Determine the object type that the first image includes；
It is the second image of different quality by the first image transcoding；
Second image is subjected to down-sampling processing, obtains the third image different from the first image resolution ratio；
According to the object type, the first image and the third image are split respectively, it is described right to obtain
As the image block of the corresponding different resolution of classification；
It is trained, is used using image block of the deep learning model to the corresponding different resolution of the object type
In being handled image resolution ratio and deep learning model parameter corresponding with the object type, picture quality.
Compared with prior art, the application has the following advantages:
Image processing method, image processing apparatus and corresponding electronic equipment provided by the present application, by will be to be processed
Image is divided into image block according to object type, and according between object type, picture quality and deep learning model parameter
Corresponding relationship, the corresponding deep learning model parameter of quality of the object type, the image to be processed is determined, according to depth
Degree learning model parameter determines deep learning model, is handled using determining deep learning model the first image block, energy
Second image block different from the first image block resolution ratio is accessed, the demand for carrying out resolution processes to image is met.
Generation method, the generating means of image resolution ratio processing parameter of image resolution ratio processing parameter provided by the present application
And corresponding electronic equipment, by being object type by the first image third image segmentation different with its corresponding resolution ratio
The image block of corresponding different resolution, recycle deep learning model above-mentioned image block is trained, generate with it is described right
As classification, the corresponding deep learning model parameter of picture quality, so as to be carried out using deep learning model parameter to image
Resolution processes, to meet the needs of carrying out resolution processes to image.
Detailed description of the invention
Fig. 1 is a kind of flow chart for image processing method that the application first embodiment provides.
Fig. 2 is the generation schematic diagram for the super-resolution image that the application first embodiment provides.
Fig. 3 is a kind of flow chart of the generation method for image resolution ratio processing parameter that the application second embodiment provides.
Fig. 4 is a kind of generation schematic diagram for image resolution ratio processing parameter that the application second embodiment provides.
Fig. 5 is a kind of schematic diagram for image processing apparatus that the application 3rd embodiment provides.
Fig. 6 is a kind of schematic diagram of the generating means for image resolution ratio processing parameter that the application fourth embodiment provides.
Fig. 7 is the schematic diagram for a kind of electronic equipment that the 5th embodiment of the application provides.
Fig. 8 is the schematic diagram for another electronic equipment that the application sixth embodiment provides.
Specific embodiment
Many details are explained in the following description in order to fully understand the application.But the application can be with
Much it is different from other way described herein to implement, those skilled in the art can be without prejudice to the application intension the case where
Under do similar popularization, therefore the application is not limited by following public specific implementation.
The application first embodiment provides a kind of image processing method.Fig. 1, Fig. 2 are please referred to, Fig. 1 is shown according to this
A kind of flow chart for image processing method that the embodiment of application provides.Fig. 2 shows what the application first embodiment provided to surpass
The generation schematic diagram of image in different resolution.It is described in detail below in conjunction with Fig. 1, Fig. 2.
It should be noted that the application first embodiment uses image processing method provided by the present application by low resolution figure
As being converted to high-definition picture, the application first embodiment can also in some application fields in for resolution ratio is higher
Image is converted to the lower image of resolution ratio.
As shown in Figure 1, in step s101, determining the object type and object type corresponding that image to be processed includes
One image block.
Image to be processed, which can be, carries out the image acquired when the shooting of image using the lower camera of hardware configuration, due to
These picture sizes are limited, and resolution ratio is lower, cause the visual effect generated not ideal enough, so needing to carry out super-resolution to it
Rate processing, is converted into the higher image of resolution ratio.Image to be processed is also possible to by configuring lower video camera acquisition
Video frame.Image to be processed can also refer to the lower image of the resolution ratio transmitted in net cast.In other embodiments, to
Processing image can also refer to the higher image of resolution ratio.
The object type can refer to that object category, object category refer to the classification of object, for example, automobile, aircraft, ox etc..
It may include an object type in image to be processed, it is also possible to including multiple object type.
The object type that the image to be processed includes can refer to the object category that image to be processed includes.For example, if
It include automobile, bicycle, ox in image to be processed, then the object type that image to be processed includes is automobile, bicycle, ox.
When the object type that image to be processed includes is object category, corresponding first image block of the object type can
Think the image block of one complete object of display, or the image block in display object parts region.For example, image block can be with
Being includes an automobile or the image block for only including automotive wheel.
Corresponding first image block of the object type, can be an image block or multiple images block.If wait locate
Reason object is divided into multiple images block, then the quantity of the first image block can be multiple, if object to be processed is divided into
One image block, then the quantity of the first image block can be one.
As shown in Figure 1, in step s 102, assessing the quality of image to be processed.
Specifically, can use the abundant degree in edge that Sobel boundary operator calculates image to be processed, image border Yue Feng
Rich quality is higher.The quality of image to be processed can be divided into several grades: 90-100 points, 80-90 in the application first embodiment
Point, 70-80 point, 70 points or less.
As shown in Figure 1, in step s 103, according between object type, picture quality and deep learning model parameter
Corresponding relationship determines the corresponding deep learning model parameter of quality of the object type, the image to be processed.
Wherein, the deep learning model parameter is to make when carrying out resolution processes to image block using deep learning model
Parameter.
About the generating process of the corresponding relationship between object type, picture quality and deep learning model parameter, please join
According to the introduction of the application second embodiment.
In practical applications, deep learning model can be convolutional network model, and deep learning model parameter can be benefit
The parameter used when carrying out resolution processes to image block with convolutional network model.
As shown in Fig. 2, conv is convolutional layer, PReLU (parametric Rectified Linear in convolutional network
It Unit) is nonlinear activation function layer, Deconv is warp lamination (amplification for realizing resolution ratio).Overall network structure are as follows:
Conv(5,d,1)-PReLU-Conv(1,s,d)-PReLU-Conv(3,s,s)-Conv(3,s,s)-Conv(3,s,
s)-Conv(3,s,s)-PReLU-Conv(1,d,s)-PReLU-Deconv(9,1,d).Wherein, Conv (f, n, c) and Deconv
F in (f, n, c) indicates the size of convolution kernel, and n indicates the number of convolution kernel, and c indicates the port number of output.Wherein, d and s
Setting can carry out compromise selection according to computation complexity and effect, and bigger d and s indicate better oversubscription image effect, together
When corresponding bigger computation complexity.
Table 1 shows a kind of implementation of the corresponding relationship between object type, picture quality and convolutional network model parameter
Example.
| Serial number | Object category | Picture quality | Convolutional network model parameter |
| 1 | Ox | 90-100 points | Parameter 1-1 |
| 80-90 points | Parameter 1-2 | ||
| 70-80 points | Parameter 1-3 | ||
| 70 points or less | Parameter 1-4 | ||
| 2 | Automobile | 90-100 points | Parameter 2-1 |
| 80-90 points | Parameter 2-2 | ||
| 70-80 points | Parameter 2-3 | ||
| 70 points or less | Parameter 2-4 |
Table 1
It is 80-90 points by automobile, picture quality of the object type of an image, it can be right according to shown in table 1
It should be related to, determine that object type is automobile, picture quality is that the corresponding deep learning model parameter of 80-90 timesharing is parameter 2-2.
As shown in Figure 1, in step S104, according to deep learning model parameter, using deep learning model to described
One image block carries out resolution processes, obtains second image block different from the first image block resolution ratio.
As one embodiment, the resolution ratio of the second image block can be higher than the resolution ratio of the first image block, such situation
Under, it is that super-resolution processing has been carried out to the first image block.The super-resolution processing refers to, by low resolution (Low
Resolution, LR) image pass through certain algorithm and promoted to high-resolution (High Resolution, HR).High-resolution
Image has higher pixel density, more detailed information, finer and smoother image quality.In step S104, according to deep learning
Model parameter carries out resolution processes to the first image block using deep learning model, obtains and the first image block
The second different image block of resolution ratio, realizes the conversion of low-resolution image to high-definition picture.
It should be noted that the resolution ratio of the second image block can also be lower than first figure as another embodiment
As the resolution ratio of block, the application first embodiment realizes the conversion of high-definition picture to low-resolution image at this time.
Preferably, the image to be processed can be divided into multiple first image blocks；It is described according to deep learning model
Parameter carries out resolution processes to the first image block using deep learning model, obtains differentiating with the first image block
The second different image block of rate, comprising: respectively according to deep learning model parameter, using deep learning model to each first figure
As block progress resolution processes, the second corresponding from each first image block and different resolution ratio image block is obtained；To own
Second image block merges, and obtains the image different from the image resolution ratio to be processed.
For example, image to be processed is divided into 4 the first image blocks, respectively the first image block A, the first image block B, first
Image block C, the first image block D, respectively according to deep learning model parameter, using deep learning model to first image block A, B,
C, D carry out resolution processes, obtain second corresponding from first image block A, B, C, D and different resolution ratio image block A1, B1,
C1, D1, then second image block A1, B1, C1, D1 is merged, obtain the image different from the image resolution ratio to be processed.
In the specific implementation, in some cases, image to be processed is divided in multiple first image blocks, only need to be to portion
The conversion for dividing the first image block to carry out resolution ratio, the first image block of other do not need to carry out the conversion of resolution ratio.Preferably,
The image to be processed is divided into multiple first image blocks；It is described according to deep learning model parameter, utilize deep learning mould
Type carries out resolution processes to the first image block, obtains second image block different from the first image block resolution ratio,
Include: according to deep learning model parameter, using deep learning model to a part first in the multiple first image block
Image block carries out resolution processes, respectively obtains the second figures corresponding from a part of first image block and that resolution ratio is different
As block；All first image blocks not processed in all second image blocks and the multiple first image block are merged,
Obtain the image different from the image resolution ratio to be processed.
For example, image to be processed is divided into 4 the first image blocks, respectively the first image block A, the first image block B, first
Image block C, the first image block D, wherein the first image block A and the first image block C need to carry out resolution processes, the first image
Block B and the first image block D do not need carry out resolution processes, then will according to deep learning model parameter, utilize deep learning mould
Type carries out resolution processes to the first image block A, C, obtains second different image block A1, C1 of resolution ratio, then by second
Image block A1, C1, merge with first image block B, D, obtain the image different from the image resolution ratio to be processed.
In addition to handling the first image block as a whole, obtain different from the first image block resolution ratio
Outside second image block, the first image block can also be divided into multiple first subimage blocks, respectively to each first subgraph
As block is handled to obtain the second different subimage block of corresponding resolution ratio, then multiple second subimage blocks are merged, is obtained
Second image block.Preferably, it is described according to deep learning model parameter, using deep learning model to the first image
Block carries out resolution processes, obtains second image block different from the first image block resolution ratio, comprising: according to deep learning
Model parameter carries out resolution processes to the multiple first subimage block using deep learning model, obtains and the first subgraph
As different multiple second subimage blocks of block resolution ratio；The multiple second subimage block is merged, second image is obtained
Block.
For example, the first image block is divided into 4 the first subimage blocks, respectively the first subimage block A, the first subimage block
B, the first subimage block C, the first subimage block D, according to deep learning model parameter, using deep learning model to described first
Subimage block A, B, C, D carry out resolution processes, obtain second subimage block A1, B1, C1, D1 of different resolution, then by the
Two subimage block A1, B1, C1, D1 merge, and obtain the second image block.
It should be noted that the sequence of step S101 and S102 are unrestricted, in the specific implementation, step can also be first carried out
Rapid S102, then execute step S101.
It is introduced by taking Fig. 2 as an example below and image is converted to from low-resolution image by high-resolution using the application first embodiment
The process of rate image, Fig. 2 shows the generation schematic diagrames for the super-resolution image that the application first embodiment provides.
As shown in Fig. 2, in step s 201, carrying out image object point to the image 21 (image to be processed) for needing oversubscription
Class generates the first image block.In step S202, image quality measure is carried out to the image 21 for needing oversubscription；Later, from data
The corresponding relationship between object classification, picture quality classification and convolutional network model parameter is obtained in model library 22, determines object
Classification and the corresponding convolutional network model parameter of picture quality；Finally, according to convolutional network model parameter, utilizing convolutional network mould
Type 23 handles the first image block, generates high resolution in the second image block (oversubscription of output of the first image block
Resolution image 24).
The application second embodiment provides a kind of generation method of image resolution ratio processing parameter.Please refer to Fig. 3 and figure
4, Fig. 3 show a kind of process of the generation method of the image resolution ratio processing parameter provided according to the application second embodiment
Figure.Fig. 4 shows a kind of generation schematic diagram of the image resolution ratio processing parameter provided according to the application second embodiment.Below
It is described in detail in conjunction with Fig. 3 and Fig. 4.
As shown in figure 3, determining the object type that the first image includes in step S301.
The first image can be high-resolution (high quality/lossless coding) image.Original high resolution in Fig. 4
Image is properly termed as the first image.
The object type is often referred to object category, and object category refers to the classification of object, for example, automobile, aircraft, ox etc..
It may include an object type in first image, it is also possible to including multiple object type.
The object type that the first image includes can refer to the object category that the first image includes.For example, if first
It include automobile, bicycle, ox in image, then the object type that the first image includes is automobile, bicycle, ox.
It determines the object type that the first image includes, can determine image by the way that the first image is detected and divided
In include object type the first image can be determined using the part Faster-RCNN convolutional network model in specific implementation
In include object type, can also adopt with other methods determine the first image in include object type.
As shown in figure 3, being the second image of different quality by the first image transcoding in step s 302.
The quality of image to be processed is divided into several grades according to the height of picture quality in the application second embodiment:
90-100 points, 80-90 points, 70-80 points, 70 points or less.In the specific implementation, in addition to the quality divided rank of the present embodiment use
Outside mode, the division of picture quality can be carried out using other quality grade compartmentalization modes.
As shown in figure 3, second image is carried out down-sampling processing, is obtained and first figure in step S303
As the different third image of resolution ratio.
Down-sampling is referred to as down-sampled or downscaled images.For the image of a width N*M, if down-sampled system
Number is k, then be in the image of N*M each row and column take point to form piece image every k point.
Down-sampling coefficient used by down-sampling processing can carry out Image Super-resolution with using the deep learning model
The super-resolution coefficient of rate processing is related.For example, if deep learning model carries out the super-resolution of Image Super Resolution Processing
Coefficient is 2, i.e., 2 times of super-resolution processing is carried out to image, then down-sampling coefficient can be 2.
As shown in figure 3, in step s 304, according to the object type, respectively to the first image and the third
Image is split, and obtains the image block of the corresponding different resolution of the object type.
When the object type is object category, the corresponding image block of the object type can be one complete object of display
The image block of body, or the image block in display object parts region.For example, image block can be one complete vapour of display
Vehicle or the only image block of display automobile wheel.
For example, if the object type that the first image includes is object category A, object category B, object category C, object type
First image is split by other D, the corresponding image block of each classification then according to object category, obtain image block A1, B1,
C1, D1, if third image is the third image for 4 different resolutions that the second image down sampling of different quality obtains, third
Image is respectively third image 3-1,3-2,3-3,3-4, then is divided into each of 4 third images third image respectively
Image block A3-1, B3-1, C3-1, D3-1 corresponding with A1, B1, C1, D1；A3-2,B3-2,C3-2,D3-2；A3-3,B3-3,
C3-3,D3-3；A3-4,B3-4,C3-4,D3-4.
As shown in figure 3, in step S205, using deep learning model to the corresponding different resolution of the object type
Image block be trained, obtain for being handled image resolution ratio and corresponding with the object type, picture quality
Deep learning model parameter.
It is described to obtain for being handled image resolution ratio and depth corresponding with the object type, picture quality
Learning model parameter, comprising: establish the corresponding relationship between the object type, picture quality and deep learning model parameter.
The deep learning model is convolutional network model, and the deep learning model parameter is convolutional network model ginseng
Number.
As shown in figure 4, in convolutional network: conv indicates convolutional layer, PReLU (parametric Rectified
Linear Unit) it is nonlinear activation function layer, Deconv is warp lamination, (amplification for realizing resolution ratio).Overall network
Structure summarizes are as follows:
Conv(5,d,1)-PReLU-Conv(1,s,d)-PReLU-Conv(3,s,s)-Conv(3,s,s)-Conv(3,s,
s)-Conv(3,s,s)-PReLU-Conv(1,d,s)-PReLU-Deconv(9,1,d).Wherein, Conv (f, n, c) and Deconv
F in (f, n, c) indicates the size of convolution kernel, and n indicates the number of convolution kernel, and c indicates the port number of output.Here, d and s
Setting can carry out compromise selection according to computation complexity and effect, and bigger d and s indicate better oversubscription image effect, together
When corresponding bigger computation complexity.
The process of the generation of the application second embodiment image resolution ratio processing parameter is introduced by taking Fig. 4 as an example below.
As shown in figure 4, the classification for carrying out object to the first image (original high-resolution image 41) is true in step S401
The object category that fixed first image includes.
In step S402, by the second image that the first image transcoding is different quality (including picture quality 90-100 point,
80-90 points, 70-80 points, 70 points of images below).
In step S403, second image is subjected to down-sampling processing, obtains the of resolution ratio lower than the first image
Three images.
Later, according to the object category, the first image and the third image is split respectively, obtain institute
State the image block of the corresponding different resolution of object category.
Finally, being instructed using image block of the convolutional network model 42 to the corresponding different resolution of the object category
Practice, obtain for being handled image resolution ratio and convolutional network parameter corresponding with the object category, picture quality,
By it according to index storage into model database 43.
Corresponding with a kind of image processing method of above-mentioned offer, the application 3rd embodiment additionally provides a kind of image
Processing unit.As shown in figure 5, image processing apparatus includes: the first image block determination unit 501, image quality measure unit
502, model parameter determination unit 503, the second image block acquiring unit 504.
First image block determination unit 501, for determining that object type that image to be processed includes and object type are corresponding
The first image block；
Image quality measure unit 502, for assessing the quality of image to be processed；
Model parameter determination unit 503, for according between object type, picture quality and deep learning model parameter
Corresponding relationship determines the corresponding deep learning model parameter of quality of the object type, the image to be processed, wherein institute
Stating deep learning model parameter is the parameter used when carrying out resolution processes to image block using deep learning model；
Second image block acquiring unit 504, for according to deep learning model parameter, using deep learning model to described
First image block carries out resolution processes, obtains second image block different from the first image block resolution ratio.
Optionally, described device further include:
First subimage block cutting unit, for the first image block to be divided into multiple first subimage blocks；
It is described that the first image block is carried out at resolution ratio according to deep learning model parameter, using deep learning model
Reason, obtains second image block different from the first image block resolution ratio, comprising:
Resolution ratio is carried out to the multiple first subimage block according to deep learning model parameter, using deep learning model
Processing, obtains multiple second subimage blocks different from the first subimage block resolution ratio；
The multiple second subimage block is merged, second image block is obtained.
Optionally, the image to be processed is divided into multiple first image blocks；
It is described that the first image block is carried out at resolution ratio according to deep learning model parameter, using deep learning model
Reason, obtains second image block different from the first image block resolution ratio, comprising:
Each first image block is carried out at resolution ratio according to deep learning model parameter, using deep learning model respectively
Reason, obtains the second corresponding from each first image block and different resolution ratio image block；
All second image blocks are merged, the image different from the image resolution ratio to be processed is obtained.
Optionally, the image to be processed is divided into multiple first image blocks；
It is described that the first image block is carried out at resolution ratio according to deep learning model parameter, using deep learning model
Reason, obtains second image block different from the first image block resolution ratio, comprising:
According to deep learning model parameter, using deep learning model to a part in the multiple first image block
One image block carries out resolution processes, respectively obtains corresponding from a part of first image block and different resolution ratio second
Image block；
All first image blocks not processed in all second image blocks and the multiple first image block are closed
And obtain the image different from the image resolution ratio to be processed.
Optionally, the object type is object category.
Optionally, corresponding first image block of the object type is the image block for showing a complete object, Huo Zhewei
Show the image block in object parts region.
Optionally, the deep learning model is convolutional network model, and the deep learning model parameter is to utilize convolution
The parameter that network model uses when carrying out resolution processes to image block.
Optionally, the high resolution of second image block is in the resolution ratio of the first image block.
Optionally, the high resolution of second subimage block is in the resolution ratio of first subimage block.
It should be noted that can be referred to for the detailed description for the image processing apparatus that the application 3rd embodiment provides
To the associated description of the application first embodiment, which is not described herein again.
It is corresponding with a kind of generation method of image resolution ratio processing parameter of above-mentioned offer, the application fourth embodiment
Additionally provide the generating means of image resolution ratio processing parameter.
As shown in fig. 6, the generating means of image resolution ratio processing parameter include: object type determination unit 601, image turn
Code unit 602, third image generation unit 603, image block generation unit 604, model parameter generation unit 605.
Object type determination unit 601, the object type for including for determining the first image；
Image transcoding unit 602, for being the second image of different quality by the first image transcoding；
Third image generation unit 603 obtains and first figure for second image to be carried out down-sampling processing
As the different third image of resolution ratio；
Image block generation unit 604 is used for according to the object type, respectively to the first image and the third figure
As being split, the image block of the corresponding different resolution of the object type is obtained；
Model parameter generation unit 605, for utilizing the different resolutions corresponding to the object type of deep learning model
The image block of rate is trained, and is obtained for being handled image resolution ratio and corresponding with the object type, picture quality
Deep learning model parameter.
Optionally, described to obtain for being handled image resolution ratio and corresponding with the object type, picture quality
Deep learning model parameter, comprising:
Establish the corresponding relationship between the object type, picture quality and deep learning model parameter.
Optionally, down-sampling coefficient used by the down-sampling is handled carries out image with using the deep learning model
The super-resolution coefficient of super-resolution processing is related.
Optionally, the object type is object category.
Optionally, described image block be include the image block of a complete object, or being includes object parts region
Image block.
Optionally, the deep learning model is convolutional network model, and the deep learning model parameter is convolutional network
Model parameter.
It should be noted that the generating means of the image resolution ratio processing parameter provided for the application fourth embodiment
Detailed description can be with the associated description of reference pair the application second embodiment, and which is not described herein again.
The 5th embodiment of the application provides a kind of electronic equipment, as shown in fig. 7, the electronic equipment includes:
Processor 701；And
Memory 702, for the program of image processing method, which, which is powered and passes through the processor, runs the image
After the program of processing method, following step is executed:
Determine the object type and corresponding first image block of object type that image to be processed includes；
Assess the quality of image to be processed；
According to the corresponding relationship between object type, picture quality and deep learning model parameter, the object class is determined
Not, the corresponding deep learning model parameter of the quality of the image to be processed；
The first image block is handled according to deep learning model parameter, using deep learning model, obtain with
The second different image block of the first image block resolution ratio.
Optionally, the electronic equipment further include: the first image block is divided into multiple first subimage blocks；
It is described that the first image block is carried out at resolution ratio according to deep learning model parameter, using deep learning model
Reason, obtains second image block different from the first image block resolution ratio, comprising:
Resolution ratio is carried out to the multiple first subimage block according to deep learning model parameter, using deep learning model
Processing, obtains multiple second subimage blocks different from the first subimage block resolution ratio；
The multiple second subimage block is merged, second image block is obtained.
Optionally, the image to be processed is divided into multiple first image blocks；
It is described that the first image block is carried out at resolution ratio according to deep learning model parameter, using deep learning model
Reason, obtains second image block different from the first image block resolution ratio, comprising:
Each first image block is carried out at resolution ratio according to deep learning model parameter, using deep learning model respectively
Reason, obtains the second corresponding from each first image block and different resolution ratio image block；
All second image blocks are merged, the image different from the image resolution ratio to be processed is obtained.
Optionally, the image to be processed is divided into multiple first image blocks；
It is described that the first image block is carried out at resolution ratio according to deep learning model parameter, using deep learning model
Reason, obtains second image block different from the first image block resolution ratio, comprising:
According to deep learning model parameter, using deep learning model to a part in the multiple first image block
One image block carries out resolution processes, respectively obtains corresponding from a part of first image block and different resolution ratio second
Image block；
All first image blocks not processed in all second image blocks and the multiple first image block are closed
And obtain the image different from the image resolution ratio to be processed.
Optionally, the object type is object category.
Optionally, corresponding first image block of the object type is the image block for showing a complete object, Huo Zhewei
Show the image block in object parts region.
Optionally, the deep learning model is convolutional network model, and the deep learning model parameter is to utilize convolution
The parameter that network model uses when carrying out resolution processes to image block.
Optionally, the high resolution of second image block is in the resolution ratio of the first image block.
Optionally, the high resolution of second subimage block is in the resolution ratio of first subimage block.
It should be noted that can be with reference pair sheet for the detailed description of the electronic equipment of the 5th embodiment of the application offer
Apply for the associated description of first embodiment, which is not described herein again.
The application sixth embodiment provides another electronic equipment, as shown in figure 8, the electronic equipment includes:
Processor 801；And
Memory 802, the program of the generation method for image resolution ratio processing parameter, the equipment are powered and pass through described
After processor runs the program of the generation method of the image resolution ratio processing parameter, following step is executed:
Determine the object type that the first image includes；
It is the second image of different quality by the first image transcoding；
Second image is subjected to down-sampling processing, obtains the third image different from the first image resolution ratio；
According to the object type, the first image and the third image are split respectively, it is described right to obtain
As the image block of the corresponding different resolution of classification；
It is trained, is used using image block of the deep learning model to the corresponding different resolution of the object type
In being handled image resolution ratio and deep learning model parameter corresponding with the object type, picture quality.
Optionally, described to obtain for being handled image resolution ratio and corresponding with the object type, picture quality
Deep learning model parameter, comprising:
Establish the corresponding relationship between the object type, picture quality and deep learning model parameter.
Optionally, down-sampling coefficient used by the down-sampling is handled carries out image with using the deep learning model
The super-resolution coefficient of super-resolution processing is related.
Optionally, the object type is object category.
Optionally, described image block be include the image block of a complete object, or being includes object parts region
Image block.
Optionally, the deep learning model is convolutional network model, and the deep learning model parameter is convolutional network
Model parameter.
It should be noted that can be with reference pair sheet for the detailed description of the electronic equipment of the application sixth embodiment offer
Apply for the associated description of second embodiment, which is not described herein again.
Although the application is disclosed as above with preferred embodiment, it is not for limiting the application, any this field skill
Art personnel are not departing from spirit and scope, can make possible variation and modification, therefore the guarantor of the application
Shield range should be subject to the range that the claim of this application defined.
In a typical configuration, calculating equipment includes one or more processors (CPU), input/output interface, net
Network interface and memory.
Memory may include the non-volatile memory in computer-readable medium, random access memory (RAM) and/or
The forms such as Nonvolatile memory, such as read-only memory (ROM) or flash memory (flash RAM).Memory is computer-readable medium
Example.
Computer-readable medium includes permanent and non-permanent, removable and non-removable media can be by any method
Or technology come realize information store.Information can be computer readable instructions, data structure, the module of program or other data.
The example of the storage medium of computer includes, but are not limited to phase change memory (PRAM), static random access memory (SRAM), moves
State random access memory (DRAM), other kinds of random access memory (RAM), read-only memory (ROM), electric erasable
Programmable read only memory (EEPROM), flash memory or other memory techniques, read-only disc read only memory (CD-ROM) (CD-ROM),
Digital versatile disc (DVD) or other optical storage, magnetic cassettes, tape magnetic disk storage or other magnetic storage devices
Or any other non-transmission medium, can be used for storage can be accessed by a computing device information.As defined in this article, it calculates
Machine readable medium does not include non-temporary computer readable media (transitory media), such as the data-signal and carrier wave of modulation.
It will be understood by those skilled in the art that embodiments herein can provide as method, system or computer program product.
Therefore, complete hardware embodiment, complete software embodiment or embodiment combining software and hardware aspects can be used in the application
Form.It is deposited moreover, the application can be used to can be used in the computer that one or more wherein includes computer usable program code
The shape for the computer program product implemented on storage media (including but not limited to magnetic disk storage, CD-ROM, optical memory etc.)
Formula.
Claims (19)
1. a kind of image processing method characterized by comprising
Determine the object type and corresponding first image block of object type that image to be processed includes；
Assess the quality of image to be processed；
According to the corresponding relationship between object type, picture quality and deep learning model parameter, the object type, institute are determined
State the corresponding deep learning model parameter of quality of image to be processed, wherein the deep learning model parameter is to utilize depth
The parameter that learning model uses when carrying out resolution processes to image block；
Resolution processes are carried out to the first image block according to deep learning model parameter, using deep learning model, are obtained
Second image block different from the first image block resolution ratio.
2. the method according to claim 1, wherein further include: the first image block is divided into multiple
One subimage block；
It is described that resolution processes are carried out to the first image block according to deep learning model parameter, using deep learning model,
Obtain second image block different from the first image block resolution ratio, comprising:
The multiple first subimage block is carried out at resolution ratio according to deep learning model parameter, using deep learning model
Reason, obtains multiple second subimage blocks different from the first subimage block resolution ratio；
The multiple second subimage block is merged, second image block is obtained.
3. the method according to claim 1, wherein the image to be processed is divided into multiple first images
Block；
It is described that resolution processes are carried out to the first image block according to deep learning model parameter, using deep learning model,
Obtain second image block different from the first image block resolution ratio, comprising:
Resolution processes are carried out to each first image block according to deep learning model parameter, using deep learning model respectively,
Obtain the second corresponding from each first image block and different resolution ratio image block；
All second image blocks are merged, the image different from the image resolution ratio to be processed is obtained.
4. the method according to claim 1, wherein the image to be processed is divided into multiple first images
Block；
It is described that resolution processes are carried out to the first image block according to deep learning model parameter, using deep learning model,
Obtain second image block different from the first image block resolution ratio, comprising:
According to deep learning model parameter, using deep learning model to the first figure of a part in the multiple first image block
As block progress resolution processes, the second images corresponding from a part of first image block and that resolution ratio is different are respectively obtained
Block；
All first image blocks not processed in all second image blocks and the multiple first image block are merged, are obtained
To the image different from the image resolution ratio to be processed.
5. the method according to claim 1, wherein the object type is object category.
6. according to the method described in claim 5, it is characterized in that, corresponding first image block of the object type is display one
The image block of a complete object, or the image block to show object parts region.
7. the method according to claim 1, wherein the deep learning model be convolutional network model, it is described
Deep learning model parameter is the parameter used when carrying out resolution processes to image block using convolutional network model.
8. the method according to claim 1, wherein the high resolution of second image block is in first figure
As the resolution ratio of block.
9. the method according to claim 1, wherein the high resolution of second subimage block is in described first
The resolution ratio of subimage block.
10. a kind of generation method of image resolution ratio processing parameter characterized by comprising
Determine the object type that the first image includes；
It is the second image of different quality by the first image transcoding；
Second image is subjected to down-sampling processing, obtains the third image different from the first image resolution ratio；
According to the object type, the first image and the third image are split respectively, obtain the object class
The image block of not corresponding different resolution；
Be trained using image block of the deep learning model to the corresponding different resolution of the object type, obtain for pair
Image resolution ratio is handled and deep learning model parameter corresponding with the object type, picture quality.
11. according to the method described in claim 10, it is characterized in that, it is described obtain for being handled image resolution ratio,
And deep learning model parameter corresponding with the object type, picture quality, comprising:
Establish the corresponding relationship between the object type, picture quality and deep learning model parameter.
12. according to the method described in claim 10, it is characterized in that, the down-sampling handle used by down-sampling coefficient with
The super-resolution coefficient for carrying out Image Super Resolution Processing using the deep learning model is related.
13. according to the method described in claim 10, it is characterized in that, the object type is object category.
14. according to the method described in claim 10, it is characterized in that, described image block is the image for including a complete object
Block, or be the image block for including object parts region.
15. according to the method described in claim 10, it is characterized in that, the deep learning model is convolutional network model, institute
Stating deep learning model parameter is convolutional network model parameter.
16. a kind of image processing apparatus characterized by comprising
First image block determination unit, for determining object type and corresponding first figure of object type that image to be processed includes
As block；
Image quality measure unit, for assessing the quality of image to be processed；
Model parameter determination unit, for according to the corresponding pass between object type, picture quality and deep learning model parameter
System, determines the corresponding deep learning model parameter of quality of the object type, the image to be processed, wherein the depth
Learning model parameter is the parameter used when carrying out resolution processes to image block using deep learning model；
Second image block acquiring unit, for according to deep learning model parameter, using deep learning model to first figure
As block progress resolution processes, second image block different from the first image block resolution ratio is obtained.
17. a kind of generating means of image resolution ratio processing parameter characterized by comprising
Object type determination unit, the object type for including for determining the first image；
Image transcoding unit, for being the second image of different quality by the first image transcoding；
Third image generation unit obtains differentiating with the first image for second image to be carried out down-sampling processing
The different third image of rate；
Image block generation unit, for being carried out to the first image and the third image respectively according to the object type
Segmentation, obtains the image block of the corresponding different resolution of the object type；
Model parameter generation unit, for the image using deep learning model to the corresponding different resolution of the object type
Block is trained, and is obtained for being handled image resolution ratio and depth corresponding with the object type, picture quality
Practise model parameter.
18. a kind of electronic equipment characterized by comprising
Processor；And
Memory, for the program of image processing method, which, which is powered and passes through the processor, runs the image processing method
After the program of method, following step is executed:
Determine the object type and corresponding first image block of object type that image to be processed includes；
Assess the quality of image to be processed；
According to the corresponding relationship between object type, picture quality and deep learning model parameter, the object type, institute are determined
State the corresponding deep learning model parameter of quality of image to be processed；
The first image block is handled according to deep learning model parameter, using deep learning model, obtain with it is described
The second different image block of first image block resolution ratio.
19. a kind of electronic equipment characterized by comprising
Processor；And
Memory, the program of the generation method for image resolution ratio processing parameter, the equipment are powered and pass through the processor
After the program for running the generation method of the image resolution ratio processing parameter, following step is executed:
Determine the object type that the first image includes；
It is the second image of different quality by the first image transcoding；
Second image is subjected to down-sampling processing, obtains the third image different from the first image resolution ratio；
According to the object type, the first image and the third image are split respectively, obtain the object class
The image block of not corresponding different resolution；
Be trained using image block of the deep learning model to the corresponding different resolution of the object type, obtain for pair
Image resolution ratio is handled and deep learning model parameter corresponding with the object type, picture quality.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201810210845.1A CN110278486A (en) | 2018-03-14 | 2018-03-14 | Image procossing, the generation method of image resolution ratio processing parameter and device |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201810210845.1A CN110278486A (en) | 2018-03-14 | 2018-03-14 | Image procossing, the generation method of image resolution ratio processing parameter and device |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN110278486A true CN110278486A (en) | 2019-09-24 |
Family
ID=67957785
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201810210845.1A Pending CN110278486A (en) | 2018-03-14 | 2018-03-14 | Image procossing, the generation method of image resolution ratio processing parameter and device |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN110278486A (en) |
Cited By (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN111031346A (en) * | 2019-10-28 | 2020-04-17 | 网宿科技股份有限公司 | Method and device for enhancing video image quality |
Citations (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| JP2002170112A (en) * | 2000-12-04 | 2002-06-14 | Minolta Co Ltd | Computer readable recording medium recording resolution conversion program, and resolution conversion apparatus and method |
| CN107481188A (en) * | 2017-06-23 | 2017-12-15 | 珠海经济特区远宏科技有限公司 | A kind of image super-resolution reconstructing method |
| CN107578377A (en) * | 2017-08-31 | 2018-01-12 | 北京飞搜科技有限公司 | A kind of super-resolution image reconstruction method and system based on deep learning |
-
2018
- 2018-03-14 CN CN201810210845.1A patent/CN110278486A/en active Pending
Patent Citations (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| JP2002170112A (en) * | 2000-12-04 | 2002-06-14 | Minolta Co Ltd | Computer readable recording medium recording resolution conversion program, and resolution conversion apparatus and method |
| CN107481188A (en) * | 2017-06-23 | 2017-12-15 | 珠海经济特区远宏科技有限公司 | A kind of image super-resolution reconstructing method |
| CN107578377A (en) * | 2017-08-31 | 2018-01-12 | 北京飞搜科技有限公司 | A kind of super-resolution image reconstruction method and system based on deep learning |
Cited By (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN111031346A (en) * | 2019-10-28 | 2020-04-17 | 网宿科技股份有限公司 | Method and device for enhancing video image quality |
| CN111031346B (en) * | 2019-10-28 | 2021-11-02 | 网宿科技股份有限公司 | Method and device for enhancing video image quality |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| Gu et al. | Hplflownet: Hierarchical permutohedral lattice flownet for scene flow estimation on large-scale point clouds | |
| Li et al. | SPM-BP: Sped-up PatchMatch belief propagation for continuous MRFs | |
| US10268901B2 (en) | Quasi-parametric optical flow estimation | |
| US20180324465A1 (en) | Edge-aware spatio-temporal filtering and optical flow estimation in real time | |
| US20140253746A1 (en) | Depth mask assisted video stabilization | |
| CN102682446A (en) | Method and apparatus for generating a dense depth map using an adaptive joint bilateral filter | |
| CN102651127A (en) | Image processing method and image processing system for super-resolution reconstruction | |
| CN107231566A (en) | A kind of video transcoding method, device and system | |
| US20160239597A1 (en) | Apparatus and method for performing finite element computation | |
| EP3985575A1 (en) | Three-dimensional information processing method and apparatus | |
| CN114022799A (en) | Self-supervision monocular depth estimation method and device | |
| CN104735360A (en) | Method and device for optical field image processing | |
| Xi et al. | Super resolution reconstruction algorithm of video image based on deep self encoding learning | |
| US11769291B2 (en) | Method and device for rendering point cloud-based data | |
| CN106326920B (en) | Off-line synchronization method and device for telemetering data and video image data | |
| CN110278486A (en) | Image procossing, the generation method of image resolution ratio processing parameter and device | |
| CN110278487A (en) | Image processing method, device and equipment | |
| CN108573510A (en) | A kind of grating map vectorization method and equipment | |
| CN112801883A (en) | Image processing method, image processing device, electronic equipment and computer readable storage medium | |
| CN109522532B (en) | Method and device for calculating selection range of line drawing frame | |
| US8611700B2 (en) | Distance map-based warping of binary images | |
| JP2023092446A (en) | Cargo counting method and apparatus, computer apparatus, and storage medium | |
| CN104517273A (en) | Image super-resolution processing method and apparatus | |
| CN110378390B (en) | Multitask graph classification method | |
| CN109951705B (en) | Reference frame synthesis method and device for vehicle object coding in surveillance video |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| REG | Reference to a national code |
Ref country code: HK Ref legal event code: DE Ref document number: 40013102 Country of ref document: HK |
|
| RJ01 | Rejection of invention patent application after publication | ||
| RJ01 | Rejection of invention patent application after publication |
Application publication date: 20190924 |