CN110278486A - Image procossing, the generation method of image resolution ratio processing parameter and device - Google Patents

Image procossing, the generation method of image resolution ratio processing parameter and device Download PDF

Info

Publication number
CN110278486A
CN110278486A CN201810210845.1A CN201810210845A CN110278486A CN 110278486 A CN110278486 A CN 110278486A CN 201810210845 A CN201810210845 A CN 201810210845A CN 110278486 A CN110278486 A CN 110278486A
Authority
CN
China
Prior art keywords
image
learning model
deep learning
block
image block
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Pending
Application number
CN201810210845.1A
Other languages
Chinese (zh)
Inventor
杨江
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Alibaba Group Holding Ltd
Original Assignee
Alibaba Group Holding Ltd
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Alibaba Group Holding Ltd filed Critical Alibaba Group Holding Ltd
Priority to CN201810210845.1A priority Critical patent/CN110278486A/en
Publication of CN110278486A publication Critical patent/CN110278486A/en
Pending legal-status Critical Current

Links

Classifications

    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06FELECTRIC DIGITAL DATA PROCESSING
    • G06F18/00Pattern recognition
    • G06F18/20Analysing
    • G06F18/24Classification techniques
    • HELECTRICITY
    • H04ELECTRIC COMMUNICATION TECHNIQUE
    • H04NPICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N21/00Selective content distribution, e.g. interactive television or video on demand [VOD]
    • H04N21/40Client devices specifically adapted for the reception of or interaction with content, e.g. set-top-box [STB]; Operations thereof
    • H04N21/43Processing of content or additional data, e.g. demultiplexing additional data from a digital video stream; Elementary client operations, e.g. monitoring of home network or synchronising decoder's clock; Client middleware
    • H04N21/44Processing of video elementary streams, e.g. splicing a video clip retrieved from local storage with an incoming video stream or rendering scenes according to encoded video stream scene graphs
    • H04N21/4402Processing of video elementary streams, e.g. splicing a video clip retrieved from local storage with an incoming video stream or rendering scenes according to encoded video stream scene graphs involving reformatting operations of video signals for household redistribution, storage or real-time display
    • H04N21/440263Processing of video elementary streams, e.g. splicing a video clip retrieved from local storage with an incoming video stream or rendering scenes according to encoded video stream scene graphs involving reformatting operations of video signals for household redistribution, storage or real-time display by altering the spatial resolution, e.g. for displaying on a connected PDA

Landscapes

  • Engineering & Computer Science (AREA)
  • Data Mining & Analysis (AREA)
  • Theoretical Computer Science (AREA)
  • Computer Vision & Pattern Recognition (AREA)
  • Evolutionary Computation (AREA)
  • Artificial Intelligence (AREA)
  • Bioinformatics & Cheminformatics (AREA)
  • Bioinformatics & Computational Biology (AREA)
  • Signal Processing (AREA)
  • Evolutionary Biology (AREA)
  • Life Sciences & Earth Sciences (AREA)
  • Physics & Mathematics (AREA)
  • General Engineering & Computer Science (AREA)
  • General Physics & Mathematics (AREA)
  • Multimedia (AREA)
  • Image Analysis (AREA)
  • Image Processing (AREA)

Abstract

This application discloses a kind of image processing methods, which comprises determines the object type and corresponding first image block of object type that image to be processed includes;Assess the quality of image to be processed;According to the corresponding relationship between object type, picture quality and deep learning model parameter, determine the corresponding deep learning model parameter of quality of the object type, the image to be processed, wherein, the deep learning model parameter is the parameter used when carrying out resolution processes to image block using deep learning model;Resolution processes are carried out to the first image block according to deep learning model parameter, using deep learning model, obtain second image block different from the first image block resolution ratio.Using the above method, the demand that resolution processes are carried out to image is met.

Description

Image procossing, the generation method of image resolution ratio processing parameter and device
Technical field
This application involves computer image processing technology fields, and in particular to a kind of image processing method and device, this Shen Please it is related to the generation method and device and a kind of electronic equipment of a kind of image resolution ratio processing parameter simultaneously.
Background technique
Due to the limitation of image capture device hardware or the influence of shooting condition, acquired image size is caused to have Limit, the visual effect of generation are not ideal enough.If carrying out resolution processes to image, the visual effect of image can be improved.
In addition, in net cast or short field of video applications, it usually needs biggish traffic overhead.If to image into Row resolution processes can then save traffic overhead.
Therefore, in some scenarios, there is the demand that resolution processes are carried out to image.
Summary of the invention
The application provides a kind of image processing method, to meet the needs of carrying out resolution processes to image.
The application provides a kind of image processing method, which comprises
Determine the object type and corresponding first image block of object type that image to be processed includes;
Assess the quality of image to be processed;
According to the corresponding relationship between object type, picture quality and deep learning model parameter, the object class is determined Not, the corresponding deep learning model parameter of the quality of the image to be processed, wherein the deep learning model parameter is to utilize The parameter that deep learning model uses when carrying out resolution processes to image block;
Resolution processes are carried out to the first image block according to deep learning model parameter, using deep learning model, Obtain second image block different from the first image block resolution ratio.
Optionally, the method also includes: the first image block is divided into multiple first subimage blocks;
It is described that the first image block is carried out at resolution ratio according to deep learning model parameter, using deep learning model Reason, obtains second image block different from the first image block resolution ratio, comprising:
Resolution ratio is carried out to the multiple first subimage block according to deep learning model parameter, using deep learning model Processing, obtains multiple second subimage blocks different from the first subimage block resolution ratio;
The multiple second subimage block is merged, second image block is obtained.
Optionally, the image to be processed is divided into multiple first image blocks;
It is described that the first image block is carried out at resolution ratio according to deep learning model parameter, using deep learning model Reason, obtains second image block different from the first image block resolution ratio, comprising:
Each first image block is carried out at resolution ratio according to deep learning model parameter, using deep learning model respectively Reason, obtains the second corresponding from each first image block and different resolution ratio image block;
All second image blocks are merged, the image different from the image resolution ratio to be processed is obtained.
Optionally, the image to be processed is divided into multiple first image blocks;
It is described that the first image block is carried out at resolution ratio according to deep learning model parameter, using deep learning model Reason, obtains second image block different from the first image block resolution ratio, comprising:
According to deep learning model parameter, using deep learning model to a part in the multiple first image block One image block carries out resolution processes, respectively obtains corresponding from a part of first image block and different resolution ratio second Image block;
All first image blocks not processed in all second image blocks and the multiple first image block are closed And obtain the image different from the image resolution ratio to be processed.
Optionally, the object type is object category.
Optionally, corresponding first image block of the object type is the image block for showing a complete object, Huo Zhewei Show the image block in object parts region.
Optionally, the deep learning model is convolutional network model, and the deep learning model parameter is to utilize convolution The parameter that network model uses when carrying out resolution processes to image block.
Optionally, the high resolution of second image block is in the resolution ratio of the first image block.
Optionally, the high resolution of second subimage block is in the resolution ratio of first subimage block.
The application also provides a kind of generation method of image resolution ratio processing parameter, which comprises
Determine the object type that the first image includes;
It is the second image of different quality by the first image transcoding;
Second image is subjected to down-sampling processing, obtains the third image different from the first image resolution ratio;
According to the object type, the first image and the third image are split respectively, it is described right to obtain As the image block of the corresponding different resolution of classification;
It is trained, is used using image block of the deep learning model to the corresponding different resolution of the object type In being handled image resolution ratio and deep learning model parameter corresponding with the object type, picture quality.
Optionally, described to obtain for being handled image resolution ratio and corresponding with the object type, picture quality Deep learning model parameter, comprising:
Establish the corresponding relationship between the object type, picture quality and deep learning model parameter.
Optionally, down-sampling coefficient used by the down-sampling is handled carries out image with using the deep learning model The super-resolution coefficient of super-resolution processing is related.
Optionally, the object type is object category.
Optionally, described image block be include the image block of a complete object, or being includes object parts region Image block.
Optionally, the deep learning model is convolutional network model, and the deep learning model parameter is convolutional network Model parameter.
In addition the application provides a kind of image processing apparatus, described device includes:
First image block determination unit, for determining object type and object type corresponding that image to be processed includes One image block;
Image quality measure unit, for assessing the quality of image to be processed;
Model parameter determination unit, for according to pair between object type, picture quality and deep learning model parameter It should be related to, determine the corresponding deep learning model parameter of quality of the object type, the image to be processed, wherein described Deep learning model parameter is the parameter used when carrying out resolution processes to image block using deep learning model;
Second image block acquiring unit, for according to deep learning model parameter, using deep learning model to described the One image block carries out resolution processes, obtains second image block different from the first image block resolution ratio.
In addition the application provides a kind of generating means of image resolution ratio processing parameter, described device includes:
Object type determination unit, the object type for including for determining the first image;
Image transcoding unit, for being the second image of different quality by the first image transcoding;
Third image generation unit, for will second image progress down-sampling processing, obtain and the first image The different third image of resolution ratio;
Image block generation unit is used for according to the object type, respectively to the first image and the third image It is split, obtains the image block of the corresponding different resolution of the object type;
Model parameter generation unit, for utilizing deep learning model to the corresponding different resolution of the object type Image block is trained, and is obtained for being handled image resolution ratio and depth corresponding with the object type, picture quality Spend learning model parameter.
The application also provides a kind of electronic equipment characterized by comprising
Processor;And
Memory, for the program of image processing method, which is powered and passes through the processor and run at the image After the program of reason method, following step is executed:
Determine the object type and corresponding first image block of object type that image to be processed includes;
Assess the quality of image to be processed;
According to the corresponding relationship between object type, picture quality and deep learning model parameter, the object class is determined Not, the corresponding deep learning model parameter of the quality of the image to be processed;
The first image block is handled according to deep learning model parameter, using deep learning model, obtain with The second different image block of the first image block resolution ratio.
In addition the application provides a kind of electronic equipment characterized by comprising
Processor;And
Memory, the program of the generation method for image resolution ratio processing parameter, the equipment are powered and pass through the place After the program for the generation method that reason device runs the image resolution ratio processing parameter, following step is executed:
Determine the object type that the first image includes;
It is the second image of different quality by the first image transcoding;
Second image is subjected to down-sampling processing, obtains the third image different from the first image resolution ratio;
According to the object type, the first image and the third image are split respectively, it is described right to obtain As the image block of the corresponding different resolution of classification;
It is trained, is used using image block of the deep learning model to the corresponding different resolution of the object type In being handled image resolution ratio and deep learning model parameter corresponding with the object type, picture quality.
Compared with prior art, the application has the following advantages:
Image processing method, image processing apparatus and corresponding electronic equipment provided by the present application, by will be to be processed Image is divided into image block according to object type, and according between object type, picture quality and deep learning model parameter Corresponding relationship, the corresponding deep learning model parameter of quality of the object type, the image to be processed is determined, according to depth Degree learning model parameter determines deep learning model, is handled using determining deep learning model the first image block, energy Second image block different from the first image block resolution ratio is accessed, the demand for carrying out resolution processes to image is met.
Generation method, the generating means of image resolution ratio processing parameter of image resolution ratio processing parameter provided by the present application And corresponding electronic equipment, by being object type by the first image third image segmentation different with its corresponding resolution ratio The image block of corresponding different resolution, recycle deep learning model above-mentioned image block is trained, generate with it is described right As classification, the corresponding deep learning model parameter of picture quality, so as to be carried out using deep learning model parameter to image Resolution processes, to meet the needs of carrying out resolution processes to image.
Detailed description of the invention
Fig. 1 is a kind of flow chart for image processing method that the application first embodiment provides.
Fig. 2 is the generation schematic diagram for the super-resolution image that the application first embodiment provides.
Fig. 3 is a kind of flow chart of the generation method for image resolution ratio processing parameter that the application second embodiment provides.
Fig. 4 is a kind of generation schematic diagram for image resolution ratio processing parameter that the application second embodiment provides.
Fig. 5 is a kind of schematic diagram for image processing apparatus that the application 3rd embodiment provides.
Fig. 6 is a kind of schematic diagram of the generating means for image resolution ratio processing parameter that the application fourth embodiment provides.
Fig. 7 is the schematic diagram for a kind of electronic equipment that the 5th embodiment of the application provides.
Fig. 8 is the schematic diagram for another electronic equipment that the application sixth embodiment provides.
Specific embodiment
Many details are explained in the following description in order to fully understand the application.But the application can be with Much it is different from other way described herein to implement, those skilled in the art can be without prejudice to the application intension the case where Under do similar popularization, therefore the application is not limited by following public specific implementation.
The application first embodiment provides a kind of image processing method.Fig. 1, Fig. 2 are please referred to, Fig. 1 is shown according to this A kind of flow chart for image processing method that the embodiment of application provides.Fig. 2 shows what the application first embodiment provided to surpass The generation schematic diagram of image in different resolution.It is described in detail below in conjunction with Fig. 1, Fig. 2.
It should be noted that the application first embodiment uses image processing method provided by the present application by low resolution figure As being converted to high-definition picture, the application first embodiment can also in some application fields in for resolution ratio is higher Image is converted to the lower image of resolution ratio.
As shown in Figure 1, in step s101, determining the object type and object type corresponding that image to be processed includes One image block.
Image to be processed, which can be, carries out the image acquired when the shooting of image using the lower camera of hardware configuration, due to These picture sizes are limited, and resolution ratio is lower, cause the visual effect generated not ideal enough, so needing to carry out super-resolution to it Rate processing, is converted into the higher image of resolution ratio.Image to be processed is also possible to by configuring lower video camera acquisition Video frame.Image to be processed can also refer to the lower image of the resolution ratio transmitted in net cast.In other embodiments, to Processing image can also refer to the higher image of resolution ratio.
The object type can refer to that object category, object category refer to the classification of object, for example, automobile, aircraft, ox etc.. It may include an object type in image to be processed, it is also possible to including multiple object type.
The object type that the image to be processed includes can refer to the object category that image to be processed includes.For example, if It include automobile, bicycle, ox in image to be processed, then the object type that image to be processed includes is automobile, bicycle, ox.
When the object type that image to be processed includes is object category, corresponding first image block of the object type can Think the image block of one complete object of display, or the image block in display object parts region.For example, image block can be with Being includes an automobile or the image block for only including automotive wheel.
Corresponding first image block of the object type, can be an image block or multiple images block.If wait locate Reason object is divided into multiple images block, then the quantity of the first image block can be multiple, if object to be processed is divided into One image block, then the quantity of the first image block can be one.
As shown in Figure 1, in step s 102, assessing the quality of image to be processed.
Specifically, can use the abundant degree in edge that Sobel boundary operator calculates image to be processed, image border Yue Feng Rich quality is higher.The quality of image to be processed can be divided into several grades: 90-100 points, 80-90 in the application first embodiment Point, 70-80 point, 70 points or less.
As shown in Figure 1, in step s 103, according between object type, picture quality and deep learning model parameter Corresponding relationship determines the corresponding deep learning model parameter of quality of the object type, the image to be processed.
Wherein, the deep learning model parameter is to make when carrying out resolution processes to image block using deep learning model Parameter.
About the generating process of the corresponding relationship between object type, picture quality and deep learning model parameter, please join According to the introduction of the application second embodiment.
In practical applications, deep learning model can be convolutional network model, and deep learning model parameter can be benefit The parameter used when carrying out resolution processes to image block with convolutional network model.
As shown in Fig. 2, conv is convolutional layer, PReLU (parametric Rectified Linear in convolutional network It Unit) is nonlinear activation function layer, Deconv is warp lamination (amplification for realizing resolution ratio).Overall network structure are as follows:
Conv(5,d,1)-PReLU-Conv(1,s,d)-PReLU-Conv(3,s,s)-Conv(3,s,s)-Conv(3,s, s)-Conv(3,s,s)-PReLU-Conv(1,d,s)-PReLU-Deconv(9,1,d).Wherein, Conv (f, n, c) and Deconv F in (f, n, c) indicates the size of convolution kernel, and n indicates the number of convolution kernel, and c indicates the port number of output.Wherein, d and s Setting can carry out compromise selection according to computation complexity and effect, and bigger d and s indicate better oversubscription image effect, together When corresponding bigger computation complexity.
Table 1 shows a kind of implementation of the corresponding relationship between object type, picture quality and convolutional network model parameter Example.
Serial number Object category Picture quality Convolutional network model parameter
1 Ox 90-100 points Parameter 1-1
80-90 points Parameter 1-2
70-80 points Parameter 1-3
70 points or less Parameter 1-4
2 Automobile 90-100 points Parameter 2-1
80-90 points Parameter 2-2
70-80 points Parameter 2-3
70 points or less Parameter 2-4
Table 1
It is 80-90 points by automobile, picture quality of the object type of an image, it can be right according to shown in table 1 It should be related to, determine that object type is automobile, picture quality is that the corresponding deep learning model parameter of 80-90 timesharing is parameter 2-2.
As shown in Figure 1, in step S104, according to deep learning model parameter, using deep learning model to described One image block carries out resolution processes, obtains second image block different from the first image block resolution ratio.
As one embodiment, the resolution ratio of the second image block can be higher than the resolution ratio of the first image block, such situation Under, it is that super-resolution processing has been carried out to the first image block.The super-resolution processing refers to, by low resolution (Low Resolution, LR) image pass through certain algorithm and promoted to high-resolution (High Resolution, HR).High-resolution Image has higher pixel density, more detailed information, finer and smoother image quality.In step S104, according to deep learning Model parameter carries out resolution processes to the first image block using deep learning model, obtains and the first image block The second different image block of resolution ratio, realizes the conversion of low-resolution image to high-definition picture.
It should be noted that the resolution ratio of the second image block can also be lower than first figure as another embodiment As the resolution ratio of block, the application first embodiment realizes the conversion of high-definition picture to low-resolution image at this time.
Preferably, the image to be processed can be divided into multiple first image blocks;It is described according to deep learning model Parameter carries out resolution processes to the first image block using deep learning model, obtains differentiating with the first image block The second different image block of rate, comprising: respectively according to deep learning model parameter, using deep learning model to each first figure As block progress resolution processes, the second corresponding from each first image block and different resolution ratio image block is obtained;To own Second image block merges, and obtains the image different from the image resolution ratio to be processed.
For example, image to be processed is divided into 4 the first image blocks, respectively the first image block A, the first image block B, first Image block C, the first image block D, respectively according to deep learning model parameter, using deep learning model to first image block A, B, C, D carry out resolution processes, obtain second corresponding from first image block A, B, C, D and different resolution ratio image block A1, B1, C1, D1, then second image block A1, B1, C1, D1 is merged, obtain the image different from the image resolution ratio to be processed.
In the specific implementation, in some cases, image to be processed is divided in multiple first image blocks, only need to be to portion The conversion for dividing the first image block to carry out resolution ratio, the first image block of other do not need to carry out the conversion of resolution ratio.Preferably, The image to be processed is divided into multiple first image blocks;It is described according to deep learning model parameter, utilize deep learning mould Type carries out resolution processes to the first image block, obtains second image block different from the first image block resolution ratio, Include: according to deep learning model parameter, using deep learning model to a part first in the multiple first image block Image block carries out resolution processes, respectively obtains the second figures corresponding from a part of first image block and that resolution ratio is different As block;All first image blocks not processed in all second image blocks and the multiple first image block are merged, Obtain the image different from the image resolution ratio to be processed.
For example, image to be processed is divided into 4 the first image blocks, respectively the first image block A, the first image block B, first Image block C, the first image block D, wherein the first image block A and the first image block C need to carry out resolution processes, the first image Block B and the first image block D do not need carry out resolution processes, then will according to deep learning model parameter, utilize deep learning mould Type carries out resolution processes to the first image block A, C, obtains second different image block A1, C1 of resolution ratio, then by second Image block A1, C1, merge with first image block B, D, obtain the image different from the image resolution ratio to be processed.
In addition to handling the first image block as a whole, obtain different from the first image block resolution ratio Outside second image block, the first image block can also be divided into multiple first subimage blocks, respectively to each first subgraph As block is handled to obtain the second different subimage block of corresponding resolution ratio, then multiple second subimage blocks are merged, is obtained Second image block.Preferably, it is described according to deep learning model parameter, using deep learning model to the first image Block carries out resolution processes, obtains second image block different from the first image block resolution ratio, comprising: according to deep learning Model parameter carries out resolution processes to the multiple first subimage block using deep learning model, obtains and the first subgraph As different multiple second subimage blocks of block resolution ratio;The multiple second subimage block is merged, second image is obtained Block.
For example, the first image block is divided into 4 the first subimage blocks, respectively the first subimage block A, the first subimage block B, the first subimage block C, the first subimage block D, according to deep learning model parameter, using deep learning model to described first Subimage block A, B, C, D carry out resolution processes, obtain second subimage block A1, B1, C1, D1 of different resolution, then by the Two subimage block A1, B1, C1, D1 merge, and obtain the second image block.
It should be noted that the sequence of step S101 and S102 are unrestricted, in the specific implementation, step can also be first carried out Rapid S102, then execute step S101.
It is introduced by taking Fig. 2 as an example below and image is converted to from low-resolution image by high-resolution using the application first embodiment The process of rate image, Fig. 2 shows the generation schematic diagrames for the super-resolution image that the application first embodiment provides.
As shown in Fig. 2, in step s 201, carrying out image object point to the image 21 (image to be processed) for needing oversubscription Class generates the first image block.In step S202, image quality measure is carried out to the image 21 for needing oversubscription;Later, from data The corresponding relationship between object classification, picture quality classification and convolutional network model parameter is obtained in model library 22, determines object Classification and the corresponding convolutional network model parameter of picture quality;Finally, according to convolutional network model parameter, utilizing convolutional network mould Type 23 handles the first image block, generates high resolution in the second image block (oversubscription of output of the first image block Resolution image 24).
The application second embodiment provides a kind of generation method of image resolution ratio processing parameter.Please refer to Fig. 3 and figure 4, Fig. 3 show a kind of process of the generation method of the image resolution ratio processing parameter provided according to the application second embodiment Figure.Fig. 4 shows a kind of generation schematic diagram of the image resolution ratio processing parameter provided according to the application second embodiment.Below It is described in detail in conjunction with Fig. 3 and Fig. 4.
As shown in figure 3, determining the object type that the first image includes in step S301.
The first image can be high-resolution (high quality/lossless coding) image.Original high resolution in Fig. 4 Image is properly termed as the first image.
The object type is often referred to object category, and object category refers to the classification of object, for example, automobile, aircraft, ox etc.. It may include an object type in first image, it is also possible to including multiple object type.
The object type that the first image includes can refer to the object category that the first image includes.For example, if first It include automobile, bicycle, ox in image, then the object type that the first image includes is automobile, bicycle, ox.
It determines the object type that the first image includes, can determine image by the way that the first image is detected and divided In include object type the first image can be determined using the part Faster-RCNN convolutional network model in specific implementation In include object type, can also adopt with other methods determine the first image in include object type.
As shown in figure 3, being the second image of different quality by the first image transcoding in step s 302.
The quality of image to be processed is divided into several grades according to the height of picture quality in the application second embodiment: 90-100 points, 80-90 points, 70-80 points, 70 points or less.In the specific implementation, in addition to the quality divided rank of the present embodiment use Outside mode, the division of picture quality can be carried out using other quality grade compartmentalization modes.
As shown in figure 3, second image is carried out down-sampling processing, is obtained and first figure in step S303 As the different third image of resolution ratio.
Down-sampling is referred to as down-sampled or downscaled images.For the image of a width N*M, if down-sampled system Number is k, then be in the image of N*M each row and column take point to form piece image every k point.
Down-sampling coefficient used by down-sampling processing can carry out Image Super-resolution with using the deep learning model The super-resolution coefficient of rate processing is related.For example, if deep learning model carries out the super-resolution of Image Super Resolution Processing Coefficient is 2, i.e., 2 times of super-resolution processing is carried out to image, then down-sampling coefficient can be 2.
As shown in figure 3, in step s 304, according to the object type, respectively to the first image and the third Image is split, and obtains the image block of the corresponding different resolution of the object type.
When the object type is object category, the corresponding image block of the object type can be one complete object of display The image block of body, or the image block in display object parts region.For example, image block can be one complete vapour of display Vehicle or the only image block of display automobile wheel.
For example, if the object type that the first image includes is object category A, object category B, object category C, object type First image is split by other D, the corresponding image block of each classification then according to object category, obtain image block A1, B1, C1, D1, if third image is the third image for 4 different resolutions that the second image down sampling of different quality obtains, third Image is respectively third image 3-1,3-2,3-3,3-4, then is divided into each of 4 third images third image respectively Image block A3-1, B3-1, C3-1, D3-1 corresponding with A1, B1, C1, D1;A3-2,B3-2,C3-2,D3-2;A3-3,B3-3, C3-3,D3-3;A3-4,B3-4,C3-4,D3-4.
As shown in figure 3, in step S205, using deep learning model to the corresponding different resolution of the object type Image block be trained, obtain for being handled image resolution ratio and corresponding with the object type, picture quality Deep learning model parameter.
It is described to obtain for being handled image resolution ratio and depth corresponding with the object type, picture quality Learning model parameter, comprising: establish the corresponding relationship between the object type, picture quality and deep learning model parameter.
The deep learning model is convolutional network model, and the deep learning model parameter is convolutional network model ginseng Number.
As shown in figure 4, in convolutional network: conv indicates convolutional layer, PReLU (parametric Rectified Linear Unit) it is nonlinear activation function layer, Deconv is warp lamination, (amplification for realizing resolution ratio).Overall network Structure summarizes are as follows:
Conv(5,d,1)-PReLU-Conv(1,s,d)-PReLU-Conv(3,s,s)-Conv(3,s,s)-Conv(3,s, s)-Conv(3,s,s)-PReLU-Conv(1,d,s)-PReLU-Deconv(9,1,d).Wherein, Conv (f, n, c) and Deconv F in (f, n, c) indicates the size of convolution kernel, and n indicates the number of convolution kernel, and c indicates the port number of output.Here, d and s Setting can carry out compromise selection according to computation complexity and effect, and bigger d and s indicate better oversubscription image effect, together When corresponding bigger computation complexity.
The process of the generation of the application second embodiment image resolution ratio processing parameter is introduced by taking Fig. 4 as an example below.
As shown in figure 4, the classification for carrying out object to the first image (original high-resolution image 41) is true in step S401 The object category that fixed first image includes.
In step S402, by the second image that the first image transcoding is different quality (including picture quality 90-100 point, 80-90 points, 70-80 points, 70 points of images below).
In step S403, second image is subjected to down-sampling processing, obtains the of resolution ratio lower than the first image Three images.
Later, according to the object category, the first image and the third image is split respectively, obtain institute State the image block of the corresponding different resolution of object category.
Finally, being instructed using image block of the convolutional network model 42 to the corresponding different resolution of the object category Practice, obtain for being handled image resolution ratio and convolutional network parameter corresponding with the object category, picture quality, By it according to index storage into model database 43.
Corresponding with a kind of image processing method of above-mentioned offer, the application 3rd embodiment additionally provides a kind of image Processing unit.As shown in figure 5, image processing apparatus includes: the first image block determination unit 501, image quality measure unit 502, model parameter determination unit 503, the second image block acquiring unit 504.
First image block determination unit 501, for determining that object type that image to be processed includes and object type are corresponding The first image block;
Image quality measure unit 502, for assessing the quality of image to be processed;
Model parameter determination unit 503, for according between object type, picture quality and deep learning model parameter Corresponding relationship determines the corresponding deep learning model parameter of quality of the object type, the image to be processed, wherein institute Stating deep learning model parameter is the parameter used when carrying out resolution processes to image block using deep learning model;
Second image block acquiring unit 504, for according to deep learning model parameter, using deep learning model to described First image block carries out resolution processes, obtains second image block different from the first image block resolution ratio.
Optionally, described device further include:
First subimage block cutting unit, for the first image block to be divided into multiple first subimage blocks;
It is described that the first image block is carried out at resolution ratio according to deep learning model parameter, using deep learning model Reason, obtains second image block different from the first image block resolution ratio, comprising:
Resolution ratio is carried out to the multiple first subimage block according to deep learning model parameter, using deep learning model Processing, obtains multiple second subimage blocks different from the first subimage block resolution ratio;
The multiple second subimage block is merged, second image block is obtained.
Optionally, the image to be processed is divided into multiple first image blocks;
It is described that the first image block is carried out at resolution ratio according to deep learning model parameter, using deep learning model Reason, obtains second image block different from the first image block resolution ratio, comprising:
Each first image block is carried out at resolution ratio according to deep learning model parameter, using deep learning model respectively Reason, obtains the second corresponding from each first image block and different resolution ratio image block;
All second image blocks are merged, the image different from the image resolution ratio to be processed is obtained.
Optionally, the image to be processed is divided into multiple first image blocks;
It is described that the first image block is carried out at resolution ratio according to deep learning model parameter, using deep learning model Reason, obtains second image block different from the first image block resolution ratio, comprising:
According to deep learning model parameter, using deep learning model to a part in the multiple first image block One image block carries out resolution processes, respectively obtains corresponding from a part of first image block and different resolution ratio second Image block;
All first image blocks not processed in all second image blocks and the multiple first image block are closed And obtain the image different from the image resolution ratio to be processed.
Optionally, the object type is object category.
Optionally, corresponding first image block of the object type is the image block for showing a complete object, Huo Zhewei Show the image block in object parts region.
Optionally, the deep learning model is convolutional network model, and the deep learning model parameter is to utilize convolution The parameter that network model uses when carrying out resolution processes to image block.
Optionally, the high resolution of second image block is in the resolution ratio of the first image block.
Optionally, the high resolution of second subimage block is in the resolution ratio of first subimage block.
It should be noted that can be referred to for the detailed description for the image processing apparatus that the application 3rd embodiment provides To the associated description of the application first embodiment, which is not described herein again.
It is corresponding with a kind of generation method of image resolution ratio processing parameter of above-mentioned offer, the application fourth embodiment Additionally provide the generating means of image resolution ratio processing parameter.
As shown in fig. 6, the generating means of image resolution ratio processing parameter include: object type determination unit 601, image turn Code unit 602, third image generation unit 603, image block generation unit 604, model parameter generation unit 605.
Object type determination unit 601, the object type for including for determining the first image;
Image transcoding unit 602, for being the second image of different quality by the first image transcoding;
Third image generation unit 603 obtains and first figure for second image to be carried out down-sampling processing As the different third image of resolution ratio;
Image block generation unit 604 is used for according to the object type, respectively to the first image and the third figure As being split, the image block of the corresponding different resolution of the object type is obtained;
Model parameter generation unit 605, for utilizing the different resolutions corresponding to the object type of deep learning model The image block of rate is trained, and is obtained for being handled image resolution ratio and corresponding with the object type, picture quality Deep learning model parameter.
Optionally, described to obtain for being handled image resolution ratio and corresponding with the object type, picture quality Deep learning model parameter, comprising:
Establish the corresponding relationship between the object type, picture quality and deep learning model parameter.
Optionally, down-sampling coefficient used by the down-sampling is handled carries out image with using the deep learning model The super-resolution coefficient of super-resolution processing is related.
Optionally, the object type is object category.
Optionally, described image block be include the image block of a complete object, or being includes object parts region Image block.
Optionally, the deep learning model is convolutional network model, and the deep learning model parameter is convolutional network Model parameter.
It should be noted that the generating means of the image resolution ratio processing parameter provided for the application fourth embodiment Detailed description can be with the associated description of reference pair the application second embodiment, and which is not described herein again.
The 5th embodiment of the application provides a kind of electronic equipment, as shown in fig. 7, the electronic equipment includes:
Processor 701;And
Memory 702, for the program of image processing method, which, which is powered and passes through the processor, runs the image After the program of processing method, following step is executed:
Determine the object type and corresponding first image block of object type that image to be processed includes;
Assess the quality of image to be processed;
According to the corresponding relationship between object type, picture quality and deep learning model parameter, the object class is determined Not, the corresponding deep learning model parameter of the quality of the image to be processed;
The first image block is handled according to deep learning model parameter, using deep learning model, obtain with The second different image block of the first image block resolution ratio.
Optionally, the electronic equipment further include: the first image block is divided into multiple first subimage blocks;
It is described that the first image block is carried out at resolution ratio according to deep learning model parameter, using deep learning model Reason, obtains second image block different from the first image block resolution ratio, comprising:
Resolution ratio is carried out to the multiple first subimage block according to deep learning model parameter, using deep learning model Processing, obtains multiple second subimage blocks different from the first subimage block resolution ratio;
The multiple second subimage block is merged, second image block is obtained.
Optionally, the image to be processed is divided into multiple first image blocks;
It is described that the first image block is carried out at resolution ratio according to deep learning model parameter, using deep learning model Reason, obtains second image block different from the first image block resolution ratio, comprising:
Each first image block is carried out at resolution ratio according to deep learning model parameter, using deep learning model respectively Reason, obtains the second corresponding from each first image block and different resolution ratio image block;
All second image blocks are merged, the image different from the image resolution ratio to be processed is obtained.
Optionally, the image to be processed is divided into multiple first image blocks;
It is described that the first image block is carried out at resolution ratio according to deep learning model parameter, using deep learning model Reason, obtains second image block different from the first image block resolution ratio, comprising:
According to deep learning model parameter, using deep learning model to a part in the multiple first image block One image block carries out resolution processes, respectively obtains corresponding from a part of first image block and different resolution ratio second Image block;
All first image blocks not processed in all second image blocks and the multiple first image block are closed And obtain the image different from the image resolution ratio to be processed.
Optionally, the object type is object category.
Optionally, corresponding first image block of the object type is the image block for showing a complete object, Huo Zhewei Show the image block in object parts region.
Optionally, the deep learning model is convolutional network model, and the deep learning model parameter is to utilize convolution The parameter that network model uses when carrying out resolution processes to image block.
Optionally, the high resolution of second image block is in the resolution ratio of the first image block.
Optionally, the high resolution of second subimage block is in the resolution ratio of first subimage block.
It should be noted that can be with reference pair sheet for the detailed description of the electronic equipment of the 5th embodiment of the application offer Apply for the associated description of first embodiment, which is not described herein again.
The application sixth embodiment provides another electronic equipment, as shown in figure 8, the electronic equipment includes:
Processor 801;And
Memory 802, the program of the generation method for image resolution ratio processing parameter, the equipment are powered and pass through described After processor runs the program of the generation method of the image resolution ratio processing parameter, following step is executed:
Determine the object type that the first image includes;
It is the second image of different quality by the first image transcoding;
Second image is subjected to down-sampling processing, obtains the third image different from the first image resolution ratio;
According to the object type, the first image and the third image are split respectively, it is described right to obtain As the image block of the corresponding different resolution of classification;
It is trained, is used using image block of the deep learning model to the corresponding different resolution of the object type In being handled image resolution ratio and deep learning model parameter corresponding with the object type, picture quality.
Optionally, described to obtain for being handled image resolution ratio and corresponding with the object type, picture quality Deep learning model parameter, comprising:
Establish the corresponding relationship between the object type, picture quality and deep learning model parameter.
Optionally, down-sampling coefficient used by the down-sampling is handled carries out image with using the deep learning model The super-resolution coefficient of super-resolution processing is related.
Optionally, the object type is object category.
Optionally, described image block be include the image block of a complete object, or being includes object parts region Image block.
Optionally, the deep learning model is convolutional network model, and the deep learning model parameter is convolutional network Model parameter.
It should be noted that can be with reference pair sheet for the detailed description of the electronic equipment of the application sixth embodiment offer Apply for the associated description of second embodiment, which is not described herein again.
Although the application is disclosed as above with preferred embodiment, it is not for limiting the application, any this field skill Art personnel are not departing from spirit and scope, can make possible variation and modification, therefore the guarantor of the application Shield range should be subject to the range that the claim of this application defined.
In a typical configuration, calculating equipment includes one or more processors (CPU), input/output interface, net Network interface and memory.
Memory may include the non-volatile memory in computer-readable medium, random access memory (RAM) and/or The forms such as Nonvolatile memory, such as read-only memory (ROM) or flash memory (flash RAM).Memory is computer-readable medium Example.
Computer-readable medium includes permanent and non-permanent, removable and non-removable media can be by any method Or technology come realize information store.Information can be computer readable instructions, data structure, the module of program or other data. The example of the storage medium of computer includes, but are not limited to phase change memory (PRAM), static random access memory (SRAM), moves State random access memory (DRAM), other kinds of random access memory (RAM), read-only memory (ROM), electric erasable Programmable read only memory (EEPROM), flash memory or other memory techniques, read-only disc read only memory (CD-ROM) (CD-ROM), Digital versatile disc (DVD) or other optical storage, magnetic cassettes, tape magnetic disk storage or other magnetic storage devices Or any other non-transmission medium, can be used for storage can be accessed by a computing device information.As defined in this article, it calculates Machine readable medium does not include non-temporary computer readable media (transitory media), such as the data-signal and carrier wave of modulation.
It will be understood by those skilled in the art that embodiments herein can provide as method, system or computer program product. Therefore, complete hardware embodiment, complete software embodiment or embodiment combining software and hardware aspects can be used in the application Form.It is deposited moreover, the application can be used to can be used in the computer that one or more wherein includes computer usable program code The shape for the computer program product implemented on storage media (including but not limited to magnetic disk storage, CD-ROM, optical memory etc.) Formula.

Claims (19)

1. a kind of image processing method characterized by comprising
Determine the object type and corresponding first image block of object type that image to be processed includes;
Assess the quality of image to be processed;
According to the corresponding relationship between object type, picture quality and deep learning model parameter, the object type, institute are determined State the corresponding deep learning model parameter of quality of image to be processed, wherein the deep learning model parameter is to utilize depth The parameter that learning model uses when carrying out resolution processes to image block;
Resolution processes are carried out to the first image block according to deep learning model parameter, using deep learning model, are obtained Second image block different from the first image block resolution ratio.
2. the method according to claim 1, wherein further include: the first image block is divided into multiple One subimage block;
It is described that resolution processes are carried out to the first image block according to deep learning model parameter, using deep learning model, Obtain second image block different from the first image block resolution ratio, comprising:
The multiple first subimage block is carried out at resolution ratio according to deep learning model parameter, using deep learning model Reason, obtains multiple second subimage blocks different from the first subimage block resolution ratio;
The multiple second subimage block is merged, second image block is obtained.
3. the method according to claim 1, wherein the image to be processed is divided into multiple first images Block;
It is described that resolution processes are carried out to the first image block according to deep learning model parameter, using deep learning model, Obtain second image block different from the first image block resolution ratio, comprising:
Resolution processes are carried out to each first image block according to deep learning model parameter, using deep learning model respectively, Obtain the second corresponding from each first image block and different resolution ratio image block;
All second image blocks are merged, the image different from the image resolution ratio to be processed is obtained.
4. the method according to claim 1, wherein the image to be processed is divided into multiple first images Block;
It is described that resolution processes are carried out to the first image block according to deep learning model parameter, using deep learning model, Obtain second image block different from the first image block resolution ratio, comprising:
According to deep learning model parameter, using deep learning model to the first figure of a part in the multiple first image block As block progress resolution processes, the second images corresponding from a part of first image block and that resolution ratio is different are respectively obtained Block;
All first image blocks not processed in all second image blocks and the multiple first image block are merged, are obtained To the image different from the image resolution ratio to be processed.
5. the method according to claim 1, wherein the object type is object category.
6. according to the method described in claim 5, it is characterized in that, corresponding first image block of the object type is display one The image block of a complete object, or the image block to show object parts region.
7. the method according to claim 1, wherein the deep learning model be convolutional network model, it is described Deep learning model parameter is the parameter used when carrying out resolution processes to image block using convolutional network model.
8. the method according to claim 1, wherein the high resolution of second image block is in first figure As the resolution ratio of block.
9. the method according to claim 1, wherein the high resolution of second subimage block is in described first The resolution ratio of subimage block.
10. a kind of generation method of image resolution ratio processing parameter characterized by comprising
Determine the object type that the first image includes;
It is the second image of different quality by the first image transcoding;
Second image is subjected to down-sampling processing, obtains the third image different from the first image resolution ratio;
According to the object type, the first image and the third image are split respectively, obtain the object class The image block of not corresponding different resolution;
Be trained using image block of the deep learning model to the corresponding different resolution of the object type, obtain for pair Image resolution ratio is handled and deep learning model parameter corresponding with the object type, picture quality.
11. according to the method described in claim 10, it is characterized in that, it is described obtain for being handled image resolution ratio, And deep learning model parameter corresponding with the object type, picture quality, comprising:
Establish the corresponding relationship between the object type, picture quality and deep learning model parameter.
12. according to the method described in claim 10, it is characterized in that, the down-sampling handle used by down-sampling coefficient with The super-resolution coefficient for carrying out Image Super Resolution Processing using the deep learning model is related.
13. according to the method described in claim 10, it is characterized in that, the object type is object category.
14. according to the method described in claim 10, it is characterized in that, described image block is the image for including a complete object Block, or be the image block for including object parts region.
15. according to the method described in claim 10, it is characterized in that, the deep learning model is convolutional network model, institute Stating deep learning model parameter is convolutional network model parameter.
16. a kind of image processing apparatus characterized by comprising
First image block determination unit, for determining object type and corresponding first figure of object type that image to be processed includes As block;
Image quality measure unit, for assessing the quality of image to be processed;
Model parameter determination unit, for according to the corresponding pass between object type, picture quality and deep learning model parameter System, determines the corresponding deep learning model parameter of quality of the object type, the image to be processed, wherein the depth Learning model parameter is the parameter used when carrying out resolution processes to image block using deep learning model;
Second image block acquiring unit, for according to deep learning model parameter, using deep learning model to first figure As block progress resolution processes, second image block different from the first image block resolution ratio is obtained.
17. a kind of generating means of image resolution ratio processing parameter characterized by comprising
Object type determination unit, the object type for including for determining the first image;
Image transcoding unit, for being the second image of different quality by the first image transcoding;
Third image generation unit obtains differentiating with the first image for second image to be carried out down-sampling processing The different third image of rate;
Image block generation unit, for being carried out to the first image and the third image respectively according to the object type Segmentation, obtains the image block of the corresponding different resolution of the object type;
Model parameter generation unit, for the image using deep learning model to the corresponding different resolution of the object type Block is trained, and is obtained for being handled image resolution ratio and depth corresponding with the object type, picture quality Practise model parameter.
18. a kind of electronic equipment characterized by comprising
Processor;And
Memory, for the program of image processing method, which, which is powered and passes through the processor, runs the image processing method After the program of method, following step is executed:
Determine the object type and corresponding first image block of object type that image to be processed includes;
Assess the quality of image to be processed;
According to the corresponding relationship between object type, picture quality and deep learning model parameter, the object type, institute are determined State the corresponding deep learning model parameter of quality of image to be processed;
The first image block is handled according to deep learning model parameter, using deep learning model, obtain with it is described The second different image block of first image block resolution ratio.
19. a kind of electronic equipment characterized by comprising
Processor;And
Memory, the program of the generation method for image resolution ratio processing parameter, the equipment are powered and pass through the processor After the program for running the generation method of the image resolution ratio processing parameter, following step is executed:
Determine the object type that the first image includes;
It is the second image of different quality by the first image transcoding;
Second image is subjected to down-sampling processing, obtains the third image different from the first image resolution ratio;
According to the object type, the first image and the third image are split respectively, obtain the object class The image block of not corresponding different resolution;
Be trained using image block of the deep learning model to the corresponding different resolution of the object type, obtain for pair Image resolution ratio is handled and deep learning model parameter corresponding with the object type, picture quality.
CN201810210845.1A 2018-03-14 2018-03-14 Image procossing, the generation method of image resolution ratio processing parameter and device Pending CN110278486A (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN201810210845.1A CN110278486A (en) 2018-03-14 2018-03-14 Image procossing, the generation method of image resolution ratio processing parameter and device

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN201810210845.1A CN110278486A (en) 2018-03-14 2018-03-14 Image procossing, the generation method of image resolution ratio processing parameter and device

Publications (1)

Publication Number Publication Date
CN110278486A true CN110278486A (en) 2019-09-24

Family

ID=67957785

Family Applications (1)

Application Number Title Priority Date Filing Date
CN201810210845.1A Pending CN110278486A (en) 2018-03-14 2018-03-14 Image procossing, the generation method of image resolution ratio processing parameter and device

Country Status (1)

Country Link
CN (1) CN110278486A (en)

Cited By (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN111031346A (en) * 2019-10-28 2020-04-17 网宿科技股份有限公司 Method and device for enhancing video image quality

Citations (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JP2002170112A (en) * 2000-12-04 2002-06-14 Minolta Co Ltd Computer readable recording medium recording resolution conversion program, and resolution conversion apparatus and method
CN107481188A (en) * 2017-06-23 2017-12-15 珠海经济特区远宏科技有限公司 A kind of image super-resolution reconstructing method
CN107578377A (en) * 2017-08-31 2018-01-12 北京飞搜科技有限公司 A kind of super-resolution image reconstruction method and system based on deep learning

Patent Citations (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JP2002170112A (en) * 2000-12-04 2002-06-14 Minolta Co Ltd Computer readable recording medium recording resolution conversion program, and resolution conversion apparatus and method
CN107481188A (en) * 2017-06-23 2017-12-15 珠海经济特区远宏科技有限公司 A kind of image super-resolution reconstructing method
CN107578377A (en) * 2017-08-31 2018-01-12 北京飞搜科技有限公司 A kind of super-resolution image reconstruction method and system based on deep learning

Cited By (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN111031346A (en) * 2019-10-28 2020-04-17 网宿科技股份有限公司 Method and device for enhancing video image quality
CN111031346B (en) * 2019-10-28 2021-11-02 网宿科技股份有限公司 Method and device for enhancing video image quality

Similar Documents

Publication Publication Date Title
Li et al. SPM-BP: Sped-up PatchMatch belief propagation for continuous MRFs
US9100574B2 (en) Depth mask assisted video stabilization
US20170161565A1 (en) Quasi-parametric optical flow estimation
US20180324465A1 (en) Edge-aware spatio-temporal filtering and optical flow estimation in real time
CN103235949B (en) Image point of interest detection method and device
CN102682446A (en) Method and apparatus for generating a dense depth map using an adaptive joint bilateral filter
CN102651127A (en) Image processing method and image processing system for super-resolution reconstruction
CN107231566A (en) A kind of video transcoding method, device and system
US11769291B2 (en) Method and device for rendering point cloud-based data
EP3985575A1 (en) Three-dimensional information processing method and apparatus
CN112070137B (en) Training data set generation method, target object detection method and related equipment
CN118247290B (en) Construction method and application of 3D panorama segmentation model of semantic-space fusion transducer
Xi et al. Super resolution reconstruction algorithm of video image based on deep self encoding learning
CN105608684B (en) The acceleration method and system of bilateral digital picture wave filter
CN106326920B (en) Off-line synchronization method and device for telemetering data and video image data
CN103839233B (en) Restoration method for blurred image caused by camera shaking
CN110278486A (en) Image procossing, the generation method of image resolution ratio processing parameter and device
Kim et al. Latent transformations neural network for object view synthesis
CN110278487A (en) Image processing method, device and equipment
CN109522532B (en) Method and device for calculating selection range of line drawing frame
CN115393868B (en) Text detection method, device, electronic equipment and storage medium
US8611700B2 (en) Distance map-based warping of binary images
JP2023092446A (en) Cargo counting method and apparatus, computer apparatus, and storage medium
CN110728710A (en) Visual mileage calculation method, device and computer readable storage medium
CN115239559A (en) Depth map super-resolution method and system for fusion view synthesis

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination
REG Reference to a national code

Ref country code: HK

Ref legal event code: DE

Ref document number: 40013102

Country of ref document: HK

RJ01 Rejection of invention patent application after publication
RJ01 Rejection of invention patent application after publication

Application publication date: 20190924