CN111489322A - Method and device for adding sky filter to static picture - Google Patents
Method and device for adding sky filter to static picture Download PDFInfo
- Publication number
- CN111489322A CN111489322A CN202010273375.0A CN202010273375A CN111489322A CN 111489322 A CN111489322 A CN 111489322A CN 202010273375 A CN202010273375 A CN 202010273375A CN 111489322 A CN111489322 A CN 111489322A
- Authority
- CN
- China
- Prior art keywords
- image
- sky
- neural network
- picture
- segmentation
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Granted
Links
- 238000000034 method Methods 0.000 title claims abstract description 39
- 230000003068 static effect Effects 0.000 title abstract description 8
- 238000013528 artificial neural network Methods 0.000 claims abstract description 105
- 238000012545 processing Methods 0.000 claims abstract description 44
- 239000000463 material Substances 0.000 claims abstract description 30
- 238000012937 correction Methods 0.000 claims abstract description 19
- 238000002156 mixing Methods 0.000 claims abstract description 18
- 230000007797 corrosion Effects 0.000 claims abstract description 14
- 238000005260 corrosion Methods 0.000 claims abstract description 14
- 230000011218 segmentation Effects 0.000 claims description 106
- 238000012549 training Methods 0.000 claims description 55
- 230000000694 effects Effects 0.000 claims description 21
- 238000013507 mapping Methods 0.000 claims description 15
- 230000006870 function Effects 0.000 claims description 10
- 230000008569 process Effects 0.000 claims description 10
- 239000011159 matrix material Substances 0.000 claims description 7
- 230000004913 activation Effects 0.000 claims description 4
- 210000003746 feather Anatomy 0.000 claims description 4
- 238000011176 pooling Methods 0.000 claims description 4
- 230000001747 exhibiting effect Effects 0.000 claims description 3
- 238000005070 sampling Methods 0.000 claims description 3
- 238000000926 separation method Methods 0.000 claims description 3
- 238000003062 neural network model Methods 0.000 abstract description 2
- 238000010586 diagram Methods 0.000 description 11
- 238000004590 computer program Methods 0.000 description 7
- 238000003860 storage Methods 0.000 description 3
- 230000007704 transition Effects 0.000 description 3
- 238000009877 rendering Methods 0.000 description 2
- 238000006467 substitution reaction Methods 0.000 description 2
- 238000012360 testing method Methods 0.000 description 2
- 230000003044 adaptive effect Effects 0.000 description 1
- 230000004075 alteration Effects 0.000 description 1
- 230000009286 beneficial effect Effects 0.000 description 1
- 230000008859 change Effects 0.000 description 1
- 239000003086 colorant Substances 0.000 description 1
- 230000007812 deficiency Effects 0.000 description 1
- 238000009499 grossing Methods 0.000 description 1
- 238000004519 manufacturing process Methods 0.000 description 1
- 230000001537 neural effect Effects 0.000 description 1
- 238000010606 normalization Methods 0.000 description 1
- 230000003287 optical effect Effects 0.000 description 1
- 238000005457 optimization Methods 0.000 description 1
- XLYOFNOQVPJJNP-UHFFFAOYSA-N water Substances O XLYOFNOQVPJJNP-UHFFFAOYSA-N 0.000 description 1
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T5/00—Image enhancement or restoration
- G06T5/50—Image enhancement or restoration using two or more images, e.g. averaging or subtraction
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/08—Learning methods
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T5/00—Image enhancement or restoration
- G06T5/20—Image enhancement or restoration using local operators
- G06T5/30—Erosion or dilatation, e.g. thinning
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T7/00—Image analysis
- G06T7/10—Segmentation; Edge detection
- G06T7/11—Region-based segmentation
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/10—Image acquisition modality
- G06T2207/10004—Still image; Photographic image
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/20—Special algorithmic details
- G06T2207/20081—Training; Learning
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/20—Special algorithmic details
- G06T2207/20084—Artificial neural networks [ANN]
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/20—Special algorithmic details
- G06T2207/20212—Image combination
- G06T2207/20221—Image fusion; Image merging
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/30—Subject of image; Context of image processing
- G06T2207/30181—Earth observation
- G06T2207/30192—Weather; Meteorology
-
- Y—GENERAL TAGGING OF NEW TECHNOLOGICAL DEVELOPMENTS; GENERAL TAGGING OF CROSS-SECTIONAL TECHNOLOGIES SPANNING OVER SEVERAL SECTIONS OF THE IPC; TECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS
- Y02—TECHNOLOGIES OR APPLICATIONS FOR MITIGATION OR ADAPTATION AGAINST CLIMATE CHANGE
- Y02A—TECHNOLOGIES FOR ADAPTATION TO CLIMATE CHANGE
- Y02A90/00—Technologies having an indirect contribution to adaptation to climate change
- Y02A90/10—Information and communication technologies [ICT] supporting adaptation to climate change, e.g. for weather forecasting or climate simulation
Landscapes
- Engineering & Computer Science (AREA)
- Physics & Mathematics (AREA)
- Theoretical Computer Science (AREA)
- General Physics & Mathematics (AREA)
- General Health & Medical Sciences (AREA)
- Molecular Biology (AREA)
- Biophysics (AREA)
- Computational Linguistics (AREA)
- Data Mining & Analysis (AREA)
- Evolutionary Computation (AREA)
- Artificial Intelligence (AREA)
- Biomedical Technology (AREA)
- Computing Systems (AREA)
- General Engineering & Computer Science (AREA)
- Life Sciences & Earth Sciences (AREA)
- Mathematical Physics (AREA)
- Software Systems (AREA)
- Health & Medical Sciences (AREA)
- Computer Vision & Pattern Recognition (AREA)
- Image Processing (AREA)
Abstract
The invention relates to a method and a device for adding a sky filter to a static picture, wherein the method comprises the steps of loading a neural network parameter model into a mobile client, acquiring an original image and determining an image to be processed; inputting an image to be processed into a neural network parameter model, and converting the image into an image with a gray sky area to obtain a first image; performing expansion corrosion operation on the first image to obtain a second image; performing Gaussian blur processing on the second image by adopting a Gaussian convolution kernel to obtain a third image; carrying out color correction on each pixel in the original image to obtain a fourth image; and linearly mixing the sky material image for replacement, the third image and the fourth image to obtain an image with a sky filter. According to the method, the neural network model is constructed, the original image is segmented and processed, and the beautified sky image is finally output.
Description
Technical Field
The invention belongs to the technical field of image processing, and particularly relates to a method and a device for adding a sky filter to a static picture.
Background
The mode of carrying out filter mixing of colors to sky in the image on the market at present has: firstly, some sky segmentation algorithms based on YCRBCR space, HSV space and RGB space. However, the method mainly stays in the stages of thesis and demo testing, and the segmentation effect is poor in a large number of picture tests based on life scenes; the algorithm is generally used for identifying whether a sky area exists on a photo in the market and performing certain HDR light optimization, and under the use scenes, the accuracy and the speed of the sky segmentation area are not high.
In addition, due to the method, the operation speed is low, the obtained edge feeling of the sky area is strong, the segmentation effect of the image in the scenes of sea-sky connection, green water, green mountains and the like is not accurate enough, and the condition that the non-sky area with similar color is segmented by mistake can occur. If the algorithm is directly used, the sky area of the divided picture is directly subjected to color filling beautification, the sky area has obvious color difference compared with other normally imaged areas, the edge transition effect is rigid, the image expression effect is poor and unnatural, the phenomenon of pause and the like in rendering occurs, and the beautified sky area cannot be naturally fused with the original image, so that the effect is abrupt.
Disclosure of Invention
In view of the above, the present invention provides a method and an apparatus for adding a sky filter to a static picture to overcome the deficiencies of the prior art, so as to solve the problems of low accuracy of segmentation between a sky region and a non-sky region, low operation speed, hard sky edge transition effect, and unnatural overall image effect in the prior art.
In order to achieve the purpose, the invention adopts the following technical scheme: a method of applying a sky filter to a still picture, comprising:
training a neural network to obtain a neural network parameter model;
loading the neural network parameter model into a mobile client, acquiring an original image by using the mobile client, and determining an image to be processed according to the original image;
inputting the image to be processed into the neural network parameter model, and converting the image to be processed into an image with a sky area in a gray scale by the neural network parameter model to obtain a first image;
performing expansion corrosion operation on the first image to obtain a second image;
performing Gaussian blur processing on the second image to obtain a third image;
performing color correction on each pixel in the original image by adopting a color search mapping algorithm to obtain a fourth image;
and acquiring a sky material image for replacement, and linearly mixing the sky material image for replacement, the third image and the fourth image to acquire an image with a sky filter.
Further, the training the neural network to obtain the neural network parameter model includes:
acquiring a plurality of sky pictures, and determining a sky picture to be processed according to the sky picture;
processing the sky picture to be processed to obtain a mask segmentation image; the mask segmentation image is a black and white image, a white area represents a sky area, and a black area represents a non-sky area;
processing the mask segmentation image by adopting a matting algorithm to obtain an alpha channel segmentation image;
adjusting the sky picture, the mask segmentation map and the alpha channel segmentation map to preset sizes to form training data;
inputting the training data into a neural network for processing to obtain a fine alpha channel segmentation graph;
initializing a neural network, calculating the image data loss between the predicted alpha channel segmentation graph and the obtained fine alpha channel segmentation graph by adopting a cross entropy loss function, and updating neural network parameters by adopting a self-adaptive estimation matrix algorithm according to the image data loss;
and dynamically adjusting the learning rate in the training process, training for multiple times until the neural network converges, and storing the neural network parameters to obtain a neural network parameter model when the neural network prediction result enables the junction of the sky region and the non-sky region to have a smooth effect.
Further, the determining an image to be processed according to the original image includes:
and adjusting the size of the original image to a preset size, and determining the image adjusted to the preset size as an image to be processed.
Further, the performing color correction on each pixel in the original image by using a color lookup mapping algorithm to obtain a fourth image includes:
obtaining L UT lookup tables corresponding to the replaced sky material images;
and carrying out color correction on each pixel in the original image by adopting a color lookup mapping algorithm according to the L UT lookup table to obtain a fourth image.
Further, the linearly mixing the sky material image for replacement, the third image and the fourth image to obtain an image with a sky filter includes:
performing three-channel separation on the third image to obtain an R channel, a G channel and a B channel, and calculating a G channel component of each pixel value in the third image;
when the G channel component is in the range of 0-1, the fourth image and the sky material image for replacement are linearly mixed according to the proportion of the G channel component;
an image exhibiting a sky filter effect is acquired.
Further, the acquiring a plurality of sky pictures and determining a to-be-processed sky picture according to the sky picture includes:
acquiring a plurality of sky pictures;
marking out sky areas of a plurality of the sky pictures;
and determining the sky picture marked with the sky area as a to-be-processed sky picture.
Further, the neural network includes: a segmentation module and a feathering module;
the segmentation module includes: the device comprises a pooling layer, a convolution layer, a batch standardization layer, an activation layer, an upper sampling layer and a Softmax layer;
the feathering module includes: convolutional layers and Sigmoid layers.
Further, the inputting the training data into a neural network for processing to obtain a fine alpha channel segmentation map includes:
inputting a sky picture and a mask segmentation picture in training data into a segmentation module for processing to obtain a rough mask segmentation picture;
inputting the coarse mask segmentation map and an alpha segmentation map in the training data into a feather module for processing to obtain a fine alpha channel segmentation map.
The embodiment of the application provides a device for adding a sky filter to a static picture, which comprises:
the training module is used for training the neural network to obtain a neural network parameter model;
the acquisition module is used for loading the neural network parameter model into a mobile client, acquiring an original image by using the mobile client, and determining an image to be processed according to the original image;
the neural network parameter model is used for converting the image to be processed into an image with a sky area in gray scale to obtain a first image;
the expansion corrosion module is used for performing expansion corrosion operation on the first image to obtain a second image;
the Gaussian blur module is used for carrying out Gaussian blur processing on the second image to obtain a third image;
the color correction module is used for performing color correction on each pixel in the original image by adopting a color searching and mapping algorithm to obtain a fourth image;
and the linear mixing module is used for acquiring a sky material image for replacement, and linearly mixing the sky material image for replacement, the third image and the fourth image to acquire an image with a sky filter.
Further, the obtaining module includes:
the system comprises a first acquisition unit, a second acquisition unit and a processing unit, wherein the first acquisition unit is used for acquiring a plurality of sky pictures and determining a sky picture to be processed according to the sky picture;
the processing unit is used for processing the sky picture to be processed to obtain a mask segmentation map; the mask segmentation image is a black and white image, a white area represents a sky area, and a black area represents a non-sky area;
the computing unit is used for processing the mask segmentation image by adopting a matting algorithm to obtain an alpha channel segmentation image;
the adjusting unit is used for adjusting the sky picture, the mask segmentation map and the alpha channel segmentation map to preset sizes so as to form training data;
the training unit is used for inputting the training data into a neural network for processing to obtain a fine alpha channel segmentation graph;
the updating unit is used for initializing the neural network, calculating the image data loss between the predicted alpha channel segmentation graph and the obtained fine alpha channel segmentation graph by adopting a cross entropy loss function, and updating the neural network parameters by adopting a self-adaptive estimation matrix algorithm according to the image data loss;
and the second acquisition unit is used for dynamically adjusting the learning rate in the training process and carrying out multiple times of training until the neural network converges, and storing the neural network parameters to acquire the neural network parameter model when the neural network prediction result enables the junction of the sky area and the non-sky area to have a smooth effect.
By adopting the technical scheme, the invention can achieve the following beneficial effects:
according to the method, the neural network model is constructed, the original image is segmented and processed, and the beautified sky image is finally output, so that the problems that the sky area has obvious chromatic aberration, the edge transition effect is harsh, the image expression effect is poor and unnatural, the rendering is blocked and the like compared with other normally imaged areas are solved.
Drawings
In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings used in the description of the embodiments or the prior art will be briefly described below, it is obvious that the drawings in the following description are only some embodiments of the present invention, and for those skilled in the art, other drawings can be obtained according to the drawings without creative efforts.
FIG. 1 is a schematic diagram illustrating a method for applying a sky filter to a still picture according to the present invention;
FIG. 2 is a flow chart illustrating a method for applying a sky filter to a still picture according to the present invention;
FIG. 3 is a schematic diagram of the steps of constructing a neural network parameter model according to the present invention;
FIG. 4 is a schematic view of a sky filter device for still pictures according to the present invention;
fig. 5 is a schematic structural diagram of an apparatus for adding a sky filter to a still picture according to another embodiment of the present invention.
Detailed Description
In order to make the objects, technical solutions and advantages of the present invention more apparent, the technical solutions of the present invention will be described in detail below. It is to be understood that the described embodiments are merely exemplary of the invention, and not restrictive of the full scope of the invention. All other embodiments, which can be derived by a person skilled in the art from the examples given herein without any inventive step, are within the scope of the present invention.
A specific method for applying a sky filter to a still picture provided in an embodiment of the present application will be described with reference to the accompanying drawings.
As shown in fig. 1, the present application provides a method for adding a sky filter to a still picture, comprising:
s101, training a neural network to obtain a neural network parameter model;
specifically, as shown in fig. 2, the neural network is trained by using multiple training data, and finally, an optimal neural network parameter model is obtained.
S102, loading the neural network parameter model into a mobile client, acquiring an original image by using the mobile client, and determining an image to be processed according to the original image;
and loading the trained neural network parameter model into a mobile client, shooting a picture by the mobile client, or acquiring the picture from a picture library stored in the mobile client as an original image, wherein the original image is a landscape or street view two-dimensional image obtained by shooting software or an image product, and adjusting the size of the original image into an image with a preset size so as to conveniently process the image, wherein the image with the preset size is the image to be processed.
Wherein, mobile client is the terminal that is equipped with the camera, for example: the mobile client can be a mobile phone, a tablet computer and the like.
S103, inputting the image to be processed into the neural network parameter model, wherein the neural network parameter model converts the image to be processed into an image with a sky area in a gray scale to obtain a first image;
and inputting the image adjusted to the preset size into a neural network parameter model to obtain a sky area mask gray image, which is called a first image.
S104, performing expansion corrosion operation on the first image to obtain a second image;
and performing expansion corrosion operation on the first image to expand the range of a certain degree of a local area with suddenly changed gray scale and gradually change the local area to obtain a second image.
S105, performing Gaussian blur processing on the second image by adopting a Gaussian convolution kernel to obtain a third image;
s106, performing color correction on each pixel in the original image by adopting a color searching and mapping algorithm to obtain a fourth image;
and obtaining an L UT lookup table corresponding to the sky material image for replacement in the file system, and performing color correction on each pixel in the original image by adopting a color lookup mapping algorithm to enable the original image to present a specific atmosphere toning effect to obtain a fourth image.
And S107, acquiring a sky material image for replacement, and linearly mixing the sky material image for replacement, the third image and the fourth image to acquire an image with a sky filter.
And acquiring a sky material image for replacement in a file system, and performing linear mixing of G channel component proportion on the fourth image and the sky material image for replacement in the range of 0-1 by adopting the G channel component of each pixel value in the third image to enable the image to present the effect of a sky filter, thereby obtaining the image with the sky filter.
In some embodiments, as shown in fig. 3, the training the neural network to obtain the neural network parameter model includes:
s301, acquiring a plurality of sky pictures, and determining a sky picture to be processed according to the sky picture;
the method comprises the steps of obtaining a plurality of sky pictures in a file system, marking sky areas in all the sky pictures, wherein the sky pictures marked with the sky areas are to-be-processed sky pictures.
S302, processing the sky picture to be processed to obtain a mask segmentation image; the mask segmentation image is a black and white image, a white area represents a sky area, and a black area represents a non-sky area;
and (4) carrying out segmentation processing on the sky picture marked with the sky area, namely segmenting the sky area and the non-sky area to obtain a mask segmentation image.
S303, processing the mask segmentation image by adopting a matting algorithm to obtain an alpha channel segmentation image;
and (4) segmenting details and highlighting edges of the sky region and the non-sky region through a matting algorithm to obtain an alpha channel segmentation map of the sky region and the non-sky region.
S304, adjusting the sky picture, the mask segmentation map and the alpha channel segmentation map to preset sizes to form training data;
the sizes of the sky picture, the mask segmentation map and the alpha channel segmentation map are adjusted to be the sky picture with a preset size, wherein the preset size is preset by a user and is adjusted to be enlarged or reduced, the mask segmentation map is the mask segmentation map corresponding to the sky picture, and the sky picture, the mask segmentation map and the alpha channel segmentation map which are adjusted to be the preset size are training data.
S305, inputting the training data into a neural network for processing to obtain a fine alpha channel segmentation graph;
and inputting the sky picture, the mask segmentation map and the alpha channel segmentation map which are adjusted to be in preset sizes into a neural network, and outputting a fine alpha channel segmentation map after the training data is processed by the neural network.
S306, initializing a neural network, calculating the image data loss between the predicted alpha channel segmentation graph and the obtained fine alpha channel segmentation graph by adopting a cross entropy loss function, and updating the neural network parameters by adopting a self-adaptive estimation matrix algorithm according to the image data loss;
and updating the neural network parameters by adopting a self-adaptive estimation matrix algorithm to optimize the neural parameter model, so that the training accuracy is improved.
S307, dynamically adjusting the learning rate in the training process, carrying out multiple times of training until the neural network converges, and storing the neural network parameters to obtain a neural network parameter model when the neural network prediction result enables the boundary of the sky region and the non-sky region to have a smooth effect.
In some embodiments, the determining an image to be processed from the original image includes:
and adjusting the size of the original image to a preset size, and determining the image adjusted to the preset size as an image to be processed.
The preset size is a size set in advance, and a user can set the preset size according to actual conditions, and the preset size is not limited in the application.
In some embodiments, the performing color correction on each pixel in the original image by using a color lookup mapping algorithm to obtain a fourth image includes:
obtaining L UT lookup tables corresponding to the replaced sky material images;
and carrying out color correction on each pixel in the original image by adopting a color lookup mapping algorithm according to the L UT lookup table to obtain a fourth image.
In some embodiments, the linearly mixing the sky material image for replacement, the third image and the fourth image to obtain an image with a sky filter includes:
performing three-channel separation on the third image to obtain an R channel, a G channel and a B channel, and calculating a G channel component of each pixel value in the third image;
when the G channel component is in the range of 0-1, the fourth image and the sky material image for replacement are linearly mixed according to the proportion of the G channel component;
an image exhibiting a sky filter effect is acquired.
Specifically, three channels of the third image are separated to obtain an R channel, a G channel and a B channel, a G channel component of each pixel value in the third image is calculated, and the fourth image and the sky material image for replacement are subjected to linear mixing of G channel component proportion within the range of 0-1, so that the image presents the effect of a sky filter.
In some embodiments, the obtaining a plurality of sky pictures and determining a sky picture to be processed according to the sky picture includes:
acquiring a plurality of sky pictures;
marking out sky areas of a plurality of the sky pictures;
and determining the sky picture marked with the sky area as a to-be-processed sky picture.
In some embodiments, the neural network comprises: a segmentation module and a feathering module;
the segmentation module includes: the device comprises a pooling layer, a convolution layer, a batch standardization layer, an activation layer, an upper sampling layer and a Softmax layer;
the feathering module includes: convolutional layers and Sigmoid layers.
Preferably, the inputting the training data into a neural network for processing to obtain a fine alpha channel segmentation map includes:
inputting a sky picture and a mask segmentation picture in training data into a segmentation module for processing to obtain a rough mask segmentation picture;
inputting the coarse mask segmentation map and an alpha segmentation map in the training data into a feather module for processing to obtain a fine alpha channel segmentation map.
Specifically, the neural network comprises a segmentation module and an emergence module, wherein the segmentation module comprises a pooling layer, a convolution layer, a batch normalization layer, an activation layer, an upsampling layer and a Softmax layer, the emergence module comprises a convolution layer and a Sigmoid layer,
the sky picture in the training data and the corresponding mask segmentation graph are processed by a segmentation module to generate a rough mask segmentation graph, and then the rough mask segmentation graph and an alpha channel segmentation graph in the training data are input into a feather module to output a fine alpha channel segmentation graph.
As shown in fig. 4, an embodiment of the present application provides an apparatus for adding a sky filter to a still picture, including:
a training module 401, configured to train a neural network to obtain a neural network parameter model;
an obtaining module 402, configured to load the neural network parameter model into a mobile client, obtain an original image by using the mobile client, and determine an image to be processed according to the original image;
a neural network module 403, configured to input the image to be processed into the neural network parameter model, where the neural network parameter model converts the image to be processed into an image with a sky area in a gray scale, so as to obtain a first image;
an expansion and corrosion module 404, configured to perform expansion and corrosion operation on the first image to obtain a second image;
a gaussian blur module 405, configured to perform gaussian blur processing on the second image to obtain a third image;
a color correction module 406, configured to perform color correction on each pixel in the original image by using a color lookup mapping algorithm to obtain a fourth image;
and the linear mixing module 407 is configured to acquire an image of the sky material for replacement, and perform linear mixing on the image of the sky material for replacement, the third image, and the fourth image to acquire an image with a sky filter.
The working principle of the device for adding the sky filter to the static picture is that a training module 401 trains a neural network to obtain a neural network parameter model; the obtaining module 402 loads the neural network parameter model into a mobile client, obtains an original image by using the mobile client, and determines an image to be processed according to the original image; the neural network module 403 inputs the image to be processed into the neural network parameter model, and the neural network parameter model converts the image to be processed into an image with a gray sky area, so as to obtain a first image; the expansion corrosion module 404 performs expansion corrosion operation on the first image to obtain a second image; the gaussian blurring module 405 performs gaussian blurring on the second image to obtain a third image; the color correction module 406 performs color correction on each pixel in the original image by using a color search mapping algorithm to obtain a fourth image; the linear mixing module 407 obtains a sky material image for replacement, and linearly mixes the sky material image for replacement, the third image and the fourth image to obtain an image with a sky filter.
In some embodiments, as shown in fig. 5, the obtaining module 401 includes:
a first obtaining unit 501, configured to obtain multiple sky pictures and determine a sky picture to be processed according to the sky picture;
a processing unit 502, configured to process the sky picture to be processed to obtain a mask segmentation map; the mask segmentation image is a black and white image, a white area represents a sky area, and a black area represents a non-sky area;
a calculating unit 503, configured to process the mask segmentation map by using a matting algorithm to obtain an alpha channel segmentation map;
an adjusting unit 504, configured to adjust the sky picture, the mask segmentation map, and the alpha channel segmentation map to a preset size to form training data;
a training unit 505, configured to input the training data into a neural network for processing, so as to obtain a fine alpha channel segmentation map;
an updating unit 506, configured to initialize a neural network, calculate an image data loss between the predicted alpha channel segmentation map and the obtained fine alpha channel segmentation map by using a cross entropy loss function, and update a neural network parameter by using an adaptive estimation matrix algorithm according to the image data loss;
and a second obtaining unit 507, configured to dynamically adjust a learning rate in a training process, perform multiple training until a neural network converges, and store a neural network parameter and obtain a neural network parameter model when a neural network prediction result enables a boundary between a sky region and a non-sky region to have a smoothing effect.
In summary, the method and apparatus for adding a sky filter to a static picture provided by the present application includes training a neural network to obtain a neural network parameter model; loading the neural network parameter model into a mobile client, acquiring an original image by using the mobile client, and determining an image to be processed according to the original image; inputting the image to be processed into the neural network parameter model, and converting the image to be processed into an image with a sky area in a gray scale by the neural network parameter model to obtain a first image; performing expansion corrosion operation on the first image to obtain a second image; performing Gaussian blur processing on the second image to obtain a third image; performing color correction on each pixel in the original image by adopting a color search mapping algorithm to obtain a fourth image; the method comprises the steps of obtaining a sky material image for replacement, and carrying out linear mixing on the sky material image for replacement, a third image and a fourth image to obtain an image with a sky filter.
It is to be understood that the embodiments of the method provided above correspond to the embodiments of the apparatus described above, and the corresponding specific contents may be referred to each other, which is not described herein again.
As will be appreciated by one skilled in the art, embodiments of the present application may be provided as a method, system, or computer program product. Accordingly, the present application may take the form of an entirely hardware embodiment, an entirely software embodiment or an embodiment combining software and hardware aspects. Furthermore, the present application may take the form of a computer program product embodied on one or more computer-usable storage media (including, but not limited to, disk storage, optical storage, and the like) having computer-usable program code embodied therein.
The present application is described with reference to flowchart illustrations and/or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of the application. It will be understood that each flow and/or block of the flow diagrams and/or block diagrams, and combinations of flows and/or blocks in the flow diagrams and/or block diagrams, can be implemented by computer program instructions. These computer program instructions may be provided to a processor of a general purpose computer, special purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, create means for implementing the functions specified in the flowchart flow or flows and/or block diagram block or blocks.
These computer program instructions may also be stored in a computer-readable memory that can direct a computer or other programmable data processing apparatus to function in a particular manner, such that the instructions stored in the computer-readable memory produce an article of manufacture including instruction means which implement the function specified in the flowchart flow or flows and/or block diagram block or blocks.
These computer program instructions may also be loaded onto a computer or other programmable data processing apparatus to cause a series of operational steps to be performed on the computer or other programmable apparatus to produce a computer implemented process such that the instructions which execute on the computer or other programmable apparatus provide steps for implementing the functions specified in the flowchart flow or flows and/or block diagram block or blocks.
The above description is only for the specific embodiments of the present invention, but the scope of the present invention is not limited thereto, and any person skilled in the art can easily conceive of the changes or substitutions within the technical scope of the present invention, and all the changes or substitutions should be covered within the scope of the present invention. Therefore, the protection scope of the present invention shall be subject to the protection scope of the appended claims.
Claims (10)
1. A method for applying a sky filter to a still picture, comprising:
training a neural network to obtain a neural network parameter model;
loading the neural network parameter model into a mobile client, acquiring an original image by using the mobile client, and determining an image to be processed according to the original image;
inputting the image to be processed into the neural network parameter model, and converting the image to be processed into an image with a sky area in a gray scale by the neural network parameter model to obtain a first image;
performing expansion corrosion operation on the first image to obtain a second image;
performing Gaussian blur processing on the second image to obtain a third image;
performing color correction on each pixel in the original image by adopting a color search mapping algorithm to obtain a fourth image;
and acquiring a sky material image for replacement, and linearly mixing the sky material image for replacement, the third image and the fourth image to acquire an image with a sky filter.
2. The method of claim 1, wherein training the neural network to obtain a neural network parametric model comprises:
acquiring a plurality of sky pictures, and determining a sky picture to be processed according to the sky picture;
processing the sky picture to be processed to obtain a mask segmentation image; the mask segmentation image is a black and white image, a white area represents a sky area, and a black area represents a non-sky area;
processing the mask segmentation image by adopting a matting algorithm to obtain an alpha channel segmentation image;
adjusting the sky picture, the mask segmentation map and the alpha channel segmentation map to preset sizes to form training data;
inputting the training data into a neural network for processing to obtain a fine alpha channel segmentation graph;
initializing a neural network, calculating the image data loss between the predicted alpha channel segmentation graph and the obtained fine alpha channel segmentation graph by adopting a cross entropy loss function, and updating neural network parameters by adopting a self-adaptive estimation matrix algorithm according to the image data loss;
and dynamically adjusting the learning rate in the training process, training for multiple times until the neural network converges, and storing the neural network parameters to obtain a neural network parameter model when the neural network prediction result enables the junction of the sky region and the non-sky region to have a smooth effect.
3. The method of claim 1, wherein determining the image to be processed from the original image comprises:
and adjusting the size of the original image to a preset size, and determining the image adjusted to the preset size as an image to be processed.
4. The method of claim 1, wherein the color correcting each pixel in the original image using a color lookup mapping algorithm to obtain a fourth image comprises:
obtaining L UT lookup tables corresponding to the replaced sky material images;
and carrying out color correction on each pixel in the original image by adopting a color lookup mapping algorithm according to the L UT lookup table to obtain a fourth image.
5. The method of claim 1, wherein the linearly blending the image of the sky material for replacement, the third image, and the fourth image to obtain an image with a sky filter comprises:
performing three-channel separation on the third image to obtain an R channel, a G channel and a B channel, and calculating a G channel component of each pixel value in the third image;
when the G channel component is in the range of 0-1, the fourth image and the sky material image for replacement are linearly mixed according to the proportion of the G channel component;
an image exhibiting a sky filter effect is acquired.
6. The method of claim 2, wherein the obtaining a plurality of sky pictures and determining a sky picture to be processed from the sky picture comprises:
acquiring a plurality of sky pictures;
marking out sky areas of a plurality of the sky pictures;
and determining the sky picture marked with the sky area as a to-be-processed sky picture.
7. The method of claim 1, wherein the neural network comprises: a segmentation module and a feathering module;
the segmentation module includes: the device comprises a pooling layer, a convolution layer, a batch standardization layer, an activation layer, an upper sampling layer and a Softmax layer;
the feathering module includes: convolutional layers and Sigmoid layers.
8. The method of claim 7, wherein inputting the training data into a neural network for processing to obtain a fine alpha channel segmentation map comprises:
inputting a sky picture and a mask segmentation picture in training data into a segmentation module for processing to obtain a rough mask segmentation picture;
inputting the coarse mask segmentation map and an alpha segmentation map in the training data into a feather module for processing to obtain a fine alpha channel segmentation map.
9. An apparatus for applying a sky filter to a still picture, comprising:
the training module is used for training the neural network to obtain a neural network parameter model;
the acquisition module is used for loading the neural network parameter model into a mobile client, acquiring an original image by using the mobile client, and determining an image to be processed according to the original image;
the neural network parameter model is used for converting the image to be processed into an image with a sky area in gray scale to obtain a first image;
the expansion corrosion module is used for performing expansion corrosion operation on the first image to obtain a second image;
the Gaussian blur module is used for carrying out Gaussian blur processing on the second image to obtain a third image;
the color correction module is used for performing color correction on each pixel in the original image by adopting a color searching and mapping algorithm to obtain a fourth image;
and the linear mixing module is used for acquiring a sky material image for replacement, and linearly mixing the sky material image for replacement, the third image and the fourth image to acquire an image with a sky filter.
10. The apparatus of claim 9, wherein the obtaining module comprises:
the system comprises a first acquisition unit, a second acquisition unit and a processing unit, wherein the first acquisition unit is used for acquiring a plurality of sky pictures and determining a sky picture to be processed according to the sky picture;
the processing unit is used for processing the sky picture to be processed to obtain a mask segmentation map; the mask segmentation image is a black and white image, a white area represents a sky area, and a black area represents a non-sky area;
the computing unit is used for processing the mask segmentation image by adopting a matting algorithm to obtain an alpha channel segmentation image;
the adjusting unit is used for adjusting the sky picture, the mask segmentation map and the alpha channel segmentation map to preset sizes so as to form training data;
the training unit is used for inputting the training data into a neural network for processing to obtain a fine alpha channel segmentation graph;
the updating unit is used for initializing the neural network, calculating the image data loss between the predicted alpha channel segmentation graph and the obtained fine alpha channel segmentation graph by adopting a cross entropy loss function, and updating the neural network parameters by adopting a self-adaptive estimation matrix algorithm according to the image data loss;
and the second acquisition unit is used for dynamically adjusting the learning rate in the training process and carrying out multiple times of training until the neural network converges, and storing the neural network parameters to acquire the neural network parameter model when the neural network prediction result enables the junction of the sky area and the non-sky area to have a smooth effect.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202010273375.0A CN111489322B (en) | 2020-04-09 | 2020-04-09 | Method and device for adding sky filter to static picture |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202010273375.0A CN111489322B (en) | 2020-04-09 | 2020-04-09 | Method and device for adding sky filter to static picture |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN111489322A true CN111489322A (en) | 2020-08-04 |
| CN111489322B CN111489322B (en) | 2023-05-26 |
Family
ID=71791869
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202010273375.0A Active CN111489322B (en) | 2020-04-09 | 2020-04-09 | Method and device for adding sky filter to static picture |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN111489322B (en) |
Cited By (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN112634165A (en) * | 2020-12-29 | 2021-04-09 | 广州光锥元信息科技有限公司 | Method and device for image adaptation VI environment |
| CN113034514A (en) * | 2021-03-19 | 2021-06-25 | 影石创新科技股份有限公司 | Sky region segmentation method and device, computer equipment and storage medium |
| CN113240599A (en) * | 2021-05-10 | 2021-08-10 | Oppo广东移动通信有限公司 | Image toning method and device, computer-readable storage medium and electronic equipment |
| CN113486271A (en) * | 2021-07-06 | 2021-10-08 | 网易（杭州）网络有限公司 | Method and device for determining dominant hue of image and electronic terminal |
| CN115908596B (en) * | 2021-08-20 | 2023-11-24 | 荣耀终端有限公司 | Image processing method and electronic equipment |
Citations (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| WO2017084391A1 (en) * | 2015-11-19 | 2017-05-26 | 广景视睿科技（深圳）有限公司 | Method and system for performing fluorescence processing on an image of a projector |
| CN109003237A (en) * | 2018-07-03 | 2018-12-14 | 深圳岚锋创视网络科技有限公司 | Sky filter method, device and the portable terminal of panoramic picture |
| CN109325495A (en) * | 2018-09-21 | 2019-02-12 | 南京邮电大学 | A kind of crop image segmentation system and method based on deep neural network modeling |
| WO2019180742A1 (en) * | 2018-03-21 | 2019-09-26 | Artificial Learning Systems India Private Limited | System and method for retinal fundus image semantic segmentation |
| CN110610526A (en) * | 2019-08-12 | 2019-12-24 | 江苏大学 | Method for segmenting monocular portrait and rendering depth of field based on WNET |
| WO2020015470A1 (en) * | 2018-07-16 | 2020-01-23 | Oppo广东移动通信有限公司 | Image processing method and apparatus, mobile terminal, and computer-readable storage medium |
-
2020
- 2020-04-09 CN CN202010273375.0A patent/CN111489322B/en active Active
Patent Citations (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| WO2017084391A1 (en) * | 2015-11-19 | 2017-05-26 | 广景视睿科技（深圳）有限公司 | Method and system for performing fluorescence processing on an image of a projector |
| WO2019180742A1 (en) * | 2018-03-21 | 2019-09-26 | Artificial Learning Systems India Private Limited | System and method for retinal fundus image semantic segmentation |
| CN109003237A (en) * | 2018-07-03 | 2018-12-14 | 深圳岚锋创视网络科技有限公司 | Sky filter method, device and the portable terminal of panoramic picture |
| WO2020015470A1 (en) * | 2018-07-16 | 2020-01-23 | Oppo广东移动通信有限公司 | Image processing method and apparatus, mobile terminal, and computer-readable storage medium |
| CN109325495A (en) * | 2018-09-21 | 2019-02-12 | 南京邮电大学 | A kind of crop image segmentation system and method based on deep neural network modeling |
| CN110610526A (en) * | 2019-08-12 | 2019-12-24 | 江苏大学 | Method for segmenting monocular portrait and rendering depth of field based on WNET |
Cited By (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN112634165A (en) * | 2020-12-29 | 2021-04-09 | 广州光锥元信息科技有限公司 | Method and device for image adaptation VI environment |
| CN112634165B (en) * | 2020-12-29 | 2024-03-26 | 广州光锥元信息科技有限公司 | Method and device for image adaptation VI environment |
| CN113034514A (en) * | 2021-03-19 | 2021-06-25 | 影石创新科技股份有限公司 | Sky region segmentation method and device, computer equipment and storage medium |
| CN113240599A (en) * | 2021-05-10 | 2021-08-10 | Oppo广东移动通信有限公司 | Image toning method and device, computer-readable storage medium and electronic equipment |
| CN113486271A (en) * | 2021-07-06 | 2021-10-08 | 网易（杭州）网络有限公司 | Method and device for determining dominant hue of image and electronic terminal |
| CN115908596B (en) * | 2021-08-20 | 2023-11-24 | 荣耀终端有限公司 | Image processing method and electronic equipment |
Also Published As
| Publication number | Publication date |
|---|---|
| CN111489322B (en) | 2023-05-26 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| Wang et al. | Gladnet: Low-light enhancement network with global awareness | |
| CN111489322B (en) | Method and device for adding sky filter to static picture | |
| US10580120B2 (en) | Image filtering based on image gradients | |
| CN109191395B (en) | Image contrast enhancement method, device, equipment and storage medium | |
| US8035871B2 (en) | Determining target luminance value of an image using predicted noise amount | |
| JP2004537791A (en) | Method and system for modifying a digital image in consideration of noise | |
| CN109658330B (en) | Color development adjusting method and device | |
| CN111383232B (en) | Matting method, matting device, terminal equipment and computer readable storage medium | |
| KR20150031241A (en) | A device and a method for color harmonization of an image | |
| CN114331873B (en) | Non-uniform illumination color image correction method based on region division | |
| CN113039576A (en) | Image enhancement system and method | |
| RU2697627C1 (en) | Method of correcting illumination of an object on an image in a sequence of images and a user's computing device which implements said method | |
| WO2022194079A1 (en) | Sky region segmentation method and apparatus, computer device, and storage medium | |
| CN114862725B (en) | Method and device for realizing motion perception fuzzy special effect based on optical flow method | |
| CN108596992B (en) | Rapid real-time lip gloss makeup method | |
| CN114862722B (en) | Image brightness enhancement implementation method and processing terminal | |
| CN113706393A (en) | Video enhancement method, device, equipment and storage medium | |
| CN111010605B (en) | Method for displaying video picture-in-picture window | |
| JP2008011286A (en) | Image processing program and image processor | |
| KR100791374B1 (en) | Method and apparatus for image adaptive color adjustment of pixel in color gamut | |
| US20220398704A1 (en) | Intelligent Portrait Photography Enhancement System | |
| JP2015158737A (en) | Image processor and image processing method | |
| US11200708B1 (en) | Real-time color vector preview generation | |
| CN116977190A (en) | Image processing method, apparatus, device, storage medium, and program product | |
| KR20140138046A (en) | Method and device for processing a picture |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |