CN111489322B - Method and device for adding sky filter to static picture - Google Patents
Method and device for adding sky filter to static picture Download PDFInfo
- Publication number
- CN111489322B CN111489322B CN202010273375.0A CN202010273375A CN111489322B CN 111489322 B CN111489322 B CN 111489322B CN 202010273375 A CN202010273375 A CN 202010273375A CN 111489322 B CN111489322 B CN 111489322B
- Authority
- CN
- China
- Prior art keywords
- image
- sky
- neural network
- segmentation map
- processed
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
- 238000000034 method Methods 0.000 title claims abstract description 36
- 230000003068 static effect Effects 0.000 title abstract description 7
- 238000013528 artificial neural network Methods 0.000 claims abstract description 109
- 238000012545 processing Methods 0.000 claims abstract description 43
- 239000000463 material Substances 0.000 claims abstract description 30
- 238000002156 mixing Methods 0.000 claims abstract description 21
- 238000012937 correction Methods 0.000 claims abstract description 20
- 238000005260 corrosion Methods 0.000 claims abstract description 14
- 230000007797 corrosion Effects 0.000 claims abstract description 14
- 230000011218 segmentation Effects 0.000 claims description 106
- 238000012549 training Methods 0.000 claims description 56
- 230000000694 effects Effects 0.000 claims description 21
- 238000013507 mapping Methods 0.000 claims description 15
- 230000032669 eclosion Effects 0.000 claims description 12
- 230000006870 function Effects 0.000 claims description 10
- 230000008569 process Effects 0.000 claims description 10
- 239000011159 matrix material Substances 0.000 claims description 7
- 230000003044 adaptive effect Effects 0.000 claims description 5
- 230000004913 activation Effects 0.000 claims description 4
- 238000010606 normalization Methods 0.000 claims description 4
- 238000011176 pooling Methods 0.000 claims description 4
- 238000000926 separation method Methods 0.000 claims description 4
- 238000003062 neural network model Methods 0.000 abstract description 2
- 238000010586 diagram Methods 0.000 description 10
- 238000004590 computer program Methods 0.000 description 7
- 238000003860 storage Methods 0.000 description 3
- 230000007704 transition Effects 0.000 description 3
- 238000009877 rendering Methods 0.000 description 2
- 238000012360 testing method Methods 0.000 description 2
- 230000004075 alteration Effects 0.000 description 1
- 230000009286 beneficial effect Effects 0.000 description 1
- 230000008859 change Effects 0.000 description 1
- 239000003086 colorant Substances 0.000 description 1
- 238000004519 manufacturing process Methods 0.000 description 1
- 230000001537 neural effect Effects 0.000 description 1
- 230000003287 optical effect Effects 0.000 description 1
- 238000005457 optimization Methods 0.000 description 1
- 238000006467 substitution reaction Methods 0.000 description 1
- XLYOFNOQVPJJNP-UHFFFAOYSA-N water Substances O XLYOFNOQVPJJNP-UHFFFAOYSA-N 0.000 description 1
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T5/00—Image enhancement or restoration
- G06T5/50—Image enhancement or restoration using two or more images, e.g. averaging or subtraction
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/08—Learning methods
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T5/00—Image enhancement or restoration
- G06T5/20—Image enhancement or restoration using local operators
- G06T5/30—Erosion or dilatation, e.g. thinning
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T7/00—Image analysis
- G06T7/10—Segmentation; Edge detection
- G06T7/11—Region-based segmentation
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/10—Image acquisition modality
- G06T2207/10004—Still image; Photographic image
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/20—Special algorithmic details
- G06T2207/20081—Training; Learning
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/20—Special algorithmic details
- G06T2207/20084—Artificial neural networks [ANN]
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/20—Special algorithmic details
- G06T2207/20212—Image combination
- G06T2207/20221—Image fusion; Image merging
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/30—Subject of image; Context of image processing
- G06T2207/30181—Earth observation
- G06T2207/30192—Weather; Meteorology
-
- Y—GENERAL TAGGING OF NEW TECHNOLOGICAL DEVELOPMENTS; GENERAL TAGGING OF CROSS-SECTIONAL TECHNOLOGIES SPANNING OVER SEVERAL SECTIONS OF THE IPC; TECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS
- Y02—TECHNOLOGIES OR APPLICATIONS FOR MITIGATION OR ADAPTATION AGAINST CLIMATE CHANGE
- Y02A—TECHNOLOGIES FOR ADAPTATION TO CLIMATE CHANGE
- Y02A90/00—Technologies having an indirect contribution to adaptation to climate change
- Y02A90/10—Information and communication technologies [ICT] supporting adaptation to climate change, e.g. for weather forecasting or climate simulation
Landscapes
- Engineering & Computer Science (AREA)
- Physics & Mathematics (AREA)
- Theoretical Computer Science (AREA)
- General Physics & Mathematics (AREA)
- Data Mining & Analysis (AREA)
- Evolutionary Computation (AREA)
- Artificial Intelligence (AREA)
- Biomedical Technology (AREA)
- Biophysics (AREA)
- Computational Linguistics (AREA)
- Health & Medical Sciences (AREA)
- Life Sciences & Earth Sciences (AREA)
- General Health & Medical Sciences (AREA)
- Molecular Biology (AREA)
- Computing Systems (AREA)
- General Engineering & Computer Science (AREA)
- Mathematical Physics (AREA)
- Software Systems (AREA)
- Computer Vision & Pattern Recognition (AREA)
- Image Processing (AREA)
Abstract
The invention relates to a method and a device for adding a sky filter to a static picture, wherein the method comprises the steps of loading a neural network parameter model to a mobile client, acquiring an original image and determining an image to be processed; inputting an image to be processed into a neural network parameter model, and converting the image into an image with gray scale in a sky area to obtain a first image; performing expansion corrosion operation on the first image to obtain a second image; performing Gaussian blur processing on the second image by adopting Gaussian convolution check to obtain a third image; performing color correction on each pixel in the original image to obtain a fourth image; and linearly mixing the sky material image for replacement, the third image and the fourth image to obtain an image with a sky filter. According to the invention, the neural network model is constructed, the original image is segmented and processed, and the beautified sky image is finally output.
Description
Technical Field
The invention belongs to the technical field of image processing, and particularly relates to a method and a device for adding a sky filter to a static picture.
Background
The current methods for carrying out filter color matching on the sky in the image in the market are as follows: some sky segmentation algorithms based on YCRBCR space, HSV space, RGB space are first used. However, the method is mainly remained in the paper and demo test stage, and the segmentation effect is poor in a large number of picture tests based on life scenes; the algorithm is usually used for identifying whether a sky area exists on a photo in the market, and performs certain HDR light optimization, and under the use scenes, the accuracy and the speed of the sky segmentation area are not too high.
In addition, due to the method, the operation speed is low, the obtained sky area has strong edge sense, and the situation of wrong segmentation of the non-sky areas with similar colors can occur when the segmentation effect in pictures is not accurate enough under the scenes of sea-sky connection, green water, blue mountain and the like. If the algorithm is directly used, the sky area of the segmented picture is directly filled and beautified in color, obvious color difference can occur in the sky area compared with other normally imaged areas, the edge transition effect is hard, the image expression effect is poor and unnatural, the phenomenon of clamping and the like occurs in rendering, the beautified sky area and the original picture cannot be fused naturally, and the effect is abrupt.
Disclosure of Invention
In view of the above, the present invention aims to overcome the shortcomings of the prior art, and provide a method and a device for adding a sky filter to a still picture, so as to solve the problems of low accuracy in segmentation of a sky region and a non-sky region, low operation speed, hard sky edge transition effect and unnatural overall image effect in the prior art.
In order to achieve the above purpose, the invention adopts the following technical scheme: a method of adding a sky filter to a still picture, comprising:
training the neural network to obtain a neural network parameter model;
loading the neural network parameter model to a mobile client, acquiring an original image by using the mobile client, and determining an image to be processed according to the original image;
inputting the image to be processed into the neural network parameter model, and converting the image to be processed into an image with gray scale in a sky area by the neural network parameter model to obtain a first image;
performing expansion corrosion operation on the first image to obtain a second image;
carrying out Gaussian blur processing on the second image to obtain a third image;
performing color correction on each pixel in the original image by adopting a color search mapping algorithm to obtain a fourth image;
and acquiring a sky material image for replacement, and linearly mixing the sky material image for replacement, the third image and the fourth image to acquire an image with a sky filter.
Further, the training the neural network to obtain the neural network parameter model includes:
acquiring a plurality of sky pictures, and determining a sky picture to be processed according to the sky pictures;
processing the sky picture to be processed to obtain a mask segmentation map; the mask segmentation map is a black-and-white picture, a white area represents a sky area, and a black area represents a non-sky area;
processing the mask segmentation map by adopting an image matting algorithm to obtain an alpha channel segmentation map;
adjusting the sky picture, the mask segmentation map and the alpha channel segmentation map to a preset size to form training data;
inputting the training data into a neural network for processing to obtain a fine alpha channel segmentation map;
initializing a neural network, calculating the image data loss between a predicted alpha channel segmentation map and an obtained fine alpha channel segmentation map by adopting a cross entropy loss function, and updating the neural network parameters by adopting a self-adaptive estimation matrix algorithm according to the image data loss;
and dynamically adjusting the learning rate in the training process and training for a plurality of times until the neural network converges, and storing the neural network parameters when the neural network prediction result enables the intersection of the sky area and the non-sky area to show a smooth effect, so as to obtain a neural network parameter model.
Further, the determining an image to be processed according to the original image includes:
and adjusting the size of the original image to a preset size, and determining the image adjusted to the preset size as an image to be processed.
Further, the performing color correction on each pixel in the original image by using a color lookup mapping algorithm to obtain a fourth image includes:
obtaining an LUT lookup table corresponding to the sky material image for replacement;
and carrying out color correction on each pixel in the original image by adopting a color lookup mapping algorithm according to the LUT lookup table to obtain a fourth image.
Further, the linearly mixing the sky material image for replacement, the third image and the fourth image to obtain an image with a sky filter, including:
three-channel separation is carried out on the third image, an R channel, a G channel and a B channel are obtained, and G channel components of each pixel value in the third image are calculated;
when the G channel component is in the range of 0-1, linearly mixing the fourth image and the sky material image for replacement according to the G channel component proportion;
and acquiring an image showing the sky filter effect.
Further, the obtaining a plurality of sky pictures and determining a sky picture to be processed according to the sky pictures includes:
acquiring a plurality of sky pictures;
marking sky areas of a plurality of sky pictures;
and determining the sky picture marked with the sky area as a sky picture to be processed.
Further, the neural network includes: a segmentation module and an eclosion module;
the segmentation module comprises: pooling layer, convolution layer, batch normalization layer, activation layer, upsampling layer, and Softmax layer;
the eclosion module comprises: a convolution layer and a Sigmoid layer.
Further, the step of inputting the training data into a neural network for processing to obtain a fine alpha channel segmentation map includes:
inputting the sky pictures and the mask segmentation pictures in the training data into a segmentation module for processing to obtain a rough mask segmentation picture;
and inputting the rough mask segmentation map and the alpha segmentation map in the training data to an eclosion module for processing to obtain a fine alpha channel segmentation map.
The embodiment of the application provides a device for adding a sky filter to a static picture, which comprises:
the training module is used for training the neural network to obtain a neural network parameter model;
the acquisition module is used for loading the neural network parameter model to a mobile client, acquiring an original image by using the mobile client, and determining an image to be processed according to the original image;
the neural network module is used for inputting the image to be processed into the neural network parameter model, and the neural network parameter model converts the image to be processed into an image with gray scale in a sky area to obtain a first image;
the expansion corrosion module is used for carrying out expansion corrosion operation on the first image to obtain a second image;
the Gaussian blur module is used for carrying out Gaussian blur processing on the second image to obtain a third image;
the color correction module is used for carrying out color correction on each pixel in the original image by adopting a color search mapping algorithm to obtain a fourth image;
and the linear mixing module is used for acquiring the sky material image for replacement, and linearly mixing the sky material image for replacement, the third image and the fourth image to acquire an image with a sky filter.
Further, the obtaining module includes:
the first acquisition unit is used for acquiring a plurality of sky pictures and determining the sky pictures to be processed according to the sky pictures;
the processing unit is used for processing the sky pictures to be processed to obtain mask segmentation graphs; the mask segmentation map is a black-and-white picture, a white area represents a sky area, and a black area represents a non-sky area;
the computing unit is used for processing the mask segmentation map by adopting an image matting algorithm to obtain an alpha channel segmentation map;
the adjusting unit is used for adjusting the sky picture, the mask segmentation map and the alpha channel segmentation map to a preset size to form training data;
the training unit is used for inputting the training data into a neural network for processing to obtain a fine alpha channel segmentation graph;
the updating unit is used for initializing a neural network, calculating the image data loss between the predicted alpha channel segmentation map and the obtained fine alpha channel segmentation map by adopting a cross entropy loss function, and updating the neural network parameters by adopting an adaptive estimation matrix algorithm according to the image data loss;
the second acquisition unit is used for dynamically adjusting the learning rate in the training process and performing multiple training until the neural network converges, and storing the neural network parameters when the neural network prediction result enables the intersection of the sky area and the non-sky area to show a smooth effect, so as to acquire a neural network parameter model.
By adopting the technical scheme, the invention has the following beneficial effects:
according to the invention, the neural network model is constructed, the original image is segmented and processed, and the beautified sky image is finally output, so that the problems that the sky area has obvious chromatic aberration compared with other normally imaged areas, the edge transition effect is hard, the image representation effect is poor and unnatural, the phenomenon of clamping and the like occurs in rendering are solved.
Drawings
In order to more clearly illustrate the embodiments of the invention or the technical solutions in the prior art, the drawings that are required in the embodiments or the description of the prior art will be briefly described, it being obvious that the drawings in the following description are only some embodiments of the invention, and that other drawings may be obtained according to these drawings without inventive effort for a person skilled in the art.
FIG. 1 is a schematic diagram showing steps of a method for adding a sky filter to a still picture according to the present invention;
FIG. 2 is a flow chart of a method for adding a sky filter to a still picture according to the present invention;
FIG. 3 is a schematic diagram of the steps for constructing a neural network parametric model according to the present invention;
FIG. 4 is a schematic diagram of a device for adding a sky filter to a still picture according to the present invention;
fig. 5 is a schematic diagram showing another structure of a sky filter device for adding a still picture according to the present invention.
Detailed Description
In order to make the objects, technical solutions and advantages of the present invention more apparent, the technical solutions of the present invention will be described in detail below. It will be apparent that the described embodiments are only some, but not all, embodiments of the invention. All other embodiments, based on the examples herein, which are within the scope of the invention as defined by the claims, will be within the scope of the invention as defined by the claims.
A specific method for adding a sky filter to a still picture according to an embodiment of the present application will be described with reference to the accompanying drawings.
As shown in fig. 1, the present application provides a method for adding a sky filter to a still picture, including:
s101, training a neural network to obtain a neural network parameter model;
specifically, as shown in fig. 2, the neural network is trained by using multiple training data, and finally, an optimal neural network parameter model is obtained.
S102, loading the neural network parameter model to a mobile client, acquiring an original image by using the mobile client, and determining an image to be processed according to the original image;
loading the trained neural network parameter model into a mobile client, taking a picture by a user through the mobile client or acquiring the picture from a picture library stored in the mobile client as an original image, namely, the original image is a landscape or street view two-dimensional image obtained through photographing software or image products, adjusting the size of the original image into an image with a preset size so as to facilitate processing of the image, wherein the image with the preset size is the image to be processed.
The mobile client is a terminal provided with a camera, for example: the mobile client may be a cell phone, tablet computer, etc.
S103, inputting the image to be processed into the neural network parameter model, and converting the image to be processed into an image with gray scale in a sky area by the neural network parameter model to obtain a first image;
and inputting the image adjusted to be of a preset size into a neural network parameter model to obtain a sky area mask gray image, which is called a first image.
S104, performing expansion corrosion operation on the first image to obtain a second image;
and performing expansion corrosion operation on the first image to expand the range of the local area with the gray scale suddenly changed to a certain extent and slow down the change, so as to obtain a second image.
S105, performing Gaussian blur processing on the second image by adopting Gaussian convolution check to obtain a third image;
s106, performing color correction on each pixel in the original image by adopting a color search mapping algorithm to obtain a fourth image;
and obtaining an LUT lookup table corresponding to the sky material image for replacement in a file system, and carrying out color correction on each pixel in the original image by adopting a color lookup mapping algorithm to enable the original image to present a specific atmosphere color matching effect, so as to obtain a fourth image.
And S107, acquiring a sky material image for replacement, and linearly mixing the sky material image for replacement, the third image and the fourth image to acquire an image with a sky filter.
And acquiring a sky material image for replacement in a file system, adopting a G channel component of each pixel value in the third image, and linearly mixing the fourth image and the sky material image for replacement in a G channel component proportion within a range of 0-1 to enable a picture to show the effect of a sky filter, thereby obtaining the image with the sky filter.
In some embodiments, as shown in fig. 3, the training the neural network to obtain the neural network parameter model includes:
s301, acquiring a plurality of sky pictures, and determining a sky picture to be processed according to the sky pictures;
and acquiring a plurality of sky pictures in the file system, marking the sky areas in all the sky pictures, and obtaining the sky picture marked with the sky areas as the sky picture to be processed.
S302, processing the sky picture to be processed to obtain a mask segmentation map; the mask segmentation map is a black-and-white picture, a white area represents a sky area, and a black area represents a non-sky area;
and (3) dividing the sky picture marked with the sky region, namely dividing the sky region and the non-sky region to obtain a mask division map.
S303, processing the mask segmentation map by adopting an image matting algorithm to obtain an alpha channel segmentation map;
and carrying out segmentation details and salient edges on the sky area and the non-sky area through an image matting algorithm to obtain an alpha channel segmentation map of the sky area and the non-sky area.
S304, adjusting the sky picture, the mask segmentation map and the alpha channel segmentation map to a preset size to form training data;
and adjusting the sizes of the sky picture, the mask segmentation map and the alpha channel segmentation map to be sky pictures with preset sizes, wherein the preset sizes are preset by a user, the sizes are adjusted to be enlarged or reduced for the pictures, the mask segmentation map is a mask segmentation map corresponding to the sky pictures, and the sky pictures, the mask segmentation map and the alpha channel segmentation map which are adjusted to be the preset sizes are training data.
S305, inputting the training data into a neural network for processing to obtain a fine alpha channel segmentation map;
inputting the sky picture, the mask segmentation map and the alpha channel segmentation map which are adjusted to be of preset sizes into a neural network, and outputting a fine alpha channel segmentation map after the neural network processes the training data.
S306, initializing a neural network, calculating the image data loss between a predicted alpha channel segmentation map and an obtained fine alpha channel segmentation map by adopting a cross entropy loss function, and updating the neural network parameters by adopting an adaptive estimation matrix algorithm according to the image data loss;
and updating the neural network parameters by adopting an adaptive estimation matrix algorithm to optimize the neural parameter model, so that the training accuracy is improved.
S307, dynamically adjusting the learning rate in the training process and training for a plurality of times until the neural network converges, and storing the neural network parameters when the neural network prediction result enables the intersection of the sky area and the non-sky area to show a smooth effect, so as to obtain a neural network parameter model.
In some embodiments, the determining an image to be processed according to the original image includes:
and adjusting the size of the original image to a preset size, and determining the image adjusted to the preset size as an image to be processed.
The preset size is a size set in advance, and can be set by a user according to actual conditions, which is not limited herein.
In some embodiments, the performing color correction on each pixel in the original image by using a color lookup mapping algorithm to obtain a fourth image includes:
obtaining an LUT lookup table corresponding to the sky material image for replacement;
and carrying out color correction on each pixel in the original image by adopting a color lookup mapping algorithm according to the LUT lookup table to obtain a fourth image.
In some embodiments, the linearly mixing the sky material image for replacement, the third image, and the fourth image to obtain an image with a sky filter includes:
three-channel separation is carried out on the third image, an R channel, a G channel and a B channel are obtained, and G channel components of each pixel value in the third image are calculated;
when the G channel component is in the range of 0-1, linearly mixing the fourth image and the sky material image for replacement according to the G channel component proportion;
and acquiring an image showing the sky filter effect.
Specifically, three channels of separation are carried out on the third image, an R channel, a G channel and a B channel are obtained, a G channel component of each pixel value in the third image is calculated, and in the range of 0-1, linear mixing of the proportion of the G channel component is carried out on the fourth image and the sky material image for replacement, so that the effect of the sky filter is displayed on the picture.
In some embodiments, the obtaining a plurality of sky pictures and determining a sky picture to be processed according to the sky pictures includes:
acquiring a plurality of sky pictures;
marking sky areas of a plurality of sky pictures;
and determining the sky picture marked with the sky area as a sky picture to be processed.
In some embodiments, the neural network comprises: a segmentation module and an eclosion module;
the segmentation module comprises: pooling layer, convolution layer, batch normalization layer, activation layer, upsampling layer, and Softmax layer;
the eclosion module comprises: a convolution layer and a Sigmoid layer.
Preferably, the step of inputting the training data into a neural network for processing to obtain a fine alpha channel segmentation map includes:
inputting the sky pictures and the mask segmentation pictures in the training data into a segmentation module for processing to obtain a rough mask segmentation picture;
and inputting the rough mask segmentation map and the alpha segmentation map in the training data to an eclosion module for processing to obtain a fine alpha channel segmentation map.
Specifically, the neural network comprises a segmentation module and an eclosion module, wherein the segmentation module comprises a pooling layer, a convolution layer, a batch normalization layer, an activation layer, an upsampling layer and a Softmax layer, wherein the eclosion module comprises a convolution layer and a Sigmoid layer,
the sky pictures in the training data and the corresponding mask segmentation graphs are processed by a segmentation module to generate rough mask segmentation graphs, and then the rough mask segmentation graphs and the alpha channel segmentation graphs in the training data are input into an eclosion module to output fine alpha channel segmentation graphs.
As shown in fig. 4, an embodiment of the present application provides an apparatus for adding a sky filter to a still picture, including:
the training module 401 is configured to train the neural network to obtain a neural network parameter model;
an acquisition module 402, configured to load the neural network parameter model into a mobile client, acquire an original image using the mobile client, and determine an image to be processed according to the original image;
the neural network module 403 is configured to input the image to be processed into the neural network parameter model, where the neural network parameter model converts the image to be processed into an image with a gray scale in a sky area, so as to obtain a first image;
the expansion corrosion module 404 is configured to perform an expansion corrosion operation on the first image to obtain a second image;
the gaussian blur module 405 is configured to perform gaussian blur processing on the second image to obtain a third image;
a color correction module 406, configured to perform color correction on each pixel in the original image by using a color lookup mapping algorithm, so as to obtain a fourth image;
the linear mixing module 407 is configured to obtain an image of the sky material for replacement, and perform linear mixing on the image of the sky material for replacement, the third image, and the fourth image to obtain an image with a sky filter.
The working principle of the device for adding the sky filter to the static picture provided by the application is that a training module 401 trains a neural network to obtain a neural network parameter model; the acquisition module 402 loads the neural network parameter model to a mobile client, acquires an original image by using the mobile client, and determines an image to be processed according to the original image; the neural network module 403 inputs the image to be processed into the neural network parameter model, and the neural network parameter model converts the image to be processed into an image with gray scale in a sky area, so as to obtain a first image; the expansion corrosion module 404 performs expansion corrosion operation on the first image to obtain a second image; the gaussian blur module 405 performs gaussian blur processing on the second image to obtain a third image; the color correction module 406 performs color correction on each pixel in the original image by using a color lookup mapping algorithm to obtain a fourth image; the linear mixing module 407 acquires an image of the sky material for replacement, and performs linear mixing on the image of the sky material for replacement, the third image and the fourth image to acquire an image with a sky filter.
In some embodiments, as shown in fig. 5, the obtaining module 401 includes:
a first obtaining unit 501, configured to obtain a plurality of sky pictures, and determine a sky picture to be processed according to the sky pictures;
a processing unit 502, configured to process the sky image to be processed to obtain a mask segmentation map; the mask segmentation map is a black-and-white picture, a white area represents a sky area, and a black area represents a non-sky area;
a calculating unit 503, configured to process the mask segmentation map by using a matting algorithm to obtain an alpha channel segmentation map;
an adjusting unit 504, configured to adjust the sky image, the mask segmentation map, and the alpha channel segmentation map to a preset size to form training data;
the training unit 505 is configured to input the training data into a neural network for processing, so as to obtain a fine alpha channel segmentation map;
an updating unit 506, configured to initialize a neural network, calculate an image data loss between the predicted alpha channel segmentation map and the obtained fine alpha channel segmentation map using a cross entropy loss function, and update a neural network parameter using an adaptive estimation matrix algorithm according to the image data loss;
the second obtaining unit 507 is configured to dynamically adjust the learning rate and perform multiple training in the training process until the neural network converges, and store the neural network parameters when the neural network prediction result makes the intersection between the sky area and the non-sky area show a smooth effect, so as to obtain a neural network parameter model.
In summary, the method and the device for adding the sky filter to the static picture provided by the application comprise the steps of training a neural network to obtain a neural network parameter model; loading the neural network parameter model to a mobile client, acquiring an original image by using the mobile client, and determining an image to be processed according to the original image; inputting the image to be processed into the neural network parameter model, and converting the image to be processed into an image with gray scale in a sky area by the neural network parameter model to obtain a first image; performing expansion corrosion operation on the first image to obtain a second image; carrying out Gaussian blur processing on the second image to obtain a third image; performing color correction on each pixel in the original image by adopting a color search mapping algorithm to obtain a fourth image; the method comprises the steps of obtaining a sky material image for replacement, linearly mixing the sky material image for replacement, a third image and a fourth image, and obtaining an image with a sky filter.
It can be understood that the above-provided method embodiments correspond to the above-described apparatus embodiments, and corresponding specific details may be referred to each other and will not be described herein.
It will be appreciated by those skilled in the art that embodiments of the present application may be provided as a method, system, or computer program product. Accordingly, the present application may take the form of an entirely hardware embodiment, an entirely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, the present application may take the form of a computer program product embodied on one or more computer-usable storage media (including, but not limited to, magnetic disk storage, optical storage, and the like) having computer-usable program code embodied therein.
The present application is described with reference to flowchart illustrations and/or block diagrams of methods, apparatus (systems) and computer program products according to embodiments of the application. It will be understood that each flow and/or block of the flowchart illustrations and/or block diagrams, and combinations of flows and/or blocks in the flowchart illustrations and/or block diagrams, can be implemented by computer program instructions. These computer program instructions may be provided to a processor of a general purpose computer, special purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, create means for implementing the functions specified in the flowchart flow or flows and/or block diagram block or blocks.
These computer program instructions may also be stored in a computer-readable memory that can direct a computer or other programmable data processing apparatus to function in a particular manner, such that the instructions stored in the computer-readable memory produce an article of manufacture including instruction means which implement the function specified in the flowchart flow or flows and/or block diagram block or blocks.
These computer program instructions may also be loaded onto a computer or other programmable data processing apparatus to cause a series of operational steps to be performed on the computer or other programmable apparatus to produce a computer implemented process such that the instructions which execute on the computer or other programmable apparatus provide steps for implementing the functions specified in the flowchart flow or flows and/or block diagram block or blocks.
The foregoing is merely illustrative of the present invention, and the present invention is not limited thereto, and any person skilled in the art will readily recognize that variations or substitutions are within the scope of the present invention. Therefore, the protection scope of the present invention shall be subject to the protection scope of the claims.
Claims (8)
1. A method of adding a sky filter to a still picture, comprising:
training the neural network to obtain a neural network parameter model;
training the neural network to obtain a neural network parameter model, including:
acquiring a plurality of sky pictures, and determining a sky picture to be processed according to the sky pictures;
processing the sky picture to be processed to obtain a mask segmentation map; the mask segmentation map is a black-and-white picture, a white area represents a sky area, and a black area represents a non-sky area;
processing the mask segmentation map by adopting an image matting algorithm to obtain an alpha channel segmentation map;
adjusting the sky picture, the mask segmentation map and the alpha channel segmentation map to a preset size to form training data;
inputting the training data into a neural network for processing to obtain a fine alpha channel segmentation map;
initializing a neural network, calculating the image data loss between a predicted alpha channel segmentation map and an obtained fine alpha channel segmentation map by adopting a cross entropy loss function, and updating the neural network parameters by adopting a self-adaptive estimation matrix algorithm according to the image data loss;
dynamically adjusting the learning rate in the training process and training for a plurality of times until the neural network converges, and storing the neural network parameters when the neural network prediction result enables the intersection of the sky area and the non-sky area to show a smooth effect, so as to obtain a neural network parameter model;
loading the neural network parameter model to a mobile client, acquiring an original image by using the mobile client, and determining an image to be processed according to the original image;
inputting the image to be processed into the neural network parameter model, and converting the image to be processed into an image with gray scale in a sky area by the neural network parameter model to obtain a first image;
performing expansion corrosion operation on the first image to obtain a second image;
carrying out Gaussian blur processing on the second image to obtain a third image;
performing color correction on each pixel in the original image by adopting a color search mapping algorithm to obtain a fourth image;
and acquiring a sky material image for replacement, and linearly mixing the sky material image for replacement, the third image and the fourth image to acquire an image with a sky filter.
2. The method of claim 1, wherein said determining an image to be processed from said original image comprises:
and adjusting the size of the original image to a preset size, and determining the image adjusted to the preset size as the image to be processed.
3. The method of claim 1, wherein performing color correction on each pixel in the original image using a color look-up mapping algorithm to obtain a fourth image comprises:
obtaining an LUT lookup table corresponding to the sky material image for replacement;
and carrying out color correction on each pixel in the original image by adopting a color lookup mapping algorithm according to the LUT lookup table to obtain a fourth image.
4. The method of claim 1, wherein the linearly blending the image of the sky material for replacement, the third image, and the fourth image to obtain an image having a sky filter comprises:
three-channel separation is carried out on the third image, an R channel, a G channel and a B channel are obtained, and G channel components of each pixel value in the third image are calculated;
when the G channel component is in the range of 0-1, linearly mixing the fourth image and the sky material image for replacement according to the G channel component proportion;
and acquiring an image showing the sky filter effect.
5. The method of claim 1, wherein the obtaining a plurality of sky pictures and determining a sky picture to be processed from the sky pictures comprises:
acquiring a plurality of sky pictures;
marking sky areas of a plurality of sky pictures;
and determining the sky picture marked with the sky area as a sky picture to be processed.
6. The method of claim 1, wherein the neural network comprises: a segmentation module and an eclosion module;
the segmentation module comprises: pooling layer, convolution layer, batch normalization layer, activation layer, upsampling layer, and Softmax layer;
the eclosion module comprises: a convolution layer and a Sigmoid layer.
7. The method of claim 6, wherein said inputting said training data into a neural network for processing results in a refined alpha channel segmentation map, comprising:
inputting the sky pictures and the mask segmentation pictures in the training data into a segmentation module for processing to obtain a rough mask segmentation picture;
and inputting the rough mask segmentation map and the alpha segmentation map in the training data to an eclosion module for processing to obtain a fine alpha channel segmentation map.
8. An apparatus for adding a sky filter to a still picture, comprising:
the training module is used for training the neural network to obtain a neural network parameter model;
the training module comprises:
the first acquisition unit is used for acquiring a plurality of sky pictures and determining the sky pictures to be processed according to the sky pictures;
the processing unit is used for processing the sky pictures to be processed to obtain mask segmentation graphs; the mask segmentation map is a black-and-white picture, a white area represents a sky area, and a black area represents a non-sky area;
the computing unit is used for processing the mask segmentation map by adopting an image matting algorithm to obtain an alpha channel segmentation map;
the adjusting unit is used for adjusting the sky picture, the mask segmentation map and the alpha channel segmentation map to a preset size to form training data;
the training unit is used for inputting the training data into a neural network for processing to obtain a fine alpha channel segmentation graph;
the updating unit is used for initializing a neural network, calculating the image data loss between the predicted alpha channel segmentation map and the obtained fine alpha channel segmentation map by adopting a cross entropy loss function, and updating the neural network parameters by adopting an adaptive estimation matrix algorithm according to the image data loss;
the second acquisition unit is used for dynamically adjusting the learning rate in the training process and performing multiple training until the neural network converges, and storing the neural network parameters when the neural network prediction result enables the intersection of the sky area and the non-sky area to show a smooth effect, so as to acquire a neural network parameter model;
the acquisition module is used for loading the neural network parameter model to a mobile client, acquiring an original image by using the mobile client, and determining an image to be processed according to the original image;
the neural network module is used for inputting the image to be processed into the neural network parameter model, and the neural network parameter model converts the image to be processed into an image with gray scale in a sky area to obtain a first image;
the expansion corrosion module is used for carrying out expansion corrosion operation on the first image to obtain a second image;
the Gaussian blur module is used for carrying out Gaussian blur processing on the second image to obtain a third image;
the color correction module is used for carrying out color correction on each pixel in the original image by adopting a color search mapping algorithm to obtain a fourth image;
and the linear mixing module is used for acquiring the sky material image for replacement, and linearly mixing the sky material image for replacement, the third image and the fourth image to acquire an image with a sky filter.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202010273375.0A CN111489322B (en) | 2020-04-09 | 2020-04-09 | Method and device for adding sky filter to static picture |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202010273375.0A CN111489322B (en) | 2020-04-09 | 2020-04-09 | Method and device for adding sky filter to static picture |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN111489322A CN111489322A (en) | 2020-08-04 |
| CN111489322B true CN111489322B (en) | 2023-05-26 |
Family
ID=71791869
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202010273375.0A Active CN111489322B (en) | 2020-04-09 | 2020-04-09 | Method and device for adding sky filter to static picture |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN111489322B (en) |
Families Citing this family (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN112634165B (en) * | 2020-12-29 | 2024-03-26 | 广州光锥元信息科技有限公司 | Method and device for image adaptation VI environment |
| CN113034514A (en) * | 2021-03-19 | 2021-06-25 | 影石创新科技股份有限公司 | Sky region segmentation method and device, computer equipment and storage medium |
| CN113240599A (en) * | 2021-05-10 | 2021-08-10 | Oppo广东移动通信有限公司 | Image toning method and device, computer-readable storage medium and electronic equipment |
| CN113486271A (en) * | 2021-07-06 | 2021-10-08 | 网易（杭州）网络有限公司 | Method and device for determining dominant hue of image and electronic terminal |
| CN115908596B (en) * | 2021-08-20 | 2023-11-24 | 荣耀终端有限公司 | Image processing method and electronic equipment |
Citations (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN109325495A (en) * | 2018-09-21 | 2019-02-12 | 南京邮电大学 | A kind of crop image segmentation system and method based on deep neural network modeling |
Family Cites Families (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN105472361B (en) * | 2015-11-19 | 2018-03-27 | 广景视睿科技（深圳）有限公司 | A kind of method and system of projector image fluorescence processing |
| WO2019180742A1 (en) * | 2018-03-21 | 2019-09-26 | Artificial Learning Systems India Private Limited | System and method for retinal fundus image semantic segmentation |
| CN109003237A (en) * | 2018-07-03 | 2018-12-14 | 深圳岚锋创视网络科技有限公司 | Sky filter method, device and the portable terminal of panoramic picture |
| CN108900769B (en) * | 2018-07-16 | 2020-01-10 | Oppo广东移动通信有限公司 | Image processing method, image processing device, mobile terminal and computer readable storage medium |
| CN110610526B (en) * | 2019-08-12 | 2023-09-22 | 江苏大学 | Method for segmenting monocular image and rendering depth of field based on WNET |
-
2020
- 2020-04-09 CN CN202010273375.0A patent/CN111489322B/en active Active
Patent Citations (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN109325495A (en) * | 2018-09-21 | 2019-02-12 | 南京邮电大学 | A kind of crop image segmentation system and method based on deep neural network modeling |
Also Published As
| Publication number | Publication date |
|---|---|
| CN111489322A (en) | 2020-08-04 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN111489322B (en) | Method and device for adding sky filter to static picture | |
| JP2004537791A (en) | Method and system for modifying a digital image in consideration of noise | |
| CN109817170B (en) | Pixel compensation method and device and terminal equipment | |
| CN110335330B (en) | Image simulation generation method and system, deep learning algorithm training method and electronic equipment | |
| CN113039576A (en) | Image enhancement system and method | |
| US11750935B2 (en) | Systems and methods of image enhancement | |
| CN114862722B (en) | Image brightness enhancement implementation method and processing terminal | |
| WO2022194079A1 (en) | Sky region segmentation method and apparatus, computer device, and storage medium | |
| CN116368811A (en) | Saliency-based capture or image processing | |
| EP4261784A1 (en) | Image processing method and apparatus based on artificial intelligence, and electronic device, computer-readable storage medium and computer program product | |
| CN113870099B (en) | Picture color conversion method, device, equipment and readable storage medium | |
| CN113706393A (en) | Video enhancement method, device, equipment and storage medium | |
| CN112149745B (en) | Method, device, equipment and storage medium for determining difficult example sample | |
| CN113112422B (en) | Image processing method, device, electronic equipment and computer readable medium | |
| CN110580696A (en) | Multi-exposure image fast fusion method for detail preservation | |
| CN112218005A (en) | Video editing method based on artificial intelligence | |
| US20220398704A1 (en) | Intelligent Portrait Photography Enhancement System | |
| CN111462158A (en) | Image processing method and device, intelligent device and storage medium | |
| KR20070063781A (en) | Method and apparatus for image adaptive color adjustment of pixel in color gamut | |
| CN115205168A (en) | Image processing method, device, electronic equipment, storage medium and product | |
| CN116977190A (en) | Image processing method, apparatus, device, storage medium, and program product | |
| CN110647898B (en) | Image processing method, image processing device, electronic equipment and computer storage medium | |
| CN109242750B (en) | Picture signature method, picture matching method, device, equipment and storage medium | |
| Zhou et al. | Saliency preserving decolorization | |
| CN111476731A (en) | Image correction method, image correction device, storage medium and electronic equipment |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |