CN113706372A - Automatic cutout model establishing method and system - Google Patents
Automatic cutout model establishing method and system Download PDFInfo
- Publication number
- CN113706372A CN113706372A CN202110962940.9A CN202110962940A CN113706372A CN 113706372 A CN113706372 A CN 113706372A CN 202110962940 A CN202110962940 A CN 202110962940A CN 113706372 A CN113706372 A CN 113706372A
- Authority
- CN
- China
- Prior art keywords
- picture
- training
- model
- data set
- matting
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Granted
Links
- 238000000034 method Methods 0.000 title claims abstract description 44
- 230000011218 segmentation Effects 0.000 claims abstract description 64
- 230000006870 function Effects 0.000 claims description 16
- 238000004590 computer program Methods 0.000 claims description 12
- 238000012163 sequencing technique Methods 0.000 claims description 4
- 230000003190 augmentative effect Effects 0.000 claims description 3
- 230000003247 decreasing effect Effects 0.000 claims description 3
- 238000010586 diagram Methods 0.000 description 13
- 238000012986 modification Methods 0.000 description 6
- 230000004048 modification Effects 0.000 description 6
- 238000000605 extraction Methods 0.000 description 4
- 239000011159 matrix material Substances 0.000 description 4
- 238000006748 scratching Methods 0.000 description 3
- 230000002393 scratching effect Effects 0.000 description 3
- 230000004075 alteration Effects 0.000 description 2
- 230000000694 effects Effects 0.000 description 2
- 238000001914 filtration Methods 0.000 description 2
- 230000004927 fusion Effects 0.000 description 2
- 238000002372 labelling Methods 0.000 description 2
- 239000000463 material Substances 0.000 description 2
- 230000007423 decrease Effects 0.000 description 1
- 238000005516 engineering process Methods 0.000 description 1
- 238000004519 manufacturing process Methods 0.000 description 1
- 238000005065 mining Methods 0.000 description 1
- 230000003287 optical effect Effects 0.000 description 1
- 238000007781 pre-processing Methods 0.000 description 1
- 238000006467 substitution reaction Methods 0.000 description 1
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T3/00—Geometric image transformations in the plane of the image
- G06T3/04—Context-preserving transformations, e.g. by using an importance map
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T7/00—Image analysis
- G06T7/10—Segmentation; Edge detection
- G06T7/11—Region-based segmentation
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T7/00—Image analysis
- G06T7/10—Segmentation; Edge detection
- G06T7/194—Segmentation; Edge detection involving foreground-background segmentation
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/10—Image acquisition modality
- G06T2207/10004—Still image; Photographic image
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/10—Image acquisition modality
- G06T2207/10016—Video; Image sequence
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/20—Special algorithmic details
- G06T2207/20081—Training; Learning
-
- Y—GENERAL TAGGING OF NEW TECHNOLOGICAL DEVELOPMENTS; GENERAL TAGGING OF CROSS-SECTIONAL TECHNOLOGIES SPANNING OVER SEVERAL SECTIONS OF THE IPC; TECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS
- Y02—TECHNOLOGIES OR APPLICATIONS FOR MITIGATION OR ADAPTATION AGAINST CLIMATE CHANGE
- Y02T—CLIMATE CHANGE MITIGATION TECHNOLOGIES RELATED TO TRANSPORTATION
- Y02T10/00—Road transport of goods or passengers
- Y02T10/10—Internal combustion engine [ICE] based vehicles
- Y02T10/40—Engine management systems
Landscapes
- Engineering & Computer Science (AREA)
- Physics & Mathematics (AREA)
- General Physics & Mathematics (AREA)
- Theoretical Computer Science (AREA)
- Computer Vision & Pattern Recognition (AREA)
- Image Analysis (AREA)
- Processing Or Creating Images (AREA)
Abstract
The invention discloses an automatic cutout model establishing method, medium, equipment and a system, wherein the method comprises the following steps: acquiring a historical picture, and generating a training data set according to the historical picture; training a segmentation model according to the training data set so as to generate a corresponding triple picture according to an original picture in the training data set through the segmentation model obtained through training; and training a matting model according to the original pictures in the training data set and the triple pictures corresponding to the original pictures to generate the matting model. The matting model established by the automatic matting model establishing method can automatically matte the foreground part according to the original picture input by the user, so that the editing time required by the user matting is shortened, and the matting difficulty is reduced; meanwhile, the stability of the matting result is ensured.
Description
The application is a divisional application with the application number of 2020106111751 and the application date of 2020, 6.30.3 entitled "automatic cutout method and system".
Technical Field
The invention relates to the technical field of image processing, in particular to an automatic cutout model establishing method, a computer readable storage medium, a computer device and an automatic cutout model establishing system.
Background
Matting (image background removal) refers to the accurate extraction of foreground objects in still pictures or video picture sequences, and is a key technology in many image editions.
In the related art, in the process of matting a still picture or a video picture, a foreground part in the still picture or the video picture is mostly scratched manually, and this process consumes a large amount of editing time of a user and the operation doorsill is high. Meanwhile, due to artificial uncertainty, the matting result is unstable, and the final matting result is possibly not fine enough.
Disclosure of Invention
The present invention is directed to solving, at least to some extent, one of the technical problems in the art described above. Therefore, one objective of the present invention is to provide an automatic cutout model establishing method, where the cutout model established by the automatic cutout model establishing method can perform automatic cutout of the foreground part according to the original picture input by the user, so as to reduce the editing time required by the user cutout and reduce the cutout difficulty; meanwhile, the stability of the matting result is ensured.
A second object of the invention is to propose a computer-readable storage medium.
A third object of the invention is to propose a computer device.
The fourth purpose of the invention is to provide an automatic cutout model building system.
In order to achieve the above object, an embodiment of a first aspect of the present invention provides an automatic cutout model establishing method, including the following steps: acquiring a historical picture, and generating a training data set according to the historical picture; training a segmentation model according to the training data set so as to generate a corresponding triple picture according to an original picture in the training data set through the segmentation model obtained through training; and training a matting model according to the original pictures in the training data set and the triple pictures corresponding to the original pictures to generate the matting model.
According to the automatic cutout method, firstly, a historical picture is obtained, and a training data set is generated according to the historical picture; secondly, training a segmentation model according to the training data set so as to generate a corresponding triple picture according to the original picture in the training data set by the segmentation model obtained through training; and then, training a matting model according to the original pictures in the training data set and the triple pictures corresponding to the original pictures to generate the matting model. Then, acquiring a picture to be scratched, and inputting the picture to be scratched into the segmentation model so as to generate a triple picture corresponding to the picture to be scratched through the segmentation model; then, inputting the picture to be scratched and the triple picture corresponding to the picture to be scratched into the scratch model, so as to generate a graphic mask corresponding to the picture to be scratched through the scratch model, and automatically scratching the picture to be scratched according to the graphic mask; therefore, automatic cutout of the foreground part is realized according to the original picture input by the user, the editing time required by the cutout of the user is shortened, and the cutout difficulty is reduced; meanwhile, the stability of the matting result is ensured.
In addition, the automatic matting model establishing method proposed according to the above embodiment of the present invention may also have the following additional technical features:
optionally, the step of training the segmentation model according to the training data set includes:
setting the initial learning rate to 0.001, and decreasing the learning rate according to a polynomial; the training period is X, and the loss function adopts cross entropy; the network weights continuously update the gradient information through back propagation to complete the training of the segmentation model.
Optionally, the step of training the segmentation model according to the training data set further includes:
in each iteration process, sequencing each pixel according to a loss function, and if the sequencing is advanced, considering that the sample error is large and the weight needs to be increased for key learning; furthermore, an error interval is set, so that only the samples in the error interval are calculated by the loss function, and the training efficiency of the segmentation model is improved.
Optionally, the step of training the segmentation model according to the training data set further includes:
the training data set is augmented according to the following formula: i ═ alpha × Fg + (1-alpha) × Bg, where Fg is the original RGB picture of the training dataset, alpha is the corresponding mask, Bg is the background dataset candidate picture, I is the synthesized new picture
Optionally, the step of generating, by the segmentation model obtained through training, a corresponding triple picture according to the original picture in the training data set includes:
generating a multi-scale feature corresponding to the original picture according to the original picture, and fusing the multi-scale feature to generate a feature layer corresponding to the original picture;
and performing fine-grained segmentation according to the original picture and the feature layer corresponding to the original picture to generate a triple picture corresponding to the original picture.
To achieve the above object, a second embodiment of the present invention provides a computer-readable storage medium, on which an automatic matting program is stored, and the automatic matting program, when executed by a processor, implements the automatic matting model establishing method as described above.
According to the computer-readable storage medium of the embodiment of the invention, the automatic cutout program is stored, so that when the processor executes the automatic cutout model establishing program, the automatic cutout model establishing method is realized, automatic cutout of the foreground part is realized according to the original picture input by the user, the editing time required by the user cutout is reduced, and the cutout difficulty is reduced; meanwhile, the stability of the matting result is ensured.
To achieve the above object, a third embodiment of the present invention provides a computer device, which includes a memory, a processor, and a computer program stored in the memory and running on the processor, and when the processor executes the computer program, the automatic cutout model building method as described above is implemented.
In order to achieve the above object, a fourth aspect of the present invention provides an automatic matting model establishing system, including: the acquisition module is used for acquiring a historical picture and generating a training data set according to the historical picture; the first training module is used for training a segmentation model according to the training data set so as to generate a corresponding triple picture according to an original picture in the training data set through the segmentation model obtained through training; and the second training module is used for training a cutout model according to the original pictures in the training data set and the triple pictures corresponding to the original pictures so as to generate the cutout model.
The automatic cutout model can automatically cutout the foreground part according to the original picture input by the user, so that the editing time required by the cutout of the user is shortened, and the cutout difficulty is reduced; meanwhile, the stability of the sectional drawing result is ensured
Drawings
FIG. 1 is a schematic flow diagram of an automatic matting method according to an embodiment of the invention;
FIG. 2 is a block diagram schematic diagram of an automatic matting system according to an embodiment of the invention;
fig. 3 is a block schematic diagram of an automatic matting system according to another embodiment of the invention.
Detailed Description
Reference will now be made in detail to embodiments of the present invention, examples of which are illustrated in the accompanying drawings, wherein like or similar reference numerals refer to the same or similar elements or elements having the same or similar function throughout. The embodiments described below with reference to the drawings are illustrative and intended to be illustrative of the invention and are not to be construed as limiting the invention.
In the related art, when a still picture or a video picture is subjected to matting, a large amount of editing time of a user needs to be wasted, an operation doorsill is high, and meanwhile, the matting result is unstable; according to the automatic cutout method, firstly, a historical picture is obtained, and a training data set is generated according to the historical picture; secondly, training a segmentation model according to the training data set so as to generate a corresponding triple picture according to the original picture in the training data set by the segmentation model obtained through training; then, training a cutout model according to the original pictures in the training data set and the triple pictures corresponding to the original pictures to generate the cutout model; then, acquiring a picture to be scratched, and inputting the picture to be scratched into the segmentation model so as to generate a triple picture corresponding to the picture to be scratched through the segmentation model; then, inputting the picture to be scratched and the triple picture corresponding to the picture to be scratched into the scratch model, so as to generate a graphic mask corresponding to the picture to be scratched through the scratch model, and automatically scratching the picture to be scratched according to the graphic mask; therefore, automatic cutout of the foreground part is realized according to the original picture input by the user, the editing time required by the cutout of the user is shortened, and the cutout difficulty is reduced; meanwhile, the stability of the matting result is ensured.
In order to better understand the above technical solutions, exemplary embodiments of the present invention will be described in more detail below with reference to the accompanying drawings. While exemplary embodiments of the invention are shown in the drawings, it should be understood that the invention can be embodied in various forms and should not be limited to the embodiments set forth herein. Rather, these embodiments are provided so that this disclosure will be thorough and complete, and will fully convey the scope of the invention to those skilled in the art.
In order to better understand the technical solution, the technical solution will be described in detail with reference to the drawings and the specific embodiments.
Fig. 1 is a schematic flow diagram of an automatic matting method according to an embodiment of the present invention, as shown in fig. 1, the automatic matting method includes the following steps:
s101, acquiring a historical picture, and generating a training data set according to the historical picture.
That is, a history picture (e.g., a portrait picture, a commodity picture, an animal picture, a vehicle picture, etc.) is acquired, and a training data set for model training is generated from the history picture.
In some embodiments, before generating the training data set according to the historical pictures, to improve the training effect of the subsequent model training, the method further includes: calculating the signal-to-noise ratio corresponding to each historical picture, and filtering the historical pictures according to the signal-to-noise ratios; and marking the significance foreground in the filtered historical picture so as to generate a training data set according to the marked historical picture.
That is, firstly, calculating a signal-to-noise ratio corresponding to each historical picture, and filtering out the pictures with blur and excessive noise in the historical pictures according to the signal-to-noise ratio; then, labeling the filtered historical pictures to label out a significant foreground (such as a portrait, a commodity, an animal, a vehicle and the like in the pictures); further, a training data set may be generated from the labeled historical pictures.
In some embodiments, in order to reduce the difficulty of acquiring the training data set and increase the number of samples of the training data set to improve the training effect of the model, after labeling the significance foreground in the filtered historical picture, the method further includes: and acquiring a background data set, and randomly replacing the background in the marked historical picture according to the background data set to generate a corresponding extended sample so as to generate a training data set according to the marked historical picture and the extended sample.
That is, first, a background data set (i.e., a set of background pictures available for replacement) is obtained; then, the background in the history picture is randomly replaced according to the background data set and the labeled history picture to generate a new picture (i.e. an expansion sample), so that the training data set is expanded.
As an example, the training data set is augmented according to the following formula: i ═ alpha × Fg + (1-alpha) × Bg, where Fg is the original RGB picture of the training dataset, alpha is the corresponding mask, Bg is the background dataset candidate picture, I is the synthesized new picture.
And S102, training a segmentation model according to the training data set so as to generate a corresponding triple picture according to the original picture in the training data set by the segmentation model obtained through training.
That is, a segmentation model is trained according to a training data set to obtain a segmentation model, and the segmentation model can generate a triplet image corresponding to an input image according to the image.
There are various ways to train the segmentation model based on the training data set.
As an example, the initial learning rate is set to 0.001, and the learning rate is decreased in accordance with a polynomial; the training period is X, and the loss function adopts cross entropy; the network weights continuously update the gradient information through back propagation to complete the training of the segmentation model.
In some embodiments, in order to improve the training efficiency of the segmentation model, difficult sample mining may be adopted, that is, in each iteration process, each pixel is sorted according to a loss function, and if the sorting is advanced, the sample error is considered to be large, and the weight needs to be increased for key learning; furthermore, an error interval can be set, so that the loss function only calculates samples in the error interval, and the training efficiency of the segmentation model is improved.
In some embodiments, the generating, by the trained segmentation model, a corresponding triplet picture from an original picture in the training dataset includes: generating a multi-scale feature corresponding to an original picture according to the original picture, and fusing the multi-scale feature to generate a feature layer corresponding to the original picture; and performing fine-grained segmentation according to the original picture and the characteristic layer corresponding to the original picture to generate a triple picture corresponding to the original picture.
That is to say, the segmentation model includes a coarse-grained identification module and a fine-grained identification module, where the coarse-grained identification module includes a feature extraction layer and a feature fusion layer, and the feature extraction layer is configured to perform feature extraction on an input original picture to extract multi-scale features corresponding to the original picture; the feature fusion layer is used for splicing features of different scales (namely multi-scale features) to generate a feature layer corresponding to the original picture; the fine-grained identification module is used for performing fine-grained segmentation on the feature layer and the original picture output by the coarse-grained identification module (namely optimizing a coarse-grained result); to generate a triple picture (including a foreground-background-transition region) corresponding to the original picture.
In some embodiments, the fine-grained identification module may employ a lightweight UNet structure.
In some embodiments, in order to overcome the problem of insufficient depth convolution receptive field, after generating the feature layer corresponding to the original picture, the method further includes: extracting the pixel characteristics corresponding to each pixel in the original picture, calculating a similarity matrix among the pixels, and calculating an information gain value among the pixels according to the pixel characteristics and the similarity matrix so as to update the characteristic layer according to the information gain value.
That is, the coarse-grained module of the segmentation model further includes a pixel association module, where the pixel association module is configured to extract a pixel feature T corresponding to each pixel, and calculate a similarity matrix R between pixels by using a point-product algorithm, so that an information gain value V between the pixel and each of the other pixels except for the pixel can be calculated according to the similarity matrix R and the pixel feature T; furthermore, the characteristic layer can be updated according to the information gain value so as to overcome the problem of insufficient depth convolution receptive field.
S103, training a matting model according to an original picture in the training data set and a triple picture corresponding to the original picture to generate the matting model.
As an example, since the cutout model focuses on learning the edge basic features, a UNet structure can be adopted to fuse the shallow feature geometric features and the high-level semantic features, the loss function of the model is the average error between the prediction mask and the actual mask, the initial learning rate is set to 0.001, and the learning rate decreases progressively according to a polynomial; the training period is X; the network weights continuously update the gradient information by back propagation.
In some embodiments, to ensure consistency of the segmentation model and the matting model input and output data, the entire matting system can also be fine-tuned.
As an example, the learning rate is fixed to 0.0001, the training period is Y, the coarse-grained identification module monitors loss and adopts cross entropy, the loss function of the matting model also adopts L1 regression error, and the network weight continuously updates gradient information through back propagation of the two to complete the fine adjustment process of the matting system.
And S104, acquiring the picture to be scratched, inputting the picture to be scratched into the segmentation model, and generating a triple picture corresponding to the picture to be scratched through the segmentation model.
And S105, inputting the picture to be subjected to matting and the triple picture corresponding to the picture to be subjected to matting into a matting model, generating a graphic mask corresponding to the picture to be subjected to matting through the matting model, and automatically matting the picture to be subjected to matting according to the graphic mask.
That is, after a segmentation model and a matting model are obtained through training, when a user needs to perform automatic matting on a picture, firstly, a picture to be subjected to matting is obtained and is input into the segmentation model, and the segmentation model generates a corresponding triple picture according to the input picture to be subjected to matting; then, inputting the obtained triple picture and the corresponding picture to be scratched into a scratch model, and generating a graphic mask corresponding to the picture to be scratched according to the input by the scratch model; then, the picture to be scratched can be automatically scratched according to the corresponding picture mask to obtain the required foreground image.
In summary, according to the automatic matting method of the embodiment of the invention, firstly, a history picture is obtained, and a training data set is generated according to the history picture; secondly, training a segmentation model according to the training data set so as to generate a corresponding triple picture according to the original picture in the training data set by the segmentation model obtained through training; then, training a cutout model according to the original pictures in the training data set and the triple pictures corresponding to the original pictures to generate the cutout model; then, acquiring a picture to be scratched, and inputting the picture to be scratched into the segmentation model so as to generate a triple picture corresponding to the picture to be scratched through the segmentation model; then, inputting the picture to be scratched and the triple picture corresponding to the picture to be scratched into the scratch model, so as to generate a graphic mask corresponding to the picture to be scratched through the scratch model, and automatically scratching the picture to be scratched according to the graphic mask; therefore, automatic cutout of the foreground part is realized according to the original picture input by the user, the editing time required by the cutout of the user is shortened, and the cutout difficulty is reduced; meanwhile, the stability of the matting result is ensured.
In order to implement the foregoing embodiments, the present invention further provides a computer-readable storage medium, on which an automatic matting program is stored, and when being executed by a processor, the automatic matting program implements the automatic matting method as described above.
According to the computer-readable storage medium of the embodiment of the invention, the automatic cutout program is stored, so that the processor can realize the automatic cutout method when executing the automatic cutout program, thereby realizing automatic cutout of the foreground part according to the original picture input by the user, reducing the editing time required by the user cutout and reducing the cutout difficulty; meanwhile, the stability of the matting result is ensured.
In order to implement the foregoing embodiments, an embodiment of the present invention further provides a computer device, including a memory, a processor, and a computer program stored in the memory and executable on the processor, where the processor executes the computer program to implement the automatic matting method as described above.
According to the computer equipment provided by the embodiment of the invention, the automatic matting program is stored through the memory, so that the processor can realize the automatic matting method when executing the automatic matting program, thereby realizing automatic matting of a foreground part according to an original picture input by a user, reducing editing time required by user matting and reducing matting difficulty; meanwhile, the stability of the matting result is ensured.
In order to implement the above embodiment, an embodiment of the present invention further provides an automatic cutout system, as shown in fig. 2, the automatic cutout system includes: an acquisition module 10, a first training module 20, a second training module 30, and an automatic matting module 40.
The acquisition module 10 is configured to acquire a history picture and generate a training data set according to the history picture;
the first training module 20 is configured to train a segmentation model according to a training data set, so that a corresponding triple picture is generated according to an original picture in the training data set by the segmentation model obtained through training;
the second training module 30 is configured to perform a matting model training according to an original picture in a training data set and a triplet picture corresponding to the original picture to generate a matting model;
the automatic matting module 40 is configured to obtain a picture to be scratched and input the picture to be scratched into a segmentation model, so as to generate a triplet picture corresponding to the picture to be scratched through the segmentation model;
the automatic matting module 40 is further configured to input the to-be-matting picture and the triplet picture corresponding to the to-be-matting picture into the matting model, so as to generate a graphic mask corresponding to the to-be-matting picture through the matting model, and perform automatic matting on the to-be-matting picture according to the graphic mask.
In some embodiments, as shown in fig. 3, the automatic matting system proposed by the embodiment of the present invention further includes: and the preprocessing module 50 is configured to calculate a signal-to-noise ratio corresponding to each historical picture, filter the historical pictures according to the signal-to-noise ratios, and label a significant foreground in the filtered historical pictures, so as to generate a training data set according to the labeled historical pictures.
In some embodiments, as shown in fig. 3, the automatic matting system proposed by the embodiment of the present invention further includes: and the sample expansion module 60 is configured to obtain a background data set, and randomly replace the background in the labeled history picture according to the background data set to generate a corresponding expansion sample, so as to generate a training data set according to the labeled history picture and the expansion sample.
It should be noted that the above description about the automatic cutout method in fig. 1 is also applicable to the automatic cutout system, and is not repeated herein.
In summary, according to the automatic matting system of the embodiment of the invention, the acquisition module is arranged to acquire the historical picture, and generate the training data set according to the historical picture; the first training module is used for training a segmentation model according to the training data set so as to generate a corresponding triple picture according to an original picture in the training data set through the segmentation model obtained through training; the second training module is used for training a matting model according to the original pictures in the training data set and the triple pictures corresponding to the original pictures to generate the matting model; the automatic matting module is used for acquiring a picture to be matting and inputting the picture to be matting into the segmentation model so as to generate a triple picture corresponding to the picture to be matting through the segmentation model; the automatic matting module is further configured to input the to-be-matting picture and the triplet picture corresponding to the to-be-matting picture into the matting model, so as to generate a graphic mask corresponding to the to-be-matting picture through the matting model, and perform automatic matting on the to-be-matting picture according to the graphic mask; therefore, automatic cutout of the foreground part is realized according to the original picture input by the user, the editing time required by the cutout of the user is shortened, and the cutout difficulty is reduced; meanwhile, the stability of the matting result is ensured.
As will be appreciated by one skilled in the art, embodiments of the present invention may be provided as a method, system, or computer program product. Accordingly, the present invention may take the form of an entirely hardware embodiment, an entirely software embodiment or an embodiment combining software and hardware aspects. Furthermore, the present invention may take the form of a computer program product embodied on one or more computer-usable storage media (including, but not limited to, disk storage, CD-ROM, optical storage, and the like) having computer-usable program code embodied therein.
The present invention is described with reference to flowchart illustrations and/or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of the invention. It will be understood that each flow and/or block of the flow diagrams and/or block diagrams, and combinations of flows and/or blocks in the flow diagrams and/or block diagrams, can be implemented by computer program instructions. These computer program instructions may be provided to a processor of a general purpose computer, special purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, create means for implementing the functions specified in the flowchart flow or flows and/or block diagram block or blocks.
These computer program instructions may also be stored in a computer-readable memory that can direct a computer or other programmable data processing apparatus to function in a particular manner, such that the instructions stored in the computer-readable memory produce an article of manufacture including instruction means which implement the function specified in the flowchart flow or flows and/or block diagram block or blocks.
These computer program instructions may also be loaded onto a computer or other programmable data processing apparatus to cause a series of operational steps to be performed on the computer or other programmable apparatus to produce a computer implemented process such that the instructions which execute on the computer or other programmable apparatus provide steps for implementing the functions specified in the flowchart flow or flows and/or block diagram block or blocks.
It should be noted that in the claims, any reference signs placed between parentheses shall not be construed as limiting the claim. The word "comprising" does not exclude the presence of elements or steps not listed in a claim. The word "a" or "an" preceding an element does not exclude the presence of a plurality of such elements. The invention may be implemented by means of hardware comprising several distinct elements, and by means of a suitably programmed computer. In the unit claims enumerating several means, several of these means may be embodied by one and the same item of hardware. The usage of the words first, second and third, etcetera do not indicate any ordering. These words may be interpreted as names.
While preferred embodiments of the present invention have been described, additional variations and modifications in those embodiments may occur to those skilled in the art once they learn of the basic inventive concepts. Therefore, it is intended that the appended claims be interpreted as including preferred embodiments and all such alterations and modifications as fall within the scope of the invention.
It will be apparent to those skilled in the art that various changes and modifications may be made in the present invention without departing from the spirit and scope of the invention. Thus, if such modifications and variations of the present invention fall within the scope of the claims of the present invention and their equivalents, the present invention is also intended to include such modifications and variations.
In the description of the present invention, it is to be understood that the terms "first", "second" and the like are used for descriptive purposes only and are not to be construed as indicating or implying relative importance or implying any number of technical features indicated. Thus, a feature defined as "first" or "second" may explicitly or implicitly include one or more of that feature. In the description of the present invention, "a plurality" means two or more unless specifically defined otherwise.
In the present invention, unless otherwise expressly stated or limited, the terms "mounted," "connected," "secured," and the like are to be construed broadly and can, for example, be fixedly connected, detachably connected, or integrally formed; can be mechanically or electrically connected; either directly or indirectly through intervening media, either internally or in any other relationship. The specific meanings of the above terms in the present invention can be understood by those skilled in the art according to specific situations.
In the present invention, unless otherwise expressly stated or limited, the first feature "on" or "under" the second feature may be directly contacting the first and second features or indirectly contacting the first and second features through an intermediate. Also, a first feature "on," "over," and "above" a second feature may be directly or diagonally above the second feature, or may simply indicate that the first feature is at a higher level than the second feature. A first feature being "under," "below," and "beneath" a second feature may be directly under or obliquely under the first feature, or may simply mean that the first feature is at a lesser elevation than the second feature.
In the description herein, references to the description of the term "one embodiment," "some embodiments," "an example," "a specific example," or "some examples," etc., mean that a particular feature, structure, material, or characteristic described in connection with the embodiment or example is included in at least one embodiment or example of the invention. In this specification, the schematic representations of the terms used above should not be understood to necessarily refer to the same embodiment or example. Furthermore, the particular features, structures, materials, or characteristics described may be combined in any suitable manner in any one or more embodiments or examples. Furthermore, various embodiments or examples and features of different embodiments or examples described in this specification can be combined and combined by one skilled in the art without contradiction.
Although embodiments of the present invention have been shown and described above, it is understood that the above embodiments are exemplary and should not be construed as limiting the present invention, and that variations, modifications, substitutions and alterations can be made to the above embodiments by those of ordinary skill in the art within the scope of the present invention.
Claims (8)
1. An automatic cutout model establishing method is characterized by comprising the following steps:
acquiring a historical picture, and generating a training data set according to the historical picture;
training a segmentation model according to the training data set so as to generate a corresponding triple picture according to an original picture in the training data set through the segmentation model obtained through training;
and training a matting model according to the original pictures in the training data set and the triple pictures corresponding to the original pictures to generate the matting model.
2. The method of automatic matting model building according to claim 1, characterized in that the step of training a segmentation model from the training data set comprises:
setting the initial learning rate to 0.001, and decreasing the learning rate according to a polynomial; the training period is X, and the loss function adopts cross entropy; the network weights continuously update the gradient information through back propagation to complete the training of the segmentation model.
3. The method for automatic matting model building according to claim 2 wherein the step of training a segmentation model from the training data set further comprises:
in each iteration process, sequencing each pixel according to a loss function, and if the sequencing is advanced, considering that the sample error is large and the weight needs to be increased for key learning; furthermore, an error interval is set, so that only the samples in the error interval are calculated by the loss function, and the training efficiency of the segmentation model is improved.
4. The method for automatic matting model building according to claim 2 wherein the step of training a segmentation model from the training data set further comprises:
the training data set is augmented according to the following formula: i ═ alpha × Fg + (1-alpha) × Bg, where Fg is the original RGB picture of the training dataset, alpha is the corresponding mask, Bg is the background dataset candidate picture, I is the synthesized new picture.
5. The method of automatic matting model establishment according to any one of claims 1-3 characterised in that the step of generating a corresponding triplet of pictures from original pictures in a training dataset by a trained segmentation model comprises:
generating a multi-scale feature corresponding to the original picture according to the original picture, and fusing the multi-scale feature to generate a feature layer corresponding to the original picture;
and performing fine-grained segmentation according to the original picture and the feature layer corresponding to the original picture to generate a triple picture corresponding to the original picture.
6. A computer-readable storage medium, having stored thereon an automatic matting program that when executed by a processor implements an automatic matting model establishing method according to any one of claims 1 to 5.
7. A computer device comprising a memory, a processor, and a computer program stored on the memory and executable on the processor, wherein the processor implements the automatic matting model establishing method according to any one of claims 1 to 5 when executing the program.
8. An automatic cutout model building system, comprising:
the acquisition module is used for acquiring a historical picture and generating a training data set according to the historical picture;
the first training module is used for training a segmentation model according to the training data set so as to generate a corresponding triple picture according to an original picture in the training data set through the segmentation model obtained through training;
and the second training module is used for training a cutout model according to the original pictures in the training data set and the triple pictures corresponding to the original pictures so as to generate the cutout model.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202110962940.9A CN113706372B (en) | 2020-06-30 | 2020-06-30 | Automatic matting model building method and system |
Applications Claiming Priority (2)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202010611175.1A CN111784564B (en) | 2020-06-30 | 2020-06-30 | Automatic image matting method and system |
| CN202110962940.9A CN113706372B (en) | 2020-06-30 | 2020-06-30 | Automatic matting model building method and system |
Related Parent Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202010611175.1A Division CN111784564B (en) | 2020-06-30 | 2020-06-30 | Automatic image matting method and system |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN113706372A true CN113706372A (en) | 2021-11-26 |
| CN113706372B CN113706372B (en) | 2024-07-05 |
Family
ID=72760421
Family Applications (2)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202010611175.1A Active CN111784564B (en) | 2020-06-30 | 2020-06-30 | Automatic image matting method and system |
| CN202110962940.9A Active CN113706372B (en) | 2020-06-30 | 2020-06-30 | Automatic matting model building method and system |
Family Applications Before (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202010611175.1A Active CN111784564B (en) | 2020-06-30 | 2020-06-30 | Automatic image matting method and system |
Country Status (2)
| Country | Link |
|---|---|
| CN (2) | CN111784564B (en) |
| WO (1) | WO2022001464A1 (en) |
Cited By (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN114359324A (en) * | 2021-12-27 | 2022-04-15 | 北京艺源酷科技有限公司 | Automatic cutout method based on U2-Net |
Families Citing this family (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN111784564B (en) * | 2020-06-30 | 2022-05-10 | 稿定（厦门）科技有限公司 | Automatic image matting method and system |
| CN113012169B (en) * | 2021-03-22 | 2023-07-07 | 深圳市人工智能与机器人研究院 | Full-automatic image matting method based on non-local attention mechanism |
| CN113259605A (en) * | 2021-04-22 | 2021-08-13 | 清华珠三角研究院 | Video matting method, system and storage medium based on prediction foreground mask prediction |
| CN114529574B (en) * | 2022-02-23 | 2024-07-12 | 平安科技（深圳）有限公司 | Image matting method and device based on image segmentation, computer equipment and medium |
Citations (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN109255334A (en) * | 2018-09-27 | 2019-01-22 | 中国电子科技集团公司第五十四研究所 | Remote sensing image terrain classification method based on deep learning semantic segmentation network |
| CN109461167A (en) * | 2018-11-02 | 2019-03-12 | Oppo广东移动通信有限公司 | The training method of image processing model scratches drawing method, device, medium and terminal |
| CN109712145A (en) * | 2018-11-28 | 2019-05-03 | 山东师范大学 | A kind of image matting method and system |
| CN110322468A (en) * | 2019-06-04 | 2019-10-11 | 广东工业大学 | A kind of automatic edit methods of image |
| CN110543841A (en) * | 2019-08-21 | 2019-12-06 | 中科视语(北京)科技有限公司 | Pedestrian re-identification method, system, electronic device and medium |
| CN111046732A (en) * | 2019-11-11 | 2020-04-21 | 华中师范大学 | Pedestrian re-identification method based on multi-granularity semantic analysis and storage medium |
Family Cites Families (20)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US8792718B2 (en) * | 2012-06-29 | 2014-07-29 | Adobe Systems Incorporated | Temporal matte filter for video matting |
| EP2869265A1 (en) * | 2013-10-30 | 2015-05-06 | Thomson Licensing | Method and apparatus for alpha matting |
| WO2017177371A1 (en) * | 2016-04-12 | 2017-10-19 | Xiaogang Wang | Method and system for object re-identification |
| CN108460770B (en) * | 2016-12-13 | 2020-03-10 | 华为技术有限公司 | Matting method and device |
| CN108062756B (en) * | 2018-01-29 | 2020-04-14 | 重庆理工大学 | Image semantic segmentation method based on deep full convolution network and conditional random field |
| CN110148102B (en) * | 2018-02-12 | 2022-07-15 | 腾讯科技（深圳）有限公司 | Image synthesis method, advertisement material synthesis method and device |
| CN108960499B (en) * | 2018-06-27 | 2021-11-23 | 东华大学 | Garment fashion trend prediction system integrating visual and non-visual features |
| CN109035253A (en) * | 2018-07-04 | 2018-12-18 | 长沙全度影像科技有限公司 | A kind of stingy drawing method of the deep learning automated graphics of semantic segmentation information guiding |
| US10692221B2 (en) * | 2018-07-13 | 2020-06-23 | Adobe Inc. | Automatic trimap generation and image segmentation |
| CN108961303B (en) * | 2018-07-23 | 2021-05-07 | 北京旷视科技有限公司 | Image processing method and device, electronic equipment and computer readable medium |
| CN109145815B (en) * | 2018-08-21 | 2022-05-03 | 深圳大学 | SAR target recognition method and device, computer equipment and storage medium |
| CN109145922B (en) * | 2018-09-10 | 2022-03-29 | 成都品果科技有限公司 | Automatic cutout system |
| CN111160380B (en) * | 2018-11-07 | 2024-10-15 | 华为技术有限公司 | Method for generating video analysis model and video analysis system |
| CN110245665B (en) * | 2019-05-13 | 2023-06-06 | 天津大学 | Image semantic segmentation method based on attention mechanism |
| CN110400323B (en) * | 2019-07-30 | 2020-11-24 | 上海艾麒信息科技股份有限公司 | Automatic cutout system, method and device |
| CN111223106B (en) * | 2019-10-28 | 2022-08-09 | 稿定（厦门）科技有限公司 | Full-automatic portrait mask matting method and system |
| CN110889855B (en) * | 2019-10-28 | 2022-05-20 | 公安部交通管理科学研究所 | Certificate photo matting method and system based on end-to-end convolution neural network |
| CN111161277B (en) * | 2019-12-12 | 2023-04-18 | 中山大学 | Natural image matting method based on deep learning |
| CN111275732B (en) * | 2020-01-16 | 2023-05-02 | 北京师范大学珠海分校 | Foreground object image segmentation method based on depth convolution neural network |
| CN111784564B (en) * | 2020-06-30 | 2022-05-10 | 稿定（厦门）科技有限公司 | Automatic image matting method and system |
-
2020
- 2020-06-30 CN CN202010611175.1A patent/CN111784564B/en active Active
- 2020-06-30 CN CN202110962940.9A patent/CN113706372B/en active Active
-
2021
- 2021-05-20 WO PCT/CN2021/094860 patent/WO2022001464A1/en active Application Filing
Patent Citations (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN109255334A (en) * | 2018-09-27 | 2019-01-22 | 中国电子科技集团公司第五十四研究所 | Remote sensing image terrain classification method based on deep learning semantic segmentation network |
| CN109461167A (en) * | 2018-11-02 | 2019-03-12 | Oppo广东移动通信有限公司 | The training method of image processing model scratches drawing method, device, medium and terminal |
| CN109712145A (en) * | 2018-11-28 | 2019-05-03 | 山东师范大学 | A kind of image matting method and system |
| CN110322468A (en) * | 2019-06-04 | 2019-10-11 | 广东工业大学 | A kind of automatic edit methods of image |
| CN110543841A (en) * | 2019-08-21 | 2019-12-06 | 中科视语(北京)科技有限公司 | Pedestrian re-identification method, system, electronic device and medium |
| CN111046732A (en) * | 2019-11-11 | 2020-04-21 | 华中师范大学 | Pedestrian re-identification method based on multi-granularity semantic analysis and storage medium |
Cited By (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN114359324A (en) * | 2021-12-27 | 2022-04-15 | 北京艺源酷科技有限公司 | Automatic cutout method based on U2-Net |
Also Published As
| Publication number | Publication date |
|---|---|
| CN113706372B (en) | 2024-07-05 |
| CN111784564A (en) | 2020-10-16 |
| CN111784564B (en) | 2022-05-10 |
| WO2022001464A1 (en) | 2022-01-06 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN111784564B (en) | Automatic image matting method and system | |
| Lee et al. | Copy-and-paste networks for deep video inpainting | |
| CN110400323B (en) | Automatic cutout system, method and device | |
| US11017586B2 (en) | 3D motion effect from a 2D image | |
| US10354392B2 (en) | Image guided video semantic object segmentation method and apparatus | |
| CN103262119B (en) | For the method and system that image is split | |
| CN111369581A (en) | Image processing method, device, equipment and storage medium | |
| CN112001407A (en) | Model iterative training method and system based on automatic labeling | |
| CN107506792B (en) | Semi-supervised salient object detection method | |
| CN110866938B (en) | Full-automatic video moving object segmentation method | |
| CN110570435A (en) | method and device for carrying out damage segmentation on vehicle damage image | |
| CN110599453A (en) | Panel defect detection method and device based on image fusion and equipment terminal | |
| CN110827312A (en) | Learning method based on cooperative visual attention neural network | |
| CN112084859A (en) | Building segmentation method based on dense boundary block and attention mechanism | |
| CN113111716A (en) | Remote sensing image semi-automatic labeling method and device based on deep learning | |
| CN116363374B (en) | Image semantic segmentation network continuous learning method, system, equipment and storage medium | |
| CN113706562B (en) | Image segmentation method, device and system and cell segmentation method | |
| CN113870315A (en) | Training method of action migration model and action migration method | |
| CN115424164A (en) | Method and system for constructing scene self-adaptive video data set | |
| CN113450363B (en) | Meta-learning cell nucleus segmentation system and method based on label correction | |
| CN114298179A (en) | Data processing method, device and equipment | |
| CN113222903A (en) | Full-section histopathology image analysis method and system | |
| CN112614149A (en) | Semantic synthesis method based on instance segmentation | |
| CN108810319A (en) | Image processing apparatus and image processing method | |
| CN115578260B (en) | Attention method and system for directional decoupling of image super-resolution |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |