CN109636711A - Comic book generation method, device and computer readable storage medium - Google Patents

Comic book generation method, device and computer readable storage medium Download PDF

Info

Publication number
CN109636711A
CN109636711A CN201811279094.5A CN201811279094A CN109636711A CN 109636711 A CN109636711 A CN 109636711A CN 201811279094 A CN201811279094 A CN 201811279094A CN 109636711 A CN109636711 A CN 109636711A
Authority
CN
China
Prior art keywords
images
several
image
cartoon
video frame
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Granted
Application number
CN201811279094.5A
Other languages
Chinese (zh)
Other versions
CN109636711B (en
Inventor
纪纲
余雪亭
徐毅刚
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Beijing Qihoo Technology Co Ltd
Original Assignee
Beijing Qihoo Technology Co Ltd
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Beijing Qihoo Technology Co Ltd filed Critical Beijing Qihoo Technology Co Ltd
Priority to CN201811279094.5A priority Critical patent/CN109636711B/en
Publication of CN109636711A publication Critical patent/CN109636711A/en
Application granted granted Critical
Publication of CN109636711B publication Critical patent/CN109636711B/en
Active legal-status Critical Current
Anticipated expiration legal-status Critical

Links

Classifications

    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06TIMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T3/00Geometric image transformations in the plane of the image
    • G06T3/04Context-preserving transformations, e.g. by using an importance map
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06FELECTRIC DIGITAL DATA PROCESSING
    • G06F18/00Pattern recognition
    • G06F18/20Analysing
    • G06F18/22Matching criteria, e.g. proximity measures
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06FELECTRIC DIGITAL DATA PROCESSING
    • G06F18/00Pattern recognition
    • G06F18/20Analysing
    • G06F18/24Classification techniques
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06TIMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T7/00Image analysis
    • G06T7/0002Inspection of images, e.g. flaw detection
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06TIMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00Indexing scheme for image analysis or image enhancement
    • G06T2207/10Image acquisition modality
    • G06T2207/10016Video; Image sequence
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06TIMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00Indexing scheme for image analysis or image enhancement
    • G06T2207/20Special algorithmic details
    • G06T2207/20081Training; Learning
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06TIMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00Indexing scheme for image analysis or image enhancement
    • G06T2207/20Special algorithmic details
    • G06T2207/20212Image combination
    • G06T2207/20221Image fusion; Image merging
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06TIMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00Indexing scheme for image analysis or image enhancement
    • G06T2207/30Subject of image; Context of image processing
    • G06T2207/30168Image quality inspection

Landscapes

  • Engineering & Computer Science (AREA)
  • Theoretical Computer Science (AREA)
  • Data Mining & Analysis (AREA)
  • General Physics & Mathematics (AREA)
  • Physics & Mathematics (AREA)
  • Computer Vision & Pattern Recognition (AREA)
  • Bioinformatics & Cheminformatics (AREA)
  • Evolutionary Biology (AREA)
  • Evolutionary Computation (AREA)
  • Bioinformatics & Computational Biology (AREA)
  • General Engineering & Computer Science (AREA)
  • Artificial Intelligence (AREA)
  • Life Sciences & Earth Sciences (AREA)
  • Quality & Reliability (AREA)
  • Image Processing (AREA)
  • Image Analysis (AREA)

Abstract

The present invention discloses a kind of comic book generation method, device and computer readable storage medium and determines that video to be converted, the video to be converted are made of several video frame images in the comic book generation method;Determine that image definition meets several target images of preset condition in several described video frame images;Several described target images are subjected to cartoon style conversion, generate several cartoon images;Several described cartoon images are synthesized into comic book according to preset format.In above scheme, Video Quality Metric to be converted can also be made comic book according to the idea of oneself even if user does not have foundation of painting for comic book, so that caricature creates simpler easy realization by selecting video to be converted by user.

Description

Comic book generation method, device and computer readable storage medium
Technical field
The present invention relates to field of computer technology more particularly to a kind of comic book generation methods, device and computer-readable Storage medium.
Background technique
With the development of science and technology, the popularity rate of electronic equipment is higher and higher, and the reading form of user is also changed Become, more user's selections are read using electronic equipment.In order to meet the reading requirement of user, many caricatures are used It is issued on line, user can browse caricature by electronic equipment, considerably increase the convenience of reading.
In the prior art, caricature is usually the pre-rendered good caricature content of caricature author on line, then unrestrained by what is drawn Draw content uploading publication, it is therefore desirable to which caricature author there must be certain foundation of painting, for the common use of not foundation of painting For family, caricature creation cannot achieve.
Summary of the invention
In view of the above problems, it proposes on the present invention overcomes the above problem or at least be partially solved in order to provide one kind State comic book generation method, device and the computer readable storage medium of problem.
In a first aspect, this specification embodiment provides a kind of comic book generation method, comprising:
Determine that video to be converted, the video to be converted are made of several video frame images;
Determine that image definition meets several target images of preset condition in several described video frame images;
Several described target images are subjected to cartoon style conversion, generate several cartoon images;
Several described cartoon images are synthesized into comic book according to preset format.
Optionally, after determination video to be converted, the method also includes:
According to trained image classification model, the image of every width video frame images in several described video frame images is determined Classification;
Determine that described image classification belongs to several video frame images of pre-set image category set, as image to be converted;
Described several target images determined image definition in several described video frame images and meet preset condition, Include:
The target image is determined in several described images to be converted.
Optionally, described several mesh determined image definition in several described video frame images and meet preset condition Logo image, comprising:
Every width video frame images in several described video frame images are normalized, normalization brightness system is obtained Number;
Gauss Distribution Fitting is carried out to the normalization luminance factor, obtain the features of every width video frame images to Amount;
According to the feature vector of every width video frame images and trained image quality measure model, institute is determined State the clarity score value of every width video frame images;
The video frame images that the clarity score value meets the preset condition are determined as target image.
Optionally, several described target images are subjected to cartoon style conversion described, before generating several cartoon images, The method also includes:
According to the picture material of width target image every in several described target images, several described target images are gone It handles again, obtains image after several duplicate removals;
It is described that several described target images are subjected to cartoon style conversion, generate several cartoon images, comprising:
Image after several described duplicate removals is subjected to cartoon style conversion, generates several described cartoon images.
Optionally, in described several target images according to every width target image picture material, to several described mesh Logo image carries out duplicate removal processing, obtains image after several duplicate removals, comprising:
According to the picture material of width target image every in several described target images, determine between any two width target image Image similarity;
According to described image similarity, multiple groups similar image set is determined, wherein be directed to the multiple groups similar image set Every group of similar image set in include several similar images, the described image similarity between any two width similar image is equal Greater than threshold value;
The image definition of several similar images described according to included in every group of similar image set, to described Several described similar images in every group of similar image set carry out duplicate removal processing, obtain image after several described duplicate removals.
Optionally, several described target images are subjected to cartoon style conversion described, after generating several cartoon images, The method also includes:
According to the picture material of several cartoon images, determination and every width cartoon image pair in several described cartoon images That answers matches literary information;
The predeterminated position in every width cartoon image is added to literary information by described.
Optionally, described that several described target images are subjected to cartoon style conversion, generate several cartoon images, comprising:
Cartoon style conversion is carried out to several described target images using confrontation network, generates several described cartoon images.
Optionally, the preset format includes default caricature lattice distribution, and described several cartoon images by described in are according to default Format synthesizes comic book, comprising:
Determine the temporal information of video frame images corresponding with every width cartoon image of several cartoon images;
By several described cartoon images according to the temporal information sequencing according to the default caricature lattice distribution into Row sequence, generates the comic book.
Second aspect, this specification embodiment provide a kind of comic book generating means, comprising:
Video determining module, for determining that video to be converted, the video to be converted are made of several video frame images;
Target image determining module, for determining that image definition meets default item in several described video frame images Several target images of part;
Image conversion module generates several cartoon images for several described target images to be carried out cartoon style conversion;
Synthesis module, for several described cartoon images to be synthesized comic book according to preset format.
Optionally, described device further include:
Categorization module, for according to trained image classification model, determining every width view in several described video frame images The image category of frequency frame image;
First determining module, for determining that described image classification belongs to several video frame figures of pre-set image category set Picture, as image to be converted;
The target image determining module is used for:
The target image is determined in several described images to be converted.
Optionally, the target image determining module is used for:
Every width video frame images in several described video frame images are normalized, normalization brightness system is obtained Number;
Gauss Distribution Fitting is carried out to the normalization luminance factor, obtain the features of every width video frame images to Amount;
According to the feature vector of every width video frame images and trained image quality measure model, institute is determined State the clarity score value of every width video frame images;
The video frame images that the clarity score value meets the preset condition are determined as target image.
Optionally, described device further include:
Deduplication module, for the picture material according to every width target image in several described target images, to it is described several Target image carries out duplicate removal processing, obtains image after several duplicate removals;
Described image conversion module is used for:
Image after several described duplicate removals is subjected to cartoon style conversion, generates several described cartoon images.
Optionally, the deduplication module is used for:
According to the clarity of width target image every in several described target images, determine between any two width target image Image similarity;
According to described image similarity, multiple groups similar image set is determined, wherein be directed to the multiple groups similar image set Every group of similar image set in include several similar images, the described image similarity between any two width similar image is equal Greater than threshold value;
The image definition of several similar images described according to included in every group of similar image set, to described Several described similar images in every group of similar image set carry out duplicate removal processing, obtain image after several described duplicate removals.
Optionally, described device further include:
Second determining module, for the picture material according to several cartoon images, determining and several described caricature figures Every width cartoon image is corresponding with literary information as in;
Text adding module, for being added to the predeterminated position in every width cartoon image with literary information for described.
Optionally, described image conversion module is used for:
Processing module, for carrying out cartoon style conversion to several described target images using confrontation network, described in generation Several cartoon images.
Optionally, the preset format includes default caricature lattice distribution, and the synthesis module is used for:
Determine the temporal information of video frame images corresponding with every width cartoon image of several cartoon images;
By several described cartoon images according to the temporal information sequencing according to the default caricature lattice distribution into Row sequence, generates the comic book.
The third aspect, this specification embodiment provide a kind of comic book generating means, including memory, processor and storage On a memory and the computer program that can run on a processor, the processor execute the step of any of the above-described the method Suddenly.
Fourth aspect, this specification embodiment provide a kind of computer readable storage medium, are stored thereon with computer journey Sequence, when which is executed by processor the step of realization any of the above-described the method.
This specification embodiment has the beneficial effect that:
In the scheme of this specification embodiment, comic book can be obtained by Video Quality Metric, in this specification embodiment In the comic book generation method of offer, the material for generating comic book, the view to be converted are determined by determination video to be converted Frequency is made of several video frame images;Determine that image definition meets the more of preset condition in several described video frame images Width target image;Several described target images are subjected to cartoon style conversion, generate several cartoon images;It will several described caricatures Image synthesizes comic book according to preset format.Therefore, in above scheme, user can be by selecting video to be converted, will be wait turn Comic book can also be made according to the idea of oneself even if user does not have foundation of painting for comic book by changing Video Quality Metric, so that Caricature creates simpler easy realization.
Detailed description of the invention
By reading the following detailed description of the preferred embodiment, various other advantages and benefits are common for this field Technical staff will become clear.The drawings are only for the purpose of illustrating a preferred embodiment, and is not considered as to the present invention Limitation.And throughout the drawings, the same reference numbers will be used to refer to the same parts.In the accompanying drawings:
Fig. 1 is a kind of flow chart for comic book generation method that this specification embodiment first aspect provides;
Fig. 2 is the flow chart that target image is determined by image definition that this specification embodiment provides;
Fig. 3 is the schematic diagram for the comic book generating means that this specification embodiment second aspect provides;
Fig. 4 is the schematic diagram for the comic book generating means that this specification embodiment third aspect provides.
Specific embodiment
This specification embodiment discloses a kind of comic book generation method, device and computer readable storage medium, can Comic book is generated by video, comic book can also be made even if the user of not foundation of painting, so that caricature creation is more It is simple easily to realize.Comic book generation method comprises determining that video to be converted, and the video to be converted is by several video frame images groups At;Determine that image definition meets several target images of preset condition in several described video frame images;It will be described more Width target image carries out cartoon style conversion, generates several cartoon images;Several described cartoon images are closed according to preset format At comic book.
Technical solution of the present invention is described in detail below by attached drawing and specific embodiment, it should be understood that the application Specific features in embodiment and embodiment are the detailed description to technical scheme, rather than to present techniques The restriction of scheme, in the absence of conflict, the technical characteristic in the embodiment of the present application and embodiment can be combined with each other.
The terms "and/or", only a kind of incidence relation for describing affiliated partner, indicates that there may be three kinds of passes System, for example, A and/or B, can indicate: individualism A exists simultaneously A and B, these three situations of individualism B.In addition, herein Middle character "/" typicallys represent the relationship that forward-backward correlation object is a kind of "or".
Embodiment
In a first aspect, this specification embodiment provides a kind of comic book generation method, as shown in Figure 1, real for this specification A kind of flow chart of comic book generation method of example offer is provided, method includes the following steps:
Step S11: determine that video to be converted, the video to be converted are made of several video frame images;
In this specification embodiment, video to be converted is the video for generating comic book, and video to be converted can be use Family passes through the video that electronic equipment is voluntarily shot, and is also possible to the video that user is obtained by downloading, is also possible to other videos. Video to be converted is made of several video frame images, and each width video frame images are a static image.
S12: determine that image definition meets several target images of preset condition in several described video frame images;
It should be understood that the quality for several video frame images that video to be converted is included is different, some video frames Image is lower, and image effect is relatively fuzzy, can be low by picture quality in order to guarantee the display effect of the comic book ultimately produced Video frame images are filtered, and determine that image definition meets the video frame images of preset condition.
In this specification embodiment, image definition can be obtained by a variety of calculations, for example, by using gradient function, Variance function, quotient function etc..Calculate the image definition for every width video frame images that video to be converted is included, and by every width figure Image sharpness is compared with preset condition, will meet the video frame images of preset condition as target image.Preset condition can To be selected according to the actual situation, in one embodiment, preset condition is more than or equal to clarity threshold.
Step S13: several described target images are subjected to cartoon style conversion, generate several cartoon images;
In this specification embodiment, after target image has been determined, preprocessing process can be carried out to target image, it will Pretreated target image carries out cartoon style conversion, generates cartoon image.The mode of cartoon style conversion can be according to reality Border is selected, and in one embodiment, can carry out style conversion by way of fighting network.
Step S14: several described cartoon images are synthesized into comic book according to preset format.
It should be understood that comic book may include multiple format, such as when comic book is four lattice caricature, four lattice caricatures Format is comprising there are four picture lattices, and when comic book overflows for item, the unrestrained format of item be the long figure of a width vertically read, long to scheme In may include multiple picture lattices, when comic book overflows for page, the unrestrained format of page be include multipage, may include in every page There are multiple picture lattices.According to the format of comic book, several cartoon images are filled in picture lattice, synthesize comic book.
Optionally, after determination video to be converted, the method also includes: according to trained image classification mould Type determines the image category of every width video frame images in several described video frame images;It is default to determine that described image classification belongs to Several video frame images of image category set, as image to be converted;It is described to be determined in several described video frame images Image definition meets several target images of preset condition, comprising: the target is determined in several described images to be converted Image.
It should be understood that the picture material for several video frame images that video to be converted includes can cover many aspects, For example, video to be converted is the video that user plays basketball in sports ground, in the video, the content of some video frame images Content for blue sky, some video frame images be ground, some video frame images content be sports ground ambient enviroment, some view Frequency frame image is that user plays basketball etc..What these picture materials had can be used to generate comic book, and some generates comic book Effect it is little, for example, in the above example, according to the video that user plays basketball in sports ground, picture material is ground Therefore video frame images do not need to appear in comic book, the video frame images that picture material is ground can be removed.
In this specification embodiment, the available video frame images of comic book are generated in order to filter out, can first be regarded several Frequency frame image carries out image classification, will generate comic book and be not necessarily to the image category used exclusion.Image classification can be according to preparatory Trained image classification model is handled, such as CNN disaggregated model, using several video frame images as the defeated of model Enter, image classification model can export the classification results of every width video frame images, i.e., the image category of every width video frame images.
Based on the image category of every width video frame images, the video that image category belongs to pre-set image category set is selected Frame image, pre-set image category set can be user oneself setting, be also possible to the photographed subject according to video to be converted Come what is determined, the more image category of the video frame images for including can also be, for example, counting under each image category and including Quantity is greater than the image category of a threshold value as pre-set image classification by the quantity of video frame images.Pre-set image class will be met The video frame images that do not gather are screened as image to be converted, are further determined by clarity for being overflow The target image of painting style lattice conversion.
As shown in Fig. 2, for the flow chart for determining target image by image definition that this specification embodiment provides, packet Include following steps.
S21: being normalized every width video frame images in several described video frame images, and it is bright to obtain normalization Spend coefficient;
S22: Gauss Distribution Fitting is carried out to the normalization luminance factor, obtains the feature of every width video frame images Vector;
S23: according to the feature vector of every width video frame images and trained image quality measure model, really The clarity score value of fixed every width video frame images;
S24: the video frame images that the clarity score value meets the preset condition are determined as target image.
In this specification embodiment, brisque (Blind/Referenceless Image Spatial can be used Quality Evaluator, the spatial domain image quality measure algorithm of no reference) determine the clarity of video frame images.
Firstly, doing normalized to every width video frame images in several video frame images, every width video frame figure is obtained The normalization luminance factor (mean subtracted contrast normalized coefficients, MSCN) of picture.Its In, for each pixel of video frame images, a normalization luminance factor, the normalization of video frame images can be calculated Luminance factor can be the set of the normalization luminance factor of each pixel.Pass through the normalization luminance factor phase of video frame images The clarity of video frame images is determined for the departure degree of the normalization luminance factor of natural image, wherein need to illustrate , natural image is the image of untreated mistake, and the normalization brightness system of natural image meets Gaussian Profile.
Secondly, determining the feature of the normalization luminance factor of video frame images by Gaussian Profile.In one embodiment In, for every width video frame images, the related information between the unconnected pixels on four direction joined for each pixel, i.e., It joined the pixel lower section, right, leading diagonal, the pixel related information on minor diagonal four direction, use is asymmetric Generalized Gaussian distribution is fitted, and has obtained the feature vector of every width video frame images.
Finally, obtained feature vector is input in preparatory trained image quality measure model, every width view is obtained The clarity score value of frequency frame image, in one embodiment, using preparatory trained support vector machines (Support Vector Machine) it is used as image quality measure model, using feature vector as input, obtain the clear of every width video frame images Clear degree score value.
According to the clarity score value of video frame images, the video frame images that the clarity score value is met preset condition are true It is set to target image, wherein preset condition can be set according to actual needs, if preset condition is more than or equal to pre- If clarity score value.
Optionally, several described target images are subjected to cartoon style conversion described, before generating several cartoon images, The method also includes: according to the picture material of width target image every in several described target images, to several described target figures As carrying out duplicate removal processing, image after several duplicate removals is obtained;It is described that several described target images are subjected to cartoon style conversion, it generates Several cartoon images, comprising: image after several described duplicate removals is subjected to cartoon style conversion, generates several described cartoon images.
In this specification embodiment, if the time of one scene capture is longer when shooting video to be converted, then it can go out The content of several existing video frame images is same or similar.For example, if user shooting be blue sky, due to blue sky variation more Slowly, the content of the video frame images in most of shooting blue skies is relatively.But when generating comic book, do not need every width Video frame images are all added to comic book, will cause the repetition of caricature content like that, increase the redundancy of caricature, thus can will in Hold several same or similar video frame images and carry out duplicate removal processing, by image after the conduct duplicate removal for remaining image.
Optionally, in described several target images according to every width target image picture material, to several described mesh Logo image carries out duplicate removal processing, obtains image after several duplicate removals, comprising: according to width target image every in several described target images Picture material, determine the image similarity between any two width target image;According to described image similarity, multiple groups phase is determined Like image collection, wherein for several similar diagrams for including in every group of similar image set of the multiple groups similar image set Picture, the described image similarity between any two width similar image are all larger than threshold value;According in every group of similar image set The image definition for several similar images for being included, to several similar diagrams described in every group of similar image set As carrying out duplicate removal processing, image after several described duplicate removals is obtained.
It should be understood that duplicate removal processing can realize that this specification is without limitation in several ways.Implement at one In example, image similarity can be determined by comparing the content of any two width target image, come further according to image similarity true Whether this fixed two width target image is image similar in content.For example, when the similarity of two width target images is greater than 90%, really This fixed two width video frame images are close image, and similar image is determined as one group of similar image set.
For several similar images for including in every group of similar image set, can be gone according to actual needs using different Double recipe formula.For example, the reservation of one or more image can be randomly selected, the most image of picture material can also be retained, or The maximum image of target object size for including in image retain etc. by person.It in one embodiment, can be at every group It chooses the highest one or more similar image of clarity in similar image set to retain, remaining image is deleted, will be protected The similar image stayed carries out cartoon style conversion as image after duplicate removal.
Optionally, described that several described target images are subjected to cartoon style conversion, generate several cartoon images, comprising: Cartoon style conversion is carried out to several described target images using confrontation network, generates several described cartoon images.
Fighting network (Generative Adversarial Networks, GAN) is a kind of deep learning model, including Generator and discriminator.In this specification embodiment, generator is used to target image carrying out cartoon style conversion, generates caricature Whether image, discriminator will true cartoon image be much true for identifying the cartoon image of generator generation as reference Real cartoon image.In one embodiment, cartoon style conversion is carried out using CycleGAN, i.e., using target image as The input of CycleGAN, the output of CycleGAN are the cartoon image converted.
Optionally, several described target images are subjected to cartoon style conversion described, after generating several cartoon images, The method also includes: it is determining unrestrained with every width in several described cartoon images according to the picture material of several cartoon images It draws as corresponding with literary information;The predeterminated position in every width cartoon image is added to literary information by described.
In this specification embodiment, an image can be set with library, which, which matches, can wrap in library containing image Content and corresponding with literary information, in addition, be directed to every kind of picture material, it is corresponding may include with literary information it is a plurality of.For example, Image is blue sky in library with the picture material for including, it is corresponding with blue sky may include with literary information it is a plurality of, such as with text letter 1 " today is fine " is ceased, with literary information 2 " day height appoints bird to fly, and Hai Kuo is with fish dive ".It can be according in the image of cartoon image Hold, image with determine in library it is corresponding can be determined at random with literary information with literary information, can also will be with cartoon image It is corresponding all to carry out being displayed for user with literary information selecting.Certainly, user can also voluntarily input with literary information, this In without limitation.
In this specification embodiment, every width cartoon image can be set predeterminated position for show with text information.It is default Position can be default, such as predeterminated position is the upper left corner of every width cartoon image, and predeterminated position can also be random setting 's.Certainly, user can be with self-setting predeterminated position, for example, the predeterminated position to default carries out drag operation, so that with text The display position of information is met the needs of users.
Optionally, the preset format includes default caricature lattice distribution, and described several cartoon images by described in are according to default Format synthesizes comic book, comprising: determine corresponding with every width cartoon image of several cartoon images video frame images when Between information;Several described cartoon images are carried out according to the sequencing of the temporal information according to the default caricature lattice distribution Sequence, generates the comic book.
Generally, the event shot in video to be converted develops sequentially in time, in order to guarantee comic book Caricature content meets the Development Logic of event, the temporal information of several available cartoon images, and several cartoon images are pressed It is filled in default caricature lattice according to the sequencing of time.Comic book can wrap containing multiple default caricature lattice, preset unrestrained The distribution of frame, which can be, to be pre-set, for example, the unrestrained caricature lattice of item be it is vertically disposed, preset the size of caricature lattice It can be and set, or be automatically adjusted according to the size of content object in cartoon image, this specification Embodiment is without limitation.
Second aspect, based on the same inventive concept, this specification embodiment provide a kind of comic book generating means, please refer to Fig. 3, comprising:
Video determining module 31, for determining that video to be converted, the video to be converted are made of several video frame images;
Target image determining module 32, for determining that it is default that image definition meets in several described video frame images Several target images of condition;
Image conversion module 33 generates several caricature figures for several described target images to be carried out cartoon style conversion Picture;
Synthesis module 34, for several described cartoon images to be synthesized comic book according to preset format.
In a kind of optional implementation, described device further include:
Categorization module, for according to trained image classification model, determining every width view in several described video frame images The image category of frequency frame image;
First determining module, for determining that described image classification belongs to several video frame figures of pre-set image category set Picture, as image to be converted;
Target image determining module 32 is used for:
The target image is determined in several described images to be converted.
In a kind of optional implementation, target image determining module 32 is used for:
Every width video frame images in several described video frame images are normalized, normalization brightness system is obtained Number;
Gauss Distribution Fitting is carried out to the normalization luminance factor, obtain the features of every width video frame images to Amount;
According to the feature vector of every width video frame images and trained image quality measure model, institute is determined State the clarity score value of every width video frame images;
The video frame images that the clarity score value meets the preset condition are determined as target image.
In a kind of optional implementation, described device further include:
Deduplication module, for the picture material according to every width target image in several described target images, to it is described several Target image carries out duplicate removal processing, obtains image after several duplicate removals;
Image conversion synthesis module 33 is used for:
Image after several described duplicate removals is subjected to cartoon style conversion, generates several described cartoon images.
In a kind of optional implementation, the deduplication module is used for:
According to the clarity of width target image every in several described target images, determine between any two width target image Image similarity;
According to described image similarity, multiple groups similar image set is determined, wherein be directed to the multiple groups similar image set Every group of similar image set in include several similar images, the described image similarity between any two width similar image is equal Greater than threshold value;
The image definition of several similar images described according to included in every group of similar image set, to described Several described similar images in every group of similar image set carry out duplicate removal processing, obtain image after several described duplicate removals.
In a kind of optional implementation, described device further include:
Second determining module, for the picture material according to several cartoon images, determining and several described caricature figures Every width cartoon image is corresponding with literary information as in;
Text adding module, for being added to the predeterminated position in every width cartoon image with literary information for described.
In a kind of optional implementation, image conversion module 33 is used for:
Processing module, for carrying out cartoon style conversion to several described target images using confrontation network, described in generation Several cartoon images.
In a kind of optional implementation, the preset format includes default caricature lattice distribution, and synthesis module 34 is used for:
Determine the temporal information of video frame images corresponding with every width cartoon image of several cartoon images;
By several described cartoon images according to the temporal information sequencing according to the default caricature lattice distribution into Row sequence, generates the comic book.
About above-mentioned apparatus, wherein the concrete function of modules is generated in comic book provided in an embodiment of the present invention It is described in detail in the embodiment of method, no detailed explanation will be given here.
The third aspect is based on inventive concept same as comic book generation method in previous embodiment, the present invention also provides A kind of comic book generating means, as shown in figure 4, including memory 504, processor 502 and being stored on memory 504 and can be The computer program run on processor 502, the processor 502 realize that comic book described previously generates when executing described program The step of either method method.
Wherein, in Fig. 4, bus architecture (is represented) with bus 500, and bus 500 may include any number of interconnection Bus and bridge, bus 500 will include the one or more processors represented by processor 502 and what memory 504 represented deposits The various circuits of reservoir link together.Bus 500 can also will peripheral equipment, voltage-stablizer and management circuit etc. it Various other circuits of class link together, and these are all it is known in the art, therefore, no longer carry out further to it herein Description.Bus interface 506 provides interface between bus 500 and receiver 501 and transmitter 503.Receiver 501 and transmitter 503 can be the same element, i.e. transceiver, provide the unit for communicating over a transmission medium with various other devices.Place It manages device 502 and is responsible for management bus 500 and common processing, and memory 504 can be used for storage processor 502 and execute behaviour Used data when making.
Fourth aspect, based on the inventive concept based on comic book generation method in previous embodiment, the present invention also provides A kind of computer readable storage medium, is stored thereon with computer program, which realizes described previously when being executed by processor The step of based on either comic book generation method method.
This specification is referring to the method, equipment (system) and computer program product according to this specification embodiment Flowchart and/or the block diagram describes.It should be understood that can be realized by computer program instructions every in flowchart and/or the block diagram The combination of process and/or box in one process and/or box and flowchart and/or the block diagram.It can provide these computers Processor of the program instruction to general purpose computer, special purpose computer, Embedded Processor or other programmable data processing devices To generate a machine, so that generating use by the instruction that computer or the processor of other programmable data processing devices execute In setting for the function that realization is specified in one or more flows of the flowchart and/or one or more blocks of the block diagram It is standby.
These computer program instructions, which may also be stored in, is able to guide computer or other programmable data processing devices with spy Determine in the computer-readable memory that mode works, so that it includes referring to that instruction stored in the computer readable memory, which generates, Enable the manufacture of equipment, the commander equipment realize in one box of one or more flows of the flowchart and/or block diagram or The function of being specified in multiple boxes.
These computer program instructions also can be loaded onto a computer or other programmable data processing device, so that counting Series of operation steps are executed on calculation machine or other programmable devices to generate computer implemented processing, thus in computer or The instruction executed on other programmable devices is provided for realizing in one or more flows of the flowchart and/or block diagram one The step of function of being specified in a box or multiple boxes.
Although preferred embodiments of the present invention have been described, it is created once a person skilled in the art knows basic Property concept, then additional changes and modifications may be made to these embodiments.So it includes excellent that the following claims are intended to be interpreted as It selects embodiment and falls into all change and modification of the scope of the invention.
Obviously, various changes and modifications can be made to the invention without departing from essence of the invention by those skilled in the art Mind and range.In this way, if these modifications and changes of the present invention belongs to the range of the claims in the present invention and its equivalent technologies Within, then the present invention is also intended to include these modifications and variations.
Invention additionally discloses A1, a kind of comic book generation method, which comprises
Determine that video to be converted, the video to be converted are made of several video frame images;
Determine that image definition meets several target images of preset condition in several described video frame images;
Several described target images are subjected to cartoon style conversion, generate several cartoon images;
Several described cartoon images are synthesized into comic book according to preset format.
A2, comic book generation method according to a1, after determination video to be converted, the method is also wrapped It includes:
According to trained image classification model, the image of every width video frame images in several described video frame images is determined Classification;
Determine that described image classification belongs to several video frame images of pre-set image category set, as image to be converted;
Described several target images determined image definition in several described video frame images and meet preset condition, Include:
The target image is determined in several described images to be converted.
A3, comic book generation method according to a1, it is described to determine that image is clear in several described video frame images It is clear to spend several target images for meeting preset condition, comprising:
Every width video frame images in several described video frame images are normalized, normalization brightness system is obtained Number;
Gauss Distribution Fitting is carried out to the normalization luminance factor, obtain the features of every width video frame images to Amount;
According to the feature vector of every width video frame images and trained image quality measure model, institute is determined State the clarity score value of every width video frame images;
The video frame images that the clarity score value meets the preset condition are determined as target image.
Several described target images are carried out cartoon style turn described by A4, comic book generation method according to a1 It changes, before generating several cartoon images, the method also includes:
According to the picture material of width target image every in several described target images, several described target images are gone It handles again, obtains image after several duplicate removals;
It is described that several described target images are subjected to cartoon style conversion, generate several cartoon images, comprising:
Image after several described duplicate removals is subjected to cartoon style conversion, generates several described cartoon images.
A5, comic book generation method according to a4, every width target image in described several target images according to Picture material, to several described target images carry out duplicate removal processing, obtain several duplicate removal images, comprising:
According to the picture material of width target image every in several described target images, determine between any two width target image Image similarity;
According to described image similarity, multiple groups similar image set is determined, wherein be directed to the multiple groups similar image set Every group of similar image set in include several similar images, the described image similarity between any two width similar image is equal Greater than threshold value;
The image definition of several similar images described according to included in every group of similar image set, to described Several described similar images in every group of similar image set carry out duplicate removal processing, obtain image after several described duplicate removals.
Several described target images are carried out cartoon style turn described by A6, comic book generation method according to a1 It changes, after generating several cartoon images, the method also includes:
According to the picture material of several cartoon images, determination and every width cartoon image pair in several described cartoon images That answers matches literary information;
The predeterminated position in every width cartoon image is added to literary information by described.
A7, comic book generation method according to a1, it is described that several described target images are subjected to cartoon style conversion, Generate several cartoon images, comprising:
Cartoon style conversion is carried out to several described target images using confrontation network, generates several described cartoon images.
A8, comic book generation method according to a1, the preset format includes default caricature lattice distribution, described by institute It states several cartoon images and synthesizes comic book according to preset format, comprising:
Determine the temporal information of video frame images corresponding with every width cartoon image of several cartoon images;
By several described cartoon images according to the temporal information sequencing according to the default caricature lattice distribution into Row sequence, generates the comic book.
B9, a kind of comic book generating means, described device include:
Video determining module, for determining that video to be converted, the video to be converted are made of several video frame images;
Target image determining module, for determining that image definition meets default item in several described video frame images Several target images of part;
Image conversion module generates several cartoon images for several described target images to be carried out cartoon style conversion;
Synthesis module, for several described cartoon images to be synthesized comic book according to preset format.
B10, the comic book generating means according to B9, described device further include:
Categorization module, for according to trained image classification model, determining every width view in several described video frame images The image category of frequency frame image;
First determining module, for determining that described image classification belongs to several video frame figures of pre-set image category set Picture, as image to be converted;
The target image determining module is used for:
The target image is determined in several described images to be converted.
B11, the comic book generating means according to B9, the target image determining module are used for:
Every width video frame images in several described video frame images are normalized, normalization brightness system is obtained Number;
Gauss Distribution Fitting is carried out to the normalization luminance factor, obtain the features of every width video frame images to Amount;
According to the feature vector of every width video frame images and trained image quality measure model, institute is determined State the clarity score value of every width video frame images;
The video frame images that the clarity score value meets the preset condition are determined as target image.
B12, the comic book generating means according to B9, described device further include:
Deduplication module, for the picture material according to every width target image in several described target images, to it is described several Target image carries out duplicate removal processing, obtains image after several duplicate removals;
Described image conversion module is used for:
Image after several described duplicate removals is subjected to cartoon style conversion, generates several described cartoon images.
B13, comic book generating means according to b12, the deduplication module are used for:
According to the clarity of width target image every in several described target images, determine between any two width target image Image similarity;
According to described image similarity, multiple groups similar image set is determined, wherein be directed to the multiple groups similar image set Every group of similar image set in include several similar images, the described image similarity between any two width similar image is equal Greater than threshold value;
The image definition of several similar images described according to included in every group of similar image set, to described Several described similar images in every group of similar image set carry out duplicate removal processing, obtain image after several described duplicate removals.
B14, the comic book generating means according to B9, described device further include:
Second determining module, for the picture material according to several cartoon images, determining and several described caricature figures Every width cartoon image is corresponding with literary information as in;
Text adding module, for being added to the predeterminated position in every width cartoon image with literary information for described.
B15, the comic book generating means according to B9, described image conversion module are used for:
Processing module, for carrying out cartoon style conversion to several described target images using confrontation network, described in generation Several cartoon images.
B16, the comic book generating means according to B9, the preset format include default caricature lattice distribution, the conjunction It is used at module:
Determine the temporal information of video frame images corresponding with every width cartoon image of several cartoon images;
By several described cartoon images according to the temporal information sequencing according to the default caricature lattice distribution into Row sequence, generates the comic book.
C17, a kind of comic book generating means, including memory, processor and storage are on a memory and can be in processor The step of computer program of upper operation, the processor realizes any one of 1-8 the method when executing described program.
D18, a kind of computer readable storage medium, are stored thereon with computer program, when which is executed by processor The step of realizing any one of A1-A8 the method.

Claims (10)

1. a kind of comic book generation method, which is characterized in that the described method includes:
Determine that video to be converted, the video to be converted are made of several video frame images;
Determine that image definition meets several target images of preset condition in several described video frame images;
Several described target images are subjected to cartoon style conversion, generate several cartoon images;
Several described cartoon images are synthesized into comic book according to preset format.
2. comic book generation method according to claim 1, which is characterized in that after determination video to be converted, The method also includes:
According to trained image classification model, the image class of every width video frame images in several described video frame images is determined Not;
Determine that described image classification belongs to several video frame images of pre-set image category set, as image to be converted;
Described several target images determined image definition in several described video frame images and meet preset condition, packet It includes:
The target image is determined in several described images to be converted.
3. comic book generation method according to claim 1, which is characterized in that described in several described video frame images Determine that image definition meets several target images of preset condition, comprising:
Every width video frame images in several described video frame images are normalized, normalization luminance factor is obtained;
Gauss Distribution Fitting is carried out to the normalization luminance factor, obtains the feature vector of every width video frame images;
According to the feature vector of every width video frame images and trained image quality measure model, determine described every The clarity score value of width video frame images;
The video frame images that the clarity score value meets the preset condition are determined as target image.
4. comic book generation method according to claim 1, which is characterized in that it is described will several described target images into The conversion of row cartoon style, before generating several cartoon images, the method also includes:
According to the picture material of width target image every in several described target images, several described target images are carried out at duplicate removal Reason, obtains image after several duplicate removals;
It is described that several described target images are subjected to cartoon style conversion, generate several cartoon images, comprising:
Image after several described duplicate removals is subjected to cartoon style conversion, generates several described cartoon images.
5. comic book generation method according to claim 4, which is characterized in that in described several target images according to The picture material of every width target image carries out duplicate removal processing to several described target images, obtains several duplicate removal images, comprising:
According to the picture material of width target image every in several described target images, the figure between any two width target image is determined As similarity;
According to described image similarity, multiple groups similar image set is determined, wherein for the every of the multiple groups similar image set Several similar images for including in similar image set are organized, the described image similarity between any two width similar image is all larger than Threshold value;
The image definition of several similar images described according to included in every group of similar image set, to described every group Several described similar images in similar image set carry out duplicate removal processing, obtain image after several described duplicate removals.
6. comic book generation method according to claim 1, which is characterized in that it is described will several described target images into The conversion of row cartoon style, after generating several cartoon images, the method also includes:
According to the picture material of several cartoon images, determination is corresponding with every width cartoon image in several described cartoon images With literary information;
The predeterminated position in every width cartoon image is added to literary information by described.
7. comic book generation method according to claim 1, which is characterized in that described to carry out several described target images Cartoon style conversion, generates several cartoon images, comprising:
Cartoon style conversion is carried out to several described target images using confrontation network, generates several described cartoon images.
8. a kind of comic book generating means, which is characterized in that described device includes:
Video determining module, for determining that video to be converted, the video to be converted are made of several video frame images;
Target image determining module, for determining that image definition meets preset condition in several described video frame images Several target images;
Image conversion module generates several cartoon images for several described target images to be carried out cartoon style conversion;
Synthesis module, for several described cartoon images to be synthesized comic book according to preset format.
9. a kind of comic book generating means, which is characterized in that including memory, processor and store on a memory and can locate The computer program run on reason device, the processor realize any one of claim 1-7 the method when executing described program The step of.
10. a kind of computer readable storage medium, which is characterized in that be stored thereon with computer program, the program is by processor The step of any one of claim 1-7 the method is realized when execution.
CN201811279094.5A 2018-10-30 2018-10-30 Cartoon album generating method, cartoon album generating device and computer readable storage medium Active CN109636711B (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN201811279094.5A CN109636711B (en) 2018-10-30 2018-10-30 Cartoon album generating method, cartoon album generating device and computer readable storage medium

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN201811279094.5A CN109636711B (en) 2018-10-30 2018-10-30 Cartoon album generating method, cartoon album generating device and computer readable storage medium

Publications (2)

Publication Number Publication Date
CN109636711A true CN109636711A (en) 2019-04-16
CN109636711B CN109636711B (en) 2024-09-17

Family

ID=66066901

Family Applications (1)

Application Number Title Priority Date Filing Date
CN201811279094.5A Active CN109636711B (en) 2018-10-30 2018-10-30 Cartoon album generating method, cartoon album generating device and computer readable storage medium

Country Status (1)

Country Link
CN (1) CN109636711B (en)

Cited By (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN111429341A (en) * 2020-03-27 2020-07-17 咪咕文化科技有限公司 Video processing method, video processing equipment and computer readable storage medium
CN117252966A (en) * 2023-11-20 2023-12-19 湖南快乐阳光互动娱乐传媒有限公司 Dynamic cartoon generation method and device, storage medium and electronic equipment

Citations (6)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
KR20090007909A (en) * 2007-07-16 2009-01-21 연세대학교 산학협력단 Method and apparatus for creating caricature video
CN102014252A (en) * 2010-12-06 2011-04-13 无敌科技(西安)有限公司 Display system and method for converting image video into pictures with image illustration
US20120257876A1 (en) * 2011-04-07 2012-10-11 Infosys Technologies, Ltd. Method and system for generating at least one of: comic strips and storyboards from videos
CN104244113A (en) * 2014-10-08 2014-12-24 中国科学院自动化研究所 Method for generating video abstract on basis of deep learning technology
US20170242833A1 (en) * 2016-02-20 2017-08-24 ComicFlix, Inc. Systems and Methods to Generate Comic Books or Graphic Novels from Videos
CN108320319A (en) * 2018-02-02 2018-07-24 广东蜂助手网络技术股份有限公司 A kind of caricature synthetic method, device, equipment and computer readable storage medium

Patent Citations (6)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
KR20090007909A (en) * 2007-07-16 2009-01-21 연세대학교 산학협력단 Method and apparatus for creating caricature video
CN102014252A (en) * 2010-12-06 2011-04-13 无敌科技(西安)有限公司 Display system and method for converting image video into pictures with image illustration
US20120257876A1 (en) * 2011-04-07 2012-10-11 Infosys Technologies, Ltd. Method and system for generating at least one of: comic strips and storyboards from videos
CN104244113A (en) * 2014-10-08 2014-12-24 中国科学院自动化研究所 Method for generating video abstract on basis of deep learning technology
US20170242833A1 (en) * 2016-02-20 2017-08-24 ComicFlix, Inc. Systems and Methods to Generate Comic Books or Graphic Novels from Videos
CN108320319A (en) * 2018-02-02 2018-07-24 广东蜂助手网络技术股份有限公司 A kind of caricature synthetic method, device, equipment and computer readable storage medium

Non-Patent Citations (5)

* Cited by examiner, † Cited by third party
Title
ANISH MITTAL等: "No-Reference Image Quality Assessment in the Spatial Domain", 《IEEE TRANSACTIONS ON IMAGE PROCESSING》, vol. 21, no. 12, pages 1 *
MENG WANG等: "Movie2Comics:Towards a Lively Video Content Presentation", 《IEEE TRANSACTIONS ON MULTIMEDIA》, vol. 14, no. 3, pages 4 - 6 *
WEI-TA CHU等: "Optimized Comics-Based Storytelling for Temporal Image Sequences", 《IEEE TRANSACTIONS ON MULTIMEDIA》, vol. 17, no. 2, pages 201 - 125 *
卢倩雯等: "基于生成对抗网络的漫画草稿图简化", 《自动化学报》, vol. 44, no. 05, pages 840 - 854 *
罗新高: "基于SimHash的海量视频检索研究", 《中国优秀硕士论文全文数据库 信息科技辑》, pages 3 *

Cited By (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN111429341A (en) * 2020-03-27 2020-07-17 咪咕文化科技有限公司 Video processing method, video processing equipment and computer readable storage medium
CN111429341B (en) * 2020-03-27 2023-08-18 咪咕文化科技有限公司 Video processing method, device and computer readable storage medium
CN117252966A (en) * 2023-11-20 2023-12-19 湖南快乐阳光互动娱乐传媒有限公司 Dynamic cartoon generation method and device, storage medium and electronic equipment
CN117252966B (en) * 2023-11-20 2024-01-30 湖南快乐阳光互动娱乐传媒有限公司 Dynamic cartoon generation method and device, storage medium and electronic equipment

Also Published As

Publication number Publication date
CN109636711B (en) 2024-09-17

Similar Documents

Publication Publication Date Title
CN109670558B (en) Digital image completion using deep learning
US11334971B2 (en) Digital image completion by learning generation and patch matching jointly
Song et al. Objectstitch: Object compositing with diffusion model
US11417130B2 (en) System and method for facilitating graphic-recognition training of a recognition model
CN103988202B (en) Image attraction based on index and search
CN110097086A (en) Image generates model training method, image generating method, device, equipment and storage medium
EP3779891A1 (en) Method and device for training neural network model, and method and device for generating time-lapse photography video
US11620480B2 (en) Learning method, computer program, classifier, and generator
CN106575450A (en) Augmented reality content rendering via albedo models, systems and methods
CN115735230A (en) View synthesis robust to unconstrained image data
CN110832583A (en) System and method for generating a summary storyboard from a plurality of image frames
CN114339409B (en) Video processing method, device, computer equipment and storage medium
US11393144B2 (en) System and method for rendering an image
JP7247587B2 (en) Image style conversion device, image style conversion method, and program
CN105122272A (en) Automatic curation of digital images
Song et al. Objectstitch: Generative object compositing
CN108604389A (en) continuous depth ordering image synthesis
CN108259949A (en) Method, apparatus and electronic equipment are recommended in a kind of advertisement
WO2024131565A1 (en) Garment image extraction method and apparatus, and device, medium and product
CN109636711A (en) Comic book generation method, device and computer readable storage medium
US8407575B1 (en) Video content summary
Sun et al. Learning adaptive patch generators for mask-robust image inpainting
CN109510943A (en) Method and apparatus for shooting image
CN108833989A (en) A kind of method and apparatus that barrage is generated and shown
CN115937358A (en) Image processing method and device, electronic device and storage medium

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination
GR01 Patent grant
GR01 Patent grant