CN113990356A - Book generation method, book generation equipment and storage medium - Google Patents

Book generation method, book generation equipment and storage medium Download PDF

Info

Publication number
CN113990356A
CN113990356A CN202010670728.0A CN202010670728A CN113990356A CN 113990356 A CN113990356 A CN 113990356A CN 202010670728 A CN202010670728 A CN 202010670728A CN 113990356 A CN113990356 A CN 113990356A
Authority
CN
China
Prior art keywords
information
story
character
role
book
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Granted
Application number
CN202010670728.0A
Other languages
Chinese (zh)
Other versions
CN113990356B (en
Inventor
王鹏
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
TCL Technology Group Co Ltd
Original Assignee
TCL Technology Group Co Ltd
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by TCL Technology Group Co Ltd filed Critical TCL Technology Group Co Ltd
Priority to CN202010670728.0A priority Critical patent/CN113990356B/en
Publication of CN113990356A publication Critical patent/CN113990356A/en
Application granted granted Critical
Publication of CN113990356B publication Critical patent/CN113990356B/en
Active legal-status Critical Current
Anticipated expiration legal-status Critical

Links

Images

Classifications

    • G—PHYSICS
    • G11—INFORMATION STORAGE
    • G11B—INFORMATION STORAGE BASED ON RELATIVE MOVEMENT BETWEEN RECORD CARRIER AND TRANSDUCER
    • G11B27/00—Editing; Indexing; Addressing; Timing or synchronising; Monitoring; Measuring tape travel
    • G11B27/02—Editing, e.g. varying the order of information signals recorded on, or reproduced from, record carriers
    • G11B27/031—Electronic editing of digitised analogue information signals, e.g. audio or video signals
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F18/00—Pattern recognition
    • G06F18/20—Analysing
    • G06F18/24—Classification techniques
    • G06F18/241—Classification techniques relating to the classification model, e.g. parametric or non-parametric approaches
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/04—Architecture, e.g. interconnection topology
    • G06N3/045—Combinations of networks
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/08—Learning methods

Landscapes

  • Engineering & Computer Science (AREA)
  • Theoretical Computer Science (AREA)
  • Physics & Mathematics (AREA)
  • Data Mining & Analysis (AREA)
  • Evolutionary Computation (AREA)
  • Life Sciences & Earth Sciences (AREA)
  • Artificial Intelligence (AREA)
  • General Physics & Mathematics (AREA)
  • General Engineering & Computer Science (AREA)
  • Computing Systems (AREA)
  • Software Systems (AREA)
  • Molecular Biology (AREA)
  • Computational Linguistics (AREA)
  • Biophysics (AREA)
  • Biomedical Technology (AREA)
  • Mathematical Physics (AREA)
  • General Health & Medical Sciences (AREA)
  • Health & Medical Sciences (AREA)
  • Bioinformatics & Cheminformatics (AREA)
  • Bioinformatics & Computational Biology (AREA)
  • Computer Vision & Pattern Recognition (AREA)
  • Evolutionary Biology (AREA)
  • Multimedia (AREA)
  • Machine Translation (AREA)

Abstract

The invention provides a book generating method, book generating equipment and a storage medium, wherein a story segment set and a role information set contained in text information of a target file are determined through the text information contained in the target file; determining an emotion label set corresponding to the role in the story segment according to the descriptive text information of the role characteristics in the role information set and the story segment contained in the story segment set; wherein the emotion label set comprises emotion labels of all characters in all story segments; and obtaining a book file corresponding to the target file according to the story segment set, the role information set and the emotion label set. The invention utilizes the language processing technology and the image generation technology to convert the existing story file into the book file, thereby overcoming the problem of lacking of the book file containing drawing.

Description

Book generation method, book generation equipment and storage medium
Technical Field
The present invention relates to the field of image processing technologies, and in particular, to a book generation method, a book generation device, and a storage medium.
Background
At present, reading is a daily habit of people, the most popular image type in many books is a book containing painting works, some pictures in the books are interlinked cartoon pictures compiled by using cartoon skills and styles, and some pictures are scene pictures for expressing scenes, and the books are very popular among readers due to interesting stories and beautiful pictures. The making of the picture in the current book containing the picture is usually performed by manual drawing of an author, the creation efficiency is low, and the creation of the painting works needs stronger painting skills, so that the current book containing the painting works in the book is deficient in resources and can not meet the requirements of readers.
Therefore, the prior art is subject to further improvement.
Disclosure of Invention
In view of the defects of the prior art, the invention aims to provide a book generation method, book generation equipment and a storage medium, which can quickly convert the existing book works only containing text information into the book works containing drawing works.
The technical scheme of the invention is as follows:
in a first aspect, the present embodiment provides a book generating method, including:
determining a story segment set and a role information set contained in the text information according to the text information contained in the target file; the story segment set comprises at least one story segment, and the role information set comprises description character information of role characteristics of at least one role;
determining an emotion label set corresponding to the role in the story segment according to the descriptive text information of the role characteristics in the role information set and the story segment contained in the story segment set; wherein the emotion label set comprises emotion labels of all characters in all story segments;
and obtaining a book file corresponding to the target file according to the story segment set, the role information set and the emotion label set.
Optionally, the step of determining a story segment set corresponding to the target file according to the text information includes:
inputting the text information into a trained story information extraction model to obtain a story segment set output by the story information extraction model; the story information extraction model is obtained by training based on the corresponding relation between a text sample information set and a plurality of story segments contained in the text sample information set.
Optionally, the story information extraction model includes: the device comprises an intermediate vector extraction module, a classifier module and a fragment collection module;
the step of inputting the text information into the trained story information extraction model to obtain the story segment set output by the story information extraction model comprises the following steps:
inputting the character information into the intermediate vector extraction module to obtain intermediate vector information output by the intermediate vector extraction module;
inputting the intermediate vector information into the classifier module to obtain a story quantity value which is output by the classifier module and corresponds to the text information;
and inputting the text information, the intermediate vector information and the story quantity value into the segment collection module to obtain a story segment set which is output by the segment collection module and corresponds to the text information.
Optionally, the step of determining the role information set contained in the text information according to the text information contained in the target file includes:
inputting the character information into a trained role information extraction model to obtain a role information set output by the role information extraction model; the character information extraction model is obtained by training based on the corresponding relation between a character sample information set and description character information of a plurality of character features contained in the character sample information set.
Optionally, the role information extraction model includes: a time sequence module and a feature extraction module;
the step of inputting the text information into the trained character information extraction model to obtain the character information set output by the character information extraction model comprises the following steps:
inputting the character information into the time sequence module to obtain a sequencing text which is output by the time sequence module and subjected to sequencing processing;
and inputting the sequencing text into the role characteristic extraction module to obtain a role information set which is output by the role characteristic extraction module and corresponds to the character information.
Optionally, the step of determining, according to the descriptive text information of the character feature in the character information set and the story segment contained in the story segment set, an emotion tag set corresponding to the character in the story segment includes:
inputting the story segment set and the role information set into a trained emotion information extraction model, and obtaining an emotion label set of a role through the emotion information extraction model, wherein the emotion information extraction model is obtained by training based on the corresponding relation among a story segment sample set, a role information sample set and a role emotion label sample set, and the role emotion label sample set is generated according to story segment samples contained in the story segment sample set and role characteristic description information contained in the role information sample set.
Optionally, the step of obtaining the book file corresponding to the target file according to the story segment set, the character information set, and the emotion tag set includes:
generating a role portrait of each role under different emotion labels according to the role information set and the emotion labels corresponding to each role;
and fusing the character portrait, the emotion labels corresponding to the characters and the story segment set to obtain a book file corresponding to the target file.
Optionally, the step of generating a character portrait of each character under different emotion labels according to the character information set and emotion labels corresponding to each character includes:
inputting the role information set and emotion labels corresponding to the roles into a trained portrait generation model to obtain role portraits of the roles output by the portrait generation model under different emotion labels, wherein the portrait generation model is obtained by training based on the corresponding relation among the role information sample set, the emotion label sample set and the role portrait sample set.
Optionally, the step of fusing the character representation, the emotion labels corresponding to the characters, and the story segment set to obtain the book file corresponding to the target file includes:
inputting the character portrait, emotion labels corresponding to the characters and the story segment set into a trained information fusion model to obtain book files output by the information fusion model and corresponding to the target file, wherein the information fusion model is obtained by training on the basis of the corresponding relationship among a character portrait sample set containing a plurality of character image samples, a character emotion label sample set containing a plurality of character emotion label samples, a story segment set containing a plurality of story segment samples and book samples, and the book samples are book samples generated according to the character portrait sample set, the character emotion label sample set and the story segment set.
Optionally, after the step of inputting the text information into the trained story information extraction model to obtain the story segment set output by the story information extraction model, the method further includes:
and inputting each story segment contained in the story segment set into the trained story refining model to obtain story segment essence output by the story refining model, and replacing each story segment with the story segment essence corresponding to the story segment essence.
In a second aspect, the present embodiment further provides a book generating device, including a processor, and a storage medium communicatively connected to the processor, wherein the storage medium is adapted to store a plurality of instructions; the processor is adapted to call instructions in the storage medium to perform the steps of implementing the book generation method.
In a third aspect, the present embodiment provides a computer-readable storage medium, wherein the computer-readable storage medium stores one or more programs, which are executable by one or more processors to implement the steps of the book generating method.
Has the advantages that: the invention provides a book generating method, book generating equipment and a storage medium, wherein a story segment set and a role information set contained in text information of a target file are determined through the text information contained in the target file; determining an emotion label set corresponding to the role in the story segment according to the descriptive text information of the role characteristics in the role information set and the story segment contained in the story segment set; wherein the emotion label set comprises emotion labels of all characters in all story segments; and obtaining a book file corresponding to the target file according to the story segment set, the role information set and the emotion label set. The invention utilizes the language processing technology and the image generation technology to convert the existing story file into the book file, thereby overcoming the problem of lacking of the book file containing drawing.
Drawings
FIG. 1 is a flow chart of the steps of the book generation method of the present invention;
FIG. 2 is a flow chart of the steps for extracting a story segment set using a story information extraction model in the method of the present invention;
FIG. 3 is a flowchart illustrating the steps of extracting a character information set using a character information extraction model according to the method of the present invention;
FIG. 4 is a flow chart of the steps of extracting emotion label sets of characters by using an emotion information extraction model in the method of the present invention;
FIG. 5 is a flowchart of the steps for generating book files using a deep network in the method of the present invention;
FIG. 6 is a schematic structural diagram of a book creation device in an embodiment of the present invention.
Detailed Description
In order to make the objects, technical solutions and advantages of the present invention clearer and clearer, the present invention is further described in detail below with reference to the accompanying drawings and examples. It should be understood that the specific embodiments described herein are merely illustrative of the invention and are not intended to limit the invention.
As used herein, the singular forms "a", "an", "the" and "the" are intended to include the plural forms as well, unless the context clearly indicates otherwise. It will be further understood that the terms "comprises" and/or "comprising," when used in this specification, specify the presence of stated features, integers, steps, operations, elements, and/or components, but do not preclude the presence or addition of one or more other features, integers, steps, operations, elements, components, and/or groups thereof. It will be understood that when an element is referred to as being "connected" or "coupled" to another element, it can be directly connected or coupled to the other element or intervening elements may also be present. Further, "connected" or "coupled" as used herein may include wirelessly connected or wirelessly coupled. As used herein, the term "and/or" includes all or any element and all combinations of one or more of the associated listed items.
It will be understood by those skilled in the art that, unless otherwise defined, all terms (including technical and scientific terms) used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this invention belongs. It will be further understood that terms, such as those defined in commonly used dictionaries, should be interpreted as having a meaning that is consistent with their meaning in the context of the prior art and will not be interpreted in an idealized or overly formal sense unless expressly so defined herein.
The inventor finds that cartoon book works in the prior art are deeply loved by readers, especially book works containing cartoons are loved by teenagers, but the existing domestic cartoon book works are scarce in resources and can not meet the requirements of readers far away, and in the manufacturing process of the cartoon book works, cartoon designers often draw cartoon roles manually, and then the cartoon roles are processed to complete the cartoon book works, so that the manufacturing efficiency is low.
In order to overcome the problems, based on the development of language processing technology and image generation technology, the method extracts story line information in a certain literary work, generates corresponding character images according to the character images in the story line, and combines the extracted story line with the character images to generate cartoon book works with the same content as the literary work. Because of the abundance of cultural background in China, a great deal of literature is suitable for being converted into book works containing drawings, such as: the conversion is carried out through a language processing technology and an image generation technology, so that the software is automatically realized, the efficiency is high, and the drawing books can be enriched, such as: cartoon resources.
The invention provides a book generation method, which comprises the steps of extracting text information contained in a certain literary work, and determining a story segment set and a role information set contained in the text information according to the extracted text information; for example: the literary work contains 6 story segments, 6 different stories are told respectively, and the 6 story segments form a story segment set. Since the story segments all relate to character images, such as the story segment 1 relates to the characters A and B, and the story segment 2 relates to the characters B and C, the description information for each character feature in each story segment constitutes a character information set. And then, according to the feature description text information corresponding to each role in each story segment, determining an emotion label set of the role, for example: in the story segment 1, the role A is happy, the role B is distressed and the like in the story segment 2, and the emotion labels corresponding to the roles in the story segments form an emotion label set of the roles. And obtaining a book file corresponding to the target file according to the story segment set, the role information set and the emotion label set. The book works with the same content as the existing story file are generated by extracting the story segment and the role information from the existing story file and fusing the story segment and the role information. And when the story segment set, the role information set and the emotion labels of the roles in the literary work are extracted, generating the cartoon book work according to the extracted information.
The method comprises the steps of uploading a file of a certain literary work to a computer, extracting literal information in the literary work through language processing software installed in the computer, obtaining story segments contained in the literal information, sequentially obtaining description information of each role characteristic from each story segment, generating emotion label sets comprising emotion labels corresponding to each role characteristic by combining the description information of the role characteristic and the story segments, inputting the description information and the emotion label sets corresponding to the role characteristic into image generation software, generating role portraits of each role under different emotion labels through the image generation software according to the received description information and the emotion label sets, and obtaining books corresponding to the literary work through the language information processing software according to the story segment sets, the emotion label sets and the role portraits.
It should be noted that the above application scenarios are only presented to facilitate understanding of the present invention, and the embodiments of the present invention are not limited in any way in this respect. Rather, embodiments of the present invention may be applied to any scenario where applicable.
Exemplary method
The embodiment provides a book generating method, as shown in fig. 2, including:
step S1, determining a story segment set and a role information set contained in the text information according to the text information contained in the target file; the story segment set comprises at least one story segment, and the role information set comprises description character information of role characteristics of at least one role.
In the step, a target file is firstly obtained, the text information of the target file is extracted, and a story segment set and a role information set contained in the story segment set are determined according to the text information of the target file.
Specifically, in the step, the target file is converted into the book file, so that preferably, the target file is a literary work such as a novel, a storytelling party, a literary famous work and the like containing a certain story line, wherein, the story is rich, and the target file is provided with a certain role and a story line and is suitable for being converted into the book work.
In this step, the character information of the whole content or part of the content of the target file can be identified through the character identification software, and the characters identified by storage form character information. The step can use the existing character information recognition software or a trained deep network model to input the content of the character information to be recognized into the deep network model, and the recognized character information is output through the deep network model.
And after the text information contained in the target file is identified, determining a story segment set and a role information set contained in the target file according to the text information.
Because the extracted text information can be subjected to semantic analysis by using semantic analysis software to obtain the story lines contained in the text information, the story lines can be divided into story segments from the analyzed story lines, and the story segments form a story segment set. And analyzing the semantic analysis result of the character information by using semantic analysis software to obtain the description information of the character features contained in the character information, and summarizing the description information of the character features into a character information set.
Specifically, the step of determining the story segment set corresponding to the target file according to the text information includes:
inputting the text information into a trained story information extraction model to obtain a story segment set output by the story information extraction model; the story information extraction model is obtained by training based on the corresponding relation between a text sample information set and a plurality of story segments contained in the text sample information set.
In the step, a story segment set corresponding to the text information is obtained by using the trained story information extraction model. The story information extraction model is a deep network model and is a story information extraction model for story information set extraction, which is obtained by inputting a character sample information set into a preset network model for multiple training.
Specifically, the text sample information contained in the text sample information set is input into a first neural network model, so that story segments output by the first neural network model and corresponding to the text sample information are obtained, and a story segment set corresponding to the text sample information set is formed by a plurality of story segments. The plurality of story segments output by the first neural network model are predicted values of story segments obtained by story segment extraction of each character sample information, error calculation is carried out on the story segment predicted values output by the first neural network model by using the true values of the story segments contained in the character sample information, errors between the predicted values and the true values are obtained according to the error calculation, and parameters of the first neural network model are optimized according to the errors. And repeating the steps of inputting the character sample information contained in the character sample information set into the first neural network model and optimizing the model parameters according to the forecast value of the story segment output by the model until the error meets the preset condition to obtain the trained story information extraction model.
Specifically, the story information extraction model includes: the device comprises an intermediate vector extraction module, a classifier module and a fragment collection module;
the step of inputting the text information into the trained story information extraction model to obtain the story segment set output by the story information extraction model comprises the following steps:
and step S111, inputting the character information into the intermediate vector extraction module to obtain intermediate vector information output by the intermediate vector extraction module.
Specifically, as shown in fig. 2, the text information is input to the intermediate vector extraction module, and the intermediate vector extraction module is used to obtain an intermediate vector associated with the text information. The intermediate vector extraction module takes the character information as input by using a transformer network structure and outputs an intermediate vector V of the character pair, wherein the intermediate vector is analysis data obtained by performing language information processing on the character information by using a natural language processing technology.
And step S112, inputting the intermediate vector information into the classifier module to obtain a story quantity value which is output by the classifier module and corresponds to the text information.
And (4) inputting the intermediate vector obtained in the step (S111) to a classifier module to obtain the number of story segments. Specifically, the classifier module is a classifier, and the classifier is used for analyzing the intermediate vector to obtain the number of story segments contained in the intermediate vector, wherein the number of story segments is formed by a cnn network.
Step S113, inputting the text information, the intermediate vector information and the story quantity value into the segment collection module to obtain a story segment set which is output by the segment collection module and corresponds to the text information.
Inputting the text information, the intermediate vector extracted in the step S111 and the story quantity value extracted in the step S112 into a segment collection module, and outputting a story segment set through the segment collection module. In one embodiment, the fragment aggregation module is comprised of a gpt2 network. The output story segment set includes a plurality of story segments. For example: generating small story-text segments of N stories from the original whole article, e.g. S ═ S1,s2,…,sNS represents a set of all story segments, and S represents each story segment.
Further, the step of determining the role information set contained in the text information according to the text information contained in the target file comprises the following steps:
inputting the character information into a trained role information extraction model to obtain a role information set output by the role information extraction model; the character information extraction model is obtained by training based on the corresponding relation between a character sample information set and description character information of a plurality of character features contained in the character sample information set.
In the step, a trained role information extraction model is used for extracting a role information set contained in the character information. The role information extraction model is a deep network model and is a role information extraction model for extracting role information, which is obtained by inputting a character sample information set into a preset network model for training for multiple times.
Specifically, the character sample information contained in the character sample information set is input into a second neural network model, so as to obtain the description information of the role characteristics output by the second neural network model and corresponding to each character sample information, and the description information of a plurality of role characteristics forms the role information set corresponding to the character sample information set. The description information of the plurality of role characteristics output by the second neural network model is a predicted value of the description information of the role characteristics extracted from each character sample information, the predicted value of the description information of the role characteristics output by the second neural network model is subjected to error calculation by using a real value of the description information of the role characteristics contained in the character sample information, an error between the predicted value and the real value is obtained according to the error calculation, and parameters of the second neural network model are optimized according to the error. And repeating the steps of inputting the character sample information contained in the character sample information set into a second neural network model, and optimizing the model parameters according to the predicted values of the character characteristic description information output by the model until the error meets the preset condition to obtain the trained character information extraction model.
In one embodiment, the character information extraction model includes: a time sequence module and a feature extraction module;
the step of inputting the text information into the trained character information extraction model to obtain the character information set output by the character information extraction model comprises the following steps:
and step S121, inputting the character information into the time sequence module to obtain a sequencing text which is output by the time sequence module and subjected to sequencing processing.
As shown in fig. 3, the text information extracted from the target file is input to the timing module, and the information contained in the text information is sequenced by the timing module, so as to ensure that the time sequence of each story contained in the text information is more accurate, specifically, in a specific application embodiment, the timing module may be implemented by using a rnn Network (Recurrent Neural Network), and the Network is trained by using the timing sequence of the general development trend of the story, so that after the text information is input into the Network, the Network structure may sequence the timing sequence contained in the input text information according to the memorized timing sequence information, and thereby output the sequenced text after sequencing.
And step S122, inputting the sequencing text into the role feature extraction module to obtain a role information set which is output by the role feature extraction module and corresponds to the character information.
And inputting the sorted text into a role feature extraction module, and outputting a role information set corresponding to the sorted text through the role feature extraction module. Specifically, in an implementation manner, the role feature extraction module may be implemented by using a cnn network, and extract description words corresponding to role features in the input word information by using the cnn network, and integrate each role description word set into a role information set.
For example: and if the character information relates to M characters in total, inputting the character information into the rnn network and the cnn network, and outputting the character information as description character information of the M character and color characteristics. Let this P ═ P1,p2,…,pMAnd P is a role information set, and represents the description text information of the individual role characteristics.
Step S2, according to the descriptive text information of the character characteristics in the character information set and the story segment contained in the story segment set, determining an emotion label set corresponding to the character in the story segment; wherein the emotion label set comprises emotion labels of the characters in the story clips.
In the step S1, the story segment set and the character information set are obtained, and because the story segments contained in the story segment set contain descriptive character information about characters, and the character information set also contains descriptive character information about character characteristics of each character, and the descriptive character information contained in the character information set and the story segment set for the same character is identical, a corresponding character can be identified from a plurality of story segments in the story segment set according to the descriptive character information about character characteristics of each character in the character information set, and an emotion label corresponding to each character can be identified according to the descriptive character information about character characteristics of each character in the story segment.
Furthermore, the character information describing the character characteristics contained in the character information set is a description of the character surface image, which is convenient for distinguishing each character in the story segment, such as: the character A is thin in image, female in gender and high in sub-height, the character description contained in the story segment is integrated into the story line, the weight of the character A is heavier than that of the character A, the information such as the image, emotion and the like of the character A appears in the story segment: and if a girl with a thin and tall body sees a certain article and then has a haha laugh, the character appearing in the story segment of the time can be identified as A and the emotion label of the character is corresponding to the smile from the descriptive text information.
In this step, the method for determining the emotion tag set according to the information contained in the story segment set and the character information set may be implemented in a processing manner based on a neural network.
Specifically, the step of determining the emotion label set corresponding to the character in the story segment according to the descriptive text information of the character feature in the character information set and the story segment contained in the story segment set includes:
inputting the story segment set and the role information set into a trained emotion information extraction model, and obtaining an emotion label set of a role through the emotion information extraction model, wherein the emotion information extraction model is obtained by training based on the corresponding relation among a story segment sample set, a role information sample set and a role emotion label sample set, and the role emotion label sample set is generated according to story segment samples contained in the story segment sample set and role characteristic description information contained in the role information sample set.
In the step, the trained emotion information extraction model is used, and the emotion labels of all the characters which appear in all story segments in a centralized manner are extracted through the emotion information extraction model. The emotion information extraction model is a deep network model, and is a trained emotion information extraction model obtained by inputting a plurality of story segments contained in a story segment sample set, description character information of a plurality of role characteristics contained in a role information sample set and a plurality of emotion label sample information contained in a role emotion label sample set into a preset third neural network model and training the third neural network model for a plurality of times.
Specifically, the story segment sample set and the role information sample set are input into a preset third neural network model, and emotion labels of all roles in all story segments output by the third neural network model are obtained. And an emotion label set consisting of emotion labels of all the characters in all story segments output by the third neural network model is a predicted value of the emotion labels of the characters in all the story segments in the story segment set and the character information set, the predicted value of the emotion label output by the third neural network model is subjected to error calculation by using a true value of the emotion label contained in the character emotion label sample, an error between the predicted value and the true value is obtained according to the error calculation, and parameters of the third neural network model are optimized according to the error. And repeating the steps of inputting the character sample information contained in the character sample information set into a third neural network model, and optimizing the model parameters according to the prediction value of the emotion label output by the model until the error meets the preset condition to obtain the trained emotion information extraction model.
In one embodiment, as shown in fig. 4, since the output of the emotion information extraction model is emotion labels of different characters, and thus belongs to the classification problem, the cnn network is selected to complete. The network input of the emotion information extraction model is the description character information of the N story segments S extracted by the story segment extraction model and the M role characteristics P extracted by the role information extraction model, and then the emotion information extraction model outputs emotion labels of M roles in the N story segments. E ═ E1,1,e1,2,…,e1,M,e2,1,…,eN,MIn which eN,MIndicating the mood of the mth character in the nth episode. If someoneNot present in a particular episode, the sentiment value is 0.
And step S3, obtaining a book file corresponding to the target file according to the story segment set, the role information set and the emotion label set.
According to the story segment set, the role information set and the emotion label set extracted in the steps, description character information of role characteristics contained in the role information set and all emotion labels contained in the emotion label set corresponding to the role information set are fused, role portraits corresponding to all roles under different emotions are obtained through an image generation method, and the role portraits of all the roles are fused into the extracted story segment set according to the emotion labels described in different story segments, so that the book file after conversion is obtained.
Specifically, the step of obtaining the book file corresponding to the target file according to the story segment set, the role information set, and the emotion tag set includes:
and step S31, generating a character portrait of each character under different emotion labels according to the character information set and the emotion labels corresponding to the characters.
And generating a character portrait under different emotion labels by using image generation software according to the description character information corresponding to each character feature in the character information set and the emotion labels corresponding to the characters.
Specifically, the step S31 of generating a character representation of each character under different emotion labels according to the character information set and emotion labels corresponding to each character includes:
step S311, inputting the role information set and emotion labels corresponding to the roles into a trained portrait generation model, and obtaining a role portrait of each role output by the portrait generation model under different emotion labels, wherein the portrait generation model is obtained by training based on the corresponding relationship among the role information sample set, the emotion label sample set and the role portrait sample set.
The process of generating the character portrait can be realized by utilizing a trained portrait generation model, namely, the character information set and emotion labels corresponding to all characters are input into the portrait generation model, and the portrait of each character under different emotions is obtained through the portrait generation model. The portrait generation model is a deep network model, and is a trained portrait generation model obtained by inputting the description character information of a plurality of character features contained in the character information sample set and the emotion label sample information contained in the character emotion label sample set into a preset fourth neural network model and training the fourth neural network model for a plurality of times.
Specifically, emotion labels of all roles in different emotions in the role information sample set and the role emotion label sample set are input into a preset fourth neural network model, and a role portrait of all the roles in different emotions output by the fourth neural network model is obtained. And a character image set consisting of the character images of each character under different moods output by the fourth neural network model is a predicted value of each character image in the character image set, the predicted value of the character image output by the fourth neural network model is subjected to error calculation by using a real value of the character image contained in the character image sample set, an error between the predicted value and the real value is obtained according to the error calculation, and parameters of the fourth neural network model are optimized according to the error. And repeating the steps of inputting the role information sample set and the role emotion label sample set into a fourth neural network model, and optimizing model parameters according to the predicted value of the role sketch output by the model until the error meets a preset condition to obtain a sketch generation model after training.
In one implementation, as shown in fig. 5, M character information and emotion information E are input to a trained image generation model, which is composed of a cnn + resnet network, and generates character images of M characters in different emotions.
And step S32, fusing the character portrait, the emotion labels corresponding to the characters and the story segment set to obtain a book file corresponding to the target file.
And respectively generating a character portrait of each character under different emotions, emotion labels corresponding to each character and a story segment set in the steps, and fusing the information to obtain a book file corresponding to the target file.
Furthermore, because each story segment in the story segment set contains different roles, different roles present different emotion labels and role portraits in each story segment, the role portraits corresponding to the emotion labels are matched with the corresponding story segments according to the emotion labels presented by each role in each story segment, and thus the book file is generated.
Specifically, the step of fusing the character portraits, the emotion labels corresponding to the characters and the story segment set to obtain the book file corresponding to the target file includes:
inputting the character portrait, emotion labels corresponding to the characters and the story segment set into a trained information fusion model to obtain book files output by the information fusion model and corresponding to the target file, wherein the information fusion model is obtained by training on the basis of the corresponding relationship among a character portrait sample set containing a plurality of character image samples, a character emotion label sample set containing a plurality of character emotion label samples, a story segment set containing a plurality of story segment samples and book samples, and the book samples are book samples generated according to the character portrait sample set, the character emotion label sample set and the story segment set.
The step can be realized by utilizing a trained information fusion model, and the role portraits, the emotion labels corresponding to the roles and the story clips are concentrated and input into the information fusion model to obtain the book files output by the information fusion model. The information fusion model is a deep network model, and is a trained information fusion model obtained by inputting the role portrait sample set, the role emotion label sample set and the story segment set into a preset fifth neural network model and training the fifth neural network model for multiple times.
Specifically, the role portrait sample set, the role emotion label sample set and the story segment set are input into a preset fifth neural network model, and a book file output by the fifth neural network model is obtained. And the book file information output by the fifth neural network model is a predicted value of cartoon information corresponding to the target file, the predicted value of the book file output by the fifth neural network model is subjected to error calculation by using the book file information real value in the book sample, the error between the predicted value and the real value is obtained according to the error calculation, and the parameter of the fifth neural network model is optimized according to the error. And repeating the steps of inputting the role portrait sample set, the role emotion label sample set and the story segment set into a preset fifth neural network model, and optimizing model parameters according to the predicted values of the book file contents output by the model until the errors meet preset conditions to obtain the trained information fusion model.
In one embodiment, the fifth neural network model generates a confrontation network, the generated story segment set, the image sample set of the character under different emotions and the emotion label sample set are input into the generation confrontation network (GAN network) together, and finally cartoon cartoons of M continuous story lines are generated, wherein the cartoon cartoons are provided with text expressions, and the character shows different emotions according to the story lines.
In order to further simplify the content contained in the book file and improve the efficiency of outputting the book file, after the step of inputting the text information into the trained story information extraction model and obtaining the story segment set output by the story information extraction model, the method further comprises the following steps:
and inputting each story segment contained in the story segment set into the trained story refining model to obtain story segment essence output by the story refining model, and replacing each story segment with the story segment essence corresponding to the story segment essence.
Namely, the N story segments extracted in the step S1 are input into the gpt2 network, and the story segments are secondarily refined to make the story segments shorter and more convenientRepresentative N segment story segment
Figure BDA0002582179240000172
The method provided by the present embodiment is further described with reference to fig. 5.
Firstly, an emotion information set E which is presented in different story segments by N story segments S, M and M role information P of character information contained in a target file and M role information is extracted.
Secondly, inputting the emotion information set E and the M role information P into a cnn + resnet network to generate role portraits of different roles under different emotion labels, and simultaneously sequentially inputting the N story segments S into a GPT2 network to obtain refined story segments corresponding to the N story segments respectively
Figure BDA0002582179240000171
Thirdly, the character portraits and the emotion information of different characters under different emotion labels are collected E, and story segments are refined
Figure BDA0002582179240000181
Inputting the information into an information fusion model (generating a confrontation network GAN), and obtaining the generated book works through the information fusion model.
The method provided by the invention can quickly convert the existing literary works into the book works containing the drawings, not only improves the generation efficiency of the book works, reduces the workload of book creation workers, but also can automatically generate the role portraits, solves the problem of the shortage of the current book works with the drawings, and has higher practical value.
Exemplary device
On the basis of the method, the embodiment also discloses book generation equipment, which comprises a processor and a storage medium in communication connection with the processor, wherein the storage medium is suitable for storing a plurality of instructions; the processor is adapted to call instructions in the storage medium to perform the steps of implementing the book generation method. The book generation device can be a mobile phone, a tablet computer or a smart television.
Specifically, as shown in fig. 6, the book generating device includes at least one processor (processor)20 and a memory (memory)22, and may further include a display 21, a communication Interface (Communications Interface)23 and a bus 24. The processor 20, the display 21, the memory 22 and the communication interface 23 can communicate with each other through the bus 24. The display screen 21 is configured to display a user guidance interface preset in the initial setting mode. The communication interface 23 may transmit information. The processor 20 may call logic instructions in the memory 22 to perform the methods in the embodiments described above.
Furthermore, the logic instructions in the memory 22 may be implemented in software functional units and stored in a computer readable storage medium when sold or used as a stand-alone product.
The memory 22, which is a computer-readable storage medium, may be configured to store a software program, a computer-executable program, such as program instructions or modules corresponding to the methods in the embodiments of the present disclosure. The processor 30 executes the functional application and data processing, i.e. implements the method in the above-described embodiments, by executing the software program, instructions or modules stored in the memory 22.
The memory 22 may include a storage program area and a storage data area, wherein the storage program area may store an operating system, an application program required for at least one function; the storage data area may store data created according to the use of the terminal device, and the like. Further, the memory 22 may include a high speed random access memory and may also include a non-volatile memory. For example, a variety of media that can store program codes, such as a usb disk, a removable hard disk, a Read-Only Memory (ROM), a Random Access Memory (RAM), a magnetic disk, or an optical disk, may also be transient storage media.
In another aspect, a computer-readable storage medium stores one or more programs, which are executable by one or more processors to implement the steps of the book generating method.
The invention provides a book generating method, book generating equipment and a storage medium, wherein a story segment set and a role information set contained in text information of a target file are determined through the text information contained in the target file; determining an emotion label set corresponding to the role in the story segment according to the descriptive text information of the role characteristics in the role information set and the story segment contained in the story segment set; the emotion label set comprises emotion labels corresponding to all characters in all story segments; and obtaining a book file corresponding to the target file according to the story segment set, the role information set and the emotion label set. The invention utilizes the language processing technology and the image generation technology to convert the existing story file into the book file, thereby overcoming the defect of lacking book files containing drawn pictures.
It should be understood that equivalents and modifications of the technical solution and inventive concept thereof may occur to those skilled in the art, and all such modifications and alterations should fall within the scope of the appended claims.

Claims (12)

1. A book generation method is characterized by comprising the following steps:
determining a story segment set and a role information set contained in the text information according to the text information contained in the target file; the story segment set comprises at least one story segment, and the role information set comprises description character information of role characteristics of at least one role;
determining an emotion label set corresponding to the role in the story segment according to the descriptive text information of the role characteristics in the role information set and the story segment contained in the story segment set; wherein the emotion label set comprises emotion labels of all characters in all story segments;
and obtaining a drawing book file corresponding to the target file according to the story segment set, the role information set and the emotion label set.
2. The book generation method of claim 1, wherein the step of determining a story segment set corresponding to the target file based on the textual information comprises:
inputting the text information into a trained story information extraction model to obtain a story segment set output by the story information extraction model; the story information extraction model is obtained by training based on the corresponding relation between a text sample information set and a plurality of story segments contained in the text sample information set.
3. The book generation method of claim 2, wherein the story information extraction model comprises: the device comprises an intermediate vector extraction module, a classifier module and a fragment collection module;
inputting the text information into a trained story information extraction model to obtain a story segment set output by the story information extraction model, wherein the story segment set comprises the following steps:
inputting the character information into the intermediate vector extraction module to obtain intermediate vector information output by the intermediate vector extraction module;
inputting the intermediate vector information into the classifier module to obtain a story quantity value which is output by the classifier module and corresponds to the text information;
and inputting the text information, the intermediate vector information and the story quantity value into the segment collection module to obtain a story segment set which is output by the segment collection module and corresponds to the text information.
4. The book generating method according to claim 2, wherein the step of determining the set of character information included in the text information of the target document based on the text information includes:
inputting the character information into a trained role information extraction model to obtain a role information set output by the role information extraction model; the character information extraction model is obtained by training based on the corresponding relation between a character sample information set and description character information of a plurality of character features contained in the character sample information set.
5. The book generation method of claim 4, wherein the role information extraction model comprises: a time sequence module and a feature extraction module;
the step of inputting the text information into the trained character information extraction model to obtain the character information set output by the character information extraction model comprises the following steps:
inputting the character information into the time sequence module to obtain a sequencing text which is output by the time sequence module and subjected to sequencing processing;
and inputting the sequencing text into the role characteristic extraction module to obtain a role information set which is output by the role characteristic extraction module and corresponds to the character information.
6. The book generation method of claim 4, wherein the step of determining the emotion label set corresponding to the character in the story segment according to the descriptive text information of the character feature in the character information set and the story segment contained in the story segment set comprises:
inputting the story segment set and the role information set into a trained emotion information extraction model, and obtaining an emotion label set of a role through the emotion information extraction model, wherein the emotion information extraction model is obtained by training based on the corresponding relation among a story segment sample set, a role information sample set and a role emotion label sample set, and the role emotion label sample set is generated according to the story segment sample contained in the story segment sample set and the description information of role characteristics contained in the role information sample set.
7. The book generation method of claim 2, wherein the step of obtaining the book file corresponding to the target file according to the story segment set, the character information set, and the emotion label set comprises:
generating a role portrait of each role under different emotion labels according to the role information set and the emotion labels corresponding to each role;
and fusing the character portrait, the emotion labels corresponding to the characters and the story segment set to obtain a book file corresponding to the target file.
8. The book generating method of claim 7, wherein the step of generating the character representation of each character under different emotion labels according to the character information set and the emotion label corresponding to each character comprises:
inputting the role information set and emotion labels corresponding to the roles into a trained portrait generation model to obtain role portraits of the roles output by the portrait generation model under different emotion labels, wherein the portrait generation model is obtained by training based on the corresponding relation among the role information sample set, the emotion label sample set and the role portrait sample set.
9. The book generation method of claim 7, wherein the step of fusing the character representation, the emotion labels corresponding to the characters, and the story segment set to obtain the book file corresponding to the target file comprises:
inputting the character portrait, emotion labels corresponding to the characters and the story segment set into a trained information fusion model to obtain book files output by the information fusion model and corresponding to the target file, wherein the information fusion model is obtained by training on the basis of the corresponding relationship among a character portrait sample set containing a plurality of character image samples, a character emotion label sample set containing a plurality of character emotion label samples, a story segment set containing a plurality of story segment samples and book samples, and the book samples are book samples generated according to the character portrait sample set, the character emotion label sample set and the story segment set.
10. The book generation method of any one of claims 2 to 9, wherein after the step of inputting the text information into the trained story information extraction model to obtain the story segment set output by the story information extraction model, the book generation method further comprises:
and inputting each story segment contained in the story segment set into the trained story refining model to obtain story segment essence output by the story refining model, and replacing each story segment with the story segment essence corresponding to the story segment essence.
11. A book generating device comprising a processor, a storage medium communicatively coupled to the processor, the storage medium adapted to store a plurality of instructions; the processor is adapted to invoke instructions in the storage medium to perform the steps of implementing the book generation method of any of the above claims 1-10.
12. A computer-readable storage medium, characterized in that the computer-readable storage medium stores one or more programs which are executable by one or more processors to implement the steps of the book generation method as claimed in any one of claims 1 to 10.
CN202010670728.0A 2020-07-13 2020-07-13 Book generation method, book generation device and storage medium Active CN113990356B (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN202010670728.0A CN113990356B (en) 2020-07-13 2020-07-13 Book generation method, book generation device and storage medium

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN202010670728.0A CN113990356B (en) 2020-07-13 2020-07-13 Book generation method, book generation device and storage medium

Publications (2)

Publication Number Publication Date
CN113990356A true CN113990356A (en) 2022-01-28
CN113990356B CN113990356B (en) 2023-05-16

Family

ID=79731308

Family Applications (1)

Application Number Title Priority Date Filing Date
CN202010670728.0A Active CN113990356B (en) 2020-07-13 2020-07-13 Book generation method, book generation device and storage medium

Country Status (1)

Country Link
CN (1) CN113990356B (en)

Citations (8)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN101382937A (en) * 2008-07-01 2009-03-11 深圳先进技术研究院 Multimedia resource processing method based on speech recognition and on-line teaching system thereof
US20110046943A1 (en) * 2009-08-19 2011-02-24 Samsung Electronics Co., Ltd. Method and apparatus for processing data
CN103873642A (en) * 2012-12-10 2014-06-18 北京三星通信技术研究有限公司 Method and device for recording call log
CN108470036A (en) * 2018-02-06 2018-08-31 北京奇虎科技有限公司 A kind of method and apparatus that video is generated based on story text
CN108764299A (en) * 2018-05-04 2018-11-06 北京物灵智能科技有限公司 Story model training and generation method, system, robot and storage device
CN109977238A (en) * 2019-04-01 2019-07-05 杭州赛鲁班网络科技有限公司 Generate the system for drawing this, method and apparatus
CN110472224A (en) * 2019-06-24 2019-11-19 深圳追一科技有限公司 Detection method, device, computer equipment and the storage medium of service quality
CN110765265A (en) * 2019-09-06 2020-02-07 平安科技（深圳）有限公司 Information classification extraction method and device, computer equipment and storage medium

Patent Citations (8)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN101382937A (en) * 2008-07-01 2009-03-11 深圳先进技术研究院 Multimedia resource processing method based on speech recognition and on-line teaching system thereof
US20110046943A1 (en) * 2009-08-19 2011-02-24 Samsung Electronics Co., Ltd. Method and apparatus for processing data
CN103873642A (en) * 2012-12-10 2014-06-18 北京三星通信技术研究有限公司 Method and device for recording call log
CN108470036A (en) * 2018-02-06 2018-08-31 北京奇虎科技有限公司 A kind of method and apparatus that video is generated based on story text
CN108764299A (en) * 2018-05-04 2018-11-06 北京物灵智能科技有限公司 Story model training and generation method, system, robot and storage device
CN109977238A (en) * 2019-04-01 2019-07-05 杭州赛鲁班网络科技有限公司 Generate the system for drawing this, method and apparatus
CN110472224A (en) * 2019-06-24 2019-11-19 深圳追一科技有限公司 Detection method, device, computer equipment and the storage medium of service quality
CN110765265A (en) * 2019-09-06 2020-02-07 平安科技（深圳）有限公司 Information classification extraction method and device, computer equipment and storage medium

Also Published As

Publication number Publication date
CN113990356B (en) 2023-05-16

Similar Documents

Publication Publication Date Title
CN107766371B (en) Text information classification method and device
CN111104498B (en) Semantic understanding method in task type dialogue system
CN111582241B (en) Video subtitle recognition method, device, equipment and storage medium
CN102193620B (en) Input method based on facial expression recognition
CN104076944A (en) Chat emoticon input method and device
CN104063683A (en) Expression input method and device based on face identification
CN104063427A (en) Expression input method and device based on semantic understanding
CN111488931A (en) Article quality evaluation method, article recommendation method and corresponding devices
CN102207950A (en) Electronic apparatus, image processing method and program
CN109376775A (en) The multi-modal sentiment analysis method of online news
CN114827752B (en) Video generation method, video generation system, electronic device and storage medium
CN111191503A (en) Pedestrian attribute identification method and device, storage medium and terminal
CN113761105A (en) Text data processing method, device, equipment and medium
CN117173497B (en) Image generation method and device, electronic equipment and storage medium
CN113920561A (en) Facial expression recognition method and device based on zero sample learning
CN115661846A (en) Data processing method and device, electronic equipment and storage medium
CN113990356B (en) Book generation method, book generation device and storage medium
CN116186310A (en) AR space labeling and displaying method fused with AI general assistant
CN116484872A (en) Multi-modal aspect emotion judging method and system based on pre-training and attention
CN115481254A (en) Method, system, readable storage medium and equipment for analyzing video effect content of movie and television play script
CN115757723A (en) Text processing method and device
CN115759293A (en) Model training method, image retrieval device and electronic equipment
CN116306506A (en) Intelligent mail template method based on content identification
CN114332476A (en) Method, device, electronic equipment, storage medium and product for identifying dimensional language
CN112287640A (en) Sequence labeling method based on Chinese character structure

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination
GR01 Patent grant
GR01 Patent grant